Skip to content

Conversation

@vasild
Copy link
Contributor

@vasild vasild commented Jun 12, 2024

Problem:

If FuzzedSock::Recv(N, MSG_PEEK) is called then N bytes would be
retrieved from the fuzz provider, saved in m_peek_data and returned
to the caller (ok).

If after this FuzzedSock::Recv(M, 0) is called where M < N
then the first M bytes from m_peek_data would be returned
to the caller (ok), but the remaining N - M bytes in m_peek_data
would be discarded/lost (not ok). They must be returned by a subsequent
Recv().

To resolve this, only remove the head N bytes from m_peek_data.


This is a followup to #30211, more specifically:

#30211 (comment)
#30211 (comment)

@DrahtBot
Copy link
Contributor

DrahtBot commented Jun 12, 2024

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage

For detailed information about the code coverage, see the test coverage report.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK marcofleon, dergoegge, brunoerg

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

@brunoerg
Copy link
Contributor

From CI:

SUMMARY: UndefinedBehaviorSanitizer: invalid-null-argument test/fuzz/util/net.cpp:219:66 
MS: 0 ; base unit: 0000000000000000000000000000000000000000
0x5c,0x57,0x5c,0x57,0x5c,0xfb,0xff,0x29,0xa5,0x6,0x6,0xff,0x29,0x35,0xda,
\\W\\W\\\373\377)\245\006\006\377)5\332
artifact_prefix='./'; Test unit written to ./crash-c88fdfee4792d18945ffb11a7a520d9f0fb5e56b
Base64: XFdcV1z7/ymlBgb/KTXa

Traceback (most recent call last):
  File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/fuzz/test_runner.py", line 411, in <module>
    main()
  File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/fuzz/test_runner.py", line 199, in main
    run_once(
  File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/fuzz/test_runner.py", line 376, in run_once
    assert len(done_stat) == 1

@vasild vasild force-pushed the fuzzedsock_unbreak_recv_peek branch from 3ecd1c6 to f479325 Compare June 13, 2024 10:08
@vasild
Copy link
Contributor Author

vasild commented Jun 13, 2024

3ecd1c632f...f4793257cd: The UB sanitizer did not like the call to memcpy(..., nullptr, 0); which happens when the fuzzer input is exhausted. Added an explicit check about that.

Copy link
Contributor

@marcofleon marcofleon Jun 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why would m_peek_data be guaranteed to be empty here? I've tested this with the I2P harness and this assert fails. The m_peek_data doesn't get erased, so there's still some random bytes from the previous Recv call. The length of the copied data is just whatever was left, which isn't equal to len.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assert is too strong! Removed and adjusted the code afterwards to expect that m_peek_data may contain some bytes.

Btw, I ran FUZZ=i2p ./src/test/fuzz/fuzz for some time and it did not crash, maybe I did not run it long enough.

Copy link
Contributor

@marcofleon marcofleon Jun 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, I'll retest.

Btw, I ran FUZZ=i2p ./src/test/fuzz/fuzz for some time and it did not crash, maybe I did not run it long enough.

Interesting, mine crashed very quickly. Did you run it with the I2P dictionary? It's a lot faster using the dict so maybe that's why.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you run it with the I2P dictionary?

No. How do I do that? Do you have a Base64 of the crash unit?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can add it to the end of the command. So FUZZ=i2p ./src/test/fuzz/fuzz -dict=DICTFILE. The dictionary (i2p.dict) is in the qa-assets repo: https://github.com/bitcoin-core/qa-assets/tree/main/fuzz_dicts. I'm running it now on the most recent commit.

vasild added 2 commits June 14, 2024 14:44
`FuzzedSock::m_peek_data` need not be an optional of a vector.
It can be just a vector whereas an empty vector denotes "no peek data".
Problem:

If `FuzzedSock::Recv(N, MSG_PEEK)` is called then `N` bytes would be
retrieved from the fuzz provider, saved in `m_peek_data` and returned
to the caller (ok).

If after this `FuzzedSock::Recv(M, 0)` is called where `M < N`
then the first `M` bytes from `m_peek_data` would be returned
to the caller (ok), but the remaining `N - M` bytes in `m_peek_data`
would be discarded/lost (not ok). They must be returned by a subsequent
`Recv()`.

To resolve this, only remove the head `N` bytes from `m_peek_data`.
@vasild vasild force-pushed the fuzzedsock_unbreak_recv_peek branch from f479325 to 4d81b4d Compare June 14, 2024 12:57
@vasild
Copy link
Contributor Author

vasild commented Jun 14, 2024

f4793257cd...4d81b4de33: rebase and resolve #30273 (comment)

Copy link
Contributor

@marcofleon marcofleon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 4d81b4d. Tested this with the I2P fuzz target and there's no loss in coverage. I think overall this is an improvement in the robustness of Recv in FuzzedSock.

Copy link
Member

@dergoegge dergoegge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review ACK 4d81b4d


// Do the latency before any of the "return" statements.
if (m_fuzzed_data_provider.ConsumeBool() && std::getenv("FUZZED_SOCKET_FAKE_LATENCY") != nullptr) {
std::this_thread::sleep_for(std::chrono::milliseconds{2});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated to this PR but manual timeouts in a fuzz test are a bad idea. If a target requires the use of GetTime (or similar) then the harness needs to set mock time (as the test is otherwise non-deterministic).

We should get rid of this and replace it with calls to SetMockTime (we might need to make mock time more granular, i.e. currently it only supports seconds iirc).

@dergoegge
Copy link
Member

CI failure looks unrelated to me

Copy link
Contributor

@brunoerg brunoerg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK 4d81b4d

@fanquake fanquake merged commit c3b446a into bitcoin:master Jul 1, 2024
@vasild vasild deleted the fuzzedsock_unbreak_recv_peek branch July 1, 2024 11:01
@bitcoin bitcoin locked and limited conversation to collaborators Jul 1, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants