Skip to content

Conversation

@achow101
Copy link
Member

@achow101 achow101 commented Apr 8, 2021

Since we want tests to run quickly, and since tests do a lot more db operations than expected we expect to see in actual usage, we disable sqlite's syncing behavior to make db operations run much faster. This syncing behavior is necessary for normal operation as it helps guarantee that data won't become lost or corrupted, but in tests, we don't care about that.

Fixes #21628

@DrahtBot
Copy link
Contributor

DrahtBot commented Apr 8, 2021

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

Copy link
Member

@maflcko maflcko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a linter comment, didn't review

Copy link
Contributor

@vasild vasild left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How much performance boost does this bring?

Copy link
Contributor

@vasild vasild left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 5c72335f0f848895079913e58534ac9488c18ed7

Locally,wallet_tests/CreateWallet test time [seconds]:

master (cb79cabd) sqlite: 63, 62, 60
master (cb79cabd) bdb:     2,  3,  3
PR     (5c72335f) sqlite:  3,  3,  3

Maybe add -unsafesqlitesync=1 to self.args[] if there is no reason to use a separate call to append_config().

@laanwj
Copy link
Member

laanwj commented Apr 12, 2021

Concept ACK. Syncing is a robustness measure, and unless you're testing the syncing itself, there is no use to have it enabled in tests (at the expense if extra testing time).

Seperately it would make sense to look into the sqlite performance issues, but this is a win.

Since we want tests to run quickly, and since tests do a lot more db
operations than expected we expect to see in actual usage, we disable
sqlite's syncing behavior to make db operations run much faster. This
syncing behavior is necessary for normal operation as it helps guarantee
that data won't become lost or corrupted, but in tests, we don't care
about that.
Copy link
Contributor

@vasild vasild left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 41f891d

@maflcko maflcko merged commit a1f0b8b into bitcoin:master Apr 13, 2021
sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Apr 13, 2021
41f891d tests: Skip SQLite fsyncs while testing (Andrew Chow)

Pull request description:

  Since we want tests to run quickly, and since tests do a lot more db operations than expected we expect to see in actual usage, we disable sqlite's syncing behavior to make db operations run much faster. This syncing behavior is necessary for normal operation as it helps guarantee that data won't become lost or corrupted, but in tests, we don't care about that.

  Fixes bitcoin#21628

ACKs for top commit:
  vasild:
    ACK 41f891d

Tree-SHA512: f36f969a182c622691ae5113573a3250e8d367437e83a1a9d3d2b55dd3a9cdf3c6474169a7bd271007bb9ce47f585aa7a6aeae6eebbaeb02d79409b02f47fd8b
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Aug 18, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Up to 100x perfomance difference for sqlite vs BDB in CreateWallet test

6 participants