Skip to content

More details for 'tail prefetch size is calculated based on'#12667

Closed
andlr wants to merge 5 commits intofacebook:mainfrom
andlr:tail-prefetch-warn
Closed

More details for 'tail prefetch size is calculated based on'#12667
andlr wants to merge 5 commits intofacebook:mainfrom
andlr:tail-prefetch-warn

Conversation

@andlr
Copy link
Copy Markdown
Contributor

@andlr andlr commented May 16, 2024

These messages indicate that SST file was created by a pre-9.0.0 RocksDB. Eventually, TailPrefetchStats might be removed, so it would be more informative if log message also included name of the affected SST file.

Issue: #12664

@facebook-github-bot
Copy link
Copy Markdown
Contributor

@hx235 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

Copy link
Copy Markdown
Contributor

@ajkr ajkr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a warning so we can track use cases that are at risk of regression when we finally remove TailPrefetchStats. The warnings and the risk can be mitigated by running compaction with a new enough RocksDB version (see #12664). If you still wanted to make a code change here, may I suggest a couple ideas:

  • As you mentioned the documentation was inadequate to know what to do. Maybe the warning text or surrounding comments could be improved.
  • Deduplicating the error could help with the spam problem. Maybe one warning per DB open saying N files are at risk.

@facebook-github-bot
Copy link
Copy Markdown
Contributor

@andlr has updated the pull request. You must reimport the pull request before landing.

@andlr andlr changed the title Use 'INFO' level for 'tail prefetch size is calculated based on' More details for 'tail prefetch size is calculated based on' May 17, 2024
@andlr
Copy link
Copy Markdown
Contributor Author

andlr commented May 17, 2024

It is a warning so we can track use cases that are at risk of regression when we finally remove TailPrefetchStats. The warnings and the risk can be mitigated by running compaction with a new enough RocksDB version (see #12664). If you still wanted to make a code change here, may I suggest a couple ideas:

  • As you mentioned the documentation was inadequate to know what to do. Maybe the warning text or surrounding comments could be improved.
  • Deduplicating the error could help with the spam problem. Maybe one warning per DB open saying N files are at risk.

Thanks for the explanation. I think in such case, it's probably enough to add name of the SST file into the log message, and a comment in code, saying that this backward compatibility won't be preserved forever

@andlr andlr requested a review from ajkr May 17, 2024 08:58
Copy link
Copy Markdown
Contributor

@ajkr ajkr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you!

@facebook-github-bot
Copy link
Copy Markdown
Contributor

@hx235 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Copy Markdown
Contributor

@andlr has updated the pull request. You must reimport the pull request before landing.

@facebook-github-bot
Copy link
Copy Markdown
Contributor

@hx235 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Copy Markdown
Contributor

@andlr has updated the pull request. You must reimport the pull request before landing.

@facebook-github-bot
Copy link
Copy Markdown
Contributor

@andlr has updated the pull request. You must reimport the pull request before landing.

@facebook-github-bot
Copy link
Copy Markdown
Contributor

@hx235 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Copy Markdown
Contributor

@hx235 merged this pull request in e4428b7.

@andlr andlr deleted the tail-prefetch-warn branch June 3, 2024 18:43
ybtsdst pushed a commit to ybtsdst/rocksdb that referenced this pull request Apr 27, 2025
…k#12667)

Summary:
These messages indicate that SST file was created by a pre-9.0.0 RocksDB. Eventually, `TailPrefetchStats` might be removed, so it would be more informative if log message also included name of the affected SST file.

Issue: facebook#12664

Pull Request resolved: facebook#12667

Reviewed By: ajkr

Differential Revision: D57464025

Pulled By: hx235

fbshipit-source-id: 12f2f2635e3092f8c29362aa132462492b5c1417
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants