Repository navigation
Propose to add note about default behaviour of TFRecord Reader to documentation #3088
Description
Hi,
I'm trying to use TFRecord Reader instead of FileReader and there is a significant difference in model accuracy. After some debugging and reading the issues it appears that the problem appears because TFRecordReader expects samples to be shuffled. In my case setting initial_fill to be equal to size of the dataset fixed the issue
The problem is, that the fact that DALI expects TFRecords to be preshuffled is only mentioned in several issues on GitHub: #2849 (comment) and #1774 (comment) while in the official documentation there is nothing about it. Such behaviour is confusing and may lead to undesired behaviour.
So I'm proposing to add information about pre-shuffle to TFRecordReader class documentation.
Activity
[-]Need note about default behaviour of TFRecord Reader[/-][+]Propose to add note about default behaviour of TFRecord Reader to documentation[/+]JanuszL commentedon Jun 24, 2021
Hi @bonlime,
I thought that it is a common practice to preshuffle TFRecord. DALI shuffling behavior is similar to one in TensorFlow itself.
If you have any particular wording in mind that would improve the documentation why not create a PR with proposed changes to this file?
We would be more than happy to review and merge such contribution.
bonlime commentedon Oct 1, 2026
Oh man it's so funny how LLM assisted PRs finally close issues open for years (not that it even bothered me ever again after the 5 years but still)