Skip to content

memory leak #4

@yueyang130

Description

@yueyang130

Hi, guys. Thanks for your great open-source work. When I use your rlds dataset built on tf.data.Dataset, I found there is memory leak when train=True. Specifically, when it's true and shuffle is called without cache, the memory gradually increases as the iteration goes.

dataset = dataset.shuffle(shuffle_buffer_size)

However, I don't understand why it happens. In my understanding, shuffle preloads a fixed size of data before iteration and then replace the used data with new data on the fly during iteration. So the memory should keep constant after preloading.
Could you provide some solution to the memory leak? If the leak is hard to solve, could you provide an approximate memory it needs to run the whole iteration during training?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions