Hi, I read through the blog you linked, and guess you're referring to this paragraph:

Free up memory using del
This is a common pitfall for new PyTorch users, and we think it isn’t documented enough.
After you’re done with some PyTorch tensor or variable, delete it using the python del operator to free up memory.

In the blog post there is no data to back up this claim. As such, I think we'd need to see some numbers that your implementation uses less memory than the current one.

Although I agree that the current implementation is a little ugly, it seems to have unnecessary for loops. How about:

self._cache = self._cache[self._replace_num:self._replace_num + remain_num] + self._replacements[:self._replace_num]

@wyli

Thank you for your quick reply!

Your question for some data is very reasonable, and I have only some anecdotal data unfortunately. I have actually implemented my method in my own project, and instead of SLURM killing my job because I was out of memory, the job ran without problems. Of course, I have not implemented nor tested your method. I will see if I can create a test that shows the difference in data use. The problem to check that thoroughly is that the garbage collector of python (which is not completely determenistic) can either be quick, which will result in a small difference, or slow, which will result in higher memory spikes.

Sounds good, I'm curious to see what your benchmarking gives. My worry is that if we can't trust the garbage collector here, then we can't trust it anywhere in the code base and will have to delete everything everywhere. My default position is therefore to trust the garbage collector!

did a quick test it seems test2 is good:

import random

def test1(cache_num=100, replace_num=20):
  _cache = list(range(cache_num))
  replace = [random.random() for x in range(replace_num)]
  remain_num: int = cache_num - replace_num
  for i in range(remain_num):
    _cache[i] = _cache[i + replace_num]
  for i in range(replace_num):
    _cache[remain_num + i] = replace[i]
  return _cache

def test2(cache_num=100, replace_num=20):
  _cache = list(range(cache_num))
  replace = [random.random() for x in range(replace_num)]

  del _cache[:replace_num]
  _cache.extend(replace)
  return _cache

def test3(cache_num=100, replace_num=20):
  _cache = list(range(cache_num))
  replace = [random.random() for x in range(replace_num)]
  remain_num: int = cache_num - replace_num
  _cache = _cache[replace_num:replace_num + remain_num] + replace[:replace_num]
  return _cache

/black
thanks @cgrain, could you address this DCO issue by signing the commits? https://github.com/Project-MONAI/MONAI/pull/2181/checks?check_run_id=2565463758

Thanks @wyli, DCO should be signed now!