Search before asking

• I searched in the issues and found nothing similar.

Motivation

Parallelly read file in AbstractFileStoreScan class plan method will run a lot of memory while files number is large.

Solution

Make memory usage limit to
'''
readManifests
.parallelStream()
.filter(this::filterManifestFileMeta)
.flatMap(m -> readManifestFileMeta(m).stream())
.filter(this::filterByStats)
.collect(Collectors.toList()))
'''

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!