Describe the problem.

When attempting to run a comparison of the MapReduce and Cholesky applications using Dask with and without ProxyStore, the ProxyStore implementations fail. Its also worth noting that these errors occur with and without the -- engine.transformer.extract-target true flag. The steps for setup for both applications were exactly the same as the TaPS application guides, and I used the data/maildir dataset for the MapReduce. The logs for both runs are attached as a file. It looks like for MapReduce all of the tasks successfully execute in the task log except for the last, the "reduce_task", and there is some future iteration issue after the store closes. For Cholesky the tasks fail on the workers, in the tasks log only "potrf" reports success.
choleskylog.txt
mapreducelog.txt

How did you install and run the code?

$ pip install -e .[cholesky]
$ python3 -m taps.run --app cholesky --engine.executor dask --app.matrix-size 10000 --app.block-size 1000 --engine.transformer proxystore --engine.transformer.connector.kind file --engine.transformer.connector.options {"store_dir": "/tmp/proxystore-cache"} --engine.transformer.extract-target true

$ curl -L https://www.cs.cmu.edu/~enron/enron_mail_20150507.tar.gz | tar -xz -C data/
$ python3 -m taps.run --app mapreduce --app.data-dir data/maildir --app.map-tasks 32 --engine.executor dask --engine.transformer proxystore --engine.transformer.connector.kind file --engine.transformer.connector.options {"store_dir": "/tmp/proxystore-cache"} --engine.transformer.extract-target true

Version

v0.2.0

Python Version

3.8.10

OS and Platform

x86 Linux