This piece of code consumes about 8GB of memory (peak), as measured by command time ./fab.py on Ubuntu 16.04 and looking at the max RSS:

#!/usr/bin/env python
import numpy as np
import dask.array as da

shape = (1000, 1000000)
dtype = np.complex64
array = da.ones(shape, dtype=dtype, chunks=(1, shape[1]))
sliced = array[:, 0:1]

if True:
    out = np.empty(sliced.shape, sliced.dtype)
    da.store(sliced, out, lock=False)
else:
    out = sliced.compute()

Changing the if True to if False reduces the max RSS to about 256MB (although it is slower), and has the same semantics (correct me if I'm wrong on that).

The script is creating an array with 1xN chunks, then takening an Mx1 slice out of it to compute. That's obviously a poor chunking scheme, but we're running into less extreme versions of this in real code where an array is stored in chunks on disk that need to cater for multiple access patterns and one particular access pattern is to use a slice width of 1 along a dimension where the chunks are much wider than 1.

I haven't dug into this at all, but my first guess would be that sliced.compute() is first computing all the chunks of sliced in memory before concatenating them, and the slicing operation takes a slice from each chunk which is a view and hence keeps the full original chunk pinned; while da.store is copying each chunk into out as it is computed.

I'm using Python 3.5 (also an issue on 2.7), dask 0.17.5, toolz 0.9.0, Ubuntu 16.04. Having read #3530 I've also tried setting MALLOC_MMAP_THRESHOLD_=16384: it doesn't fix the problem.