Pickle is significantly slower than a memory copy

My machine copies memory at 5GB/s

In [1]: b = b'0' * 1000000000

In [2]: %time len(b[1:])
CPU times: user 139 ms, sys: 63.3 ms, total: 202 ms
Wall time: 202 ms
Out[2]: 999999999

But NumPy arrays only serialize at 2.5 GB/s

In [4]: import numpy as np

In [5]: x = np.random.randint(0, 255, dtype='u1', size=1000000000)  # 1GB

In [6]: import pickle

In [7]: %time len(pickle.dumps(x, protocol=-1))
CPU times: user 309 ms, sys: 96.2 ms, total: 405 ms
Wall time: 404 ms
Out[7]: 1000000161

Why the extra time?

Versions

Python 3.4, Linux, NumPy 1.11.0

@teoliphant

The answer is that ndarray.__reduce__ uses tostring() internally (making a copy) and then pickle.dumps makes an additional copy of any data is receives from __reduce__ (writing it into a io.BytesIO, of course).

Compare:

In [22]: %time _ = x.copy()
CPU times: user 286 ms, sys: 324 ms, total: 609 ms
Wall time: 609 ms

In [23]: %time _ = x.__reduce__()
CPU times: user 296 ms, sys: 320 ms, total: 616 ms
Wall time: 615 ms

In [24]: %time _ = pickle.dumps(x, protocol=-1)
CPU times: user 606 ms, sys: 682 ms, total: 1.29 s
Wall time: 1.29 s

It might be possible to do the pickling without an additional copy, but as far as I can tell based on the current design of pickle, that would require converting numpy arrays into bytes or another builtin Python type supported by pickle without a copy (you can't pickle memory views). Unfortunately, as @teoliphant explains, converting numpy arrays into strings without a copy isn't possible.

So I guess you could either try to get first class support for memoryview objects into pickle (maybe not a bad idea) or roll your own serialization format.

For those interested in this issue, would recommend taking a look at issue ( #11161 ). Basically there is a draft PEP that proposes leveraging the Python Buffer Protocol to perform zero-copy networking. There is also a backport package, pickle5, which implements this for recent Python 3 versions.

Sorry to hijack.

As an other note, it would also be really cool to get more bandwidth in memory copy in python in general.

➤ ipython                                                                                          (owl) 
Python 3.6.6 | packaged by conda-forge | (default, Jul 26 2018, 09:53:17) 
Type 'copyright', 'credits' or 'license' for more information
IPython 6.5.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: b = b'0' * 1000000000

In [2]: %timeit len(b[1:])
414 ms ± 851 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [3]: len(b)/414E-3 / 1E9
Out[3]: 2.4154589371980677

That is a memory bandwidth of 5GB/s (read + write)

Numpy's copy seems to be a bit faster clocking in at 5.8GB/s

In [1]: import numpy as np

In [2]: a = np.zeros(10*1024*1024*1024, 'byte')

In [3]: %timeit _ = a.copy()
3.44 s ± 9.64 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [4]: 10 / 3.44
Out[4]: 2.9069767441860463

Following what this article is saying:
http://codearcana.com/posts/2013/05/18/achieving-maximum-memory-bandwidth.html
and using their benchmark https://github.com/awreece/memory-bandwidth-demo

On my machine I was able to obtain the following numbers:

➤ ./memory_profiler 
           read_memory_rep_lodsq: 11.46 GiB/s
                read_memory_loop: 14.09 GiB/s
                 read_memory_sse: 14.50 GiB/s
                 read_memory_avx: 13.04 GiB/s
        read_memory_prefetch_avx: 13.11 GiB/s
               write_memory_loop:  9.21 GiB/s
          write_memory_rep_stosq: 13.59 GiB/s
                write_memory_sse:  9.15 GiB/s
    write_memory_nontemporal_sse: 20.68 GiB/s
                write_memory_avx:  9.22 GiB/s
    write_memory_nontemporal_avx: 20.66 GiB/s
             write_memory_memset: 14.04 GiB/s
       read_memory_rep_lodsq_omp: 42.03 GiB/s
            read_memory_loop_omp: 42.06 GiB/s
             read_memory_sse_omp: 42.00 GiB/s
             read_memory_avx_omp: 41.48 GiB/s
    read_memory_prefetch_avx_omp: 41.28 GiB/s
           write_memory_loop_omp: 20.41 GiB/s
      write_memory_rep_stosq_omp: 36.22 GiB/s
            write_memory_sse_omp: 20.40 GiB/s
write_memory_nontemporal_sse_omp: 46.65 GiB/s
            write_memory_avx_omp: 20.39 GiB/s
write_memory_nontemporal_avx_omp: 51.50 GiB/s
         write_memory_memset_omp: 38.56 GiB/s

This blows any single digit performance that we get with python out the water.

PS. I had to change the size of the benchmark to 6GB since my computer has 6 cores + hyperthreading. Maybe I could have done with 3GB to get that factor of 3 in the size of the benchmark.

note that your numpy benchmark probably spends most of its time pagefaulting new memory so it is not comparable with the other benchmark.
doing following with a preallocated destination will likely be much faster:

%timeit np.copyto(dst, src)

I reach 20GiB/s on my machine (using glibc 2.27 which uses avx nontemporal writes)

@juliantaylor

Thanks @juliantaylor!

I'm up to 10GB/s copy speed so 20GB/s bandwidth. Much more respectable! (Conda forge numpy, I'm not sure what version of glibc they use).

benchmarks make my head hurt :S.

When allocating and deallocating large arrays, is numpy/python smart enough to reuse memory so it doesn't keep page faulting?

On which platform and C library are on? for copying numpy just uses that so the performance depends on the platform. On mine the threaded benchmark you linked reaches 27GiB max so numpy is pretty close.

unfortunately numpy does not cache large arrays (only small ones). gh-8783 is about using a special linux features to cache large arrays without using up all of the systems memory when its not needed. But it still needs some time spent on for review and refactoring.

I'm on an updated version of conda-forge, I think they use glibc 2.55, but I'm pretty sure they limit it to SSE2 (soon to be SSE3). Ubuntu 16.04 64 bit.

at first the 2.5GB/s (or 5GB/s) was an order of magnitude off. the 10GB/s or 20GB/s depending on how you count is much better already.

PS. if there a more appropriate place to follow up on this conversation, please let me know. separate issue / stack overflow. I don't know if this conversation derailed too much from the pickling question.

Probably worth rechecking with protocol 5 now that pickle5 will be used if found on Python 3.6+ as of NumPy 1.16.0.

Support for protocol 5 has been merged. Closing. If the copy benchmark analysis leads to another issue, please open a new one.

Navigation Menu

Repository navigation

Pickle is significantly slower than a memory copy #7544

Versions

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Pickle is significantly slower than a memory copy #7544

Description

Versions

Activity

shoyer commented on Apr 14, 2016

jakirkham commented on Aug 8, 2018

hmaarrfk commented on Sep 5, 2018

juliantaylor commented on Sep 7, 2018

hmaarrfk commented on Sep 7, 2018

juliantaylor commented on Sep 7, 2018

hmaarrfk commented on Sep 7, 2018

jakirkham commented on Jan 16, 2019

mattip commented on May 7, 2019

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions