Skip to content

Cannot pickle numpy.datetime64 scalars in Python 3.6 #11656

Closed

Description

@xhochy

Reproducing code example:

Using numpy==1.15.0 and Python 3.6

import pickle
import numpy as np
pickle.dumps(np.datetime64('2016-01-01T00:00:00.000000000'))

leads to BufferError: could not get scalar buffer information

Numpy/Python version information:

>>> import sys, numpy; print(numpy.__version__, sys.version)
1.15.0 3.6.4 | packaged by conda-forge | (default, Dec 23 2017, 16:54:01)
[GCC 4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)]

Activity

  1. tylerjereddy commented on Aug 3, 2018

    @tylerjereddy
    Contributor

    I bisected to identify the problematic commit:

    e4d678a2f5859d29a853d617e9e5bbd4b6241898 is the first bad commit
    commit e4d678a2f5859d29a853d617e9e5bbd4b6241898
    Author: Jeff VanOss <[email protected]>
    Date:   Sun Mar 25 01:29:20 2018 -0400
    
        BUG: Provide correct format in Py_buffer for scalars (#10564)
        
        Unifies scalar and ndarray pep3118 format string generation
    
    :040000 040000 42eebbd3aa7713e65ba2c7ecee910e8eea53fd1c 60e7cdc28dbd474ad4205a6628cec1f0a9749884 M	numpy
    
  2. added this to the 1.15.1 release milestone on Aug 4, 2018
  3. adaag commented on Aug 8, 2018

    @adaag

    @xhochy
    def parallel_apply(data, fun, args):
    pool = mp.Pool(mp.cpu_count() - 1)
    result = [pool.apply(fun, (data.loc[x, :], *args)) for x in data.index]
    result = [p.get() for p in result]
    end_result = pd.concat(result)
    return end_result

    from this part of code i also get following error "BufferError: could not get scalar buffer information". i tried several things to solve it but non of them worked. Any sugesstion? Thanks for help

  4. Ape commented on Aug 20, 2018

    @Ape

    Is there a release planned that would contain this fix?

  5. eric-wieser commented on Aug 20, 2018

    @eric-wieser
    Member

    image

    :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions