Skip to content

Add a __reduce__ method to Spec#25658

Merged
tgamblin merged 3 commits intospack:developfrom
alalazo:fixes/recursion_limit_pickle
Aug 28, 2021
Merged

Add a __reduce__ method to Spec#25658
tgamblin merged 3 commits intospack:developfrom
alalazo:fixes/recursion_limit_pickle

Conversation

@alalazo
Copy link
Copy Markdown
Member

@alalazo alalazo commented Aug 27, 2021

fixes #23892

Hitting the recursion limit seems to be due to the default way in which a Spec is serialized, following all the attributes. It's still not clear to me why this is related to being in an environment, but in any case we already have methods to serialize specs to
disk in JSON and YAML format. Here we use them to pickle a Spec instance too.

fixes spack#23892

The recursion limit seems to be due to the default
way in which a Spec is serialized, following all
the attributes. It's still not clear to me why this
is related to being in an environment, but in any
case we already have methods to serialize Specs to
disk in JSON and YAML format. Here we use them to
pickle a Spec instance too.
@alalazo
Copy link
Copy Markdown
Member Author

alalazo commented Aug 27, 2021

@adamjstewart This still has rough edges, but solves the issue for me in #23892 Can you confirm?

Hopefully nothing will change the package in
between serializing the spec and sending it
to the child process.
@alalazo
Copy link
Copy Markdown
Member Author

alalazo commented Aug 27, 2021

Currently this has issues with Python 2.X since we can't use a static method in Spec.__reduce__

@alalazo alalazo closed this Aug 27, 2021
@alalazo alalazo reopened this Aug 27, 2021
@alalazo alalazo marked this pull request as ready for review August 27, 2021 18:19
@alalazo alalazo added bugfix Something wasn't working, here's a fix environments labels Aug 27, 2021
Copy link
Copy Markdown
Member

@adamjstewart adamjstewart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed that this fixes the problem for me, thanks @alalazo!!!

@tgamblin tgamblin merged commit 40788cf into spack:develop Aug 28, 2021
@alalazo alalazo deleted the fixes/recursion_limit_pickle branch August 30, 2021 06:08
@haampie haampie mentioned this pull request Sep 16, 2021
11 tasks
haampie pushed a commit to haampie/spack that referenced this pull request Sep 16, 2021
* Add a __reduce__ method to Spec

fixes spack#23892

The recursion limit seems to be due to the default
way in which a Spec is serialized, following all
the attributes. It's still not clear to me why this
is related to being in an environment, but in any
case we already have methods to serialize Specs to
disk in JSON and YAML format. Here we use them to
pickle a Spec instance too.

* Downgrade to build-hash

Hopefully nothing will change the package in
between serializing the spec and sending it
to the child process.

* Add support for Python 2
haampie pushed a commit to haampie/spack that referenced this pull request Sep 17, 2021
* Add a __reduce__ method to Spec

fixes spack#23892

The recursion limit seems to be due to the default
way in which a Spec is serialized, following all
the attributes. It's still not clear to me why this
is related to being in an environment, but in any
case we already have methods to serialize Specs to
disk in JSON and YAML format. Here we use them to
pickle a Spec instance too.

* Downgrade to build-hash

Hopefully nothing will change the package in
between serializing the spec and sending it
to the child process.

* Add support for Python 2
haampie pushed a commit to haampie/spack that referenced this pull request Sep 17, 2021
* Add a __reduce__ method to Spec

fixes spack#23892

The recursion limit seems to be due to the default
way in which a Spec is serialized, following all
the attributes. It's still not clear to me why this
is related to being in an environment, but in any
case we already have methods to serialize Specs to
disk in JSON and YAML format. Here we use them to
pickle a Spec instance too.

* Downgrade to build-hash

Hopefully nothing will change the package in
between serializing the spec and sending it
to the child process.

* Add support for Python 2
haampie added a commit to haampie/spack that referenced this pull request Sep 17, 2021
@haampie haampie mentioned this pull request Sep 21, 2021
14 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Something wasn't working, here's a fix environments macOS

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RecursionError: maximum recursion depth exceeded while pickling an object

4 participants