Skip to content

Fix bug in temporary path setting (was: Clear up cache-related warnings on Windows) - #9703

Merged
pllim merged 1 commit into
astropy:masterfrom
aarchiba:cache-warning
Dec 5, 2019
Merged

pllim merged 1 commit into
astropy:masterfrom
aarchiba:cache-warning

Conversation

@aarchiba

@aarchiba aarchiba commented Nov 28, 2019 •

Copy link
Copy Markdown
Contributor

Description

#9634 suppressed a certain number of warning from the cache module. This PR is meant to track them down and test for them as appropriate so that they can be left enabled.

It turns out they all stemmed from a single bug, which was that if an error occurred at a particular moment, the command that temporarily resets the cache location fails to reset it.

Fixes #9704
Fixes #9620

@aarchiba

Copy link
Copy Markdown
Contributor Author

OK now that's really weird - non-data tests, astropy/visualization/wcsaxes/tests/test_images.py::TestBasic.test_contour_overlay are using the temporary cache created for tests in test_data.py: /tmp/pytest-of-travis/pytest-0/test_cache_dir_is_actually_a_f0/file/astropy. The temporary cache setting should be going away as soon as the test is over, it's established by a pytest fixture in test_data.py.

@aarchiba

aarchiba commented Nov 29, 2019 •

Copy link
Copy Markdown
Contributor Author

The fix for #9704 would be much clearer (and in fact the bug would not have arisen) if the context managers had been written as generators with @contextlib.contextmanager but they store data globally on the class object so it's a little weird, and they use inheritance, which is awkward to replicate with functions.

Lesson: don't use explicit __enter__ and __exit__ unless you absolutely have to.

@aarchiba aarchiba changed the title WIP: clear up cache-related warnings on Windows Clear up cache-related warnings on Windows Nov 30, 2019
@bsipocz bsipocz added this to the v4.0.1 milestone Nov 30, 2019
@pllim
pllim self-requested a review December 1, 2019 15:58
@pllim pllim added the Bug label Dec 1, 2019
@aarchiba

aarchiba commented Dec 2, 2019

Copy link
Copy Markdown
Contributor Author

@bsipocz - should I zap the changelog entry then? This does actually fix a bug in astropy.config.paths. maybe I should update the description too?

@bsipocz

bsipocz commented Dec 2, 2019

Copy link
Copy Markdown
Member

Yes, the current one has to go as none of that is user facing. However, I'm not against adding one for the config part.

But, let's wait for @pllim's review.

@pllim

pllim commented Dec 2, 2019

Copy link
Copy Markdown
Member

Sorry, this is on my list to review but I am catching up with post-gobble-gobble email pile. 🦃 😬

@pllim pllim left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread CHANGES.rst Resolved
Comment thread astropy/config/paths.py Resolved
Comment thread astropy/utils/tests/test_data.py Resolved
Comment thread astropy/utils/tests/test_data.py Resolved
@pllim pllim added the config label Dec 3, 2019
@pllim

pllim commented Dec 3, 2019

Copy link
Copy Markdown
Member

I can confirm that this PR clears the warning I saw in #9634 🎉

Once @bsipocz gives instruction on how to proceed with the change log (and maybe squashing), this is good to go. Thank you very much!

@pllim

pllim commented Dec 3, 2019

Copy link
Copy Markdown
Member

Depending on whether this or #9669 gets merged first, the other will need a rebase to rerun the CI before merge, just in case.

@bsipocz bsipocz modified the milestones: v4.0.1, v4.0 Dec 4, 2019
@aarchiba aarchiba changed the title Clear up cache-related warnings on Windows Fix bug in temporary path setting (was: Clear up cache-related warnings on Windows) Dec 4, 2019
@pllim pllim added the Affects-dev PRs and issues that do not impact an existing Astropy release label Dec 4, 2019
@pllim

pllim commented Dec 4, 2019

Copy link
Copy Markdown
Member

Please rebase and let the CI run one more time now that #9669 is merged. Thank you!

@pllim

pllim commented Dec 5, 2019

Copy link
Copy Markdown
Member

One of the job failed with some cache error. I restarted it in case it's transient.

@pllim

pllim commented Dec 5, 2019

Copy link
Copy Markdown
Member

Still failed. @aarchiba , any ideas?

astropy.utils.data.CacheDamaged: Apparently abandoned files: {'C:\\...\pytest-0\\test_check_download_cache0\\astropy\\download\\py3\\709c583b30be1ba30dab6b73157c9610'}

@pllim

pllim commented Dec 5, 2019

Copy link
Copy Markdown
Member

I manually deleted Travis cache for this PR and rerun the failed job again. We'll see.

@pllim

pllim commented Dec 5, 2019

Copy link
Copy Markdown
Member

Tests passed. @aarchiba , is that CacheDamaged error earlier of any concern? Or can we merge this?

@aarchiba

aarchiba commented Dec 5, 2019

Copy link
Copy Markdown
Contributor Author

I don't think the CacheDamaged is real - I mean it shouldn't happen, but I didn't change anything relevant to it, and it went away on a clean rerun. And the failure mode is that there is an unreferenced file left behind, which is a less-serious sort of error. I don't know where it came from. But the state of the code previous to this fix introduced some nasty dependences between test cases.

@pllim

pllim commented Dec 5, 2019

Copy link
Copy Markdown
Member

Okay, merging then. Thank you very much!

@pllim
pllim merged commit ceef2b3 into astropy:master Dec 5, 2019
@aarchiba
aarchiba deleted the cache-warning branch December 5, 2019 20:23
bsipocz pushed a commit that referenced this pull request Dec 7, 2019
Fix bug in temporary path setting (was: Clear up cache-related warnings on Windows)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Affects-dev PRs and issues that do not impact an existing Astropy release Bug config no-changelog-entry-needed utils

Projects

None yet

3 participants