Repository navigation
Make overwrite=True work with Path arguments - #17552
Conversation
|
Thank you for your contribution to Astropy! 🌌 This checklist is meant to remind the package maintainers who will review this pull request of some common things to look for.
|
| try: | ||
| t1.write(Path(filename), format="fits", overwrite=True) | ||
| except OSError as error: | ||
| pytest.fail(f"overwrite true raises {error}") |
There was a problem hiding this comment.
filename is already a path using tmp_path. No need to explicitly catch the exception because any exception or unhandled warning is going to fail the test anyway.
| try: | |
| t1.write(Path(filename), format="fits", overwrite=True) | |
| except OSError as error: | |
| pytest.fail(f"overwrite true raises {error}") | |
| t1.write(filename, format="fits", overwrite=True) |
There was a problem hiding this comment.
Same comment as above ... for some reason just using filename does not make the test fail on main...
|
I have investigated a bit more the issue on the test (see this comment) and decided to remove the check on the type of |
I have added a context manager to move to the temporary directory and come back after the test.... @pllim is that ok with you? |
| ) | ||
|
|
||
| def test_path(self, tmp_path): | ||
| with working_directory(tmp_path): |
There was a problem hiding this comment.
Like you, I am now very perplexed on why this bug only shows up when the filename has no path attached to it. 🤯
Perhaps we should wait for @saimn to review. He might have extra understanding in regards to io.fits. 🙏
| input : Table | ||
| The table to write out. | ||
| output : str | ||
| output : str or os.PathLike[str] |
There was a problem hiding this comment.
not sure what the [str] suffix means in this case ?
There was a problem hiding this comment.
os.PathLike comes in two forms: os.PathLike[bytes] and os.PathLike[str]
There was a problem hiding this comment.
perhaps a simpler way to go would be
| output : str or os.PathLike[str] | |
| output : str or pathlib.Path |
(it's more specific than it needs to, but should cover 100% of what real users need or want)
There was a problem hiding this comment.
I'm not an expert at all about paths.... I just did a git grep in astropy and there are no os.Pathlike, while 6 occurrences of os.Pathlike[str] ... this is why I put os.Pathlike[str] since ced653d for consistency
There was a problem hiding this comment.
I probably authored most of these annotations, so it is fine as is, AFAIC, I'm just offering an alternative in case @saimn has a different preference
|
Okay I've played a little with this locally and I think I got a complete solution to the issue with your test diff --git a/astropy/io/fits/tests/test_connect.py b/astropy/io/fits/tests/test_connect.py
index 969eee2691..f2eb77a33d 100644
--- a/astropy/io/fits/tests/test_connect.py
+++ b/astropy/io/fits/tests/test_connect.py
@@ -1,6 +1,4 @@
-import contextlib
import gc
-import os
import warnings
from pathlib import Path
@@ -60,21 +58,6 @@ def equal_data(a, b):
return all(np.all(a[name] == b[name]) for name in a.dtype.names)
-@contextlib.contextmanager
-def working_directory(path):
- """
- A context manager changing the working directory to the given
- path. The directory is changed back on exit.
- """
-
- cwd = os.getcwd()
- os.chdir(path)
- try:
- yield
- finally:
- os.chdir(cwd)
-
-
class TestSingleTable:
def setup_class(self):
self.data = np.array(
@@ -83,11 +66,10 @@ class TestSingleTable:
)
def test_path(self, tmp_path):
- with working_directory(tmp_path):
- filename = "temp.fits"
- t1 = Table(self.data)
- t1.write(filename, overwrite=True)
- t1.write(Path(filename), format="fits", overwrite=True)
+ filename = tmp_path / "temp.fits"
+ t1 = Table(self.data)
+ t1.write(filename, overwrite=True)
+ t1.write(Path(filename), format="fits", overwrite=True)
def test_simple(self, tmp_path):
filename = tmp_path / "test_simple.fts"
diff --git a/astropy/io/registry/core.py b/astropy/io/registry/core.py
index b7325c4beb..3c0e1fad25 100644
--- a/astropy/io/registry/core.py
+++ b/astropy/io/registry/core.py
@@ -15,7 +15,7 @@ def _expand_user_in_args(args):
# Conservatively attempt to apply `os.path.expanduser` to the first
# argument, which can be either a path or the contents of a table.
if len(args) and isinstance(args[0], PATH_TYPES):
- ex_user = os.path.expanduser(args[0])
+ ex_user = type(args[0])(os.path.expanduser(args[0]))
if ex_user != args[0] and os.path.exists(os.path.dirname(ex_user)):
args = (ex_user,) + args[1:]
return args
diff --git a/astropy/io/votable/connect.py b/astropy/io/votable/connect.py
index 8c58a9b105..68dcb63366 100644
--- a/astropy/io/votable/connect.py
+++ b/astropy/io/votable/connect.py
@@ -163,7 +163,7 @@ def write_table_votable(
)
# Check if output file already exists
- if isinstance(output, str) and os.path.exists(output):
+ if isinstance(output, (str, os.PathLike)) and os.path.exists(output):
if overwrite:
os.remove(output)
else:
diff --git a/astropy/io/votable/util.py b/astropy/io/votable/util.py
index 970a9cbd03..c8537da4f3 100644
--- a/astropy/io/votable/util.py
+++ b/astropy/io/votable/util.py
@@ -43,7 +43,7 @@ def convert_to_writable_filelike(fd, compressed=False):
-------
fd : :term:`file-like (writeable)`
"""
- if isinstance(fd, str):
+ if isinstance(fd, (str, os.PathLike)):
fd = os.path.expanduser(fd)
if fd.endswith(".gz") or compressed:
with gzip.GzipFile(filename=fd, mode="wb") as real_fd:(I checked that all tests pass locally) |
|
Re: patch -- Does this mean this PR would also add Path support to votable unified I/O if accepted? |
|
I guess so. |
|
We should keep this PR about the original bug. |
|
I'm not sure these are actually separable unfortunately. My patch appears to be needed to allow this PR's test to be rewritten in an acceptable form, but my patch also doesn't work on its own: it requires this PR's change in |
|
I also believe that this PR should keep focus on a specific bug.
Supporting |
happy to submit my patch as a follow up PR if we can finish this one without it ! |
Co-authored-by: P. L. Lim <[email protected]>
|
This time I did not forget to pull... :) |
…552-on-v7.0.x Backport PR #17552 on branch v7.0.x (Make `overwrite=True` work with `Path` arguments)
Description
This pull request takes
overwrite=Trueinto account in writing fits tables.Fixes #15350