Skip to content

Don't use cp -c if src and dst are on different devices when copying dSYMs#2766

Merged
brentleyjones merged 1 commit intobazelbuild:masterfrom
ma-oli:remove-clonefile-when-copying-dsyms
Aug 18, 2025
Merged

Don't use cp -c if src and dst are on different devices when copying dSYMs#2766
brentleyjones merged 1 commit intobazelbuild:masterfrom
ma-oli:remove-clonefile-when-copying-dsyms

Conversation

@ma-oli
Copy link
Copy Markdown
Contributor

@ma-oli ma-oli commented Aug 15, 2025

dSYMs are currently copied using cp -cp on MacOS, which means MacOS will use clonefile(). While this optimizes speed and storage, this will also fail if src and dst are not on the same filesystem. This will typically happen if the sandbox and bazel-out are on different filesystems.

@keith
Copy link
Copy Markdown
Member

keith commented Aug 15, 2025

the manpage for cp says:

     -c    copy files using clonefile(2).  Note that if clonefile(2) is not
           supported for the target filesystem, then cp will fallback to using
           copyfile(2) instead to ensure the copy still succeeds.

is this not the case? it makes it sound like it shouldn't fail, but will fallback gracefully? i think in general we probably need to keep this optimization but figure out a fallback if it doesn't work. we do that in python like this

if result != 0:
if get_errno() in (errno.EXDEV, errno.ENOTSUP):
_USE_CLONEFILE = False
shutil.copy(src, full_dest)

@ma-oli
Copy link
Copy Markdown
Contributor Author

ma-oli commented Aug 15, 2025

is this not the case? it makes it sound like it shouldn't fail, but will fallback gracefully? i think in general we probably need to keep this optimization but figure out a fallback if it doesn't work. we do that in python like this

I see the confusion indeed; but looking at the clonefile() manual, it looks like underlying filesystem does not support this call and src and dst are not on the same filesystem are two different error codes. I suspect cp -c is lenient on the first one and not on the second one.

i think in general we probably need to keep this optimization but figure out a fallback if it doesn't work

I'll update my patch

@brentleyjones
Copy link
Copy Markdown
Collaborator

Here is how rules_xcodeproj does it:

if [[ $(stat -f '%d' "$src") == $(stat -f '%d' "$dest") ]]; then
  readonly cp_cmd="cp -c"
else
  readonly cp_cmd="cp"
fi

@ma-oli ma-oli changed the title Remove clonefile() optimization when copying dSYMs Don't use cp -c if src and dst are on different devices when copying dSYMs Aug 15, 2025
@ma-oli ma-oli force-pushed the remove-clonefile-when-copying-dsyms branch 2 times, most recently from 4bc6d4f to 1e8c9d4 Compare August 15, 2025 23:14
dSYMs are currently copied using `cp -cp` on MacOS, which means MacOS
will use clonefile(). While this optimizes speed and storage, this will
also fail if src and dst are not on the same devices. We're fixing that
by testing the device with `stat` before deciding how to copy.
@ma-oli ma-oli force-pushed the remove-clonefile-when-copying-dsyms branch from 1e8c9d4 to 5adc03a Compare August 17, 2025 22:43
@brentleyjones brentleyjones merged commit 5c092c4 into bazelbuild:master Aug 18, 2025
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants