Problem

Converting Windows file attributes + ACLs to 9-bit UNIX permissions when
docker images are built from windows CLI. Concept of 'execute' doesn't
match 1:1 on Windows vs POSIX.

Scenario

1. User downloads Docker client for Windows (docker.exe)
2. User develops an "app" to run on Linux containers.
3. User runs the command docker build .
4. Context tar is created on client-side and transferred to daemon (running on Linux) for building.

Problem steps:

1. Let's say the "app" is actually something user cloned from a remote git repo.
2. Repo had a setup.sh script (chmod +xed)
3. There's no x bit on windows, so during git clone we lost that bit
4. Today all files packaged to context from windows get -rw-rw-rw- permissions.
5. This can also happen if user unzips a file or downloads from ftp etc.
6. If user tries to RUN ./setup.sh or ENTRYPOINT ./setup.sh it will fail
   due to a vague "permission denied" error because script was not chmod +xed.
7. Result: a dockerfile that builds correctly fine from OS X/Linux cli has failed on windows.
8. User will take a lot of time to figure +x bit was missing.
9. User will go add RUN chmod +x setup.sh as a dockerfile instruction if they're
   building from Windows.

Root cause 1: No 'x' (execute) bit on Windows

• Windows does not have an execute permission as the 'x' bit in Unix
• By default, almost all files are executable on windows. (different than unix)
• You can execute a binary if the extension is exe/bat/cmd/com/ps1 etc.
• Windows executable permission is more about dynamically linked libraries (dlls) etc.
• It's not widely used.
• Does not correspond to executability concept in Unix.
• Every time user grabs a +xed file from ftp/git/unzip/samba, we lose that
  unix bit on windows, because it's not the same thing.

Quoting from SAMBA book:

“Note that there is no bit to specify that a file is executable. DOS and Windows NT filesystems identify executable files by giving them the extensions .EXE, .COM, .CMD, or .BAT. [...] Samba can preserve these bits by reusing the executable permission bits of the file on the Unix side - if it is instructed to do so. Mapping these bits, however, has an unfortunate side-effect: if a Windows user stores a file in a Samba share, and you view it on Unix with the ls -al command, some of the executable bits won't mean what you'd expect them to.”

Root cause 2: Go Windows Problems

Golang's os.Stat implementation is very poor:

• It cannot figure out executability. (because it's not a file attribute on
  Windows but an ACL setting –found out through a separate syscall, not implemented in golang)
• It returns same permissions for all user/group/others sections:
  • Because there's no group/everybody concept on Windows as in chmod bits.
  • That's why it returns -rw-rw-rw- for all files on Windows and that's the permission what files copied to a docker build context gets by default.
  • This is dangerous (imagine a private key being set to -rw-rw-rw- permissions by default, that's the state we're in now).

We are hoping to address problems in golang in the long term. (fingers crossed)

Solution

1. We are going to clear all 'group/others' bits: -???------
   • Since we're clearing r from group/others, this might break multi-user docker images.
2. We are going to copy r/w bits we got from golang os.Stat().
3. We are going to add +x to all files packaged from windows. 💥
   • This way we can at least have +x on all files and can support executing ./binary or ./script.sh from shell.
   • This may break some apps that decline +xed files as input. I haven't really heard of anything like that,
     must be a really rare case. Even so user can have RUN chmod -x to remove that bit manually
   • Security issue: We are making everything executable. Could be. ❗ Please discuss.
4. We are going to add a notice for windows CLI: WARNING: Permission bits on files added to image might not be correctly set. 🆕

Proposed Fix

We modify the perm bits of the tar header created via tar.FileInfoHeader
with a simple platform-specific helper method. (see code)

We are also going to change the expected permission string
on the integration-cli test cases when executed on windows.

Result

We will end up with a docker image in which all files copied from
Windows filesystem have (1) grp/others bits cleared (2) +x added
regardless of it should have +x because it does no harm.

Behavior on Linux/Mac CLIs are unchanged.

I really appreciate your input on this. That's the only thing left prevents us from getting green on Windows client CI.

cc: @ewindisch @crosbymichael @cpuguy83 @tianon @tiborvass @jfrazelle @icecrime @johngossman @sachin-jayant-joshi @jhowardmsft