Linux Containers on Windows
Announced at Dockercon 2017, LCOW (Linux Containers on Windows) allows Linux containers to run side-by-side with Windows containers on a Windows host using Hyper-V isolation technologies. See this TechNet blog for more info.
With the recent changes in Moby to support LCOW, in order to reach parity, containerd should also be able to support LCOW. Due to the nature of running a different OS than the host, there are a number of assumptions made by containerd that no longer hold.
This proposal that outlines the major changes needed in containerd to support LCOW. This is not an exhaustive list, but outlines the key areas where changes are needed to support LCOW. This issue will be updated as more areas are discovered that need changes.
Snapshotter
Due to limitations of NTFS (valid character set, linux style ownership and permissions), it is not possible to store LCOW layers raw on disk. Instead, the files must be managed by a Linux Service Virtual Machine (Service VM from here on out) and stored on disk as a VHD. This means that the snapshotter on Windows needs to understand the context of the call (Windows container or Linux container) to decide if the call must be forwarded to be completed via the Service VM. Since the multiple snapshotter PR is merged, this seems reasonable to do.
This means that all locations that access the container's filesystem need to coordinate with the Service VM.
Expected work
Differ
The differ currently makes the assumption that the layers can be expanded out to disk. As per above, this is not possible for LCOW. The differ will need a way to do different things with different types of mounts. This could either be done by Mount type, but more likely an API addition is needed to specify a differ (or snapshotter) to complete the operation. #1017 seems to have taken a bit of a pass at doing something similar, though not currently sufficient for LCOW as far as I can tell.
Expected work
Service VM lifetime management
Along with the above, the Service VM must be created and destroyed as needed. There are two modes for Service VM lifetimes, global and per-instance. Moby currently supports global and per-instance. Global is faster, as the Service VM remains between operations and isn't started and stopped each time, and per-instance provides better multi-tenant isolation at the expense of some performance.
This Service VM must be available for use by the Differ, as well as any file access on the container layers from the host, such as a builder, or docker cp. I think the logical place for the Service VM management to live is in the Snapshotter, created at Prepare and deleted with Remove, but this means that the Differ must be tied to the snapshotter.
Expected work
Runtime
The Windows runtime should be expanded to also support LCOW. The platform of the container is determined by examining the OCI spec, and the runtime handles the rest from there. It looks to be a fairly simple change assuming that we can get the Linux image support in the snapshotter.
Expected work
Distribution/Content
TBD - Still understanding the distribution workflow and will update this issue as I understand more here. My current understanding is that Content Store should be agnostic to this data (but still needs work on Windows with or without LCOW), and only the Snapshotter/Differ should be aware of the LCOW layer details.
Linux Containers on Windows
Announced at Dockercon 2017, LCOW (Linux Containers on Windows) allows Linux containers to run side-by-side with Windows containers on a Windows host using Hyper-V isolation technologies. See this TechNet blog for more info.
With the recent changes in Moby to support LCOW, in order to reach parity, containerd should also be able to support LCOW. Due to the nature of running a different OS than the host, there are a number of assumptions made by containerd that no longer hold.
This proposal that outlines the major changes needed in containerd to support LCOW. This is not an exhaustive list, but outlines the key areas where changes are needed to support LCOW. This issue will be updated as more areas are discovered that need changes.
Snapshotter
Due to limitations of NTFS (valid character set, linux style ownership and permissions), it is not possible to store LCOW layers raw on disk. Instead, the files must be managed by a Linux Service Virtual Machine (Service VM from here on out) and stored on disk as a VHD. This means that the snapshotter on Windows needs to understand the context of the call (Windows container or Linux container) to decide if the call must be forwarded to be completed via the Service VM. Since the multiple snapshotter PR is merged, this seems reasonable to do.
This means that all locations that access the container's filesystem need to coordinate with the Service VM.
Expected work
Differ
The differ currently makes the assumption that the layers can be expanded out to disk. As per above, this is not possible for LCOW. The differ will need a way to do different things with different types of mounts. This could either be done by Mount type, but more likely an API addition is needed to specify a differ (or snapshotter) to complete the operation. #1017 seems to have taken a bit of a pass at doing something similar, though not currently sufficient for LCOW as far as I can tell.
Expected work
Service VM lifetime management
Along with the above, the Service VM must be created and destroyed as needed. There are two modes for Service VM lifetimes, global and per-instance. Moby currently supports global and per-instance. Global is faster, as the Service VM remains between operations and isn't started and stopped each time, and per-instance provides better multi-tenant isolation at the expense of some performance.
This Service VM must be available for use by the Differ, as well as any file access on the container layers from the host, such as a builder, or
docker cp. I think the logical place for the Service VM management to live is in the Snapshotter, created at Prepare and deleted with Remove, but this means that the Differ must be tied to the snapshotter.Expected work
Runtime
The Windows runtime should be expanded to also support LCOW. The platform of the container is determined by examining the OCI spec, and the runtime handles the rest from there. It looks to be a fairly simple change assuming that we can get the Linux image support in the snapshotter.
Expected work
Distribution/Content
TBD - Still understanding the distribution workflow and will update this issue as I understand more here. My current understanding is that Content Store should be agnostic to this data (but still needs work on Windows with or without LCOW), and only the Snapshotter/Differ should be aware of the LCOW layer details.