Looking at these changes in context though, I'm a bit on the fence if these locations are the right location for creating these directories; specifically the one in pkg/cio feels very out of place, but I'm a bit concerned about the general design to sprinkle some MkdirAll calls through the code; more so after looking that the CreateTopLevelDirectories is present to setup required paths as early as possible. Having such mechanisms centralised so that they're created, as early as possible, and that there's a canonical location on their permissions may be a good thing.

Looking at the linked ticket; #10502 the request

I am running multiple instances of the containerd daemon on my machine and each containerd instance has their own state directories. e.g. /run/containerd-foo and /run/containerd-bar, /run/containerd

Those locations (per above) should already be created when each of those instances starts up.

containerd-shim however creates the shim socket under the default state directory which is set to /run/containerd

This is the part that (IIUC) causes issues for you, because starting a containerd instance with /run/containerd-foo as state-dir, will still attempt to create sockets in /run/containerd. Also from that linked discussion it looks like that's currently not configurable (hence this PR). I'm slightly concerned though that we're tapering over the underlying issue (those paths not configurable and/or not tied to the actual state directory in use (/run/containerd-foo)).

I'm also interested to learn about this part of your request; i.e., is there a specific reason you must have the additional instances running before the "main" (for better words) instance?

This is an issue because I want some containerd daemon instances to start earlier than the regular containerd daemon

Given that (from all things above) it looks like "multiple instances" is still a partial implementation, but the code changes in this PR will affect standard setups, I'm wondering if (while the "custom path" implementation is not complete / fully fledged out), a different workaround would be more appropriate. Which could be (e.g.) a systemd unit / script to create the directories ahead of time (before any of the containerd instances is started).

An alternative could be to;

• have CreateTopLevelDirectories unconditionally create /run/containerd regardless of the actually configured state directory
• or have CreateTopLevelDirectories create /run/containerd if the configured State is non-standard

The latter one could be something like;

if config.State != defaults.DefaultStateDir {
	// FIXME: socket-paths in pkg/shim are hard-coded to the default location, so make sure this location is created.
	if err := sys.MkdirAllWithACL(config.State, 0o711); err != nil {
		return err
	}
}

It looks like taking that approach also makes sure that these directories are created with the same permissions and options as the default.

LGTM, thanks!