Skip to content

Commit 07bde83

Browse files
AkihiroSudaAkihiroSuda
authored
Merge pull request #5494 from dmcgowan/backport-1.5-host-documentation
[release/1.5] hosts documentation
2 parents eebc914 + 042e49f commit 07bde83

1 file changed

Lines changed: 312 additions & 0 deletions

File tree

‎docs/hosts.md‎

Lines changed: 312 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,312 @@
1+
2+
# Registry Configuration - Introduction
3+
4+
Configuring registries will be done by specifying (optionally) a `hosts.toml` file for
5+
each desired registry host in a configuration directory. **Note**: Updates under this directory
6+
do not require restarting the containerd daemon.
7+
8+
## Specifying the Configuration Directory
9+
10+
### Using Host Namespace Configs with CTR
11+
12+
When pulling via `ctr` use the `--hosts-dir` option:
13+
```
14+
ctr images pull --hosts-dir "/etc/containerd/certs.d"
15+
```
16+
17+
### CRI
18+
_The old CRI config pattern for specifying registry.mirrors and registry.configs has
19+
been **DEPRECATED**._ You should now point your registry `config_path` to the path where your
20+
`hosts.toml` files are located.
21+
22+
Modify your `config.toml` (default location: `/etc/containerd/config.toml`) as follows:
23+
```toml
24+
[plugins."io.containerd.grpc.v1.cri".registry]
25+
config_path = "/etc/containerd/certs.d"
26+
```
27+
28+
## Support for Docker's Certificate File Pattern
29+
30+
If no hosts.toml configuration exists in the host directory, it will fallback to check
31+
certificate files based on [Docker's certificate file pattern](https://docs.docker.com/engine/reference/commandline/dockerd/#insecure-registries)
32+
(".crt" files for CA certificates and ".cert"/".key" files for client certificates).
33+
34+
## Registry Host Namespace
35+
36+
A registry host is the location where container images and artifacts are sourced. These
37+
registry hosts may be local or remote and are typically accessed via http/https using the
38+
[OCI distribution specification](https://github.com/opencontainers/distribution-spec/blob/main/spec.md).
39+
A registry mirror is not a registry host but these mirrors can also be used to pull content.
40+
Registry hosts are typically refered to by their internet domain names, aka. registry
41+
host names. For example, docker.io, quay.io, gcr.io, and ghcr.io.
42+
43+
A registry host namespace is, for the purpose of containerd registry configuration, a
44+
path to the `hosts.toml` file specified by the registry host name, or ip address, and an
45+
optional port identifier. When making a pull request for an image the format is
46+
typically as follows:
47+
```
48+
pull [registry_host_name|IP address][:port][/v2][/org_path]<image_name>[:tag|@DIGEST]
49+
```
50+
51+
The registry host namespace portion is `[registry_host_name|IP address][:port]`. Example
52+
tree for docker.io:
53+
54+
```
55+
$ tree /etc/containerd/certs.d
56+
/etc/containerd/certs.d
57+
└── docker.io
58+
└── hosts.toml
59+
```
60+
61+
The `/v2` portion of the pull request format shown above refers to the version of the
62+
distribution api. If not included in the pull request, `/v2` is added by default for all
63+
clients compliant to the distribution specification linked above.
64+
65+
For example when pulling image_name:tag from a private registry named myregistry.io over
66+
port 5000:
67+
```
68+
pull myregistry.io:5000/image_name:tag
69+
```
70+
The pull will resolve to `https://myregistry.io:5000/v2/image_name:tag`
71+
72+
## Specifying Registry Credentials
73+
74+
### CTR
75+
76+
When performing image operations via `ctr` use the --help option to get a list of options you can set for specifying credentials:
77+
```
78+
ctr i pull --help
79+
...
80+
OPTIONS:
81+
--skip-verify, -k skip SSL certificate validation
82+
--plain-http allow connections using plain HTTP
83+
--user value, -u value user[:password] Registry user and password
84+
--refresh value refresh token for authorization server
85+
--hosts-dir value Custom hosts configuration directory
86+
--tlscacert value path to TLS root CA
87+
--tlscert value path to TLS client certificate
88+
--tlskey value path to TLS client key
89+
--http-dump dump all HTTP request/responses when interacting with container registry
90+
--http-trace enable HTTP tracing for registry interactions
91+
--snapshotter value snapshotter name. Empty value stands for the default value. [$CONTAINERD_SNAPSHOTTER]
92+
--label value labels to attach to the image
93+
--platform value Pull content from a specific platform
94+
--all-platforms pull content and metadata from all platforms
95+
--all-metadata Pull metadata for all platforms
96+
--print-chainid Print the resulting image's chain ID
97+
--max-concurrent-downloads value Set the max concurrent downloads for each pull (default: 0)
98+
```
99+
100+
## CRI
101+
102+
Although we have deprecated the old CRI config pattern for specifying registry.mirrors
103+
and registry.configs you can still specify your credentials via
104+
[CRI config](https://github.com/containerd/containerd/blob/master/docs/cri/registry.md#configure-registry-credentials).
105+
106+
Additionally, the containerd CRI plugin implements/supports the authentication parameters passed in through CRI pull image service requests.
107+
For example, when containerd is the container runtime implementation for `Kubernetes`, the containerd CRI plugin receives
108+
authentication credentials from kubelet as retrieved from
109+
[Kubernetes Image Pull Secrets](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/)
110+
111+
# Registry Configuration - Examples
112+
113+
### Simple (default) Host Config for Docker
114+
Here is a simple example for a default registry hosts configuration. Set
115+
`config_path = "/etc/containerd/certs.d"` in your config.toml for containerd.
116+
Make a directory tree at the config path that includes `docker.io` as a directory
117+
representing the host namespace to be configured. Then add a `hosts.toml` file
118+
in the `docker.io` to configure the host namespace. It should look like this:
119+
120+
```
121+
$ tree /etc/containerd/certs.d
122+
/etc/containerd/certs.d
123+
└── docker.io
124+
└── hosts.toml
125+
126+
$ cat /etc/containerd/certs.d/docker.io/hosts.toml
127+
server = "https://docker.io"
128+
129+
[host."https://registry-1.docker.io"]
130+
capabilities = ["pull", "resolve"]
131+
```
132+
133+
### Setup a Local Mirror for Docker
134+
135+
```
136+
server = "https://registry-1.docker.io" # Exclude this to not use upstream
137+
138+
[host."https://public-mirror.example.com"]
139+
capabilities = ["pull"] # Requires less trust, won't resolve tag to digest from this host
140+
[host."https://docker-mirror.internal"]
141+
capabilities = ["pull", "resolve"]
142+
ca = "docker-mirror.crt" # Or absolute path /etc/containerd/certs.d/docker.io/docker-mirror.crt
143+
```
144+
145+
### Bypass TLS Verification Example
146+
147+
To bypass the TLS verification for a private registry at `192.168.31.250:5000`
148+
149+
Create a path and `hosts.toml` text at the path "/etc/containerd/certs.d/docker.io/hosts.toml" with following or similar contents:
150+
151+
```toml
152+
server = "https://registry-1.docker.io"
153+
154+
[host."http://192.168.31.250:5000"]
155+
capabilities = ["pull", "resolve", "push"]
156+
skip_verify = true
157+
```
158+
159+
# hosts.toml Content Description - Detail
160+
161+
For each registry host namespace directory in your registry `config_path` you may
162+
include a `hosts.toml` configuration file. The following root level toml fields
163+
apply to the registry host namespace:
164+
165+
**Note**: All paths specified in the `hosts.toml` file may be absolute or relative
166+
to the `hosts.toml` file.
167+
168+
## server field
169+
`server` specifies the default server for this registry host namespace. When
170+
`host`(s) are specified, the hosts are tried first in the order listed.
171+
```
172+
server = "https://docker.io"
173+
```
174+
175+
## capabilities field
176+
177+
`capabilities` is an optional setting for specifying what operations a host is
178+
capable of performing. Include only the values that apply.
179+
```
180+
capabilities = ["pull", "resolve", "push"]
181+
```
182+
183+
capabilities (or Host capabilities) represent the capabilities of the registry host.
184+
This also represents the set of operations for which the registry host may be trusted
185+
to perform.
186+
187+
For example, pushing is a capability which should only be performed on an upstream
188+
source, not a mirror.
189+
190+
Resolving (the process of converting a name into a digest)
191+
must be considered a trusted operation and only done by
192+
a host which is trusted (or more preferably by secure process
193+
which can prove the provenance of the mapping).
194+
195+
A public mirror should never be trusted to do a resolve action.
196+
197+
| Registry Type | Pull | Resolve | Push |
198+
|------------------|------|---------|------|
199+
| Public Registry | yes | yes | yes |
200+
| Private Registry | yes | yes | yes |
201+
| Public Mirror | yes | no | no |
202+
| Private Mirror | yes | yes | no |
203+
204+
## ca field
205+
206+
`ca` (Certificate Authority Certification) can be set to a path or an array of
207+
paths each pointing to a ca file for use in authenticating with the registry
208+
namespace.
209+
```
210+
ca = "/etc/certs/mirror.pem"
211+
```
212+
or
213+
```
214+
ca = ["/etc/certs/test-1-ca.pem", "/etc/certs/special.pem"]
215+
```
216+
217+
## client field
218+
219+
`client` certificates are configured as follows
220+
221+
a path:
222+
```
223+
client = "/etc/certs/client.pem"
224+
```
225+
226+
an array of paths:
227+
```
228+
client = ["/etc/certs/client-1.pem", "/etc/certs/client-2.pem"]
229+
```
230+
231+
an array of pairs of paths:
232+
```
233+
client = [["/etc/certs/client.cert", "/etc/certs/client.key"],["/etc/certs/client.pem", ""]]
234+
```
235+
236+
## skip_verify field
237+
238+
`skip_verify` set this flag to `true` to skip the registry certificate
239+
verification for this registry host namespace. (Defaults to `false`)
240+
```
241+
skip_verify = false
242+
```
243+
244+
## header fields (in the toml table format)
245+
246+
`[header]` contains some number of keys where each key is to one of a string or
247+
248+
an array of strings as follows:
249+
```
250+
[header]
251+
x-custom-1 = "custom header"
252+
```
253+
254+
or
255+
```
256+
[header]
257+
x-custom-1 = ["custom header part a","part b"]
258+
```
259+
260+
or
261+
```
262+
[header]
263+
x-custom-1 = "custom header",
264+
x-custom-1-2 = "another custom header"
265+
```
266+
267+
## host field(s) (in the toml table format)
268+
269+
`[host]."https://namespace"` and `[host].http://namespace` entries in the
270+
`hosts.toml` configuration are registry namespaces used in lieu of the default
271+
registry host namespace. These hosts are sometimes called mirrors because they
272+
may contain a copy of the container images and artifacts you are attempting to
273+
retrieve from the default registry. Each `host`/`mirror` namespace is also
274+
configured in much the same way as the default registry namespace. Notably the
275+
`server` is not specified in the `host` description because it is specified in
276+
the namespace. Here are a few rough examples configuring host mirror namespaces
277+
for this registry host namespace:
278+
```
279+
[host."https://mirror.registry"]
280+
capabilities = ["pull"]
281+
ca = "/etc/certs/mirror.pem"
282+
skip_verify = false
283+
[host."https://mirror.registry".header]
284+
x-custom-2 = ["value1", "value2"]
285+
286+
[host."https://mirror-bak.registry/us"]
287+
capabilities = ["pull"]
288+
skip_verify = true
289+
290+
[host."http://mirror.registry"]
291+
capabilities = ["pull"]
292+
293+
[host."https://test-1.registry"]
294+
capabilities = ["pull", "resolve", "push"]
295+
ca = ["/etc/certs/test-1-ca.pem", "/etc/certs/special.pem"]
296+
client = [["/etc/certs/client.cert", "/etc/certs/client.key"],["/etc/certs/client.pem", ""]]
297+
298+
[host."https://test-2.registry"]
299+
client = "/etc/certs/client.pem"
300+
301+
[host."https://test-3.registry"]
302+
client = ["/etc/certs/client-1.pem", "/etc/certs/client-2.pem"]
303+
```
304+
305+
**Note**: Recursion is not supported in the specification of host mirror
306+
namespaces in the hosts.toml file. Thus the following is not allowed/supported:
307+
```
308+
[host."http://mirror.registry"]
309+
capabilities = ["pull"]
310+
[host."http://double-mirror.registry"]
311+
capabilities = ["pull"]
312+
```

0 commit comments

Comments
 (0)