Improve documentation around maintenance, building, and packaging by neersighted · Pull Request #46772 · moby/moby

neersighted · 2023-11-03T16:18:06Z

Closes convert to go modules (go.mod/go.sum) #46761

Attempt to better document the state of the project. This is incremental progress, but I'm hoping this can help "unstick" us and get a little more momentum going on documentation and improved transparency.

I'm only married to the intent of this PR, and not the specific phrasing, so please do suggest edits liberally. Likewise, I've guessed a bit as to intent/content with the branch documentation; please do suggest improvements/corrections if my mental model differs from reality (especially for those I have called out by name!).

Signed-off-by: Bjorn Neergaard <[email protected]>

Note that this entire document needs a modernization pass, both for current best practices as well as something approximating s/Docker/Moby/. In the interest of making some forward progress, we'll make minimal changes for now, however. Signed-off-by: Bjorn Neergaard <[email protected]>

Signed-off-by: Bjorn Neergaard <[email protected]>

samuelkarp · 2023-11-03T16:41:04Z

+- **Maintained** - actively developed by project maintainers; accepting contributions and backports; in-scope for security advisories 
+- **Maintained (security)** - no longer actively developed; may accept contributions and backports for critical security issues; in-scope for security advisories
+- **Unmaintained** - no longer actively developed; not accepting contributions; out-of-scope for security advisories


containerd describes these as "Active", "Extended", and "End of Life" in case you wanted to keep terminology consistent across projects.

samuelkarp · 2023-11-03T16:47:30Z

+The Moby Project is widely packaged by many independent software distributors,
+generally under the name 'docker' or 'docker.io.'
+
+A non-exhuastive list of known software packagers and distributions includes:


How exhaustive do you want this list to be? Repology has a fairly long list (though it isn't fully normalized). Beyond those lists, I also know of Bottlerocket and Google Container-Optimized OS.

This is definitely pretty initial; it definitely would be helpful to at least document say, the top 10 Linux distributions with packaging, and any non-distribution distributions that are well known.

Interesting to know that Bottlerocket has separate packaging from AL -- thanks for pointing that out! I'll try and add some more entries from the links you have provided.

cpuguy83 · 2023-11-03T16:57:44Z

 exact version for each.

+Note that this project is **not** a Go module. To properly build the project,
+`GOPATH` mode must be used. The project is migrating to Go modules, but `vendor.mod`


This seems like strange wording.
The issue with go.mod is more about people importing the project, not building it.
We absolutely use vendor.mod as a stand-in for go.mod:

moby/hack/make/.binary

Line 85 in 6fd893b

./hack/with-go-mod.sh go build -mod=vendor -modfile=vendor.mod -o "$DEST/$BINARY_FULLNAME" "${BUILDFLAGS[@]}" -ldflags "$LDFLAGS $LDFLAGS_STATIC $DOCKER_LDFLAGS" -gcflags="${GCFLAGS}" "$GO_PACKAGE"

We also don't use GOPATH mode (or rather don't require it), hence the hack/with-go-mod.sh.

There are issues with go.mod in building; while we do some trickery with vendor.mod here, it's worth noting that renaming vendor.mod to go.mod doesn't work; this is a special case where the Go toolchain requires a stub go.mod to be bent the way we want.

In general I see a lot of #44618 coming up; when it comes to advice for packagers, I think we really should express GOPATH is the supported way to build.

Our own build scripts are turning on module mode, though.
If someone builds with hack/make.sh [binary|dynbyinary] they are building with module mode.

It can be used correctly with the wrapper for sure; I just don't know that is something that we want to expose. Do you think it's worth documenting how to build in module mode (with the wrapper/correct arguments) and making that a "stable" interface for people to depend on?

hack/make.sh is the interface we have always expected people to use.
I think it is fair to mention that in this doc?

AkihiroSuda · 2023-11-04T01:52:05Z

-Any public release of a compiled binary, with the logical exception of nightly builds, should have
-a corresponding tag in the repository.
+All releases of The Moby Project should have a corresponding tag in the repository.
+The project generally attempts to adhere to [Semantic Versioning](https://semver.org) whenever possible.


Suggested change

The project generally attempts to adhere to [Semantic Versioning](https://semver.org) whenever possible.

The project generally attempts to adhere to [Semantic Versioning](https://semver.org) whenever possible since v23, and until v1.13.

v17.03-v20.10 releases were following CalVer.

AkihiroSuda · 2023-11-04T01:53:45Z

+| Branch Name                                    | Sponsoring Maintainer(s)                       | Contribution Status   | Expected End of Maintenance  | Known Distributors                                                   |
+|------------------------------------------------|------------------------------------------------|-----------------------|------------------------------|----------------------------------------------------------------------|
+| master (development branch)                    | The Moby Project [MAINTAINERS](../MAINTAINERS) | N/A                   | -                            | N/A                                                                  |
+| 24.0                                           | @thaJeztah, @rumpl, @neersighted               | Maintained            | TBD                          | [Docker, Inc.](https://docker.com)                                   |


I was for some reason expecting GitHub to linkify these; I agree we should link them to the GitHub profiles.

Hello,
are there any informations about the versions beyond 24.0? Is 24 still maintained?

24.0 is not maintained; currently maintained branches;

master / main

27.0 (which will be renamed to 27.x) - current release (v27.x.x)

25.0 (will be used as base for the next LTS version of Mirantis Container Runtime)

23.0 (branch used as base for current LTS version of Mirantis Container Runtime)

With the disclaimer that older branches depend on versions of the BuildKit dependency that are no longer maintained (the BuildKit project currently doesn't provide maintained long-lived branches)

Thank you very much :)

I should also add that the 25.0 branch is not yet used by Mirantis; normally, we consider branches with no "owner" (no maintainer involved in maintaining the branch) to be "frozen", however Mirantis maintainers have indicated their intent to use that branch for their next LTS release.

@austinvazquez (working at AWS) is not (yet) a maintainer, but requested some backports, and with the knowledge that Mirantis (who does provide maintainers) has the commitment to use that branch in future, we accepted those backports.

Is all of this information hidden in the documentation or somewhere else? I searched about 1h before asking here with many different answers but non from docker or moby

AkihiroSuda · 2023-11-04T01:56:17Z

-// around SemVer, which this repo cannot abide by as it uses CalVer.
+// There is no 'go.mod' file, as the project has not yet completed the migration
+// to a Go module. This project should be built in GOPATH mode, and not module mode,
+// until the migration is complete and this file becomes 'go.mod.'


Can we have a link to an issue to track the blockers for the migration?

I've been bugging @thaJeztah about making one; I'll link it in these places once we have one 👍

tianon

In this case, IMO good is better than perfect, and this is pretty good -- at most maybe this needs an update to that table of supported versions/branches to be unstuck and more improvements could be separate PRs?

(Right now we're in the fun state where I, a Moby maintainer and Docker employee, don't even know which versions are actively "supported" which isn't a great state to be in.)

vvoland

LGTM; as Bjorn is currently out of office, I went ahead and updated the version table

vvoland · 2024-07-26T11:49:13Z

+| Branch Name                       | Sponsoring Maintainer(s)                                | Contribution Status   | Expected End of Maintenance  | Known Distributors                  |
+|-----------------------------------|---------------------------------------------------------|-----------------------|------------------------------|-------------------------------------|
+| master (development branch)       | The Moby Project [MAINTAINERS](../MAINTAINERS)          | N/A                   | -                            | N/A                                 |
+| 27.0                              | The Moby Project [MAINTAINERS](../MAINTAINERS)          | Maintained            | After 28.0                   | [Docker, Inc.](https://docker.com)  |


Suggested change

| 27.0 | The Moby Project [MAINTAINERS](../MAINTAINERS) | Maintained | After 28.0 | [Docker, Inc.](https://docker.com) |

| 27.0 | The Moby Project [MAINTAINERS](../MAINTAINERS) | Maintained | After 28.0 | [Docker, Inc.](https://docker.com) |

I put The Moby Project as the sponsoring maintainers for the latest release branch to avoid listing every individual maintainer that happens to do the backports.
Alternative would be to create a team in the Moby org with people that do backports (which would be at least Docker Inc. maintainers?) and reference the whole group there.

@thaJeztah @cpuguy83 @corhere WDYT?

vvoland · 2024-07-26T11:49:19Z

+|-----------------------------------|---------------------------------------------------------|-----------------------|------------------------------|-------------------------------------|
+| master (development branch)       | The Moby Project [MAINTAINERS](../MAINTAINERS)          | N/A                   | -                            | N/A                                 |
+| 27.0                              | The Moby Project [MAINTAINERS](../MAINTAINERS)          | Maintained            | After 28.0                   | [Docker, Inc.](https://docker.com)  |
+| 26.1                              | @cpuguy83                                               | Maintained (security) | TBD                          | [Microsoft](https://microsoft.com)  |


@cpuguy83 PTAL

vvoland · 2024-07-26T11:49:25Z

+| master (development branch)       | The Moby Project [MAINTAINERS](../MAINTAINERS)          | N/A                   | -                            | N/A                                 |
+| 27.0                              | The Moby Project [MAINTAINERS](../MAINTAINERS)          | Maintained            | After 28.0                   | [Docker, Inc.](https://docker.com)  |
+| 26.1                              | @cpuguy83                                               | Maintained (security) | TBD                          | [Microsoft](https://microsoft.com)  |
+| 25.0                              | @corhere                                                | Maintained            | TBD                          | [Mirantis](https://mirantis.com)    |


@corhere PTAL

Also cc @austinvazquez

24.x is no longer maintained as of February 1, 2024[1]. It did not (yet?) receive a fix for CVE-2024-41110. [1] moby/moby#46772 (comment)

slonopotamus · 2024-07-26T21:19:01Z

I don't understand why this PR closes #46761 =/

tianon · 2024-07-26T22:06:41Z

I think that's because it includes language that addresses the frequently asked question that issue represents -- we do need to create something (a proper issue, documentation, a project board, etc) that tracks what needs to happen for us to make the transition (as noted both in this PR's discussion and in that issue's discussion).

24.x is no longer maintained as of February 1, 2024[1]. It did not (yet?) receive a fix for CVE-2024-41110. According to [1] 25.x will be the next LTS version, use that version to reduce risk of possible breakage. [1] moby/moby#46772 (comment)

Signed-off-by: Paweł Gronowski <[email protected]>

neersighted added area/docs area/project area/packaging labels Nov 3, 2023

neersighted added this to the 25.0.0 milestone Nov 3, 2023

neersighted requested review from AkihiroSuda, akerouanton, bsousaa, corhere, cpuguy83, estesp, laurazard, rumpl, samuelkarp, tianon and vvoland November 3, 2023 16:18

neersighted requested a review from thaJeztah as a code owner November 3, 2023 16:18

This was referenced Nov 3, 2023

Releases are very unfriendly for new people #39302

Closed

convert to go modules (go.mod/go.sum) #46761

Closed

neersighted added 3 commits November 3, 2023 10:27

project: document modern branch and tag practices

f502f49

Signed-off-by: Bjorn Neergaard <[email protected]>

project,vendor.mod: document more clearly that we are not a Go module

a52aab8

Signed-off-by: Bjorn Neergaard <[email protected]>

neersighted force-pushed the project_docs branch from fbb90c5 to a52aab8 Compare November 3, 2023 16:27

samuelkarp reviewed Nov 3, 2023

View reviewed changes

cpuguy83 reviewed Nov 3, 2023

View reviewed changes

AkihiroSuda reviewed Nov 4, 2023

View reviewed changes

thaJeztah mentioned this pull request Dec 14, 2023

README: Docker EE -> Docker Desktop or Mirantis Container Engine #46937

Closed

thaJeztah removed this from the 25.0.0 milestone Jan 3, 2024

tianon approved these changes Jul 23, 2024

View reviewed changes

teutat3s mentioned this pull request Jul 25, 2024

Why is the default docker version still at 24? NixOS/nixpkgs#328275

Closed

teutat3s mentioned this pull request Jul 26, 2024

docker: move default from 24.x to 27.x, docker_24 mark as vulnerable, docker_{25,26}: bump NixOS/nixpkgs#330109

Merged

13 tasks

vvoland approved these changes Jul 26, 2024

View reviewed changes

vvoland reviewed Jul 26, 2024

View reviewed changes

vvoland force-pushed the project_docs branch from b1d58c2 to c7c61ee Compare July 26, 2024 14:13

tianon approved these changes Jul 26, 2024

View reviewed changes

cpuguy83 approved these changes Jul 26, 2024

View reviewed changes

SuperSandro2000 pushed a commit to teutat3s/nixpkgs that referenced this pull request Jul 26, 2024

docker: move default from 24.x to 27.x

b381163

24.x is no longer maintained as of February 1, 2024[1]. It did not (yet?) receive a fix for CVE-2024-41110. [1] moby/moby#46772 (comment)

teutat3s mentioned this pull request Jul 29, 2024

[docker-engine] Set 24.0 and 26.0 EOL endoflife-date/endoflife.date#5554

Merged

teutat3s mentioned this pull request Jul 29, 2024

[24.05] docker: move default from 24.x to 25.x, docker_24 mark as vulnerable, docker_{25,26}: bump NixOS/nixpkgs#330861

Merged

13 tasks

project: update supported release branches

403f9db

Signed-off-by: Paweł Gronowski <[email protected]>

vvoland force-pushed the project_docs branch from c7c61ee to 403f9db Compare July 29, 2024 16:11

vdemeester approved these changes Jul 29, 2024

View reviewed changes

corhere approved these changes Jul 29, 2024

View reviewed changes

vvoland merged commit 89bca30 into moby:master Jul 30, 2024

tianon mentioned this pull request Jul 30, 2024

Remove versions that won't get more Docker, Inc builds docker-library/docker#506

Merged

thaJeztah added this to the 28.0.0 milestone Sep 18, 2024

	The project generally attempts to adhere to [Semantic Versioning](https://semver.org) whenever possible.
	The project generally attempts to adhere to [Semantic Versioning](https://semver.org) whenever possible since v23, and until v1.13.
	v17.03-v20.10 releases were following CalVer.

	\| 24.0 \| @thaJeztah, @rumpl, @neersighted \| Maintained \| TBD \| [Docker, Inc.](https://docker.com) \|
	\| 24.0 \| [@thaJeztah](https://github.com/thaJeztah), @rumpl, @neersighted \| Maintained \| TBD \| [Docker, Inc.](https://docker.com) \|

	\| 27.0 \| The Moby Project [MAINTAINERS](../MAINTAINERS) \| Maintained \| After 28.0 \| [Docker, Inc.](https://docker.com) \|
	\| 27.0 \| The Moby Project [MAINTAINERS](../MAINTAINERS) \| Maintained \| After 28.0 \| [Docker, Inc.](https://docker.com) \|

Conversation

neersighted commented Nov 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thaJeztah Jul 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tianon left a comment

Choose a reason for hiding this comment

Uh oh!

vvoland left a comment

Choose a reason for hiding this comment

Uh oh!

vvoland Jul 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

slonopotamus commented Jul 26, 2024

Uh oh!

tianon commented Jul 26, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants

neersighted commented Nov 3, 2023 •

edited

Loading

thaJeztah Jul 22, 2024 •

edited

Loading

vvoland Jul 26, 2024 •

edited

Loading