Skip to content

Sort the labels passed from build --labels#29624

Merged
tonistiigi merged 1 commit intomoby:masterfrom
yongtang:29619-build-label-order
Dec 29, 2016
Merged

Sort the labels passed from build --labels#29624
tonistiigi merged 1 commit intomoby:masterfrom
yongtang:29619-build-label-order

Conversation

@yongtang
Copy link
Member

- What I did

This fix tries to fix the issue in #29619 where labels passed from build --labels are not sorted.
As a result, if multiple labels have been passed, each docker build --labels A=A --labels B=B --labels C=C run will generate different layers.

- How I did it

This fix fixes the issue by sort the Labels before they are concatenated to LABEL ....

- How to verify it

An integration test has been added to cover the changes

- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

cute-kitten-865-10-400x250

This fix fixes #29619.

Signed-off-by: Yong Tang [email protected]

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: we can trim the space here, and use strings.Join() instead of the loop in L246-L248.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Done.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably this can be moved to UT, but I think it is ok to keep this IT at the moment.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @AkihiroSuda. In #29619 the issue was only triggered by docker build --label A=A --label B=B ... through command line. LABEL inside Dockerfile directly will not trigger the issue. So I think it probably make sense to have a IT for it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree this should really be a unit test. Every bug is technically invoked from either the CLI or API, but I don't think that necessarily warrants an integration test. Integration tests are great for catching regressions where the bug was because of an integration between two components.

In this case the bug seems to be in a single component, so it could be tested directly.

A rule I like to follow is that if a function is hard to unit test, it's a good sign the function should be refactored in a way that makes it easier to test. Functions that are hard to test are often hard to re-use as well. Since this function is 100 lines long, it is difficult to test the behaviour in isolation. It might not be a bad idea to split it up, making it easier to test.

Copy link
Member

@cpuguy83 cpuguy83 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tonistiigi
Copy link
Member

LGTM

@yongtang yongtang force-pushed the 29619-build-label-order branch from 4244624 to ce089dd Compare December 21, 2016 21:12
@yongtang
Copy link
Member Author

Thanks @AkihiroSuda @dnephin. The PR has been updated with unit test. The build() has been decomposed into processLabels() as build() will invoke the daemon to compete the whole build.

Please take a look.

@yongtang yongtang force-pushed the 29619-build-label-order branch from ce089dd to 1bf2c13 Compare December 21, 2016 21:20
Copy link
Member

@dnephin dnephin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@cpuguy83
Copy link
Member

builder/dockerfile/builder_test.go:39: missing argument for Fatalf("%s"): format reads arg 1, have only 0 args

@yongtang
Copy link
Member Author

Thanks @cpuguy83 The PR has been updated.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you may want to put the labels in out-of-order to test sording actually works. e.g. org.c, org.a, org.d, ...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For clarity, yes I agree.

In practice, it won't make a difference, because Go always randomizes map order; see https://nathanleclaire.com/blog/2014/04/27/a-surprising-feature-of-golang-that-colored-me-impressed/

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@briceburg The map is always randomized:

ubuntu@ubuntu:~/tmp$ cat main.go 
package main

import (
	"fmt"
)

func main() {
	labels := map[string]string{
		"org.a": "cli-a",
		"org.b": "cli-b",
		"org.c": "cli-c",
		"org.d": "cli-d",
		"org.e": "cli-e",
	}
	for k, v := range labels {
		fmt.Println(k, v)
	}
}

ubuntu@ubuntu:~/tmp$ 
ubuntu@ubuntu:~/tmp$ go run main.go 
org.a cli-a
org.b cli-b
org.c cli-c
org.d cli-d
org.e cli-e
ubuntu@ubuntu:~/tmp$ go run main.go 
org.e cli-e
org.a cli-a
org.b cli-b
org.c cli-c
org.d cli-d
ubuntu@ubuntu:~/tmp$ go run main.go 
org.d cli-d
org.e cli-e
org.a cli-a
org.b cli-b
org.c cli-c
ubuntu@ubuntu:~/tmp$ go run main.go 
org.b cli-b
org.c cli-c
org.d cli-d
org.e cli-e
org.a cli-a
ubuntu@ubuntu:~/tmp$

I updated the PR to shuffle the order though to make it clear though.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, totally unnecessary but perhaps more clear to a gomature like myself. boy do I have a lot to learn! 😄

Many thanks for sharing, and for jumping on the label issue so quickly! It's inspiring to witness this project work.

@yongtang yongtang force-pushed the 29619-build-label-order branch from bf854ad to 027ed81 Compare December 21, 2016 23:39
@coolljt0725
Copy link
Contributor

@yongtang some tests failed

This fix tries to fix the issue in 29619 where
labels passed from `build --labels` are not sorted.
As a result, if multiple labels have been passed,
each `docker build --labels A=A --labels B=B --labels C=C`
will generate different layers.

This fix fixes the issue by sort the Labels before
they are concatenated to `LABEL ...`.

A unit test has been added to cover the changes

This fix fixes 29619.

Signed-off-by: Yong Tang <[email protected]>
@yongtang yongtang force-pushed the 29619-build-label-order branch from 027ed81 to d32efdb Compare December 22, 2016 15:33
@yongtang
Copy link
Member Author

Thanks @coolljt0725 I tested locally and those tests passed. Think it might be flaky. Added an issue #29663 and will keep an eye on it.

@tonistiigi tonistiigi merged commit aacc2c8 into moby:master Dec 29, 2016
@GordonTheTurtle GordonTheTurtle added this to the 1.14.0 milestone Dec 29, 2016
@yongtang yongtang deleted the 29619-build-label-order branch December 29, 2016 23:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

labels assigned in inconsistent/random order via docker build --label

10 participants