Skip to content

Jenkinsfile: set repo and branch, to assist validate_diff()#40035

Merged
thaJeztah merged 1 commit intomoby:masterfrom
thaJeztah:do_the_right_diff_do_the_right_diff
Oct 3, 2019
Merged

Jenkinsfile: set repo and branch, to assist validate_diff()#40035
thaJeztah merged 1 commit intomoby:masterfrom
thaJeztah:do_the_right_diff_do_the_right_diff

Conversation

@thaJeztah
Copy link
Member

This is a continuation of 2a08f33 (#38655);

When running CI in other repositories (e.g. Docker's downstream docker/engine repository), or other branches, the validation scripts were calculating the list of changes based on the wrong information.

This lead to weird failures in CI in a branch where these values were not updated ':-) (CI on a pull request failed because it detected that new tests were added to the deprecated integration-cli test-suite, but the pull request did not actually make changes in that area).

This patch uses environment variables set by Jenkins to sets the correct target repository (and branch) to compare to.

@thaJeztah thaJeztah force-pushed the do_the_right_diff_do_the_right_diff branch 4 times, most recently from e855e85 to b6df91f Compare October 3, 2019 16:22
This is a continuation of 2a08f33;

When running CI in other repositories (e.g. Docker's downstream
docker/engine repository), or other branches, the validation
scripts were calculating the list of changes based on the wrong
information.

This lead to weird failures in CI in a branch where these values
were not updated ':-) (CI on a pull request failed because it detected
that new tests were added to the deprecated `integration-cli` test-suite,
but the pull request did not actually make changes in that area).

This patch uses environment variables set by Jenkins to sets the
correct target repository (and branch) to compare to.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
@thaJeztah thaJeztah force-pushed the do_the_right_diff_do_the_right_diff branch 2 times, most recently from e4a564a to 7019b60 Compare October 3, 2019 16:49
@thaJeztah
Copy link
Member Author

thaJeztah commented Oct 3, 2019

Ok, this looks good; from a debug line that I just removed, the variables look to be set properly inside the container;

GOLANG_VERSION=1.13.1
HOSTNAME=d8830fce2d2c
DOCKER_EXPERIMENTAL=1
container=docker
GO111MODULE=off
GOPATH=/go
PWD=/go/src/github.com/docker/docker
HOME=/root
VALIDATE_REPO=https://github.com/moby/moby.git
SCRIPTDIR=/go/src/github.com/docker/docker/hack/validate
DOCKER_GITCOMMIT=e4a564a5966b389b210b6c32bbb21342474a12e4
DOCKER_BUILDTAGS=apparmor seccomp selinux
DOCKER_GRAPHDRIVER=overlay2
TERM=xterm
VALIDATE_BRANCH=master
SHLVL=1
PATH=/usr/local/cli:/go/bin:/usr/local/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

@thaJeztah thaJeztah marked this pull request as ready for review October 3, 2019 17:05
@thaJeztah
Copy link
Member Author

@kolyshkin @andrewhsu @tiborvass PTAL

-e DOCKER_GITCOMMIT=${GIT_COMMIT} \
-e DOCKER_GRAPHDRIVER \
-e VALIDATE_REPO=${GIT_URL} \
-e VALIDATE_BRANCH=${CHANGE_TARGET} \
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW; I tried setting these as global variable in the Jenkinsfile, but the Git plugin does not expose it's variables at that point, so while we can get CHANGE_TARGET, we can't get GIT_URL (without additional hackery), so I went for just passing them here inline

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To clarify; I assumed Jenkins would set that env-var for the whole job, but the Git plugin only sets it after if does the checkout, so it's only present in the stages itself.

My initial attempt was to map those env-vars in the top level block;

    environment {
        DOCKER_BUILDKIT     = '1'
        DOCKER_EXPERIMENTAL = '1'
        DOCKER_GRAPHDRIVER  = 'overlay2'
        APT_MIRROR          = 'cdn-fastly.deb.debian.org'
        CHECK_CONFIG_COMMIT = '78405559cfe5987174aa2cb6463b9b2c1b917255'
        TESTDEBUG           = '0'
        TIMEOUT             = '120m'
        TIMEOUT             = '120m'
        TIMEOUT             = '120m'
        VALIDATE_REPO       = "${env.GIT_URL}"
        VALIDATE_BRANCH     = "${env.CHANGE_TARGET}""
    }

Which worked for $CHANGE_TARGET / VALIDATE_BRANCH, but VALIDATE_REPO would always be NULL

Copy link
Member

@cpuguy83 cpuguy83 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@andrewhsu andrewhsu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@kolyshkin kolyshkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

(Every time I look at repetitive things like this, I think Jenkins must have an ability to do functions, or templates, or something -- alas, I looked at it once and it felt way too complicated)

@thaJeztah
Copy link
Member Author

@kolyshkin yes (also see above on setting the env-var globally). Unfortunately that isn't there yet for the declarative pipelines; you can create an account, and vote for JENKINS-40986 though (I did that earlier today)

@thaJeztah
Copy link
Member Author

This is green; let me merge it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants