Jenkinsfile: force validate on master, and some fixes/improvements#40050
Jenkinsfile: force validate on master, and some fixes/improvements#40050thaJeztah merged 7 commits intomoby:masterfrom
Conversation
2e2cf11 to
1a3d106
Compare
Jenkinsfile
Outdated
There was a problem hiding this comment.
Env vars available;
APT_MIRROR=cdn-fastly.deb.debian.org
BRANCH_NAME=PR-40050
BUILD_DISPLAY_NAME=#2
BUILD_ID=2
BUILD_NUMBER=2
BUILD_TAG=jenkins-moby-PR-40050-2
BUILD_URL=https://ci.docker.com/public/job/moby/job/PR-40050/2/
CHANGE_AUTHOR=thaJeztah
CHANGE_AUTHOR_DISPLAY_NAME=Sebastiaan van Stijn
CHANGE_BRANCH=ci_force_validate
CHANGE_FORK=thaJeztah
CHANGE_ID=40050
CHANGE_TARGET=master
CHANGE_TITLE=Jenkinsfile: force validate steps on master
CHANGE_URL=https://github.com/moby/moby/pull/40050
CHECK_CONFIG_COMMIT=78405559cfe5987174aa2cb6463b9b2c1b917255
DOCKER_BUILDKIT=1
DOCKER_EXPERIMENTAL=1
DOCKER_GRAPHDRIVER=overlay2
EXECUTOR_NUMBER=0
GIT_BRANCH=PR-40050
GIT_COMMIT=b83c2f934e32add82cde1add189ce1612f351c28
GIT_URL=https://github.com/moby/moby.git
HOME=/home/ubuntu
HUDSON_COOKIE=e23b914d-319f-4995-9452-b4c69f21426f
HUDSON_HOME=/var/cloudbees-jenkins-distribution
HUDSON_SERVER_COOKIE=aaf6decb76ababb5
HUDSON_URL=https://ci.docker.com/public/
JENKINS_HOME=/var/cloudbees-jenkins-distribution
JENKINS_NODE_COOKIE=7007687d-1e88-4a00-8b91-95e3000f7ac0
JENKINS_SERVER_COOKIE=durable-9ee5edf5ed6d91eb1e01ee93e0a76d2f
JENKINS_URL=https://ci.docker.com/public/
JOB_BASE_NAME=PR-40050
JOB_DISPLAY_URL=https://ci.docker.com/public/job/moby/job/PR-40050/display/redirect
JOB_NAME=moby/PR-40050
JOB_URL=https://ci.docker.com/public/job/moby/job/PR-40050/
LANG=C.UTF-8
LOGNAME=ubuntu
MAIL=/var/mail/ubuntu
NODE_LABELS=amd64 app-converter docker e2e-runner e2e-runner-next ec2 linux overlay2 ubuntu ubuntu-1804 ubuntu-1804-overlay2 ubuntu-1804-overlay2 (i-0ff999de81893f11a) x86_64
NODE_NAME=ubuntu-1804-overlay2 (i-0ff999de81893f11a)
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
PWD=/home/ubuntu/workspace/moby_PR-40050
RUN_CHANGES_DISPLAY_URL=https://ci.docker.com/public/job/moby/job/PR-40050/2/display/redirect?page=changes
RUN_DISPLAY_URL=https://ci.docker.com/public/job/moby/job/PR-40050/2/display/redirect
SHELL=/bin/bash
SHLVL=1
SSH_CLIENT=10.100.0.41 49388 22
SSH_CONNECTION=10.100.0.41 49388 10.100.56.66 22
STAGE_NAME=DCO-check
TESTDEBUG=0
TIMEOUT=120m
USER=ubuntu
WORKSPACE=/home/ubuntu/workspace/moby_PR-40050
XDG_RUNTIME_DIR=/run/user/1000
XDG_SESSION_ID=3
_=/usr/bin/java
On a master job, this looks like (on windows):
Name Value
---- -----
ALLUSERSPROFILE C:\ProgramData
amd64 true
APPDATA C:\windows\system32\config\systemprofile\AppData\Roaming
APT_MIRROR cdn-fastly.deb.debian.org
BRANCH_NAME master
BUILD_DISPLAY_NAME #206
BUILD_ID 206
BUILD_NUMBER 206
BUILD_TAG jenkins-moby-master-206
BUILD_URL https://ci.docker.com/public/job/moby/job/master/206/
CHECK_CONFIG_COMMIT 78405559cfe5987174aa2cb6463b9b2c1b917255
CommonProgramFiles C:\Program Files\Common Files
CommonProgramFiles(x86) C:\Program Files (x86)\Common Files
CommonProgramW6432 C:\Program Files\Common Files
COMPUTERNAME azwin-2-42c8d0
ComSpec C:\windows\system32\cmd.exe
ConfigSequenceNumber 0
DOCKER_BUILDKIT 0
DOCKER_DUT_DEBUG 1
DOCKER_EXPERIMENTAL 1
DOCKER_GRAPHDRIVER overlay2
DriverData C:\Windows\System32\Drivers\DriverData
EXECUTOR_NUMBER 0
FQDN azwin-2-42c8d0.westus.cloudapp.azure.com
GIT_BRANCH master
GIT_COMMIT 0abbb9e4ebf1f31590209047881492f0347868e9
GIT_PREVIOUS_COMMIT ed2f50f8cdda067bad521acbe7a75b4e91e0962b
GIT_PREVIOUS_SUCCESSFUL_COMMIT de5a67156b5eed287883958b60b0c08dacfc7332
GIT_URL https://github.com/moby/moby.git
HUDSON_COOKIE fd08b14d-dd3f-4172-a652-2e0b329d7264
HUDSON_HOME /var/cloudbees-jenkins-distribution
HUDSON_SERVER_COOKIE aaf6decb76ababb5
HUDSON_URL https://ci.docker.com/public/
JAVA_HOME C:\java-1.8.0-openjdk-1.8.0.222-2.b10.ojdkbuild.windows.x86_64
JENKINS_HOME /var/cloudbees-jenkins-distribution
JENKINS_NODE_COOKIE c143233d-6b73-464b-90de-646bb4011557
JENKINS_SERVER_COOKIE durable-2f56e31ca5b2498536d5dc93c29eccaf
JENKINS_URL https://ci.docker.com/public/
JOB_BASE_NAME master
JOB_DISPLAY_URL https://ci.docker.com/public/job/moby/job/master/display/redirect
JOB_NAME moby/master
JOB_URL https://ci.docker.com/public/job/moby/job/master/
library.jps.version master
LOCALAPPDATA C:\windows\system32\config\systemprofile\AppData\Local
NODE_LABELS amd64 azure azwin-2-42c8d0 azwin-2019 docker west-us windows windows-2019 x86_64
NODE_NAME azwin-2-42c8d0
NUMBER_OF_PROCESSORS 4
OS Windows_NT
Path C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPo...
PATHEXT .COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC;.CPL
ppc64le true
PROCESSOR_ARCHITECTURE AMD64
PROCESSOR_IDENTIFIER Intel64 Family 6 Model 79 Stepping 1, GenuineIntel
PROCESSOR_LEVEL 6
PROCESSOR_REVISION 4f01
ProgramData C:\ProgramData
ProgramFiles C:\Program Files
ProgramFiles(x86) C:\Program Files (x86)
ProgramW6432 C:\Program Files
PROMPT $P$G
PSExecutionPolicyPreference Bypass
PSModulePath WindowsPowerShell\Modules;C:\Program Files\WindowsPowerShell\Modules;C:\windows\syste...
PUBLIC C:\Users\Public
RUN_CHANGES_DISPLAY_URL https://ci.docker.com/public/job/moby/job/master/206/display/redirect?page=changes
RUN_DISPLAY_URL https://ci.docker.com/public/job/moby/job/master/206/display/redirect
s390x true
skip_dco false
SKIP_VALIDATION_TESTS 1
SOURCES_DRIVE d
SOURCES_SUBDIR gopath
STAGE_NAME Run tests
SystemDrive C:
SystemRoot C:\windows
TEMP C:\windows\TEMP
TESTDEBUG 0
TESTRUN_DRIVE d
TESTRUN_SUBDIR CI
TIMEOUT 120m
TMP C:\windows\TEMP
unit_validate true
USERDOMAIN WORKGROUP
USERNAME azwin-2-42c8d0$
USERPROFILE C:\windows\system32\config\systemprofile
windir C:\windows
WINDOWS_BASE_IMAGE mcr.microsoft.com/windows/servercore
WINDOWS_BASE_IMAGE_TAG ltsc2019
windowsRS1 false
windowsRS5 true
WORKSPACE d:\gopath\src\github.com\docker\docker
And on a backport PR (on windows);
Name Value
---- -----
ALLUSERSPROFILE C:\ProgramData
APPDATA C:\windows\system32\config\systemprofile\AppData\Roaming
APT_MIRROR cdn-fastly.deb.debian.org
BRANCH_NAME PR-397
BUILD_DISPLAY_NAME #1
BUILD_ID 1
BUILD_NUMBER 1
BUILD_TAG jenkins-engine-PR-397-1
BUILD_URL https://ci.docker.com/public/job/engine/job/PR-397/1/
CHANGE_AUTHOR thaJeztah
CHANGE_AUTHOR_DISPLAY_NAME Sebastiaan van Stijn
CHANGE_BRANCH 19.03_backport_slirp4netns_sandbox
CHANGE_FORK thaJeztah
CHANGE_ID 397
CHANGE_TARGET 19.03
CHANGE_TITLE [19.03 backport] rootless: harden slirp4netns with mount namespace and seccomp
CHANGE_URL https://github.com/docker/engine/pull/397
CHECK_CONFIG_COMMIT 78405559cfe5987174aa2cb6463b9b2c1b917255
CommonProgramFiles C:\Program Files\Common Files
CommonProgramFiles(x86) C:\Program Files (x86)\Common Files
CommonProgramW6432 C:\Program Files\Common Files
COMPUTERNAME azwin-2-11bd00
ComSpec C:\windows\system32\cmd.exe
ConfigSequenceNumber 0
DOCKER_BUILDKIT 0
DOCKER_DUT_DEBUG 1
DOCKER_EXPERIMENTAL 1
DOCKER_GRAPHDRIVER overlay2
DriverData C:\Windows\System32\Drivers\DriverData
EXECUTOR_NUMBER 0
FQDN azwin-2-11bd00.westus.cloudapp.azure.com
GIT_BRANCH PR-397
GIT_COMMIT 5bd4233d7b0710005e0a520c137c64631c7e6664
GIT_URL https://github.com/docker/engine.git
HUDSON_COOKIE ba44f2b6-b0ab-41c6-a032-de0a150fce8c
HUDSON_HOME /var/cloudbees-jenkins-distribution
HUDSON_SERVER_COOKIE aaf6decb76ababb5
HUDSON_URL https://ci.docker.com/public/
JAVA_HOME C:\java-1.8.0-openjdk-1.8.0.222-2.b10.ojdkbuild.windows.x86_64
JENKINS_HOME /var/cloudbees-jenkins-distribution
JENKINS_NODE_COOKIE fe54bb84-9153-4bf2-a9b1-c678b81f40d4
JENKINS_SERVER_COOKIE durable-2f56e31ca5b2498536d5dc93c29eccaf
JENKINS_URL https://ci.docker.com/public/
JOB_BASE_NAME PR-397
JOB_DISPLAY_URL https://ci.docker.com/public/job/engine/job/PR-397/display/redirect
JOB_NAME engine/PR-397
JOB_URL https://ci.docker.com/public/job/engine/job/PR-397/
library.jps.version master
LOCALAPPDATA C:\windows\system32\config\systemprofile\AppData\Local
NODE_LABELS amd64 azure azwin-2-11bd00 azwin-2019 docker west-us windows windows-2019 x86_64
NODE_NAME azwin-2-11bd00
NUMBER_OF_PROCESSORS 4
OS Windows_NT
Path C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPo...
PATHEXT .COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC;.CPL
PROCESSOR_ARCHITECTURE AMD64
PROCESSOR_IDENTIFIER Intel64 Family 6 Model 79 Stepping 1, GenuineIntel
PROCESSOR_LEVEL 6
PROCESSOR_REVISION 4f01
ProgramData C:\ProgramData
ProgramFiles C:\Program Files
ProgramFiles(x86) C:\Program Files (x86)
ProgramW6432 C:\Program Files
PROMPT $P$G
PSExecutionPolicyPreference Bypass
PSModulePath WindowsPowerShell\Modules;C:\Program Files\WindowsPowerShell\Modules;C:\windows\syste...
PUBLIC C:\Users\Public
RUN_CHANGES_DISPLAY_URL https://ci.docker.com/public/job/engine/job/PR-397/1/display/redirect?page=changes
RUN_DISPLAY_URL https://ci.docker.com/public/job/engine/job/PR-397/1/display/redirect
SKIP_VALIDATION_TESTS 1
SOURCES_DRIVE d
SOURCES_SUBDIR gopath
STAGE_NAME Run tests
SystemDrive C:
SystemRoot C:\windows
TEMP C:\windows\TEMP
TESTDEBUG 0
TESTRUN_DRIVE d
TESTRUN_SUBDIR CI
TIMEOUT 120m
TMP C:\windows\TEMP
USERDOMAIN WORKGROUP
USERNAME azwin-2-11bd00$
USERPROFILE C:\windows\system32\config\systemprofile
windir C:\windows
WINDOWS_BASE_IMAGE mcr.microsoft.com/windows/servercore
WINDOWS_BASE_IMAGE_TAG ltsc2019
WORKSPACE d:\gopath\src\github.com\docker\docker
|
From those environment variables; differences between a "pull-request" job, and a "master" job; Pull request: "master": Differences between a "regular" pull request, and a backport; Regular PR: Backport PR: So we should force running these tests if either:
|
47505e6 to
4f83a87
Compare
|
The good news is that this works; the bad news is that we need to have #40038 merged to fix go-swagger from panicking |
0b76725 to
820d1d9
Compare
820d1d9 to
385e79f
Compare
|
@kolyshkin @cpuguy83 @tianon @andrewhsu this should be ready for review 👍 |
385e79f to
e843616
Compare
|
fixed the indentation; no other chances (CI was green before) |
e843616 to
217fa3c
Compare
217fa3c to
227c69f
Compare
|
ping @tianon @tiborvass PTAL 🤗 |
tianon
left a comment
There was a problem hiding this comment.
Overall looks good, just a few minor comments/questions. 👍
Jenkinsfile
Outdated
There was a problem hiding this comment.
I think ${CHANGE_TARGET:=master} here was intended to be ${CHANGE_TARGET:-master}? (given the script's length it really doesn't make much difference, but IMO it's better to make sure the implementation matches the intention without side-effect 😅)
hack/validate/default-seccomp
Outdated
There was a problem hiding this comment.
Should ${TEST_FORCE_VALIDATE} be ${TEST_FORCE_VALIDATE:-} here to ensure we don't get hung up by set -u if we run this script locally?
There was a problem hiding this comment.
It doesn't make much difference functionally, but I'd recommend using the (otherwise implied) -n here explicitly (if [ -n "$new_tests" ]; then). 👍
hack/validate/golangci-lint
Outdated
There was a problem hiding this comment.
Does this script ever get piped to a file or similar where these control codes will just be binary noise, or is it pretty exclusively terminal/Jenkins?
There was a problem hiding this comment.
This should only be in CI (afaik)
hack/validate/swagger
Outdated
There was a problem hiding this comment.
Same comment as above (should this be ${TEST_FORCE_VALIDATE:-} in case this is run locally?)
hack/validate/swagger-gen
Outdated
hack/validate/vendor
Outdated
This reverts commit 5f67568. Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]>
The `hack/validate/deprecate-integration-cli` script exited on success. As a result, validation steps to run afterwards would not be executed. Signed-off-by: Sebastiaan van Stijn <[email protected]>
This introduces a FORCE_VALIDATE environment variable, which allows forcing some validation steps, even if no changes were detected. Signed-off-by: Sebastiaan van Stijn <[email protected]>
On master ("non-pull-request"), force running some validation checks
(such as vendor, swagger), even if no files were changed. This allows
catching problems caused by pull-requests that were merged out-of-sequence.
Signed-off-by: Sebastiaan van Stijn <[email protected]>
The golang-ci-lint takes longest to run, so do the other
checks first; this also makes the output a bit more readable.
While at it, fix some consistency issues in the output of the
other checks.
Before this change:
TEST_FORCE_VALIDATE=1 hack/validate/default
Congratulations! All commits are properly signed with the DCO!
Congratulations! Seccomp profile generation is done correctly.
INFO [config_reader] Used config file hack/validate/golangci-lint.yml
INFO [lintersdb] Active 15 linters: [deadcode gofmt goimports golint gosec gosimple govet ineffassign misspell staticcheck structcheck typecheck unconvert unused varcheck]
...
INFO Execution took 2m12.4287302s
Congratulations! "./pkg/..." is safely isolated from internal code.
The swagger spec at "api/swagger.yaml" is valid against swagger specification 2.0
Congratulations! All api changes are done the right way.
Congratulations! All toml source files changed here have valid syntax.
Congratulations! Changelog CHANGELOG.md is well-formed.
Congratulations! Changelog CHANGELOG.md dates are in descending order.
After this change:
TEST_FORCE_VALIDATE=1 hack/validate/default
Congratulations! All commits are properly signed with the DCO!
Congratulations! Seccomp profile generation is done correctly.
Congratulations! Packages in "./pkg/..." are safely isolated from internal code.
Congratulations! The swagger spec at "api/swagger.yaml" is valid against swagger specification 2.0
Congratulations! All API changes are done the right way.
Congratulations! All TOML source files changed here have valid syntax.
Congratulations! Changelog CHANGELOG.md is well-formed.
Congratulations! Changelog CHANGELOG.md dates are in descending order.
Congratulations! No new tests were added to integration-cli.
INFO Start validation with golang-ci-lint
INFO [config_reader] Used config file hack/validate/golangci-lint.yml
INFO [lintersdb] Active 15 linters: [deadcode gofmt goimports golint gosec gosimple govet ineffassign misspell staticcheck structcheck typecheck unconvert unused varcheck]
...
INFO Execution took 2m12.4287302s
Signed-off-by: Sebastiaan van Stijn <[email protected]>
The intent of this check is to not add integration tests that use the CLI to test API functionality, so check for changes in "API" tests as well, now that the `integration/` suite has matured enough to develop new tests, or migrate existing ones. Signed-off-by: Sebastiaan van Stijn <[email protected]>
227c69f to
d3c0261
Compare
|
@tianon updated with your suggestions (thanks! ❤️) |
On master ("non-pull-request"), force running some validation checks
(such as vendor, swagger), even if no files were changed. This allows
catching problems caused by pull-requests that were merged out-of-sequence.
This also includes some additional changes and fixes;
hack/validate/deprecate-integration-clialso check for changes inintegration-cli/*_api_*.gofiles, because no new tests should be added there as wellcleaned up output;
Before this change:
After this change: