Skip to content

Jenkinsfile: force validate on master, and some fixes/improvements#40050

Merged
thaJeztah merged 7 commits intomoby:masterfrom
thaJeztah:ci_force_validate
Dec 23, 2019
Merged

Jenkinsfile: force validate on master, and some fixes/improvements#40050
thaJeztah merged 7 commits intomoby:masterfrom
thaJeztah:ci_force_validate

Conversation

@thaJeztah
Copy link
Member

@thaJeztah thaJeztah commented Oct 7, 2019

On master ("non-pull-request"), force running some validation checks
(such as vendor, swagger), even if no files were changed. This allows
catching problems caused by pull-requests that were merged out-of-sequence.

This also includes some additional changes and fixes;

  • revert 5f67568, which was temporarily added in Replace gometalinter with golangci-lint #39668, but we forgot to remove afterwards
  • fix hack/validate/deprecate-integration-cli to not exit on "success" (which prevented other tests to run after it in the same batch)
  • cleanup the output for consistency (see below)
  • make hack/validate/deprecate-integration-cli also check for changes in integration-cli/*_api_*.go files, because no new tests should be added there as well

cleaned up output;

Before this change:

TEST_FORCE_VALIDATE=1 hack/validate/default
Congratulations!  All commits are properly signed with the DCO!
Congratulations! Seccomp profile generation is done correctly.
INFO [config_reader] Used config file hack/validate/golangci-lint.yml
INFO [lintersdb] Active 15 linters: [deadcode gofmt goimports golint gosec gosimple govet ineffassign misspell staticcheck structcheck typecheck unconvert unused varcheck]
...
INFO Execution took 2m12.4287302s
Congratulations!  "./pkg/..." is safely isolated from internal code.
The swagger spec at "api/swagger.yaml" is valid against swagger specification 2.0
Congratulations! All api changes are done the right way.
Congratulations!  All toml source files changed here have valid syntax.
Congratulations!  Changelog CHANGELOG.md is well-formed.
Congratulations!  Changelog CHANGELOG.md dates are in descending order.

After this change:

TEST_FORCE_VALIDATE=1 hack/validate/default
Congratulations!  All commits are properly signed with the DCO!
Congratulations!  Seccomp profile generation is done correctly.
Congratulations!  Packages in "./pkg/..." are safely isolated from internal code.
Congratulations!  The swagger spec at "api/swagger.yaml" is valid against swagger specification 2.0
Congratulations!  All API changes are done the right way.
Congratulations!  All TOML source files changed here have valid syntax.
Congratulations!  Changelog CHANGELOG.md is well-formed.
Congratulations!  Changelog CHANGELOG.md dates are in descending order.
Congratulations!  No new tests were added to integration-cli.

INFO Start validation with golang-ci-lint
INFO [config_reader] Used config file hack/validate/golangci-lint.yml
INFO [lintersdb] Active 15 linters: [deadcode gofmt goimports golint gosec gosimple govet ineffassign misspell staticcheck structcheck typecheck unconvert unused varcheck]
...
INFO Execution took 2m12.4287302s

Jenkinsfile Outdated
Copy link
Member Author

@thaJeztah thaJeztah Oct 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Env vars available;

 APT_MIRROR=cdn-fastly.deb.debian.org
 BRANCH_NAME=PR-40050
 BUILD_DISPLAY_NAME=#2
 BUILD_ID=2
 BUILD_NUMBER=2
 BUILD_TAG=jenkins-moby-PR-40050-2
 BUILD_URL=https://ci.docker.com/public/job/moby/job/PR-40050/2/
 CHANGE_AUTHOR=thaJeztah
 CHANGE_AUTHOR_DISPLAY_NAME=Sebastiaan van Stijn
 CHANGE_BRANCH=ci_force_validate
 CHANGE_FORK=thaJeztah
 CHANGE_ID=40050
 CHANGE_TARGET=master
 CHANGE_TITLE=Jenkinsfile: force validate steps on master
 CHANGE_URL=https://github.com/moby/moby/pull/40050
 CHECK_CONFIG_COMMIT=78405559cfe5987174aa2cb6463b9b2c1b917255
 DOCKER_BUILDKIT=1
 DOCKER_EXPERIMENTAL=1
 DOCKER_GRAPHDRIVER=overlay2
 EXECUTOR_NUMBER=0
 GIT_BRANCH=PR-40050
 GIT_COMMIT=b83c2f934e32add82cde1add189ce1612f351c28
 GIT_URL=https://github.com/moby/moby.git
 HOME=/home/ubuntu
 HUDSON_COOKIE=e23b914d-319f-4995-9452-b4c69f21426f
 HUDSON_HOME=/var/cloudbees-jenkins-distribution
 HUDSON_SERVER_COOKIE=aaf6decb76ababb5
 HUDSON_URL=https://ci.docker.com/public/
 JENKINS_HOME=/var/cloudbees-jenkins-distribution
 JENKINS_NODE_COOKIE=7007687d-1e88-4a00-8b91-95e3000f7ac0
 JENKINS_SERVER_COOKIE=durable-9ee5edf5ed6d91eb1e01ee93e0a76d2f
 JENKINS_URL=https://ci.docker.com/public/
 JOB_BASE_NAME=PR-40050
 JOB_DISPLAY_URL=https://ci.docker.com/public/job/moby/job/PR-40050/display/redirect
 JOB_NAME=moby/PR-40050
 JOB_URL=https://ci.docker.com/public/job/moby/job/PR-40050/
 LANG=C.UTF-8
 LOGNAME=ubuntu
 MAIL=/var/mail/ubuntu
 NODE_LABELS=amd64 app-converter docker e2e-runner e2e-runner-next ec2 linux overlay2 ubuntu ubuntu-1804 ubuntu-1804-overlay2 ubuntu-1804-overlay2 (i-0ff999de81893f11a) x86_64
 NODE_NAME=ubuntu-1804-overlay2 (i-0ff999de81893f11a)
 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
 PWD=/home/ubuntu/workspace/moby_PR-40050
 RUN_CHANGES_DISPLAY_URL=https://ci.docker.com/public/job/moby/job/PR-40050/2/display/redirect?page=changes
 RUN_DISPLAY_URL=https://ci.docker.com/public/job/moby/job/PR-40050/2/display/redirect
 SHELL=/bin/bash
 SHLVL=1
 SSH_CLIENT=10.100.0.41 49388 22
 SSH_CONNECTION=10.100.0.41 49388 10.100.56.66 22
 STAGE_NAME=DCO-check
 TESTDEBUG=0
 TIMEOUT=120m
 USER=ubuntu
 WORKSPACE=/home/ubuntu/workspace/moby_PR-40050
 XDG_RUNTIME_DIR=/run/user/1000
 XDG_SESSION_ID=3
 _=/usr/bin/java

On a master job, this looks like (on windows):

Name                           Value                                                                                   
 ----                           -----                                                                                   
 ALLUSERSPROFILE                C:\ProgramData                                                                          
 amd64                          true                                                                                    
 APPDATA                        C:\windows\system32\config\systemprofile\AppData\Roaming                                
 APT_MIRROR                     cdn-fastly.deb.debian.org                                                               
 BRANCH_NAME                    master                                                                                  
 BUILD_DISPLAY_NAME             #206                                                                                    
 BUILD_ID                       206                                                                                     
 BUILD_NUMBER                   206                                                                                     
 BUILD_TAG                      jenkins-moby-master-206                                                                 
 BUILD_URL                      https://ci.docker.com/public/job/moby/job/master/206/                                   
 CHECK_CONFIG_COMMIT            78405559cfe5987174aa2cb6463b9b2c1b917255                                                
 CommonProgramFiles             C:\Program Files\Common Files                                                           
 CommonProgramFiles(x86)        C:\Program Files (x86)\Common Files                                                     
 CommonProgramW6432             C:\Program Files\Common Files                                                           
 COMPUTERNAME                   azwin-2-42c8d0                                                                          
 ComSpec                        C:\windows\system32\cmd.exe                                                             
 ConfigSequenceNumber           0                                                                                       
 DOCKER_BUILDKIT                0                                                                                       
 DOCKER_DUT_DEBUG               1                                                                                       
 DOCKER_EXPERIMENTAL            1                                                                                       
 DOCKER_GRAPHDRIVER             overlay2                                                                                
 DriverData                     C:\Windows\System32\Drivers\DriverData                                                  
 EXECUTOR_NUMBER                0                                                                                       
 FQDN                           azwin-2-42c8d0.westus.cloudapp.azure.com                                                
 GIT_BRANCH                     master                                                                                  
 GIT_COMMIT                     0abbb9e4ebf1f31590209047881492f0347868e9                                                
 GIT_PREVIOUS_COMMIT            ed2f50f8cdda067bad521acbe7a75b4e91e0962b                                                
 GIT_PREVIOUS_SUCCESSFUL_COMMIT de5a67156b5eed287883958b60b0c08dacfc7332                                                
 GIT_URL                        https://github.com/moby/moby.git                                                        
 HUDSON_COOKIE                  fd08b14d-dd3f-4172-a652-2e0b329d7264                                                    
 HUDSON_HOME                    /var/cloudbees-jenkins-distribution                                                     
 HUDSON_SERVER_COOKIE           aaf6decb76ababb5                                                                        
 HUDSON_URL                     https://ci.docker.com/public/                                                           
 JAVA_HOME                      C:\java-1.8.0-openjdk-1.8.0.222-2.b10.ojdkbuild.windows.x86_64                          
 JENKINS_HOME                   /var/cloudbees-jenkins-distribution                                                     
 JENKINS_NODE_COOKIE            c143233d-6b73-464b-90de-646bb4011557                                                    
 JENKINS_SERVER_COOKIE          durable-2f56e31ca5b2498536d5dc93c29eccaf                                                
 JENKINS_URL                    https://ci.docker.com/public/                                                           
 JOB_BASE_NAME                  master                                                                                  
 JOB_DISPLAY_URL                https://ci.docker.com/public/job/moby/job/master/display/redirect                       
 JOB_NAME                       moby/master                                                                             
 JOB_URL                        https://ci.docker.com/public/job/moby/job/master/                                       
 library.jps.version            master                                                                                  
 LOCALAPPDATA                   C:\windows\system32\config\systemprofile\AppData\Local                                  
 NODE_LABELS                    amd64 azure azwin-2-42c8d0 azwin-2019 docker west-us windows windows-2019 x86_64        
 NODE_NAME                      azwin-2-42c8d0                                                                          
 NUMBER_OF_PROCESSORS           4                                                                                       
 OS                             Windows_NT                                                                              
 Path                           C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPo...
 PATHEXT                        .COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC;.CPL                              
 ppc64le                        true                                                                                    
 PROCESSOR_ARCHITECTURE         AMD64                                                                                   
 PROCESSOR_IDENTIFIER           Intel64 Family 6 Model 79 Stepping 1, GenuineIntel                                      
 PROCESSOR_LEVEL                6                                                                                       
 PROCESSOR_REVISION             4f01                                                                                    
 ProgramData                    C:\ProgramData                                                                          
 ProgramFiles                   C:\Program Files                                                                        
 ProgramFiles(x86)              C:\Program Files (x86)                                                                  
 ProgramW6432                   C:\Program Files                                                                        
 PROMPT                         $P$G                                                                                    
 PSExecutionPolicyPreference    Bypass                                                                                  
 PSModulePath                   WindowsPowerShell\Modules;C:\Program Files\WindowsPowerShell\Modules;C:\windows\syste...
 PUBLIC                         C:\Users\Public                                                                         
 RUN_CHANGES_DISPLAY_URL        https://ci.docker.com/public/job/moby/job/master/206/display/redirect?page=changes      
 RUN_DISPLAY_URL                https://ci.docker.com/public/job/moby/job/master/206/display/redirect                   
 s390x                          true                                                                                    
 skip_dco                       false                                                                                   
 SKIP_VALIDATION_TESTS          1                                                                                       
 SOURCES_DRIVE                  d                                                                                       
 SOURCES_SUBDIR                 gopath                                                                                  
 STAGE_NAME                     Run tests                                                                               
 SystemDrive                    C:                                                                                      
 SystemRoot                     C:\windows                                                                              
 TEMP                           C:\windows\TEMP                                                                         
 TESTDEBUG                      0                                                                                       
 TESTRUN_DRIVE                  d                                                                                       
 TESTRUN_SUBDIR                 CI                                                                                      
 TIMEOUT                        120m                                                                                    
 TMP                            C:\windows\TEMP                                                                         
 unit_validate                  true                                                                                    
 USERDOMAIN                     WORKGROUP                                                                               
 USERNAME                       azwin-2-42c8d0$                                                                         
 USERPROFILE                    C:\windows\system32\config\systemprofile                                                
 windir                         C:\windows                                                                              
 WINDOWS_BASE_IMAGE             mcr.microsoft.com/windows/servercore                                                    
 WINDOWS_BASE_IMAGE_TAG         ltsc2019                                                                                
 windowsRS1                     false                                                                                   
 windowsRS5                     true                                                                                    
 WORKSPACE                      d:\gopath\src\github.com\docker\docker      

And on a backport PR (on windows);

Name                           Value                                                                                   
 ----                           -----                                                                                   
 ALLUSERSPROFILE                C:\ProgramData                                                                          
 APPDATA                        C:\windows\system32\config\systemprofile\AppData\Roaming                                
 APT_MIRROR                     cdn-fastly.deb.debian.org                                                               
 BRANCH_NAME                    PR-397                                                                                  
 BUILD_DISPLAY_NAME             #1                                                                                      
 BUILD_ID                       1                                                                                       
 BUILD_NUMBER                   1                                                                                       
 BUILD_TAG                      jenkins-engine-PR-397-1                                                                 
 BUILD_URL                      https://ci.docker.com/public/job/engine/job/PR-397/1/                                   
 CHANGE_AUTHOR                  thaJeztah                                                                               
 CHANGE_AUTHOR_DISPLAY_NAME     Sebastiaan van Stijn                                                                    
 CHANGE_BRANCH                  19.03_backport_slirp4netns_sandbox                                                      
 CHANGE_FORK                    thaJeztah                                                                               
 CHANGE_ID                      397                                                                                     
 CHANGE_TARGET                  19.03                                                                                   
 CHANGE_TITLE                   [19.03 backport] rootless: harden slirp4netns with mount namespace and seccomp          
 CHANGE_URL                     https://github.com/docker/engine/pull/397                                               
 CHECK_CONFIG_COMMIT            78405559cfe5987174aa2cb6463b9b2c1b917255                                                
 CommonProgramFiles             C:\Program Files\Common Files                                                           
 CommonProgramFiles(x86)        C:\Program Files (x86)\Common Files                                                     
 CommonProgramW6432             C:\Program Files\Common Files                                                           
 COMPUTERNAME                   azwin-2-11bd00                                                                          
 ComSpec                        C:\windows\system32\cmd.exe                                                             
 ConfigSequenceNumber           0                                                                                       
 DOCKER_BUILDKIT                0                                                                                       
 DOCKER_DUT_DEBUG               1                                                                                       
 DOCKER_EXPERIMENTAL            1                                                                                       
 DOCKER_GRAPHDRIVER             overlay2                                                                                
 DriverData                     C:\Windows\System32\Drivers\DriverData                                                  
 EXECUTOR_NUMBER                0                                                                                       
 FQDN                           azwin-2-11bd00.westus.cloudapp.azure.com                                                
 GIT_BRANCH                     PR-397                                                                                  
 GIT_COMMIT                     5bd4233d7b0710005e0a520c137c64631c7e6664                                                
 GIT_URL                        https://github.com/docker/engine.git                                                    
 HUDSON_COOKIE                  ba44f2b6-b0ab-41c6-a032-de0a150fce8c                                                    
 HUDSON_HOME                    /var/cloudbees-jenkins-distribution                                                     
 HUDSON_SERVER_COOKIE           aaf6decb76ababb5                                                                        
 HUDSON_URL                     https://ci.docker.com/public/                                                           
 JAVA_HOME                      C:\java-1.8.0-openjdk-1.8.0.222-2.b10.ojdkbuild.windows.x86_64                          
 JENKINS_HOME                   /var/cloudbees-jenkins-distribution                                                     
 JENKINS_NODE_COOKIE            fe54bb84-9153-4bf2-a9b1-c678b81f40d4                                                    
 JENKINS_SERVER_COOKIE          durable-2f56e31ca5b2498536d5dc93c29eccaf                                                
 JENKINS_URL                    https://ci.docker.com/public/                                                           
 JOB_BASE_NAME                  PR-397                                                                                  
 JOB_DISPLAY_URL                https://ci.docker.com/public/job/engine/job/PR-397/display/redirect                     
 JOB_NAME                       engine/PR-397                                                                           
 JOB_URL                        https://ci.docker.com/public/job/engine/job/PR-397/                                     
 library.jps.version            master                                                                                  
 LOCALAPPDATA                   C:\windows\system32\config\systemprofile\AppData\Local                                  
 NODE_LABELS                    amd64 azure azwin-2-11bd00 azwin-2019 docker west-us windows windows-2019 x86_64        
 NODE_NAME                      azwin-2-11bd00                                                                          
 NUMBER_OF_PROCESSORS           4                                                                                       
 OS                             Windows_NT                                                                              
 Path                           C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPo...
 PATHEXT                        .COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC;.CPL                              
 PROCESSOR_ARCHITECTURE         AMD64                                                                                   
 PROCESSOR_IDENTIFIER           Intel64 Family 6 Model 79 Stepping 1, GenuineIntel                                      
 PROCESSOR_LEVEL                6                                                                                       
 PROCESSOR_REVISION             4f01                                                                                    
 ProgramData                    C:\ProgramData                                                                          
 ProgramFiles                   C:\Program Files                                                                        
 ProgramFiles(x86)              C:\Program Files (x86)                                                                  
 ProgramW6432                   C:\Program Files                                                                        
 PROMPT                         $P$G                                                                                    
 PSExecutionPolicyPreference    Bypass                                                                                  
 PSModulePath                   WindowsPowerShell\Modules;C:\Program Files\WindowsPowerShell\Modules;C:\windows\syste...
 PUBLIC                         C:\Users\Public                                                                         
 RUN_CHANGES_DISPLAY_URL        https://ci.docker.com/public/job/engine/job/PR-397/1/display/redirect?page=changes      
 RUN_DISPLAY_URL                https://ci.docker.com/public/job/engine/job/PR-397/1/display/redirect                   
 SKIP_VALIDATION_TESTS          1                                                                                       
 SOURCES_DRIVE                  d                                                                                       
 SOURCES_SUBDIR                 gopath                                                                                  
 STAGE_NAME                     Run tests                                                                               
 SystemDrive                    C:                                                                                      
 SystemRoot                     C:\windows                                                                              
 TEMP                           C:\windows\TEMP                                                                         
 TESTDEBUG                      0                                                                                       
 TESTRUN_DRIVE                  d                                                                                       
 TESTRUN_SUBDIR                 CI                                                                                      
 TIMEOUT                        120m                                                                                    
 TMP                            C:\windows\TEMP                                                                         
 USERDOMAIN                     WORKGROUP                                                                               
 USERNAME                       azwin-2-11bd00$                                                                         
 USERPROFILE                    C:\windows\system32\config\systemprofile                                                
 windir                         C:\windows                                                                              
 WINDOWS_BASE_IMAGE             mcr.microsoft.com/windows/servercore                                                    
 WINDOWS_BASE_IMAGE_TAG         ltsc2019                                                                                
 WORKSPACE                      d:\gopath\src\github.com\docker\docker               

@thaJeztah
Copy link
Member Author

From those environment variables; differences between a "pull-request" job, and a "master" job;

Pull request:

BRANCH_NAME                    PR-397

"master":

BRANCH_NAME                    master

Differences between a "regular" pull request, and a backport;

Regular PR:

CHANGE_TARGET                  master

Backport PR:

CHANGE_TARGET                  19.03

So we should force running these tests if either:

BRANCH_NAME != PR-...... ("master" or "release branch" runs) or CHANGE_TARGET != master (backport pull requests to release branches)

@thaJeztah thaJeztah force-pushed the ci_force_validate branch 17 times, most recently from 47505e6 to 4f83a87 Compare October 7, 2019 16:24
@thaJeztah
Copy link
Member Author

The good news is that this works; the bad news is that we need to have #40038 merged to fix go-swagger from panicking

@thaJeztah thaJeztah force-pushed the ci_force_validate branch 3 times, most recently from 0b76725 to 820d1d9 Compare October 11, 2019 11:44
@thaJeztah thaJeztah changed the title Jenkinsfile: force validate steps on master Jenkinsfile: force validate on master, and some fixes/improvements Oct 11, 2019
@thaJeztah thaJeztah marked this pull request as ready for review October 11, 2019 18:15
@thaJeztah thaJeztah requested a review from tianon as a code owner October 11, 2019 18:15
@thaJeztah
Copy link
Member Author

@kolyshkin @cpuguy83 @tianon @andrewhsu this should be ready for review 👍

@thaJeztah
Copy link
Member Author

fixed the indentation; no other chances (CI was green before)

Copy link
Member

@cpuguy83 cpuguy83 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@thaJeztah
Copy link
Member Author

ping @tianon @tiborvass PTAL 🤗

Copy link
Member

@tianon tianon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good, just a few minor comments/questions. 👍

Jenkinsfile Outdated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think ${CHANGE_TARGET:=master} here was intended to be ${CHANGE_TARGET:-master}? (given the script's length it really doesn't make much difference, but IMO it's better to make sure the implementation matches the intention without side-effect 😅)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should ${TEST_FORCE_VALIDATE} be ${TEST_FORCE_VALIDATE:-} here to ensure we don't get hung up by set -u if we run this script locally?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't make much difference functionally, but I'd recommend using the (otherwise implied) -n here explicitly (if [ -n "$new_tests" ]; then). 👍

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this script ever get piped to a file or similar where these control codes will just be binary noise, or is it pretty exclusively terminal/Jenkins?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should only be in CI (afaik)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as above (should this be ${TEST_FORCE_VALIDATE:-} in case this is run locally?)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto above.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto. 👍

This reverts commit 5f67568.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
The `hack/validate/deprecate-integration-cli` script exited on
success. As a result, validation steps to run afterwards would
not be executed.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
This introduces a FORCE_VALIDATE environment variable, which allows
forcing some validation steps, even if no changes were detected.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
On master ("non-pull-request"), force running some validation checks
(such as vendor, swagger), even if no files were changed. This allows
catching problems caused by pull-requests that were merged out-of-sequence.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
The golang-ci-lint takes longest to run, so do the other
checks first; this also makes the output a bit more readable.

While at it, fix some consistency issues in the output of the
other checks.

Before this change:

    TEST_FORCE_VALIDATE=1 hack/validate/default
    Congratulations!  All commits are properly signed with the DCO!
    Congratulations! Seccomp profile generation is done correctly.
    INFO [config_reader] Used config file hack/validate/golangci-lint.yml
    INFO [lintersdb] Active 15 linters: [deadcode gofmt goimports golint gosec gosimple govet ineffassign misspell staticcheck structcheck typecheck unconvert unused varcheck]
    ...
    INFO Execution took 2m12.4287302s
    Congratulations!  "./pkg/..." is safely isolated from internal code.
    The swagger spec at "api/swagger.yaml" is valid against swagger specification 2.0
    Congratulations! All api changes are done the right way.
    Congratulations!  All toml source files changed here have valid syntax.
    Congratulations!  Changelog CHANGELOG.md is well-formed.
    Congratulations!  Changelog CHANGELOG.md dates are in descending order.

After this change:

    TEST_FORCE_VALIDATE=1 hack/validate/default
    Congratulations!  All commits are properly signed with the DCO!
    Congratulations!  Seccomp profile generation is done correctly.
    Congratulations!  Packages in "./pkg/..." are safely isolated from internal code.
    Congratulations!  The swagger spec at "api/swagger.yaml" is valid against swagger specification 2.0
    Congratulations!  All API changes are done the right way.
    Congratulations!  All TOML source files changed here have valid syntax.
    Congratulations!  Changelog CHANGELOG.md is well-formed.
    Congratulations!  Changelog CHANGELOG.md dates are in descending order.
    Congratulations!  No new tests were added to integration-cli.

    INFO Start validation with golang-ci-lint
    INFO [config_reader] Used config file hack/validate/golangci-lint.yml
    INFO [lintersdb] Active 15 linters: [deadcode gofmt goimports golint gosec gosimple govet ineffassign misspell staticcheck structcheck typecheck unconvert unused varcheck]
    ...
    INFO Execution took 2m12.4287302s

Signed-off-by: Sebastiaan van Stijn <[email protected]>
The intent of this check is to not add integration tests that use the
CLI to test API functionality, so check for changes in "API" tests as
well, now that the `integration/` suite has matured enough to develop
new tests, or migrate existing ones.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
@thaJeztah
Copy link
Member Author

@tianon updated with your suggestions (thanks! ❤️)

Copy link
Member

@tianon tianon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants