Provide custom sponge configs to exclude manual runs from testgrid by lidizheng · Pull Request #26283 · grpc/grpc

lidizheng · 2021-05-18T18:17:22Z

Internal counterpart cl/374442201

Sponge tests allow the test script to provide additional sponge configs in CSV to search test results.

This is implemented by PSM Sec framework for a while:

grpc/tools/internal_ci/linux/grpc_xds_k8s_install_test_driver.sh

Lines 229 to 241 in c8350cc

    
           ####################################### 
        
           # Report extra information about the job via sponge properties. 
        
           # Globals: 
        
           #   KOKORO_ARTIFACTS_DIR 
        
           #   GIT_ORIGIN_URL 
        
           #   GIT_COMMIT_SHORT 
        
           #   TESTGRID_EXCLUDE 
        
           # Arguments: 
        
           #   None 
        
           # Outputs: 
        
           #   Writes the output to stdout 
        
           #   Writes job properties to $KOKORO_ARTIFACTS_DIR/custom_sponge_config.csv 
        
           #######################################

This PR hope to bring this functionality to run_xds_tests.py as well.

The bash scripts are scattered for xDS interop tests that we have multiple languages times v2/v3. Backporting that many bash scripts might take forever. Hence, this PR choose to add the logic to the LCA of all build runs, run_xds_tests.py.

lidizheng · 2021-05-18T18:24:18Z

Test runs:

https://fusion2.corp.google.com/invocations/c01fc70b-628a-4260-a11e-696ae13155b5/targets

2021-05-18 20:59:52,366: INFO     Writing Sponge configs: ['TESTS_FORMAT_VERSION,2', 'TESTGRID_EXCLUDE,0', 'GIT_ORIGIN_URL,/var/local/jenkins/grpc', 'GIT_COMMIT_SHORT,e38cbb2']

sergiitk

Wow, that was fast! 🥇
Found an error tho.

tools/run_tests/run_xds_tests.py

lidizheng · 2021-05-18T23:41:02Z

Test: sponge/2e26cde7-e900-49a1-a6f7-70a3a20a1672

tools/run_tests/run_xds_tests.py

lidizheng

Thanks for the quick review! All comments addressed. The test link is updated.

tools/run_tests/run_xds_tests.py

sergiitk

This is great! Thank you so much! 🥇

tools/run_tests/dockerize/build_and_run_docker.sh

…rpc#26283) * Provide custom sponge configs * Correct the script * Allow test scripts to produce artifacts through Docker * Overcome the direcotry clone in our docker script * Expose TESTGRID_EXCLUDE and GIT_ORIGIN_URL through docker run * Make sanity tests happy

…26283) (#26312) * Provide custom sponge configs * Correct the script * Allow test scripts to produce artifacts through Docker * Overcome the direcotry clone in our docker script * Expose TESTGRID_EXCLUDE and GIT_ORIGIN_URL through docker run * Make sanity tests happy

…26283) (#26313) * Provide custom sponge configs * Correct the script * Allow test scripts to produce artifacts through Docker * Overcome the direcotry clone in our docker script * Expose TESTGRID_EXCLUDE and GIT_ORIGIN_URL through docker run * Make sanity tests happy

…tgrid (#26283) (#26312)" This reverts commit d529dde.

…tgrid (#26283) (#26313)" This reverts commit 54dc182.

…tgrid (#26283)" This reverts commit 113cd95.

…tgrid (#26283) (#26312)" (#26323) This reverts commit d529dde.

…tgrid (#26283)" (#26325) This reverts commit 113cd95.

…tgrid (#26283) (#26313)" (#26324) This reverts commit 54dc182.

* Bump version to v1.38.0-pre1 (#26208) * Bump version to v1.38.0-pre1 * Regenerate projects * [Backport][v1.38.x] Add EventEngine's port.h to the podspec (#26222) Backport of #26220 to v1.38.x. This is the only backport, since the EventEngine code was introduced after v1.37 was cut. This file is needed for the Objective-C cocoapod build. * fix xds resolver to add XdsClient to channel args even on errors (#26282) (#26288) Co-authored-by: Mark D. Roth <[email protected]> * Backport config update timeout change to v1.38.x (#26276) * Enable Traffic Director time tracer (#26111) * Just update the timeout (#26223) * Increase xds job timeouts (#26187) * Revert grpc_xds_k8s job timeouts back to 120 mins (#26219) * Revert grpc_xds_k8s_python timeout to 120mins (#26221) Seems to have been inadvertently increased in #26187 Co-authored-by: Eric Gribkoff <[email protected]> Co-authored-by: Yash Tibrewal <[email protected]> * Bump v1.38.x to v1.38.0 (#26317) * Bump version to v1.38.0 * Regenerate projects * Provide custom sponge configs to exclude manual runs from testgrid (#26283) (#26313) * Provide custom sponge configs * Correct the script * Allow test scripts to produce artifacts through Docker * Overcome the direcotry clone in our docker script * Expose TESTGRID_EXCLUDE and GIT_ORIGIN_URL through docker run * Make sanity tests happy * Revert "Provide custom sponge configs to exclude manual runs from testgrid (#26283) (#26313)" (#26324) This reverts commit 54dc182. * better way of building protoc aarch64 artifacts (#26423) * Backport #26430 and #26435 to v1.38.x (#26436) * workaround #26279 at the expense of binary compatibility * correctly tag aarch64 wheels as manylinux_2_24 * remove unnecessary comments left by #26430 * Backport #22964 into v1.38.x (#26448) * upgrade dotnet SDK 3 for C# build * add C# global.json * install dotnet SDK 3 on macos * upgrade C# docker images to debian buster * regenerate dockerfiles * Backport fixes from #26465 to v1.38.x (#26466) * switch remaining tests to C# buster image * remove C#'s global.json (which requires dotnet SDK 3+) * dotnet SDK3 currently breaks C# package build * regenerate dockerfiles Co-authored-by: Yash Tibrewal <[email protected]> Co-authored-by: AJ Heller <[email protected]> Co-authored-by: donnadionne <[email protected]> Co-authored-by: Mark D. Roth <[email protected]> Co-authored-by: Lidi Zheng <[email protected]> Co-authored-by: Eric Gribkoff <[email protected]> Co-authored-by: Jan Tattermusch <[email protected]>

Provide custom sponge configs

1aaa908

lidizheng added the release notes: no Indicates if PR should not be in release notes label May 18, 2021

Correct the script

92aee63

Allow test scripts to produce artifacts through Docker

e38cbb2

lidizheng marked this pull request as ready for review May 18, 2021 22:15

lidizheng requested review from ericgribkoff and sergiitk May 18, 2021 22:15

lidizheng assigned sergiitk and ericgribkoff May 18, 2021

sergiitk reviewed May 18, 2021

View reviewed changes

tools/run_tests/run_xds_tests.py Outdated Show resolved Hide resolved

Overcome the direcotry clone in our docker script

2554e3b

sergiitk reviewed May 18, 2021

View reviewed changes

tools/run_tests/run_xds_tests.py Outdated Show resolved Hide resolved

tools/run_tests/run_xds_tests.py Show resolved Hide resolved

Expose TESTGRID_EXCLUDE and GIT_ORIGIN_URL through docker run

135e942

lidizheng commented May 19, 2021

View reviewed changes

tools/run_tests/run_xds_tests.py Show resolved Hide resolved

sergiitk approved these changes May 19, 2021

View reviewed changes

tools/run_tests/dockerize/build_and_run_docker.sh Show resolved Hide resolved

Make sanity tests happy

0a1c95d

lidizheng merged commit 113cd95 into grpc:master May 19, 2021

lidizheng mentioned this pull request May 19, 2021

Backport custom sponge config to v1.30.x #26305

Closed

lidizheng mentioned this pull request May 19, 2021

Backport custom sponge config to v1.31.x #26306

Closed

lidizheng mentioned this pull request May 19, 2021

Backport custom sponge config to v1.32.x #26307

Closed

lidizheng mentioned this pull request May 19, 2021

Backport custom sponge config to v1.33.x #26308

Closed

lidizheng mentioned this pull request May 19, 2021

Backport custom sponge config to v1.34.x #26309

Closed

lidizheng mentioned this pull request May 19, 2021

Backport custom sponge config to v1.35.x #26310

Closed

lidizheng mentioned this pull request May 19, 2021

Backport custom sponge config to v1.36.x #26311

Closed

lidizheng mentioned this pull request May 19, 2021

Backport custom sponge config to v1.37.x #26312

Merged

lidizheng mentioned this pull request May 19, 2021

Backport custom sponge config to v1.38.x #26313

Merged

lidizheng added a commit that referenced this pull request May 20, 2021

Revert "Provide custom sponge configs to exclude manual runs from tes…

4dc4026

…tgrid (#26283) (#26312)" This reverts commit d529dde.

lidizheng mentioned this pull request May 20, 2021

Revert "Backport custom sponge config to v1.37.x" #26323

Merged

lidizheng added a commit that referenced this pull request May 20, 2021

Revert "Provide custom sponge configs to exclude manual runs from tes…

799737b

…tgrid (#26283) (#26313)" This reverts commit 54dc182.

lidizheng mentioned this pull request May 20, 2021

Revert "Backport custom sponge config to v1.38.x" #26324

Merged

lidizheng added a commit that referenced this pull request May 20, 2021

Revert "Provide custom sponge configs to exclude manual runs from tes…

e2b4a69

…tgrid (#26283)" This reverts commit 113cd95.

lidizheng mentioned this pull request May 20, 2021

Revert "Provide custom sponge configs to exclude manual runs from testgrid" #26325

Merged

lidizheng added a commit that referenced this pull request May 20, 2021

Revert "Provide custom sponge configs to exclude manual runs from tes…

963449d

…tgrid (#26283) (#26312)" (#26323) This reverts commit d529dde.

lidizheng added a commit that referenced this pull request May 20, 2021

Revert "Provide custom sponge configs to exclude manual runs from tes…

6f23b3f

…tgrid (#26283)" (#26325) This reverts commit 113cd95.

lidizheng added a commit that referenced this pull request May 20, 2021

Revert "Provide custom sponge configs to exclude manual runs from tes…

3f0ae86

…tgrid (#26283) (#26313)" (#26324) This reverts commit 54dc182.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Provide custom sponge configs to exclude manual runs from testgrid#26283

Provide custom sponge configs to exclude manual runs from testgrid#26283
lidizheng merged 6 commits intogrpc:masterfrom
lidizheng:custom-sponge

lidizheng commented May 18, 2021 •

edited

Loading

Uh oh!

lidizheng commented May 18, 2021 •

edited

Loading

Uh oh!

sergiitk left a comment •

edited

Loading

Uh oh!

Uh oh!

lidizheng commented May 18, 2021 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

lidizheng left a comment

Uh oh!

Uh oh!

sergiitk left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	#######################################
	# Report extra information about the job via sponge properties.
	# Globals:
	# KOKORO_ARTIFACTS_DIR
	# GIT_ORIGIN_URL
	# GIT_COMMIT_SHORT
	# TESTGRID_EXCLUDE
	# Arguments:
	# None
	# Outputs:
	# Writes the output to stdout
	# Writes job properties to $KOKORO_ARTIFACTS_DIR/custom_sponge_config.csv
	#######################################

Conversation

lidizheng commented May 18, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lidizheng commented May 18, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sergiitk left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lidizheng commented May 18, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lidizheng left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sergiitk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

lidizheng commented May 18, 2021 •

edited

Loading

lidizheng commented May 18, 2021 •

edited

Loading

sergiitk left a comment •

edited

Loading

lidizheng commented May 18, 2021 •

edited

Loading