ci: update testing pipeline for notebooks by loferris · Pull Request #1345 · googleapis/synthtool

loferris · 2022-02-03T04:38:30Z

This PR updates the notebook template to reflect parallelized tests for notebooks repos. We have also included one sample notebook with which to test the template.

parthea · 2022-03-29T21:10:38Z

Here is the output for the failing session

nox > flake8 synthtool tests autosynth integration_tests
synthtool/gcp/templates/python_notebooks_testing_pipeline/.cloud-build/execute_changed_notebooks_helper.py:76:86: E231 missing whitespace after ','
synthtool/gcp/templates/python_notebooks_testing_pipeline/.cloud-build/execute_changed_notebooks_helper.py:152:9: F841 local variable 'operation_result' is assigned to but never used
synthtool/gcp/templates/python_notebooks_testing_pipeline/.cloud-build/execute_changed_notebooks_helper.py:212:15: F541 f-string is missing placeholders
synthtool/gcp/templates/python_notebooks_testing_pipeline/.cloud-build/utils/utils.py:16:1: F401 'datetime.datetime' imported but unused
synthtool/gcp/templates/python_notebooks_testing_pipeline/.cloud-build/utils/utils.py:17:1: F401 'typing.Optional' imported but unused
synthtool/gcp/templates/python_notebooks_testing_pipeline/.cloud-build/utils/utils.py:18:1: F401 'google.cloud.storage' imported but unused
synthtool/gcp/templates/python_notebooks_testing_pipeline/.cloud-build/utils/utils.py:19:1: F401 'google.cloud.aiplatform.utils' imported but unused
synthtool/gcp/templates/python_notebooks_testing_pipeline/.cloud-build/utils/utils.py:20:1: F401 'google.auth.credentials as auth_credentials' imported but unused
nox > Command flake8 synthtool tests autosynth integration_tests failed with exit code 1
nox > Session lint-3.6 failed.

parthea · 2022-04-01T14:58:00Z

@@ -1,5 +1,5 @@
 #!/usr/bin/env python
-# # Copyright 2021 Google LLC
+# # Copyright 2022 Google LLC


Please keep original copyright year. Applies throughout

resolved in latest commit

parthea · 2022-04-01T14:59:15Z

+                except Exception as exception:
+                    print(exception)

+        print("")


Please remove print statement or add a comment to clarify the reason for keeping it.

Resolved in recent commit.

parthea · 2022-04-01T15:00:03Z

+is_dry_run = False
+
+if is_dry_run:
+    print("Starting cleanup in dry run mode...")


is_dry_run is always False, please remove test code or add a comment to clarify the reason for having it.

Removed - I think the setting fixes an immediate problem but the print statement exists for future settings of that variable. However, I tend to not include code until it's supported. @ivanmkc may have more context!

parthea · 2022-04-01T15:22:22Z

+from google.api_core import operation
+
+CLOUD_BUILD_FILEPATH = ".cloud-build/notebook-execution-test-cloudbuild-single.yaml"
+TIMEOUT_IN_SECONDS = 86400


Can you confirm that a 24 hour timeout is correct? It's better to have a lower timeout to limit resource usage in case the build is stuck.

Yes, this is a requirement for the tests at present.

parthea · 2022-04-01T15:23:45Z

+    # Print the in-progress operation
+    # print("IN PROGRESS:")
+    # print(operation.metadata)
+
+    # Print the completed status
+    # print("RESULT:", result.status)


nit: add a TODO comment if there is action needed for a developer, otherwise remove commented out code

Added in recent commit!

parthea · 2022-04-01T15:24:52Z

+    - 'python3 -m pip install -U pip && python3 -m pip freeze && python3 .cloud-build/execute_notebook_cli.py --notebook_source "${_NOTEBOOK_GCS_URI}" --output_file_or_uri "${_NOTEBOOK_OUTPUT_GCS_URI}"'
+    env:
+    - 'IS_TESTING=1'
+timeout: 86400s


Similar to above. Could we lower the timeout and increase as needed?

Not as present, but we can check in about it again with notebooks folks!

parthea · 2022-04-01T15:26:20Z

+timeout: 86400s
+options:
+  pool:
+    name: ${_PRIVATE_POOL_NAME}


Can you clarify where the environment variables are set? It may be helpful to add a README or provide a link to existing documentation to clarify what environment variables are needed.

I'd like to add that as a follow up issue, but I agree.

parthea · 2022-04-01T15:28:48Z

@@ -3,6 +3,7 @@
 # source: metadata.proto


If this change is not necessary for your PR, please could you change it in a separate PR?

Removed in recent commit.

…ud-build/execute_changed_notebooks_cli.py Co-authored-by: Anthonios Partheniou <[email protected]>

* update testing pipeline * kokoro fix * fixing linting errors * Update synthtool/gcp/templates/python_notebooks_testing_pipeline/.cloud-build/execute_changed_notebooks_cli.py Co-authored-by: Anthonios Partheniou <[email protected]> * Update synthtool/gcp/templates/python_notebooks_testing_pipeline/.cloud-build/execute_changed_notebooks_cli.py Co-authored-by: Anthonios Partheniou <[email protected]> * resolving license year * removing redundant print statements * adding developer todos * reverting protos * resetting protos to main Co-authored-by: Anthonios Partheniou <[email protected]>

* adding notebook template function * changing template path * changing template path * chore: add googleapis/yoshi-python to CODEOWNERS (#1380) * chore(python): add E231 to .flake8 ignore list (#1379) * chore: Enable Size-Label bot in all googleapis Python notebook testing (#1384) * chore: Enable Size-Label bot in all googleapis Python notebook testing repositories Auto-label T-shirt size indicator should be assigned on every new pull request in all googleapis Python notebook testing repositories * Remove product Remove product since it is by default true * chore(python): Enable size-label bot (#1383) * chore: Enable Size-Label bot in all googleapis Python repositories Auto-label T-shirt size indicator should be assigned on every new pull request in all googleapis Python repositories * Remove product Remove product since it is by default true Co-authored-by: Anthonios Partheniou <[email protected]> * chore(python): update .pre-commit-config.yaml to use black==22.3.0 (#1378) Co-authored-by: Tim Swast <[email protected]> * chore(python): Enable size-label bot (#1385) * chore: remove dependency on google-api-core (#1372) * chore(deps): update dependency google-api-core to v2.7.1 * chore: remove dependency on google-api-core Co-authored-by: Anthonios Partheniou <[email protected]> * chore(deps): update dependency numpy to v1.22.3 (#1387) * chore(deps): disable dependency dashboard (#1223) Co-authored-by: Anthonios Partheniou <[email protected]> * chore(deps): update dependency pandas to v1.4.1 (#1258) Co-authored-by: Anthonios Partheniou <[email protected]> * build(deps): bump ipython (#1336) Bumps [ipython](https://github.com/ipython/ipython) from 7.0 to 7.16.3. - [Release notes](https://github.com/ipython/ipython/releases) - [Commits](ipython/ipython@7.0.0...7.16.3) --- updated-dependencies: - dependency-name: ipython dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: remove use of googleapis-discovery (#1280) * chore: remove use of googleapis-discovery * remove constant and method for googleapis-discovery Co-authored-by: Anthonios Partheniou <[email protected]> * chore: add classifiers for python 3.7, 3.8 and 3.9 (#1247) added Support for python 3.7, 3.8, 3.9 Co-authored-by: Anthonios Partheniou <[email protected]> * chore(python): refactor unit / system test dependency install (#1294) * chore(python): refactor unit / system test dependency install Closes #1185. * chore: use editable installs for local deps * chore: don't install deps using '-e' * chore: deprecate 'unit_test_external_dependencies' * fix: install standard + main unit test deps together FBO pip resolver. Co-authored-by: Anthonios Partheniou <[email protected]> * chore(python_notebooks): update dependency black to v22 (#1396) * chore(deps): update dependency ipython to v8 (#1397) Co-authored-by: Anthonios Partheniou <[email protected]> * chore: Enable Size-Label bot in all googleapis NodeJs repositories (#1382) * chore: Enable Size-Label bot in all googleapis NodeJs repositories Auto-label T-shirt size indicator should be assigned on every new pull request in all googleapis NodeJs repositories * Remove product Remove product since it is by default true * chore(deps): update dependency setuptools to v61 (#1398) Co-authored-by: Anthonios Partheniou <[email protected]> * chore(deps): update dependency org.sonatype.plugins:nexus-staging-maven-plugin to v1.6.12 (#1388) Co-authored-by: Neenu Shaji <[email protected]> * ci: update testing pipeline for notebooks (#1345) * update testing pipeline * kokoro fix * fixing linting errors * Update synthtool/gcp/templates/python_notebooks_testing_pipeline/.cloud-build/execute_changed_notebooks_cli.py Co-authored-by: Anthonios Partheniou <[email protected]> * Update synthtool/gcp/templates/python_notebooks_testing_pipeline/.cloud-build/execute_changed_notebooks_cli.py Co-authored-by: Anthonios Partheniou <[email protected]> * resolving license year * removing redundant print statements * adding developer todos * reverting protos * resetting protos to main Co-authored-by: Anthonios Partheniou <[email protected]> * chore(deps): update actions/setup-python action to v3 (#1395) Co-authored-by: Anthonios Partheniou <[email protected]> * chore: upgrade black in noxfile.py to 22.3.0 (#1377) * chore: upgrade black to 22.3.0 * update lint also * run blacken session * ci: use click>8.0 for blacken/lint sessions * chore: fix typo Co-authored-by: nicain <[email protected]> * rebase error Co-authored-by: nicain <[email protected]> * chore(deps): update dependency pandas to v1.4.2 (#1400) * chore(deps): update dependency setuptools to v61.3.1 (#1399) Co-authored-by: Anthonios Partheniou <[email protected]> * chore(deps): update dependency nbqa to v1.3.1 (#1391) Co-authored-by: Anthonios Partheniou <[email protected]> * chore(deps): update dependency protobuf to v3.19.4 (#1389) Co-authored-by: Anthonios Partheniou <[email protected]> * chore(deps): update dependency protobuf to v3.20.0 (#1403) * chore(deps): update dependency setuptools to v62 (#1406) * chore(python): Use python 3.10.4 base image for post processor (#1405) * chore(deps): update actions/setup-node action to v3 (#1393) Co-authored-by: Jeffrey Rennie <[email protected]> * chore(python): add license header to auto-label.yaml (#1404) * chore: improve logic to configure release-please for previous major versions (#1408) * fix: allow version.py files without the library version * chore: remove obsolete file * chore: run black Co-authored-by: Anthonios Partheniou <[email protected]> * chore: Enable Size-Label bot in all googleapis Java repositories (#1381) * chore: Enable Size-Label bot in all googleapis Java repositories Auto-label T-shirt size indicator should be assigned on every new pull request in all googleapis Java repositories * Remove product Remove product since it is by default true * add license header Co-authored-by: Neenu Shaji <[email protected]> * chore(deps): update actions/checkout action to v3 (#1392) Co-authored-by: Jeffrey Rennie <[email protected]> * chore: run tests with Python 3.10 (#1407) Co-authored-by: Anthonios Partheniou <[email protected]> * chore(deps): update dependency python to v3.10.4 (#1386) Co-authored-by: Anthonios Partheniou <[email protected]> Co-authored-by: Jeffrey Rennie <[email protected]> * fix: clarify the gax-nodejs usage in README (#1352) Co-authored-by: Benjamin E. Coe <[email protected]> Co-authored-by: Jeffrey Rennie <[email protected]> * Revert "fix: clarify the gax-nodejs usage in README (#1352)" (#1409) This reverts commit e1557e4. * chore(deps): update dependency pyupgrade to v2.32.0 (#1412) * chore(deps): update dependency nbconvert to v6.5.0 (#1414) * build: make ci testing conditional on engines field in package.json, move configs to Node 12 (#1418) * build: make ci testing conditional on engines field in package.json, move configs to Node 12 Co-authored-by: Benjamin E. Coe <[email protected]> * build: sdd srs yaml file (#1419) * build: add sync-repo-settings and change branch protection * update names for template * adding type annotation Co-authored-by: Anthonios Partheniou <[email protected]> Co-authored-by: losalex <[email protected]> Co-authored-by: Tim Swast <[email protected]> Co-authored-by: WhiteSource Renovate <[email protected]> Co-authored-by: Dan Lee <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Brent Shaffer <[email protected]> Co-authored-by: Anurag Kumar <[email protected]> Co-authored-by: Tres Seaver <[email protected]> Co-authored-by: Neenu Shaji <[email protected]> Co-authored-by: nicain <[email protected]> Co-authored-by: Jeffrey Rennie <[email protected]> Co-authored-by: Bu Sun Kim <[email protected]> Co-authored-by: Summer Ji <[email protected]> Co-authored-by: Benjamin E. Coe <[email protected]> Co-authored-by: sofisl <[email protected]>

update testing pipeline

82eee5d

loferris requested review from SurferJeffAtGoogle, bcoe and chingor13 as code owners February 3, 2022 04:38

loferris requested a review from a team February 3, 2022 04:38

Merge branch 'master' into notebooks_parallel_ci

1ba8a4e

loferris changed the title ~~feat: update testing pipeline for notebooks~~ WIP feat: update testing pipeline for notebooks Feb 3, 2022

loferris changed the title ~~WIP feat: update testing pipeline for notebooks~~ feat: update testing pipeline for notebooks Feb 3, 2022

loferris requested a review from ivanmkc February 3, 2022 04:45

kokoro fix

0d02565

fixing linting errors

f1b5fef

loferris requested a review from a team March 31, 2022 21:03

Merge branch 'master' into notebooks_parallel_ci

77c969e

loferris requested a review from parthea March 31, 2022 22:28

parthea self-assigned this Apr 1, 2022

parthea approved these changes Apr 1, 2022

View reviewed changes

parthea changed the title ~~feat: update testing pipeline for notebooks~~ ci: update testing pipeline for notebooks Apr 1, 2022

parthea assigned loferris and unassigned parthea Apr 1, 2022

loferris and others added 9 commits April 1, 2022 11:19

Merge branch 'master' into notebooks_parallel_ci

e48179e

Update synthtool/gcp/templates/python_notebooks_testing_pipeline/.clo…

9b2580a

…ud-build/execute_changed_notebooks_cli.py Co-authored-by: Anthonios Partheniou <[email protected]>

Update synthtool/gcp/templates/python_notebooks_testing_pipeline/.clo…

538d391

…ud-build/execute_changed_notebooks_cli.py Co-authored-by: Anthonios Partheniou <[email protected]>

resolving license year

1c286b3

removing redundant print statements

a43dcd4

adding developer todos

0d2f80c

reverting protos

777282b

resetting protos to main

2e04f04

Merge branch 'master' into notebooks_parallel_ci

7c20a33

loferris merged commit ddcf429 into googleapis:master Apr 1, 2022

Conversation

loferris commented Feb 3, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

parthea commented Mar 29, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

loferris commented Feb 3, 2022 •

edited

Loading