Add support for `google_cloudbuildv2_connection` and `google_cloudbuildv2_repository` #6756

mariomachado94 · 2022-10-31T17:29:54Z

Adds CloudBuildv2 Repos API support to Beta provider. Also adds a field to CloudBuild (v1) trigger resource to be used with the Repos API.

These changes were already in the private providers:
https://cloud-internal-review.git.corp.google.com/c/cloud-graphite-eng/magic-modules-private-overrides/+/40250
https://cloud-internal-review.git.corp.google.com/c/cloud-graphite-eng/magic-modules-private-overrides/+/40350

Depends on DCL changes: cl/484273201

No testing is included. As secrets are needed for these resources, only manual testing will be done. (As advised on October 25th, 2022 office hours)

If this PR is for Terraform, I acknowledge that I have:

Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
Generated Terraform, and ran make test and make lint to ensure it passes unit and linter tests.
Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
Ran relevant acceptance tests (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

cloudbuildv2: Added `google_cloudbuildv2_connection` and `google_cloudbuildv2_repository` (beta)

cloudbuild: Added field `repository_event_config` to `google_cloudbuild_trigger` for use with v2 repository (beta)

rileykarson · 2022-11-01T19:29:35Z

This is failing generate-diffs, I believe due to the DCL version. We'll need to wait for #6639 to land to move forward w/ it.

mariomachado94 · 2022-11-07T17:25:40Z

@rileykarson we also want to wait for our cloudbuild API to go into public preview before we merge this, so we have 3 dependency checkpoints before merging:

New DCL: Upgrade the dcl version to v1.26.0 #6639
Related DCL changes: cl/484273201
CloudBuild public preview launches

One other PR is based on this change, and so this blocks it:

#6778

modular-magician · 2023-01-12T16:43:19Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 3 files changed, 512 insertions(+))
Terraform Beta: Diff ( 12 files changed, 2467 insertions(+), 5 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

modular-magician · 2023-01-12T17:29:53Z

Tests analytics

Total tests: 2413
Passed tests 2152
Skipped tests: 251
Failed tests: 10

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccContainerCluster_withInvalidGatewayApiConfigChannel|TestAccRegionInstanceGroupManager_stateful|TestAccCloudbuildv2Repository_GithubRepository|TestAccCloudbuildv2Repository_GheRepository|TestAccCloudbuildv2Connection_GithubConnection|TestAccCloudbuildv2Connection_GhePrivUpdateConnection|TestAccCloudbuildv2Connection_GhePrivConnection|TestAccCloudbuildv2Connection_GheConnection|TestAccCloudbuildv2Connection_GheCompleteConnection|TestAccBigtableAppProfile_bigtableAppProfileMulticlusterExample

modular-magician · 2023-01-12T17:34:23Z

Tests passed during RECORDING mode:
TestAccContainerCluster_withInvalidGatewayApiConfigChannel[Debug log]
TestAccRegionInstanceGroupManager_stateful[Debug log]
TestAccBigtableAppProfile_bigtableAppProfileMulticlusterExample[Debug log]

Tests failed during RECORDING mode:
TestAccCloudbuildv2Repository_GithubRepository[Error message] [Debug log]
TestAccCloudbuildv2Repository_GheRepository[Error message] [Debug log]
TestAccCloudbuildv2Connection_GithubConnection[Error message] [Debug log]
TestAccCloudbuildv2Connection_GhePrivUpdateConnection[Error message] [Debug log]
TestAccCloudbuildv2Connection_GhePrivConnection[Error message] [Debug log]
TestAccCloudbuildv2Connection_GheConnection[Error message] [Debug log]
TestAccCloudbuildv2Connection_GheCompleteConnection[Error message] [Debug log]

Please fix these to complete your PR
View the build log or the debug log for each test

rileykarson

I've done an initial pass through the connection resource, although not a complete one yet. It looked reasonable so far. I haven't reviewed the trigger and repository changes in much detail yet.

It looks like all the tests are failing; we'll need to fix those up to move forward. It seems like they're referencing pre-created resources, which we'll need to work out a strategy for. Are there details about what infrastructure was already set up for the DCL in a doc or comment somewhere?

Additionally, I'd recommend splitting this into three PRs, and sending them one-by-one:

the connection resource
the repository resource
the changes to trigger

Smaller changes tend to go exponentially faster- as a single unit re-verifying this entire change takes a substantial amount of time, so complete review passes may take days to get the free time for.

We can merge them as we go, which means pieces may get released in isolation. If that's undesirable we can use a branch to manage the changes, although an additional review pass will be needed at merge time to correct any drift from HEAD (if any).

rileykarson · 2023-01-13T19:34:15Z

mmv1/products/cloudbuild/terraform.yaml

+        name: "cloudbuild_trigger_repo"
+        min_version: beta
+        primary_resource_id: "repo-trigger"
+        skip_test: true


Can we amend this configuration to generate a working test? If that's impossible, what's the blocker?

rileykarson · 2023-01-13T19:34:48Z

mmv1/templates/terraform/examples/cloudbuild_trigger_repo.tf.erb

+
+provider "google-beta" {}
+
+resource "google_secret_manager_secret" "github-token-secret" {


Within an example, we use the same provider version throughout. Here, that should be provider = google-beta (This applies to all HCL files)

rileykarson · 2023-01-13T19:35:11Z

mmv1/templates/terraform/examples/cloudbuild_trigger_repo.tf.erb

@@ -0,0 +1,72 @@
+terraform {


This block & the provider definition on L8 should be superfluous & can be removed. (This applies to all HCL files)

rileykarson · 2023-01-13T19:37:59Z

tpgtools/overrides/cloudbuildv2/beta/tpgtools_product.yaml

+
+- type: PRODUCT_DOCS_SECTION
+  details:
+    docssection: Cloud Build


We map our docs sections to the endpoints being hit, rather than a subjective grouping of what a product or service is. We'll want to use the natural name "Cloud Build v2", or override to something like "Cloud Build (v2 API)" or similar. See Cloud Functions and Cloud Run & their v2s for reference.

rileykarson · 2023-01-13T19:38:18Z

tpgtools/overrides/cloudbuildv2/samples/connection/github.tf.tmpl

@@ -0,0 +1,50 @@
+terraform {


Same comments as the MMv1 example.

rileykarson · 2023-01-13T19:42:43Z

mmv1/templates/terraform/examples/cloudbuild_trigger_repo.tf.erb

+}
+
+resource "google_secret_manager_secret_iam_policy" "policy" {
+  project = google_secret_manager_secret.github-token-secret.project


We supply a project automatically, and you can omit the field unless the project is defined within the same configuration (same for all occurrences)

rileykarson · 2023-01-13T19:47:18Z

tpgtools/api/cloudbuildv2/samples/ghe_complete.connection.json

+  "name": "{{connection}}",
+  "githubEnterpriseConfig": {
+    "hostUri": "https://ghe.proctor-staging-test.com",
+    "privateKeySecretVersion":"projects/1033762898806/secrets/gcbrepos-dcltest-private-key/versions/1",


What project do the referenced secrets live in? Are they publicly accessible?

We're likely to need to do a permissions grant within the test, create the secrets we reference, or similar.

Moved to a separate PR: #7157

Adjusted to use a separate project as agreed offline.

vicpadilla · 2023-01-23T18:50:08Z

Comments addressed in new PR: #7157

rileykarson · 2023-01-27T19:34:52Z

Superseded w/ #7157

mariomachado94 requested a review from rileykarson as a code owner October 31, 2022 17:29

rileykarson changed the title ~~Main~~ Add support for google_cloudbuildv2_connection and google_cloudbuildv2_repository Nov 1, 2022

mariomachado94 mentioned this pull request Nov 3, 2022

CloudBuild V2 Connection IAM resources #6778

Merged

5 tasks

mariomachado94 force-pushed the main branch from 4d07e46 to 12ceeb3 Compare November 14, 2022 17:04

Mario Machado added 2 commits January 12, 2023 11:26

Adding cloudbuildv2 Connection and Repository to beta provider.

3aa6faf

Adds repo trigger field and example to beta provider.

60e5a67

mariomachado94 force-pushed the main branch from 12ceeb3 to 60e5a67 Compare January 12, 2023 16:28

rileykarson reviewed Jan 13, 2023

View reviewed changes

rileykarson closed this Jan 27, 2023


		provider "google-beta" {}

		resource "google_secret_manager_secret" "github-token-secret" {

Add support for google_cloudbuildv2_connection and google_cloudbuildv2_repository #6756

Add support for google_cloudbuildv2_connection and google_cloudbuildv2_repository #6756

Uh oh!

Conversation

mariomachado94 commented Oct 31, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rileykarson commented Nov 1, 2022

Uh oh!

mariomachado94 commented Nov 7, 2022

Uh oh!

modular-magician commented Jan 12, 2023

Diff report

Uh oh!

modular-magician commented Jan 12, 2023

Tests analytics

Action taken

Uh oh!

modular-magician commented Jan 12, 2023

Uh oh!

rileykarson left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vicpadilla commented Jan 23, 2023

Uh oh!

rileykarson commented Jan 27, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Add support for `google_cloudbuildv2_connection` and `google_cloudbuildv2_repository` #6756

Add support for `google_cloudbuildv2_connection` and `google_cloudbuildv2_repository` #6756

mariomachado94 commented Oct 31, 2022 •

edited

Loading