Skip to content

Commit b4cacfe

Browse files
MushuEEstshrive
authored andcommitted
tools: remove setup-gcp and hack/teardown.sh (#3505)
Supersedes #3396 by @stshrive, rebased onto current `main`. The commit keeps his authorship and sign-off. The GKE provisioning and teardown tooling now lives in https://github.com/ai-on-gke/substrate-gke under tools/setup-gcp, so core no longer carries a copy. Docs and comments that pointed at the removed files now point at substrate-gke. go mod tidy drops the GCP client modules only setup-gcp used, and vendor/ and _LICENSES/ are regenerated to match. Covers #3390, except that hack/teardown.sh is removed here as well; substrate-gke's tools/setup-gcp/teardown.sh replaces it. Part of #2304 ### Rebase notes Conflicts were mostly from the Hugo docsite move (#1974): - `tools/setup-gcp/cloud-sql.md`: deleted, as in the original PR. - `docs/guides/operator-install.md`, `docs/guides/postgres.md`: the original edits, re-applied at the new paths. - `docs/guides/observability.md`: kept main's version, since #1974 already removed the Dashboards section the original PR edited. - `cmd/kubectl-ate/README.md`: kept main's stub that points to `docs/reference/kubectl-ate.md`. That page no longer has the setup-gcp line, so the edit had nowhere to go. --------- Signed-off-by: Steven Shriver <[email protected]> Signed-off-by: Grant McCloskey <[email protected]> Co-authored-by: Steven Shriver <[email protected]> (cherry picked from commit b66e16a) Signed-off-by: Eitan Yarmush <[email protected]>
1 parent 1eeb151 commit b4cacfe

205 files changed

Lines changed: 84 additions & 159224 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.gitignore‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,6 @@ Thumbs.db
6363
# root; tools with their own go.mod land beside their source. Build with
6464
# `-o bin/<tool>`, which is ignored above.
6565
/validate-image-cache
66-
/setup-gcp
6766
/tools/apitool/apitool
6867
/tools/junittool/junittool
6968

‎AGENTS.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ It takes the Kubernetes control-plane out of the critical path to achieve lower
6666
Agent Substrate relies on the fact that agent-like applications tend to be idle most of the time to achieve heavy multiplexing.
6767

6868
For development, it's recommended to read the `README.md` and `CONTRIBUTING.md` in the root folder.
69-
See `cmd/ate-setup` and `tools/setup-gcp` for provisioning and deploying clusters and GCP resources.
69+
See `cmd/ate-setup` for deploying clusters. GCP resources are provisioned with `tools/setup-gcp` in [substrate-gke](https://github.com/ai-on-gke/substrate-gke).
7070

7171
## Repository Layout
7272

@@ -156,7 +156,7 @@ See the [metric registry](docs/guides/observability.md#the-metric-registry) sect
156156
1. Write tests for new behavior. Update tests when you change existing behavior. Consider the value of tests for future changes.
157157
2. Ensure changes do not break existing tests.
158158
3. Run `go test -race ./...` and `hack/verify-all.sh` locally before requesting a code review to catch common issues like missed copyright headers or formatting drift.
159-
4. For end-to-end tests involving the actual infrastructure, ensure you have a running cluster (setup via `hack/ate-dev-env.sh.example` and `go run ./tools/setup-gcp bootstrap`).
159+
4. For end-to-end tests involving the actual infrastructure, ensure you have a running cluster (on GKE, set up with `tools/setup-gcp` in [substrate-gke](https://github.com/ai-on-gke/substrate-gke)).
160160
5. A test that skips when a precondition is missing (Docker, a cluster, an artifact directory) must **fail** on it in CI, because a skip and a pass are the same exit code. Check for CI with a named predicate like `cmd/ateapi/internal/store/dockerenv.Required()` instead of an inline `os.Getenv("CI")`. Make sure the CI branch actually fails before merging. See `docs/dev/best-practices/ci-fail-closed.md`.
161161

162162
## Security Considerations

‎README.md‎

Lines changed: 19 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -164,9 +164,10 @@ curl -X POST -i http://localhost:8000/
164164

165165
### GKE Quickstart (Development)
166166

167-
1. Create and configure your environment file:
167+
1. Create and configure your environment file from the GKE example in
168+
[substrate-gke](https://github.com/ai-on-gke/substrate-gke/blob/main/tools/setup-gcp/ate-dev-env.sh.example) (`hack/ate-dev-env.sh.example` here is for kind):
168169
```bash
169-
cp hack/ate-dev-env.sh.example .ate-dev-env.sh
170+
curl -fsSLo .ate-dev-env.sh https://raw.githubusercontent.com/ai-on-gke/substrate-gke/main/tools/setup-gcp/ate-dev-env.sh.example
170171

171172
# Edit .ate-dev-env.sh to match your project and preferences, then source it:
172173
source .ate-dev-env.sh
@@ -177,19 +178,16 @@ curl -X POST -i http://localhost:8000/
177178
gcloud auth application-default login --project=${PROJECT_ID}
178179
```
179180

180-
3. Provision the required GCP resources (GKE cluster, GCS, and IAM bindings):
181-
```bash
182-
go run ./tools/setup-gcp bootstrap
183-
```
181+
3. Provision the required GCP resources (GKE cluster, GCS, and IAM bindings)
182+
with [`tools/setup-gcp`](https://github.com/ai-on-gke/substrate-gke/tree/main/tools/setup-gcp) in the
183+
[substrate-gke](https://github.com/ai-on-gke/substrate-gke) repository, which holds the GKE
184+
setup and teardown tooling. Its `bootstrap` command also creates the Workload
185+
Identity IAM grants that ate-api-server and atelet depend on. See
186+
[what `create iam` actually grants](https://github.com/ai-on-gke/substrate-gke/blob/main/tools/setup-gcp/README.md#what-create-iam-actually-grants)
187+
to audit them or apply them manually.
184188

185-
On a fresh project this step also creates the Workload Identity IAM grants
186-
that ate-api-server and atelet depend on. See
187-
[what `create iam` actually grants](tools/setup-gcp/README.md#what-create-iam-actually-grants)
188-
to audit them or apply them manually. If you bring your own cluster instead,
189-
it needs the PodCertificateRequest and ClusterTrustBundle APIs, which GKE
190-
1.37 and higher serve by default. On 1.36, enable them as beta APIs, then
191-
replace any 1.36 node pools created before that. See the
192-
[Create Cluster warning](tools/setup-gcp/README.md#2-create-cluster).
189+
If you bring your own cluster instead, it needs the PodCertificateRequest and
190+
ClusterTrustBundle APIs, which GKE 1.37 and higher serve by default.
193191

194192
4. Deploy the Agent Substrate system to your cluster:
195193
```bash
@@ -220,17 +218,11 @@ curl -X POST -i http://localhost:8000/
220218

221219
#### Custom Setup and Deployment
222220

223-
You can run individual setup steps to create GCP resources as needed. See `go run ./tools/setup-gcp --help` for available options. For example:
224-
```bash
225-
go run ./tools/setup-gcp create cluster
226-
go run ./tools/setup-gcp create bucket
227-
```
228-
229-
To run the PostgreSQL store backend on Cloud SQL — with IAM database
230-
authentication and no passwords — see [tools/setup-gcp/cloud-sql.md](tools/setup-gcp/cloud-sql.md).
221+
To create individual GCP resources, or to run the PostgreSQL store backend on
222+
Cloud SQL, see [`tools/setup-gcp`](https://github.com/ai-on-gke/substrate-gke/tree/main/tools/setup-gcp) in substrate-gke.
231223

232-
Similarly, you can deploy or clean up specific Agent Substrate components with
233-
`ate-setup`. See `go run ./cmd/ate-setup --help` for all options.
224+
You can deploy or cleanup specific Agent Substrate components using
225+
the installation script. See `go run ./cmd/ate-setup` for all options.
234226
```bash
235227
# Re-deploy only ate-apiserver of the ATE system
236228
go run ./cmd/ate-setup deploy apiserver
@@ -241,17 +233,13 @@ go run ./cmd/ate-setup delete all
241233

242234
#### Tearing down resources (GCP)
243235

244-
If you need to delete the resources created by the setup script, you can use the provided script `hack/teardown.sh`. This script will delete resources in the reverse order of creation and handles partial failures gracefully.
245-
246-
```bash
247-
./hack/teardown.sh [--all|--cluster-scoped]
248-
```
236+
To delete the GCP resources, run substrate-gke's
237+
[`tools/setup-gcp/teardown.sh`](https://github.com/ai-on-gke/substrate-gke/blob/main/tools/setup-gcp/README.md#teardown). It deletes resources
238+
in the reverse order of creation and handles partial failures.
249239

250240
* `--all` deletes everything, including the dashboards, the snapshot transfer service account and the project-level IAM bindings, which every Substrate cluster in the project shares.
251241
* `--cluster-scoped` deletes only this cluster, its buckets and the bindings on them. Use it while other Substrate clusters still use the project.
252242

253-
Or run individual teardown steps as needed (see `./hack/teardown.sh` for available options).
254-
255243
#### Tearing down local `kind` resources
256244

257245
If you need to delete the local `kind` cluster and its registry (if it was created by `hack/create-kind-cluster.sh`):
@@ -303,5 +291,4 @@ We provide several sample applications demonstrating Agent Substrate's capabilit
303291
will eventually ship in upstream Kubernetes (with different names).
304292
* `cmd/kubectl-ate`: A CLI tool for managing Agent Substrate resources. See its [README](docs/reference/kubectl-ate.md).
305293
* `cmd/benchmarking`: Synthetic workloads used by the load tests, including `glutton`, which consumes RAM, disk, and file descriptors on demand.
306-
* `tools/setup-gcp`: A provisioning utility to set up the necessary GCP infrastructure resources (GKE, GCS, IAM).
307294
* `demos/`: Sample applications demonstrating Agent Substrate capabilities.

‎_LICENSES/cloud.google.com/go/container/LICENSE‎

Lines changed: 0 additions & 202 deletions
This file was deleted.

0 commit comments

Comments
 (0)