Repository navigation
Initial commit of Envoy Dynamic Modules - #1535
Conversation
|
Thanks! Taking a look tomorrow |
Bowei Du (bowei)
added
kind/feature
Signed-off-by: Yan Avlasov <[email protected]>
Signed-off-by: Yan Avlasov <[email protected]>
e2c7004 to
554f311
Compare
yanavlasov
changed the title
|
FYI -- the shell will be deleted very soon. But it's ok to keep here for now. |
|
I don't know rust organization well enough -- but is it better to put rust code in a multi language repo at the top level or to nest it inside? |
|
Does Cargo.toml need to be at the root level |
| return | ||
| fi | ||
|
|
||
| # Build envoy dataplane image |
There was a problem hiding this comment.
how long does this take, on a cold CI machine?
There was a problem hiding this comment.
~5 minutes on my VM.
There was a problem hiding this comment.
Even less than that. Just ran with a cold docker cache and full image build finished in just over 2 minutes.
There was a problem hiding this comment.
CI has relatively small VMs. We can see all of the e2e jobs are failing on this PR.
There was a problem hiding this comment.
The failure was due to this problem: #1535 (comment) It is fixed after migrating to go based config
There was a problem hiding this comment.
We should consider what options we have to speed this up, e.g. caching in the actions, compiler flags, etc.
There was a problem hiding this comment.
Is this ok for now, or we need to fix before merge. Benjamin Elder (@BenTheElder) isn't this a temporary issue that will be resolved post -donation.
There was a problem hiding this comment.
That may take a bit, is there anything we can do to drive down the time? CI has become a bit of a bottleneck and 2 minutes is not a trivial addition.
There was a problem hiding this comment.
What kind of caching is available? Isn't the action basically constrained in the same way as the CI?
Benjamin Elder (BenTheElder)
left a comment
Benjamin Elder (BenTheElder)
left a comment
There was a problem hiding this comment.
we should probably have e.g. clippy?
No, think of it like |
I does not really matter, other than clarity for maintainers. I put it where all other binaries live. Open to suggestions here. |
You mean THE clippy? |
the official linter of rust ... https://github.com/rust-lang/rust-clippy |
Signed-off-by: Yan Avlasov <[email protected]>
Signed-off-by: Yan Avlasov <[email protected]>
I've run clippy on the code. In the followup PR I will add GH action to run unit tests and clippy on the changed code. |
Signed-off-by: Yan Avlasov <[email protected]>
Lior Lieberman (LiorLieberman)
left a comment
Lior Lieberman (LiorLieberman)
left a comment
There was a problem hiding this comment.
this looks good to me. this PR should be ~zero impact to envoy afaiu
will yield to Bowei to do final approval
Signed-off-by: Yan Avlasov <[email protected]>
Fixes the kind install on arm64 hosts (Apple Silicon), broken in agent-substrate#1535 - `BuildDockerfileImage` hard-codes `--platform=linux/amd64`, so the dataplane image's Rust builder stage fails with `exec /bin/sh: exec format error` on mac with arm64 The ko images already follow `KO_DEFAULTPLATFORMS`, which the kind install sets to the host architecture; the Dockerfile images did not. **Changes** - Pass `KO_DEFAULTPLATFORMS` through to `docker buildx build --platform` for both Dockerfile images (envoy dataplane, claude-code-multiplex demo), defaulting to `linux/amd64` when unset. - Pin the Envoy base image by its multi-platform index digest so `--platform` selects the matching manifest. - [x] Tested (`hack/install-ate-kind.sh --deploy-ate-system` on mac succeed)
…bstrate#1964) ## What - A pre-built install (`ate-setup deploy --image-repo REPO --image-tag TAG`) now pins `REPO/envoy-dataplane:TAG` like the ko images, instead of building it with docker buildx. - `make build-envoy-dataplane` publishes that image. - `make build-release-images` publishes every image a pre-built install needs (ko images, demos, envoy-dataplane), all tagged `$(VERSION)`. ## Why envoy-dataplane (agent-substrate#1535) is built from a Dockerfile, and ate-setup built it even for pre-built installs. Those installs have no registry to push to, so they failed on the default envoy router after the control plane was applied. It also couldn't join `ALL_IMAGES`, which only lists ko builds, so nothing published it for a release. ## Testing - New unit tests for pinning and rendering a pre-built egress manifest. The rendering test fails on `main` with the production error (`invalid tag "/envoy-dataplane:build-…"`). - `make build-release-images` pushed all 13 images under one tag, and `images.Prebuilt` pinned every one against the real registry. - Cherry-picks cleanly onto `release-0.2`. Fixes agent-substrate#1962 > It's a good idea to open an issue first for discussion. - [ ] Tests pass - [x] Appropriate changes to documentation are included in the PR --------- Co-authored-by: Anna Pendleton <[email protected]>
Collecting initial feedback on integrating Envoy dynamic modules for Substrate dataplane.
Envoy Dynamic Modules allow fast iteration and experimentation on Substrate dataplane. Dynamic modules are used to improve feature velocity. As requirements and solution are better understood they will be generalized and moved into Envoy's main repository.
Important points:
Dynamic modules introduce Rust toolchain into Substrate dev environment. This PR does not add GH actions for testing Dynamic Modules during CI builds. This will be added in a subsequent PR.
This change brings in Dockerfiles and docker buildx. It produces an image with a versioned Envoy binary (1.39) and accompanying dynamic modules.
Docker image registry URL in deployment spec for Envoy datplane is at this point hardcoded to
localhost:5001. This will break for anyone using a different registry. Fixing this will require using envsubst or equivalent. I'm open to other suggestions.Tests pass
Appropriate changes to documentation are included in the PR