Skip to content

Commit e159f28

Browse files
committed
docs(examples): add S3 via STS credentials manual test guide
Step-by-step walkthrough for testing end-to-end S3 access from an OpenShell sandbox using gateway-minted STS temporary credentials with proxy-side SigV4 re-signing. Covers AWS resource setup, supervisor image build, gateway configuration, provider lifecycle, and curl-based S3 operations inside a sandbox. Refs NVIDIA#1576
1 parent d46749f commit e159f28

1 file changed

Lines changed: 224 additions & 0 deletions

File tree

‎examples/aws-s3-sts.md‎

Lines changed: 224 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,224 @@
1+
# Manual E2E Test: S3 Access via STS Credentials
2+
3+
This guide walks through an end-to-end test of an OpenShell sandbox accessing
4+
AWS S3 using gateway-minted STS temporary credentials with proxy-side SigV4
5+
re-signing. The sandbox never sees real AWS credentials — the proxy resolves
6+
placeholders and signs requests on the fly.
7+
8+
## Prerequisites
9+
10+
- AWS CLI authenticated (`aws sts get-caller-identity` succeeds)
11+
- Podman running (`podman info` succeeds)
12+
- OpenShell built from the `feat/1576-aws-sts-with-sigv4` branch (or later,
13+
once merged)
14+
15+
## 1. Create AWS test resources
16+
17+
Create an S3 bucket and an IAM role the gateway can assume:
18+
19+
```shell
20+
BUCKET="openshell-sts-test-$(date +%s)"
21+
ACCOUNT=$(aws sts get-caller-identity --query Account --output text)
22+
23+
aws s3 mb "s3://${BUCKET}" --region us-east-1
24+
25+
aws iam create-role \
26+
--role-name openshell-sts-test-role \
27+
--assume-role-policy-document '{
28+
"Version": "2012-10-17",
29+
"Statement": [{
30+
"Effect": "Allow",
31+
"Principal": {"AWS": "arn:aws:iam::'${ACCOUNT}':root"},
32+
"Action": "sts:AssumeRole"
33+
}]
34+
}'
35+
36+
aws iam put-role-policy \
37+
--role-name openshell-sts-test-role \
38+
--policy-name s3-access \
39+
--policy-document '{
40+
"Version": "2012-10-17",
41+
"Statement": [{
42+
"Effect": "Allow",
43+
"Action": ["s3:PutObject", "s3:GetObject", "s3:ListBucket"],
44+
"Resource": ["arn:aws:s3:::'${BUCKET}'", "arn:aws:s3:::'${BUCKET}'/*"]
45+
}]
46+
}'
47+
```
48+
49+
Verify the role works:
50+
51+
```shell
52+
aws sts assume-role \
53+
--role-arn "arn:aws:iam::${ACCOUNT}:role/openshell-sts-test-role" \
54+
--role-session-name test \
55+
--query Credentials.AccessKeyId --output text
56+
```
57+
58+
## 2. Build the supervisor image
59+
60+
The supervisor image must include the SigV4 re-signing code and the updated
61+
proto definitions. Build it from the branch:
62+
63+
```shell
64+
CONTAINER_ENGINE=podman IMAGE_TAG=dev mise run build:docker:supervisor
65+
```
66+
67+
Verify the image exists locally:
68+
69+
```shell
70+
podman images | grep "openshell/supervisor.*dev"
71+
```
72+
73+
## 3. Start the gateway
74+
75+
The gateway needs AWS credentials in its environment to call `sts:AssumeRole`.
76+
Export them before starting:
77+
78+
```shell
79+
eval "$(aws configure export-credentials --format env)"
80+
```
81+
82+
The gateway must use the local supervisor image. Edit the gateway config
83+
(`~/.cache/gateway-podman/gateway.toml` or equivalent) so `supervisor_image`
84+
appears under `[openshell.gateway]`:
85+
86+
```toml
87+
[openshell.gateway]
88+
supervisor_image = "localhost/openshell/supervisor:dev"
89+
```
90+
91+
Then start the gateway:
92+
93+
```shell
94+
mise run gateway
95+
```
96+
97+
Alternatively, start the gateway binary directly with the correct config:
98+
99+
```shell
100+
eval "$(aws configure export-credentials --format env)"
101+
./target/debug/openshell-gateway \
102+
--config .cache/gateway-podman/gateway.toml \
103+
--port 18080 --log-level info --drivers podman --disable-tls \
104+
--db-url "sqlite:.cache/gateway-podman/gateway.db?mode=rwc"
105+
```
106+
107+
## 4. Configure the provider
108+
109+
In a separate terminal:
110+
111+
```shell
112+
export OPENSHELL_BASE_URL=http://localhost:18080
113+
114+
# Enable provider v2 (required for STS)
115+
openshell settings set --global --key providers_v2_enabled --value true --yes
116+
117+
# Create the provider with the aws-s3 profile
118+
openshell provider create --name s3-test --type aws-s3 \
119+
--credential AWS_ACCESS_KEY_ID=placeholder
120+
121+
# Configure STS refresh
122+
openshell provider refresh configure s3-test \
123+
--credential-key AWS_ACCESS_KEY_ID \
124+
--strategy aws-sts-assume-role \
125+
--material role_arn="arn:aws:iam::${ACCOUNT}:role/openshell-sts-test-role" \
126+
--material session_name="openshell-sandbox" \
127+
--material aws_region="us-east-1"
128+
129+
# Mint the first set of credentials
130+
openshell provider refresh rotate s3-test \
131+
--credential-key AWS_ACCESS_KEY_ID
132+
133+
# Verify
134+
openshell provider refresh status s3-test
135+
```
136+
137+
The status should show `refreshed` with an expiry ~1 hour from now.
138+
139+
## 5. Test S3 access from a sandbox
140+
141+
```shell
142+
openshell sandbox create --name s3-smoke \
143+
--provider s3-test \
144+
-- bash -c '
145+
BUCKET="<your-bucket-name>"
146+
REGION="us-east-1"
147+
CA=/etc/openshell-tls/ca-bundle.pem
148+
149+
echo "=== Upload ==="
150+
curl -s --cacert $CA -X PUT -H "Content-Type: text/plain" \
151+
-d "hello from openshell sandbox via STS" \
152+
"https://${BUCKET}.s3.${REGION}.amazonaws.com/from-sandbox.txt" \
153+
-w "\nHTTP %{http_code}\n"
154+
155+
echo ""
156+
echo "=== List ==="
157+
curl -s --cacert $CA \
158+
"https://${BUCKET}.s3.${REGION}.amazonaws.com/?list-type=2&max-keys=5" \
159+
-w "\nHTTP %{http_code}\n"
160+
161+
echo ""
162+
echo "=== Download ==="
163+
curl -s --cacert $CA \
164+
"https://${BUCKET}.s3.${REGION}.amazonaws.com/from-sandbox.txt" \
165+
-w "\nHTTP %{http_code}\n"
166+
'
167+
```
168+
169+
All three operations should return `HTTP 200`. The download should print
170+
`hello from openshell sandbox via STS`.
171+
172+
### What's happening
173+
174+
1. The gateway called `sts:AssumeRole` and stored three short-lived credentials
175+
(`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN`) in the
176+
provider record.
177+
2. The sandbox received placeholder values for these credentials as environment
178+
variables.
179+
3. Curl sent unsigned HTTP requests through the sandbox proxy's CONNECT tunnel.
180+
4. The proxy terminated TLS, stripped any existing AWS auth headers, resolved
181+
the real credentials from the `SecretResolver`, computed a fresh SigV4
182+
signature using the `aws-sigv4` crate, and forwarded the signed request to
183+
S3.
184+
5. S3 validated the signature and accepted the request.
185+
186+
The sandbox never saw real AWS credentials — only placeholders.
187+
188+
### Using `--cacert`
189+
190+
The proxy terminates TLS and presents a certificate signed by the OpenShell
191+
Sandbox CA. Curl needs `--cacert /etc/openshell-tls/ca-bundle.pem` to trust
192+
it. Python clients (boto3, requests) need `AWS_CA_BUNDLE` or
193+
`REQUESTS_CA_BUNDLE` set to the same path.
194+
195+
### Known limitation: chunked transfer encoding
196+
197+
S3 clients that use `Transfer-Encoding: chunked` (e.g., boto3 `put_object`)
198+
will fail because the SigV4 relay buffers the body by `Content-Length` only.
199+
Curl uses `Content-Length` by default and works. This will be addressed in a
200+
follow-up.
201+
202+
## 6. Clean up
203+
204+
```shell
205+
# Delete the sandbox
206+
openshell sandbox delete s3-smoke
207+
208+
# Delete AWS resources
209+
aws s3 rb "s3://${BUCKET}" --force
210+
aws iam delete-role-policy --role-name openshell-sts-test-role --policy-name s3-access
211+
aws iam delete-role --role-name openshell-sts-test-role
212+
```
213+
214+
## Troubleshooting
215+
216+
| Symptom | Cause | Fix |
217+
|---------|-------|-----|
218+
| `STS AssumeRole failed: dispatch failure` | Gateway doesn't have AWS credentials | Export credentials before starting: `eval "$(aws configure export-credentials --format env)"` |
219+
| `Policy discovery sync failed: invalid wire type` | Supervisor image doesn't have updated proto | Rebuild: `CONTAINER_ENGINE=podman IMAGE_TAG=dev mise run build:docker:supervisor` |
220+
| `CONNECT ... not permitted by policy` | Binary not in profile's `binaries` list | Use curl (in the list) or add your binary path to the policy |
221+
| `AuthorizationHeaderMalformed: region 's3' is wrong` | Old `extract_aws_region` bug | Rebuild from latest branch (fixed in `d46749fc`) |
222+
| `403 AccessDenied` from S3 | IAM role missing permissions, or STS creds expired | Check `openshell provider refresh status`; re-rotate if expired |
223+
| `SSLEOFError` from boto3 | boto3 uses chunked encoding; SigV4 relay doesn't handle it yet | Use curl instead of boto3 for now |
224+
| Supervisor uses wrong image | `supervisor_image` in wrong TOML section | Place under `[openshell.gateway]`, not `[openshell.gateway.gateway_jwt]` |

0 commit comments

Comments
 (0)