|
| 1 | +# Manual E2E Test: S3 Access via STS Credentials |
| 2 | + |
| 3 | +This guide walks through an end-to-end test of an OpenShell sandbox accessing |
| 4 | +AWS S3 using gateway-minted STS temporary credentials with proxy-side SigV4 |
| 5 | +re-signing. The sandbox never sees real AWS credentials — the proxy resolves |
| 6 | +placeholders and signs requests on the fly. |
| 7 | + |
| 8 | +## Prerequisites |
| 9 | + |
| 10 | +- AWS CLI authenticated (`aws sts get-caller-identity` succeeds) |
| 11 | +- Podman running (`podman info` succeeds) |
| 12 | +- OpenShell built from the `feat/1576-aws-sts-with-sigv4` branch (or later, |
| 13 | + once merged) |
| 14 | + |
| 15 | +## 1. Create AWS test resources |
| 16 | + |
| 17 | +Create an S3 bucket and an IAM role the gateway can assume: |
| 18 | + |
| 19 | +```shell |
| 20 | +BUCKET="openshell-sts-test-$(date +%s)" |
| 21 | +ACCOUNT=$(aws sts get-caller-identity --query Account --output text) |
| 22 | + |
| 23 | +aws s3 mb "s3://${BUCKET}" --region us-east-1 |
| 24 | + |
| 25 | +aws iam create-role \ |
| 26 | + --role-name openshell-sts-test-role \ |
| 27 | + --assume-role-policy-document '{ |
| 28 | + "Version": "2012-10-17", |
| 29 | + "Statement": [{ |
| 30 | + "Effect": "Allow", |
| 31 | + "Principal": {"AWS": "arn:aws:iam::'${ACCOUNT}':root"}, |
| 32 | + "Action": "sts:AssumeRole" |
| 33 | + }] |
| 34 | + }' |
| 35 | + |
| 36 | +aws iam put-role-policy \ |
| 37 | + --role-name openshell-sts-test-role \ |
| 38 | + --policy-name s3-access \ |
| 39 | + --policy-document '{ |
| 40 | + "Version": "2012-10-17", |
| 41 | + "Statement": [{ |
| 42 | + "Effect": "Allow", |
| 43 | + "Action": ["s3:PutObject", "s3:GetObject", "s3:ListBucket"], |
| 44 | + "Resource": ["arn:aws:s3:::'${BUCKET}'", "arn:aws:s3:::'${BUCKET}'/*"] |
| 45 | + }] |
| 46 | + }' |
| 47 | +``` |
| 48 | + |
| 49 | +Verify the role works: |
| 50 | + |
| 51 | +```shell |
| 52 | +aws sts assume-role \ |
| 53 | + --role-arn "arn:aws:iam::${ACCOUNT}:role/openshell-sts-test-role" \ |
| 54 | + --role-session-name test \ |
| 55 | + --query Credentials.AccessKeyId --output text |
| 56 | +``` |
| 57 | + |
| 58 | +## 2. Build the supervisor image |
| 59 | + |
| 60 | +The supervisor image must include the SigV4 re-signing code and the updated |
| 61 | +proto definitions. Build it from the branch: |
| 62 | + |
| 63 | +```shell |
| 64 | +CONTAINER_ENGINE=podman IMAGE_TAG=dev mise run build:docker:supervisor |
| 65 | +``` |
| 66 | + |
| 67 | +Verify the image exists locally: |
| 68 | + |
| 69 | +```shell |
| 70 | +podman images | grep "openshell/supervisor.*dev" |
| 71 | +``` |
| 72 | + |
| 73 | +## 3. Start the gateway |
| 74 | + |
| 75 | +The gateway needs AWS credentials in its environment to call `sts:AssumeRole`. |
| 76 | +Export them before starting: |
| 77 | + |
| 78 | +```shell |
| 79 | +eval "$(aws configure export-credentials --format env)" |
| 80 | +``` |
| 81 | + |
| 82 | +The gateway must use the local supervisor image. Edit the gateway config |
| 83 | +(`~/.cache/gateway-podman/gateway.toml` or equivalent) so `supervisor_image` |
| 84 | +appears under `[openshell.gateway]`: |
| 85 | + |
| 86 | +```toml |
| 87 | +[openshell.gateway] |
| 88 | +supervisor_image = "localhost/openshell/supervisor:dev" |
| 89 | +``` |
| 90 | + |
| 91 | +Then start the gateway: |
| 92 | + |
| 93 | +```shell |
| 94 | +mise run gateway |
| 95 | +``` |
| 96 | + |
| 97 | +Alternatively, start the gateway binary directly with the correct config: |
| 98 | + |
| 99 | +```shell |
| 100 | +eval "$(aws configure export-credentials --format env)" |
| 101 | +./target/debug/openshell-gateway \ |
| 102 | + --config .cache/gateway-podman/gateway.toml \ |
| 103 | + --port 18080 --log-level info --drivers podman --disable-tls \ |
| 104 | + --db-url "sqlite:.cache/gateway-podman/gateway.db?mode=rwc" |
| 105 | +``` |
| 106 | + |
| 107 | +## 4. Configure the provider |
| 108 | + |
| 109 | +In a separate terminal: |
| 110 | + |
| 111 | +```shell |
| 112 | +export OPENSHELL_BASE_URL=http://localhost:18080 |
| 113 | + |
| 114 | +# Enable provider v2 (required for STS) |
| 115 | +openshell settings set --global --key providers_v2_enabled --value true --yes |
| 116 | + |
| 117 | +# Create the provider with the aws-s3 profile |
| 118 | +openshell provider create --name s3-test --type aws-s3 \ |
| 119 | + --credential AWS_ACCESS_KEY_ID=placeholder |
| 120 | + |
| 121 | +# Configure STS refresh |
| 122 | +openshell provider refresh configure s3-test \ |
| 123 | + --credential-key AWS_ACCESS_KEY_ID \ |
| 124 | + --strategy aws-sts-assume-role \ |
| 125 | + --material role_arn="arn:aws:iam::${ACCOUNT}:role/openshell-sts-test-role" \ |
| 126 | + --material session_name="openshell-sandbox" \ |
| 127 | + --material aws_region="us-east-1" |
| 128 | + |
| 129 | +# Mint the first set of credentials |
| 130 | +openshell provider refresh rotate s3-test \ |
| 131 | + --credential-key AWS_ACCESS_KEY_ID |
| 132 | + |
| 133 | +# Verify |
| 134 | +openshell provider refresh status s3-test |
| 135 | +``` |
| 136 | + |
| 137 | +The status should show `refreshed` with an expiry ~1 hour from now. |
| 138 | + |
| 139 | +## 5. Test S3 access from a sandbox |
| 140 | + |
| 141 | +```shell |
| 142 | +openshell sandbox create --name s3-smoke \ |
| 143 | + --provider s3-test \ |
| 144 | + -- bash -c ' |
| 145 | +BUCKET="<your-bucket-name>" |
| 146 | +REGION="us-east-1" |
| 147 | +CA=/etc/openshell-tls/ca-bundle.pem |
| 148 | +
|
| 149 | +echo "=== Upload ===" |
| 150 | +curl -s --cacert $CA -X PUT -H "Content-Type: text/plain" \ |
| 151 | + -d "hello from openshell sandbox via STS" \ |
| 152 | + "https://${BUCKET}.s3.${REGION}.amazonaws.com/from-sandbox.txt" \ |
| 153 | + -w "\nHTTP %{http_code}\n" |
| 154 | +
|
| 155 | +echo "" |
| 156 | +echo "=== List ===" |
| 157 | +curl -s --cacert $CA \ |
| 158 | + "https://${BUCKET}.s3.${REGION}.amazonaws.com/?list-type=2&max-keys=5" \ |
| 159 | + -w "\nHTTP %{http_code}\n" |
| 160 | +
|
| 161 | +echo "" |
| 162 | +echo "=== Download ===" |
| 163 | +curl -s --cacert $CA \ |
| 164 | + "https://${BUCKET}.s3.${REGION}.amazonaws.com/from-sandbox.txt" \ |
| 165 | + -w "\nHTTP %{http_code}\n" |
| 166 | +' |
| 167 | +``` |
| 168 | + |
| 169 | +All three operations should return `HTTP 200`. The download should print |
| 170 | +`hello from openshell sandbox via STS`. |
| 171 | + |
| 172 | +### What's happening |
| 173 | + |
| 174 | +1. The gateway called `sts:AssumeRole` and stored three short-lived credentials |
| 175 | + (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN`) in the |
| 176 | + provider record. |
| 177 | +2. The sandbox received placeholder values for these credentials as environment |
| 178 | + variables. |
| 179 | +3. Curl sent unsigned HTTP requests through the sandbox proxy's CONNECT tunnel. |
| 180 | +4. The proxy terminated TLS, stripped any existing AWS auth headers, resolved |
| 181 | + the real credentials from the `SecretResolver`, computed a fresh SigV4 |
| 182 | + signature using the `aws-sigv4` crate, and forwarded the signed request to |
| 183 | + S3. |
| 184 | +5. S3 validated the signature and accepted the request. |
| 185 | + |
| 186 | +The sandbox never saw real AWS credentials — only placeholders. |
| 187 | + |
| 188 | +### Using `--cacert` |
| 189 | + |
| 190 | +The proxy terminates TLS and presents a certificate signed by the OpenShell |
| 191 | +Sandbox CA. Curl needs `--cacert /etc/openshell-tls/ca-bundle.pem` to trust |
| 192 | +it. Python clients (boto3, requests) need `AWS_CA_BUNDLE` or |
| 193 | +`REQUESTS_CA_BUNDLE` set to the same path. |
| 194 | + |
| 195 | +### Known limitation: chunked transfer encoding |
| 196 | + |
| 197 | +S3 clients that use `Transfer-Encoding: chunked` (e.g., boto3 `put_object`) |
| 198 | +will fail because the SigV4 relay buffers the body by `Content-Length` only. |
| 199 | +Curl uses `Content-Length` by default and works. This will be addressed in a |
| 200 | +follow-up. |
| 201 | + |
| 202 | +## 6. Clean up |
| 203 | + |
| 204 | +```shell |
| 205 | +# Delete the sandbox |
| 206 | +openshell sandbox delete s3-smoke |
| 207 | + |
| 208 | +# Delete AWS resources |
| 209 | +aws s3 rb "s3://${BUCKET}" --force |
| 210 | +aws iam delete-role-policy --role-name openshell-sts-test-role --policy-name s3-access |
| 211 | +aws iam delete-role --role-name openshell-sts-test-role |
| 212 | +``` |
| 213 | + |
| 214 | +## Troubleshooting |
| 215 | + |
| 216 | +| Symptom | Cause | Fix | |
| 217 | +|---------|-------|-----| |
| 218 | +| `STS AssumeRole failed: dispatch failure` | Gateway doesn't have AWS credentials | Export credentials before starting: `eval "$(aws configure export-credentials --format env)"` | |
| 219 | +| `Policy discovery sync failed: invalid wire type` | Supervisor image doesn't have updated proto | Rebuild: `CONTAINER_ENGINE=podman IMAGE_TAG=dev mise run build:docker:supervisor` | |
| 220 | +| `CONNECT ... not permitted by policy` | Binary not in profile's `binaries` list | Use curl (in the list) or add your binary path to the policy | |
| 221 | +| `AuthorizationHeaderMalformed: region 's3' is wrong` | Old `extract_aws_region` bug | Rebuild from latest branch (fixed in `d46749fc`) | |
| 222 | +| `403 AccessDenied` from S3 | IAM role missing permissions, or STS creds expired | Check `openshell provider refresh status`; re-rotate if expired | |
| 223 | +| `SSLEOFError` from boto3 | boto3 uses chunked encoding; SigV4 relay doesn't handle it yet | Use curl instead of boto3 for now | |
| 224 | +| Supervisor uses wrong image | `supervisor_image` in wrong TOML section | Place under `[openshell.gateway]`, not `[openshell.gateway.gateway_jwt]` | |
0 commit comments