Skip to content

Commit 62f7243

Browse files
authored
test(e2e): migrate test-credential-sanitization.sh to vitest (#5336)
## Summary Migrate `test/e2e/test-credential-sanitization.sh` with the simplest equivalent Vitest coverage. ## Related Issues Refs #5098 Refs #156 Refs #1438 ## Contract mapping - Legacy assertion: migration bundle config strips credential fields, removes `auth-profiles.json`, preserves non-secret workspace/config data, and keeps symlink traversal out of sensitive-file removal. - Replacement: `test/e2e-scenario/live/credential-sanitization.test.ts` local product-code assertions using `sanitizeConfigFile`, `stripCredentials`, `isSensitiveFile`, and `isCredentialField`. - Boundary preserved: real filesystem bundle mutation plus product sanitization code. - Legacy assertion: blueprint digest is not empty and tracks the pinned sandbox image digest. - Replacement: `credential-sanitization.test.ts` parses `nemoclaw-blueprint/blueprint.yaml` and asserts `sha256:<64hex>` plus image-digest equality. - Boundary preserved: shipped blueprint YAML. - Legacy assertion: after install/onboard, sandbox agent state does not expose `auth-profiles.json` or secret-shaped `nvapi-`/`ghp_`/`npm_` values. - Replacement: `credential-sanitization.test.ts` runs `bash install.sh --non-interactive --yes-i-accept-third-party-software`, then probes the real OpenShell sandbox filesystem from Vitest. - Boundary preserved: `install.sh`, Docker/OpenShell, real sandbox exec, `NVIDIA_API_KEY` secret redaction. ## Simplicity check - Test shape: simple live Vitest test with local helper functions. - Original runner/lane: `nightly-e2e.yaml` job `credential-sanitization-e2e`, `runs-on: ubuntu-latest`, Docker/OpenShell, `NVIDIA_API_KEY`, timeout 60 minutes. - Replacement runner: same `ubuntu-latest` class in `.github/workflows/e2e-vitest-scenarios.yaml` job `credential-sanitization-vitest`, with Docker/OpenShell and `NVIDIA_API_KEY`. - New shared helpers: none. - New framework/registry/ledger: **none**. - Workflow changes: add selective free-standing Vitest job `credential-sanitization-vitest`; legacy shell deletion and shell workflow retirement deferred to #5098 Phase 11. - Selective dispatch: `gh workflow run e2e-vitest-scenarios.yaml --repo NVIDIA/NemoClaw --ref e2e-migrate/test-credential-sanitization -f jobs=credential-sanitization-vitest -f pr_number=<PR>`. ## Verification - `npm ci --ignore-scripts` - `npm run build:cli` - `npx vitest run --project e2e-vitest-support test/e2e-scenario/support-tests/e2e-scenarios-workflow.test.ts --silent=false --reporter=default` - `NEMOCLAW_RUN_E2E_SCENARIOS=1 npx vitest run --project e2e-scenarios-live test/e2e-scenario/live/credential-sanitization.test.ts --silent=false --reporter=default` (local no-secret run: product-code assertions pass, live sandbox section skips on missing `NVIDIA_API_KEY`) - `npx vitest run test/validate-blueprint.test.ts src/lib/security/credential-filter.test.ts --silent=false --reporter=default` - `npm run typecheck:cli` - `git diff --check` - PR: #5336 - Same-runner selective run: https://github.com/NVIDIA/NemoClaw/actions/runs/27432354309 — `credential-sanitization-vitest` passed on `ubuntu-latest` with Docker/OpenShell and `NVIDIA_API_KEY` after latest main merge/dependency fix (`f2cd7cc46`). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Added a live end-to-end test that validates credential sanitization during install/onboarding, checks sandbox filesystem boundaries, ensures sensitive fields are redacted, and emits a scenario artifact summarizing assertions. * **Chores** * CI updated to add a dedicated credential-sanitization job to validation, matrix generation and PR reporting; job runs the new live scenario, handles Docker auth with fallback/retry and ensures cleanup. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
1 parent 7a8fcae commit 62f7243

4 files changed

Lines changed: 476 additions & 3 deletions

File tree

‎.github/workflows/e2e-vitest-scenarios.yaml‎

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -693,6 +693,90 @@ jobs:
693693
retention-days: 14
694694

695695

696+
credential-sanitization-vitest:
697+
needs: generate-matrix
698+
if: ${{ (inputs.jobs == '' && inputs.scenarios == '') || contains(format(',{0},', inputs.jobs), ',credential-sanitization-vitest,') || contains(format(',{0},', inputs.scenarios), ',credential-sanitization,') }}
699+
runs-on: ubuntu-latest
700+
timeout-minutes: 65
701+
env:
702+
E2E_ARTIFACT_DIR: ${{ github.workspace }}/e2e-artifacts/vitest/credential-sanitization
703+
NEMOCLAW_CLI_BIN: ${{ github.workspace }}/bin/nemoclaw.js
704+
NEMOCLAW_RUN_E2E_SCENARIOS: "1"
705+
NEMOCLAW_SANDBOX_NAME: "e2e-credential-sanitization"
706+
OPENSHELL_GATEWAY: "nemoclaw"
707+
steps:
708+
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
709+
with:
710+
persist-credentials: false
711+
712+
- name: Authenticate to Docker Hub
713+
env:
714+
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
715+
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
716+
shell: bash
717+
run: |
718+
set -euo pipefail
719+
if [[ -z "${DOCKERHUB_USERNAME}" || -z "${DOCKERHUB_TOKEN}" ]]; then
720+
echo "::notice::Docker Hub credentials not configured; continuing with anonymous pulls."
721+
exit 0
722+
fi
723+
login_succeeded=0
724+
for attempt in 1 2 3; do
725+
if echo "${DOCKERHUB_TOKEN}" | timeout 30s docker login docker.io --username "${DOCKERHUB_USERNAME}" --password-stdin; then
726+
login_succeeded=1
727+
break
728+
fi
729+
if [[ "$attempt" -lt 3 ]]; then
730+
echo "::warning::Docker Hub login attempt ${attempt} failed; retrying."
731+
sleep 5
732+
fi
733+
done
734+
if [[ "$login_succeeded" -ne 1 ]]; then
735+
echo "::warning::Docker Hub login failed after 3 attempts; continuing with anonymous pulls."
736+
fi
737+
738+
- name: Set up Node
739+
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.0.0
740+
with:
741+
node-version: 22
742+
cache: npm
743+
744+
- name: Install root dependencies
745+
run: npm ci --ignore-scripts
746+
747+
- name: Build CLI
748+
run: npm run build:cli
749+
750+
- name: Run credential sanitization live test
751+
# Migrated from test/e2e/test-credential-sanitization.sh. Preserves the
752+
# same ubuntu-latest + Docker/OpenShell + NVIDIA_API_KEY lane by running
753+
# install.sh, onboarding a real sandbox, and probing sandbox state from
754+
# Vitest while fixture redaction owns evidence logs.
755+
env:
756+
NVIDIA_API_KEY: ${{ secrets.NVIDIA_API_KEY }}
757+
run: |
758+
set -euo pipefail
759+
npx vitest run --project e2e-scenarios-live \
760+
test/e2e-scenario/live/credential-sanitization.test.ts \
761+
--silent=false --reporter=default
762+
763+
- name: Upload credential sanitization artifacts
764+
if: always()
765+
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
766+
with:
767+
name: e2e-vitest-scenarios-credential-sanitization
768+
path: e2e-artifacts/vitest/credential-sanitization/
769+
include-hidden-files: false
770+
if-no-files-found: ignore
771+
retention-days: 14
772+
773+
- name: Clean up Docker auth
774+
if: always()
775+
shell: bash
776+
run: |
777+
docker logout docker.io >/dev/null 2>&1 || true
778+
779+
696780
credential-migration-vitest:
697781
needs: generate-matrix
698782
if: ${{ (inputs.jobs == '' && inputs.scenarios == '') || contains(format(',{0},', inputs.jobs), ',credential-migration-vitest,') }}
@@ -1792,6 +1876,7 @@ jobs:
17921876
onboard-negative-paths-vitest,
17931877
skill-agent-vitest,
17941878
inference-routing-vitest,
1879+
credential-sanitization-vitest,
17951880
credential-migration-vitest,
17961881
runtime-overrides-vitest,
17971882
hermes-e2e-vitest,

0 commit comments

Comments
 (0)