Skip to content

Commit 02bfe53

Browse files
ci: drop CUDA/Python-versioned image tags (#2174)
Per-arch images are now tagged `<version>-cu<major>[-ubi10]-<arch>` instead of `<version>-cuda<X.Y>-py<X.Y>-<arch>`. Published tags are unchanged: `<version>-cu*` (`<version>a-cu*` for nightly), plus `latest-cu*` on release builds only. Also updates the placeholder-release script, removes the dead helm tag sed in `update-version.sh`, and drops the CUDA+Python tag guidance from the server skill. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Authors: - Ramakrishna Prabhu (https://github.com/ramakrishnap-nv) - https://github.com/svc-nvskills-signing Approvers: - James Lamb (https://github.com/jameslamb) - Ishika Roy (https://github.com/Iroy30) URL: #2174
1 parent 738fd0d commit 02bfe53

10 files changed

Lines changed: 65 additions & 91 deletions

File tree

‎.github/workflows/build_images.yaml‎

Lines changed: 9 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -92,7 +92,7 @@ jobs:
9292
CUDA_VER: ${{ inputs.CUDA_VER }}
9393
PYTHON_VER: ${{ inputs.PYTHON_VER }}
9494
run: |
95-
echo "CUDA_SHORT=$(echo "$CUDA_VER" | sed -E 's/([0-9]+\.[0-9]+)\.[0-9]+/\1/')" >> $GITHUB_OUTPUT
95+
echo "CUDA_MAJOR=${CUDA_VER%%.*}" >> $GITHUB_OUTPUT
9696
echo "PYTHON_SHORT=$(echo "$PYTHON_VER" | sed -E 's/([0-9]+\.[0-9]+)\.[0-9]+/\1/')" >> $GITHUB_OUTPUT
9797
- name: Build image and push to DockerHub and NGC
9898
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6.19.2
@@ -110,17 +110,16 @@ jobs:
110110
PYTHON_SHORT_VER=${{ steps.trim.outputs.PYTHON_SHORT }}
111111
CUOPT_VER=${{ inputs.CUOPT_VER }}
112112
LINUX_VER=${{ inputs.LINUX_VER }}
113-
tags: nvidia/cuopt:${{ inputs.IMAGE_TAG_PREFIX }}-cuda${{ steps.trim.outputs.CUDA_SHORT }}-py${{ steps.trim.outputs.PYTHON_SHORT }}-${{ matrix.ARCH }}
113+
tags: nvidia/cuopt:${{ inputs.IMAGE_TAG_PREFIX }}-cu${{ steps.trim.outputs.CUDA_MAJOR }}-${{ matrix.ARCH }}
114114

115115
- name: Push image to NGC
116116
env:
117117
IMAGE_TAG_PREFIX: ${{ inputs.IMAGE_TAG_PREFIX }}
118118
ARCH: ${{ matrix.ARCH }}
119-
CUDA_SHORT: ${{ steps.trim.outputs.CUDA_SHORT }}
120-
PYTHON_SHORT: ${{ steps.trim.outputs.PYTHON_SHORT }}
119+
CUDA_MAJOR: ${{ steps.trim.outputs.CUDA_MAJOR }}
121120
run: |
122-
docker tag "nvidia/cuopt:${IMAGE_TAG_PREFIX}-cuda${CUDA_SHORT}-py${PYTHON_SHORT}-${ARCH}" "nvcr.io/nvstaging/nvaie/cuopt:${IMAGE_TAG_PREFIX}-cuda${CUDA_SHORT}-py${PYTHON_SHORT}-${ARCH}"
123-
docker push "nvcr.io/nvstaging/nvaie/cuopt:${IMAGE_TAG_PREFIX}-cuda${CUDA_SHORT}-py${PYTHON_SHORT}-${ARCH}"
121+
docker tag "nvidia/cuopt:${IMAGE_TAG_PREFIX}-cu${CUDA_MAJOR}-${ARCH}" "nvcr.io/nvstaging/nvaie/cuopt:${IMAGE_TAG_PREFIX}-cu${CUDA_MAJOR}-${ARCH}"
122+
docker push "nvcr.io/nvstaging/nvaie/cuopt:${IMAGE_TAG_PREFIX}-cu${CUDA_MAJOR}-${ARCH}"
124123
125124
- name: Build UBI10 image and push to DockerHub and NGC
126125
if: ${{ startsWith(inputs.CUDA_VER, '13.') }}
@@ -135,14 +134,14 @@ jobs:
135134
build-args: |
136135
CUDA_VER=${{ inputs.CUDA_VER }}
137136
CUOPT_VER=${{ inputs.CUOPT_VER }}
138-
tags: nvidia/cuopt:${{ inputs.IMAGE_TAG_PREFIX }}-cuda${{ steps.trim.outputs.CUDA_SHORT }}-ubi10-${{ matrix.ARCH }}
137+
tags: nvidia/cuopt:${{ inputs.IMAGE_TAG_PREFIX }}-cu${{ steps.trim.outputs.CUDA_MAJOR }}-ubi10-${{ matrix.ARCH }}
139138

140139
- name: Push UBI10 image to NGC
141140
if: ${{ startsWith(inputs.CUDA_VER, '13.') }}
142141
env:
143142
IMAGE_TAG_PREFIX: ${{ inputs.IMAGE_TAG_PREFIX }}
144143
ARCH: ${{ matrix.ARCH }}
145-
CUDA_SHORT: ${{ steps.trim.outputs.CUDA_SHORT }}
144+
CUDA_MAJOR: ${{ steps.trim.outputs.CUDA_MAJOR }}
146145
run: |
147-
docker tag "nvidia/cuopt:${IMAGE_TAG_PREFIX}-cuda${CUDA_SHORT}-ubi10-${ARCH}" "nvcr.io/nvstaging/nvaie/cuopt:${IMAGE_TAG_PREFIX}-cuda${CUDA_SHORT}-ubi10-${ARCH}"
148-
docker push "nvcr.io/nvstaging/nvaie/cuopt:${IMAGE_TAG_PREFIX}-cuda${CUDA_SHORT}-ubi10-${ARCH}"
146+
docker tag "nvidia/cuopt:${IMAGE_TAG_PREFIX}-cu${CUDA_MAJOR}-ubi10-${ARCH}" "nvcr.io/nvstaging/nvaie/cuopt:${IMAGE_TAG_PREFIX}-cu${CUDA_MAJOR}-ubi10-${ARCH}"
147+
docker push "nvcr.io/nvstaging/nvaie/cuopt:${IMAGE_TAG_PREFIX}-cu${CUDA_MAJOR}-ubi10-${ARCH}"

‎.github/workflows/build_test_publish_images.yaml‎

Lines changed: 1 addition & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -150,20 +150,11 @@ jobs:
150150
registry: "nvcr.io"
151151
username: "$oauthtoken"
152152
password: ${{ secrets.CUOPT_NGC_DOCKER_KEY }}
153-
- name: Trim CUDA and Python versions
154-
id: trim
155-
env:
156-
CUDA_VER: ${{ matrix.CUDA_VER }}
157-
PYTHON_VER: ${{ matrix.PYTHON_VER }}
158-
run: |
159-
echo "CUDA_SHORT=$(echo "$CUDA_VER" | sed -E 's/([0-9]+\.[0-9]+)\.[0-9]+/\1/')" >> $GITHUB_OUTPUT
160-
echo "PYTHON_SHORT=$(echo "$PYTHON_VER" | sed -E 's/([0-9]+\.[0-9]+)\.[0-9]+/\1/')" >> $GITHUB_OUTPUT
161153
- name: Create multiarch manifest
162154
shell: bash
163155
env:
164156
CUOPT_VER: ${{ needs.compute-matrix.outputs.CUOPT_VER }}
165-
CUDA_SHORT: ${{ steps.trim.outputs.CUDA_SHORT }}
166-
PYTHON_SHORT: ${{ steps.trim.outputs.PYTHON_SHORT }}
157+
CUDA_VER: ${{ matrix.CUDA_VER }}
167158
IMAGE_TAG_PREFIX: ${{ needs.compute-matrix.outputs.IMAGE_TAG_PREFIX }}
168159
BUILD_TYPE: ${{ inputs.build_type }}
169160
run: bash ci/docker/create_multiarch_manifest.sh

‎ci/docker/create_multiarch_manifest.sh‎

Lines changed: 8 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -57,34 +57,33 @@ create_manifest() {
5757
echo "✓ Successfully created and pushed manifest: $manifest_name"
5858
}
5959

60-
CUDA_MAJOR="${CUDA_SHORT%%.*}"
60+
CUDA_MAJOR="${CUDA_VER%%.*}"
6161

6262
IS_RELEASE=false
6363
if [[ "${IMAGE_TAG_PREFIX}" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then IS_RELEASE=true; fi
6464

65-
# Short cu<major> manifests, built from the per-arch cuda<X.Y>-... images.
6665
create_short_manifests() {
67-
local short=$1 source=$2 repo
66+
local short=$1 repo
6867
for repo in nvidia/cuopt nvcr.io/nvstaging/nvaie/cuopt; do
6968
echo "=== Creating ${short} manifests in ${repo} ==="
7069
create_manifest \
7170
"${repo}:${IMAGE_TAG_PREFIX}-${short}" \
72-
"${repo}:${IMAGE_TAG_PREFIX}-${source}-amd64" \
73-
"${repo}:${IMAGE_TAG_PREFIX}-${source}-arm64"
71+
"${repo}:${IMAGE_TAG_PREFIX}-${short}-amd64" \
72+
"${repo}:${IMAGE_TAG_PREFIX}-${short}-arm64"
7473
if [[ "${IS_RELEASE}" == "true" ]]; then
7574
create_manifest \
7675
"${repo}:latest-${short}" \
77-
"${repo}:${IMAGE_TAG_PREFIX}-${source}-amd64" \
78-
"${repo}:${IMAGE_TAG_PREFIX}-${source}-arm64"
76+
"${repo}:${IMAGE_TAG_PREFIX}-${short}-amd64" \
77+
"${repo}:${IMAGE_TAG_PREFIX}-${short}-arm64"
7978
fi
8079
done
8180
}
8281

83-
create_short_manifests "cu${CUDA_MAJOR}" "cuda${CUDA_SHORT}-py${PYTHON_SHORT}"
82+
create_short_manifests "cu${CUDA_MAJOR}"
8483

8584
# UBI10 CUDA base images are only published for CUDA 13.x and later
8685
if [[ "${CUDA_MAJOR}" == "13" ]]; then
87-
create_short_manifests "cu${CUDA_MAJOR}-ubi10" "cuda${CUDA_SHORT}-ubi10"
86+
create_short_manifests "cu${CUDA_MAJOR}-ubi10"
8887
else
8988
echo "Skipping UBI10 manifests (CUDA_MAJOR='${CUDA_MAJOR}' — UBI10 base images require CUDA 13+)"
9089
fi

‎ci/release/publish_placeholder_release_images.sh‎

Lines changed: 5 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,6 @@
3232
#
3333
# Env overrides:
3434
# CUDA_VERS space-separated list of full CUDA versions (default: "12.9.0 13.3.0")
35-
# PYTHON_VERS space-separated list of full Python versions (default: "3.14.4")
3635
# RELEASE_TAG override the derived release tag (default: derived from VERSION)
3736
# NIGHTLY_TAG override the derived source nightly tag (default: "${RELEASE_TAG}a")
3837
# PUSH_LATEST set to "true" to also push "latest" manifests (default: "false")
@@ -54,7 +53,6 @@ BASE_VER=$(sed -E 's/\.0+([0-9])/\.\1/g' VERSION | tr -d '[:space:]')
5453
RELEASE_TAG="${RELEASE_TAG:-${BASE_VER}}"
5554
NIGHTLY_TAG="${NIGHTLY_TAG:-${RELEASE_TAG}a}"
5655
CUDA_VERS="${CUDA_VERS:-12.9.0 13.3.0}"
57-
PYTHON_VERS="${PYTHON_VERS:-3.14.4}"
5856
DRY_RUN="${DRY_RUN:-false}"
5957
PUSH_LATEST="${PUSH_LATEST:-false}"
6058

@@ -63,7 +61,6 @@ echo "NGC repo: ${NGC_REPO}"
6361
echo "Source tag: ${NIGHTLY_TAG} (nightly)"
6462
echo "Target tag: ${RELEASE_TAG} (placeholder release)"
6563
echo "CUDA_VERS: ${CUDA_VERS}"
66-
echo "PYTHON_VERS: ${PYTHON_VERS}"
6764
echo "PUSH_LATEST: ${PUSH_LATEST}"
6865
echo "DRY_RUN: ${DRY_RUN}"
6966
if [[ "${PUSH_LATEST}" == "true" ]]; then
@@ -130,7 +127,7 @@ create_release_manifest() {
130127
# create_multiarch_manifest.sh builds the list from them).
131128
publish_release_variant() {
132129
local manifest_tag=$1 # e.g. "${RELEASE_TAG}-cu12"
133-
local nightly_tag=$2 # e.g. "${NIGHTLY_TAG}-cuda12.9-py3.14"
130+
local nightly_tag=$2 # e.g. "${NIGHTLY_TAG}-cu12"
134131

135132
local release_amd64="${NGC_REPO}:${manifest_tag}-amd64"
136133
local release_arm64="${NGC_REPO}:${manifest_tag}-arm64"
@@ -141,12 +138,10 @@ publish_release_variant() {
141138
create_release_manifest "${NGC_REPO}:${manifest_tag}" "$release_amd64" "$release_arm64"
142139
}
143140

144-
# Release/latest use only the short cu<major> tags; the fully-qualified names are nightly-only.
145141
publish_variant_and_latest() {
146142
local alias=$1 # e.g. "cu13" or "cu13-ubi10"
147-
local nightly_tag=$2 # e.g. "${NIGHTLY_TAG}-cuda13.3-py3.14"
148143

149-
publish_release_variant "${RELEASE_TAG}-${alias}" "$nightly_tag"
144+
publish_release_variant "${RELEASE_TAG}-${alias}" "${NIGHTLY_TAG}-${alias}"
150145

151146
if [[ "${PUSH_LATEST}" == "true" ]]; then
152147
create_release_manifest \
@@ -157,22 +152,13 @@ publish_variant_and_latest() {
157152
}
158153

159154
for CUDA_VER in ${CUDA_VERS}; do
160-
CUDA_SHORT=$(echo "$CUDA_VER" | sed -E 's/([0-9]+\.[0-9]+)\.[0-9]+/\1/')
161-
CUDA_MAJOR="${CUDA_SHORT%%.*}"
155+
CUDA_MAJOR="${CUDA_VER%%.*}"
162156

163-
for PYTHON_VER in ${PYTHON_VERS}; do
164-
PYTHON_SHORT=$(echo "$PYTHON_VER" | sed -E 's/([0-9]+\.[0-9]+)\.[0-9]+/\1/')
165-
166-
publish_variant_and_latest \
167-
"cu${CUDA_MAJOR}" \
168-
"${NIGHTLY_TAG}-cuda${CUDA_SHORT}-py${PYTHON_SHORT}"
169-
done
157+
publish_variant_and_latest "cu${CUDA_MAJOR}"
170158

171159
# UBI10 base images are only published for CUDA 13.x and later
172160
if [[ "${CUDA_MAJOR}" == "13" ]]; then
173-
publish_variant_and_latest \
174-
"cu${CUDA_MAJOR}-ubi10" \
175-
"${NIGHTLY_TAG}-cuda${CUDA_SHORT}-ubi10"
161+
publish_variant_and_latest "cu${CUDA_MAJOR}-ubi10"
176162
fi
177163
done
178164

‎ci/release/update-version.sh‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -152,7 +152,6 @@ sed_runner 's/badge\/version-[0-9]\+\.[0-9]\+\.[0-9]\+-blue/badge\/version-'${NE
152152
sed_runner 's/'"cuopt_version: \"[0-9][0-9].[0-9][0-9]\""'/'"cuopt_version: \"${NEXT_SHORT_TAG}\""'/g' .github/workflows/nightly.yaml
153153

154154
# Update Helm chart files
155-
sed_runner 's/\(tag: "\)[0-9][0-9]\.[0-9]\+\.[0-9]\+\(-cuda12\.9-py3\.12"\)/\1'${DOCKER_TAG}'\2/g' helmchart/cuopt-server/values.yaml
156155
sed_runner 's/\(appVersion: \)[0-9][0-9]\.[0-9]\+\.[0-9]\+/\1'${DOCKER_TAG}'/g' helmchart/cuopt-server/Chart.yaml
157156
sed_runner 's/\(version: \)[0-9][0-9]\.[0-9]\+\.[0-9]\+/\1'${DOCKER_TAG}'/g' helmchart/cuopt-server/Chart.yaml
158157

‎skills/cuopt-server-api-python/BENCHMARK.md‎

Lines changed: 30 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -9,12 +9,12 @@ Recommended for publication based on the completed evaluation evidence in this r
99
## Evaluation Metadata
1010

1111
- Skill: `cuopt-server-api-python`
12-
- Evaluation date: 2026-09-15
12+
- Evaluation date: 2026-10-16
1313
- Evaluator version: `1.5.6`
1414
- Agents: Claude Code (`aws/anthropic/bedrock-claude-opus-4-8`), Codex (`openai/openai/gpt-5.5`)
1515
- Tasks: 8 evaluation tasks (8 positive)
16-
- Dataset digest: `sha256:89ad304517df359a8c33a1f67da3ad9df8aeae5ec234d2adbed1593bf49c4ebc` (skill-evaluator-dataset-snapshot/1)
17-
- Attempts per task: 3
16+
- Dataset digest: `sha256:c54a53a22db535b2e2101bfcd2c43a8cdfef342f5c166ba343b8fdba517c741d` (skill-evaluator-dataset-snapshot/1)
17+
- Attempts per task: 1
1818
- Environment: `k8s-sandbox`
1919
- Tier 2 evidence: required for publication
2020
- Tier 3 evidence: required for publication
@@ -35,12 +35,12 @@ The three-tier evaluation checks whether the skill:
3535

3636
| Measure | Claude Code (Baseline → Skill Uplift) | Codex (Baseline → Skill Uplift) |
3737
|---|---:|---:|
38-
| Overall | 98.3% — baseline ran, but no comparable score was available; uplift unavailable | 94.4% — baseline ran, but no comparable score was available; uplift unavailable |
39-
| Security | 100.0% → 100.0% (±0.0 points) | 90.9% → 100.0% (+9.1 points) |
40-
| Correctness | 68.0% → 100.0% (+32.0 points) | 67.3% → 100.0% (+32.7 points) |
41-
| Discoverability | 100.0% — baseline ran, but no comparable score was available; uplift unavailable | 94.4% — baseline ran, but no comparable score was available; uplift unavailable |
42-
| Effectiveness | 47.8% → 95.2% (+47.4 points) | 48.5% → 91.2% (+42.7 points) |
43-
| Efficiency | 96.1% — baseline ran, but no comparable score was available; uplift unavailable | 86.7% — baseline ran, but no comparable score was available; uplift unavailable |
38+
| Overall | 98.3% — baseline ran, but no comparable score was available; uplift unavailable | 95.9% — baseline ran, but no comparable score was available; uplift unavailable |
39+
| Security | 100.0% → 100.0% (±0.0 points) | 100.0% → 100.0% (±0.0 points) |
40+
| Correctness | 77.5% → 100.0% (+22.5 points) | 82.5% → 100.0% (+17.5 points) |
41+
| Discoverability | 100.0% — baseline ran, but no comparable score was available; uplift unavailable | 93.8% — baseline ran, but no comparable score was available; uplift unavailable |
42+
| Effectiveness | 58.1% → 94.3% (+36.2 points) | 64.1% → 90.5% (+26.4 points) |
43+
| Efficiency | 97.1% — baseline ran, but no comparable score was available; uplift unavailable | 95.1% — baseline ran, but no comparable score was available; uplift unavailable |
4444

4545
**How to read this table:** baseline is the same task attempted without the target skill. Scores are rounded to one decimal; threshold-adjacent values use additional precision so their displayed band matches the verdict. Uplift is derived from those displayed scores and shown in percentage points.
4646

@@ -52,25 +52,25 @@ Actual Tier 3 execution usage is reported for every observed agent/case pair and
5252

5353
| Agent | Dataset case | With skill | Without skill | Delta | Change | Coverage |
5454
|---|---|---:|---:|---:|---:|---|
55-
| claude-code | All cases | 990,844 | 2,338,962 | N/A | N/A | skill 8/8; base 10/10 |
56-
| claude-code | srv-py-eval-001-rest-routing-workflow | 61,737 | 31,761 | +29,976 | +94.38% | skill 1/1; base 1/1 |
57-
| claude-code | srv-py-eval-002-rest-lp-payload-fields | 159,833 | 224,737 | -64,904 | -28.88% | skill 1/1; base 1/1 |
58-
| claude-code | srv-py-eval-003-rest-milp-vs-lp | 258,072 | 30,448 | +227,624 | +747.58% | skill 1/1; base 1/1 |
59-
| claude-code | srv-py-eval-004-qp-not-supported-over-rest | 61,254 | 366,123 | -304,869 | -83.27% | skill 1/1; base 1/1 |
60-
| claude-code | srv-py-eval-005-debug-422-field-names | 61,453 | 30,853 | +30,600 | +99.18% | skill 1/1; base 1/1 |
61-
| claude-code | srv-py-eval-006-docker-deployment | 61,610 | 155,592 | -93,982 | -60.40% | skill 1/1; base 1/1 |
62-
| claude-code | srv-py-eval-007-client-hardening | 197,723 | 30,615 | +167,108 | +545.84% | skill 1/1; base 1/1 |
63-
| claude-code | srv-py-eval-008-runnable-assets | 129,162 | 1,468,833 | N/A | N/A | skill 1/1; base 3/3 |
64-
| codex | All cases | 378,610 | 1,697,844 | N/A | N/A | skill 8/8; base 11/11 |
65-
| codex | srv-py-eval-001-rest-routing-workflow | 45,986 | 193,794 | -147,808 | -76.27% | skill 1/1; base 1/1 |
66-
| codex | srv-py-eval-002-rest-lp-payload-fields | 45,220 | 47,739 | -2,519 | -5.28% | skill 1/1; base 1/1 |
67-
| codex | srv-py-eval-003-rest-milp-vs-lp | 46,076 | 42,547 | +3,529 | +8.29% | skill 1/1; base 1/1 |
68-
| codex | srv-py-eval-004-qp-not-supported-over-rest | 78,889 | 576,772 | N/A | N/A | skill 1/1; base 2/2 |
69-
| codex | srv-py-eval-005-debug-422-field-names | 28,843 | 118,005 | -89,162 | -75.56% | skill 1/1; base 1/1 |
70-
| codex | srv-py-eval-006-docker-deployment | 41,121 | 25,518 | +15,603 | +61.15% | skill 1/1; base 1/1 |
71-
| codex | srv-py-eval-007-client-hardening | 29,126 | 25,289 | +3,837 | +15.17% | skill 1/1; base 1/1 |
72-
| codex | srv-py-eval-008-runnable-assets | 63,349 | 668,180 | N/A | N/A | skill 1/1; base 3/3 |
73-
| ALL AGENTS | Dataset aggregate | 1,369,454 | 4,036,806 | N/A | N/A | skill 16/16; base 21/21 |
55+
| claude-code | All cases | 885,077 | 597,044 | +288,033 | +48.24% | skill 8/8; base 8/8 |
56+
| claude-code | srv-py-eval-001-rest-routing-workflow | 61,314 | 29,004 | +32,310 | +111.40% | skill 1/1; base 1/1 |
57+
| claude-code | srv-py-eval-002-rest-lp-payload-fields | 158,831 | 29,008 | +129,823 | +447.54% | skill 1/1; base 1/1 |
58+
| claude-code | srv-py-eval-003-rest-milp-vs-lp | 159,973 | 28,976 | +130,997 | +452.09% | skill 1/1; base 1/1 |
59+
| claude-code | srv-py-eval-004-qp-not-supported-over-rest | 60,627 | 59,196 | +1,431 | +2.42% | skill 1/1; base 1/1 |
60+
| claude-code | srv-py-eval-005-debug-422-field-names | 60,837 | 29,007 | +31,830 | +109.73% | skill 1/1; base 1/1 |
61+
| claude-code | srv-py-eval-006-docker-deployment | 60,693 | 28,988 | +31,705 | +109.37% | skill 1/1; base 1/1 |
62+
| claude-code | srv-py-eval-007-client-hardening | 162,999 | 28,980 | +134,019 | +462.45% | skill 1/1; base 1/1 |
63+
| claude-code | srv-py-eval-008-runnable-assets | 159,803 | 363,885 | -204,082 | -56.08% | skill 1/1; base 1/1 |
64+
| codex | All cases | 388,941 | 481,460 | -92,519 | -19.22% | skill 8/8; base 8/8 |
65+
| codex | srv-py-eval-001-rest-routing-workflow | 45,413 | 92,911 | -47,498 | -51.12% | skill 1/1; base 1/1 |
66+
| codex | srv-py-eval-002-rest-lp-payload-fields | 45,088 | 46,097 | -1,009 | -2.19% | skill 1/1; base 1/1 |
67+
| codex | srv-py-eval-003-rest-milp-vs-lp | 45,998 | 30,983 | +15,015 | +48.46% | skill 1/1; base 1/1 |
68+
| codex | srv-py-eval-004-qp-not-supported-over-rest | 28,544 | 46,936 | -18,392 | -39.19% | skill 1/1; base 1/1 |
69+
| codex | srv-py-eval-005-debug-422-field-names | 28,579 | 61,511 | -32,932 | -53.54% | skill 1/1; base 1/1 |
70+
| codex | srv-py-eval-006-docker-deployment | 35,800 | 32,176 | +3,624 | +11.26% | skill 1/1; base 1/1 |
71+
| codex | srv-py-eval-007-client-hardening | 79,361 | 27,750 | +51,611 | +185.99% | skill 1/1; base 1/1 |
72+
| codex | srv-py-eval-008-runnable-assets | 80,158 | 143,096 | -62,938 | -43.98% | skill 1/1; base 1/1 |
73+
| ALL AGENTS | Dataset aggregate | 1,274,018 | 1,078,504 | +195,514 | +18.13% | skill 16/16; base 16/16 |
7474

7575
Prompt tokens include cached reads, so total tokens are `prompt + completion` (cached is not added twice). The Efficiency score uses `(prompt - cached) + completion`. N/A means the relevant trajectory counters were not available; coverage is never estimated.
7676

@@ -88,8 +88,8 @@ Prompt tokens include cached reads, so total tokens are `prompt + completion` (c
8888
<summary>Show detailed findings and successful checks</summary>
8989

9090
- **MEDIUM** SCHEMA/frontmatter_field_placement: Root field 'version' is ignored; use 'metadata.version' (`skills/cuopt-server-api-python/SKILL.md`)
91-
- **MEDIUM** SECURITY/Unknown (LP3): MCP Least Privilege: Without declared permissions the skill's intent is opaque and cannot be validated. (`SKILL.md:1`)
92-
- **MEDIUM** SECURITY/Unknown (RP1): MCP Rug Pull: The Docker run command references `nvidia/cuopt:latest-cu13` (a mutable floating tag) without pinning to an immutable di (`SKILL.md:55`)
91+
- **MEDIUM** SECURITY/Unknown (LP3): MCP Least Privilege: The skill demonstrates network-capable code (HTTP requests via `requests`, Docker networking) but declares no tool scope (`SKILL.md:1`)
92+
- **MEDIUM** SECURITY/Unknown (RP1): MCP Rug Pull: The Docker run command uses `nvidia/cuopt:latest-cu13` (a floating/mutable tag). The skill itself acknowledges this risk (`SKILL.md:55`)
9393
- **MEDIUM** SECURITY/External Transmission (E1): Data Exfiltration: requests.post(f"{SERVER}/cuopt/request", json= (`SKILL.md:88`)
9494
- **MEDIUM** SECURITY/Tainted flow: 'req_id' from requests.post (line 81, network input) → requests.get (network output) (TT2): Data Flow: response = requests.get(
9595
f"{server}/cuopt/solution/{req_id}",

‎skills/cuopt-server-api-python/SKILL.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -56,9 +56,9 @@ docker run --gpus all -d -p 8000:8000 -e CUOPT_SERVER_PORT=8000 \
5656
nvidia/cuopt:latest-cu13
5757
```
5858

59-
Use `latest-cu12` or `latest-cu13` to match your driver's CUDA major version (`latest-cu13-ubi10` for a UBI10 base). Prefer these over the CUDA+Python-specific tags such as `latest-cuda12.9-py3.13` — those track a single Python line and go stale when it stops receiving builds.
59+
Use `latest-cu12` or `latest-cu13` to match your driver's CUDA major version (`latest-cu13-ubi10` for a UBI10 base).
6060

61-
For production, pin rather than float: `latest-*` tags are mutable and can silently move to a different image. Use a full release tag (`nvidia/cuopt:<release>-cuda<cuda>-py<python>`) or an immutable digest (`nvidia/cuopt@sha256:<digest>`). Check the `nvidia/cuopt` registry for available tags.
61+
For production, pin rather than float: `latest-*` tags are mutable and can silently move to a different image. Use a full release tag (`nvidia/cuopt:<release>-cu<cuda-major>`) or an immutable digest (`nvidia/cuopt@sha256:<digest>`). Check the `nvidia/cuopt` registry for available tags.
6262

6363
## Verify
6464

0 commit comments

Comments
 (0)