Skip to content

load.sh exits 0 reporting "0 failures" when 100% of requests fail AND demos/parking WorkerPool omits resources.limits (failing to trigger parking with 4 actors). #3287

Closed

Description

Summary & Root Cause

  1. Worker capacity comes from the WorkerPool's container limits. With no limits, each parking worker advertises the whole node's allocatable capacity (3920m CPU on the test nodes). A 1000m actor then fits three times per worker, so 2 workers hold 6 actors, and the demo's 4 actors never contend.
  2. load.sh targets a plaintext router port that no longer exists and sends no client cert, so every curl records 000. The verdict counts only 503 as failure, so 0 successes with 0 503s prints => 0 failures under saturation and exits 0. The total == 0 guard does not fire because 000 lines are still recorded.

Affected Code Locations

  • demos/parking/parking.yaml.tmpl:42-48 (WorkerPool/parking: replicas: 2 with the comment "Intentionally small: 2 workers, so 3+ concurrently-active actors saturate the pool", and no template.resources; the file ends at line 48)
  • demos/counter/counter.yaml.tmpl:46-63 (comparison: template.resources.limits: {cpu: "1", memory: 1Gi}; the comment notes that capacity comes from the limits)
  • demos/parking/parking-template.yaml.tmpl:29-34 (per-actor cpuMillicores: 1000, memoryBytes: 536870912; the comment says "The pool sets no worker limits, so the worker offers the node's allocatable")
  • demos/parking/load.sh:40, 45, 61 (ROUTER="http://localhost:8000", prerequisite port-forward ... svc/atenet-router 8000:80)
  • demos/parking/load.sh:108-110 (plain curl with no TLS or client cert)
  • demos/parking/load.sh:133-146 (busy counts only 503; if [[ "${busy}" -eq 0 ]] prints the success line; there is no non-zero exit for ok == 0 or other > 0)

Detailed Reproduction Guide

  1. Deploy and inspect the pool:

    go run ./cmd/ate-setup deploy demo parking
    kubectl get workerpool -n ate-demo-parking parking -o jsonpath='{.spec.replicas} resources={.spec.template.resources}'
  2. Run the load script as documented:

    kubectl port-forward -n ate-system svc/atenet-router 8000:80 &   # fails: no port 80
    ./demos/parking/load.sh -d 5; echo "exit=$?"
  3. Wake p1..p4 concurrently over mTLS (from a client pod as in S-2) and check placement:

    for a in p1 p2 p3 p4; do
      kubectl exec -n test-client test-client -- curl -sS -o /dev/null -w "$a: %{http_code} %{time_total}s\n" \
        --cacert /run/servicedns-ca/trust-bundle.pem --cert /run/podidentity/credential-bundle.pem --key /run/podidentity/credential-bundle.pem \
        -H "ate-target-actor: ate-demo-parking/$a" https://atenet-router.ate-system.svc/ &
    done; wait
    kubectl ate get workers -n ate-demo-parking

Expected vs. Actual Behavior

  • Expected: With 2 workers sized for one 1000m actor each, waking 4 actors runs 2 and parks 2. load.sh exits non-zero when requests fail to connect or none succeed.
  • Actual: All 4 actors run at once (2 per worker) with no parking. load.sh reports 0 OK, 0 503, all "other", prints the success line, and exits 0.

Observed Output & Logs

  $ kubectl get workerpool -n ate-demo-parking parking -o jsonpath='{.spec.replicas} resources={.spec.resources}'
  2 resources=
  $ ./demos/parking/load.sh -d 5       # exit 0
      total requests : 28
      200 OK         : 0
      503 unavailable: 0
      other          : 28
      => 0 failures under saturation: parking absorbed the contention.
  p1: 200 0.438695s | p3: 200 0.501199s | p4: 200 0.497656s | p2: 200 0.542469s
  parking WORKER_STATE_ACTIVE 2/1000 2000/3920 ... parking-65bf544cf-6wqrn
  parking WORKER_STATE_ACTIVE 2/1000 2000/3920 ... parking-65bf544cf-bfr9g
  parking   3/1000   3000/3920   (worker A)
  parking   1/1000   1000/3920   (worker B)

Activity

  1. changed the title [-]demos/parking WorkerPool omits resources.limits (failing to trigger parking with 4 actors) and load.sh exits 0 reporting "0 failures" when 100% of requests fail[/-] [+]load.sh exits 0 reporting "0 failures" when 100% of requests fail AND demos/parking WorkerPool omits resources.limits (failing to trigger parking with 4 actors).[/+] on Oct 27, 2026
  2. added a commit that references this issue on Nov 3, 2026
    a9fe809
  3. added a commit that references this issue on Nov 3, 2026
    ad39f20
  4. added a commit that references this issue on Nov 3, 2026
    cfa3e3f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions