Repository navigation
Commit 0ddbf78
authored
benchmarking: add --test-name to run selected orchestrator tests (agent-substrate#3464)
The orchestrator ran every entry in the `--tests` file, so running one
test meant copying it into a new file first. `--test-name NAME` runs
only the named entries and can be repeated.
The orchestrator now selects and validates the entries before it waits
for Docker and clones the repo, so a mistyped name fails right away. A
relative `--tests` path now resolves against the working directory
instead of the clone.
<!-- e.g. "AI-assisted, I reviewed and tested it." No AI co-author
trailers. -->
AI use: AI-assisted, I reviewed and tested it
<!-- Breaking change? Describe what users must do on upgrade and ask for
the `breaking-change` label. -->
- [x] Tests pass
- [x] Docs updated
Signed-off-by: Haiyan Meng <[email protected]>1 parent a1ee475 commit 0ddbf78
4 files changed
Lines changed: 52 additions & 25 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
40 | 40 | | |
41 | 41 | | |
42 | 42 | | |
43 | | - | |
| 43 | + | |
44 | 44 | | |
45 | 45 | | |
46 | 46 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
66 | 66 | | |
67 | 67 | | |
68 | 68 | | |
69 | | - | |
70 | | - | |
71 | | - | |
72 | | - | |
73 | | - | |
74 | | - | |
75 | | - | |
76 | | - | |
77 | | - | |
78 | | - | |
79 | | - | |
| 69 | + | |
| 70 | + | |
80 | 71 | | |
81 | 72 | | |
82 | 73 | | |
| |||
127 | 118 | | |
128 | 119 | | |
129 | 120 | | |
130 | | - | |
| 121 | + | |
| 122 | + | |
131 | 123 | | |
132 | 124 | | |
133 | 125 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
21 | | - | |
22 | | - | |
23 | | - | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
| |||
91 | 91 | | |
92 | 92 | | |
93 | 93 | | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
94 | 101 | | |
95 | 102 | | |
96 | 103 | | |
| |||
108 | 115 | | |
109 | 116 | | |
110 | 117 | | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
111 | 131 | | |
112 | 132 | | |
113 | 133 | | |
| |||
555 | 575 | | |
556 | 576 | | |
557 | 577 | | |
| 578 | + | |
| 579 | + | |
| 580 | + | |
| 581 | + | |
| 582 | + | |
| 583 | + | |
| 584 | + | |
| 585 | + | |
| 586 | + | |
558 | 587 | | |
559 | 588 | | |
560 | 589 | | |
| |||
574 | 603 | | |
575 | 604 | | |
576 | 605 | | |
577 | | - | |
578 | | - | |
579 | | - | |
580 | | - | |
581 | | - | |
582 | | - | |
583 | | - | |
584 | | - | |
585 | | - | |
586 | 606 | | |
587 | 607 | | |
588 | 608 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
28 | 43 | | |
29 | 44 | | |
30 | 45 | | |
| |||
0 commit comments