Repository navigation
try codspeed runners - #1079
try codspeed runners#1079
Conversation
|
Superseded by #1082 (for now). |
ccdc9ae to
c3b76db
Compare
Merging this PR will improve performance by 3.84%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Simulation | test_empty_struct_create[arm] |
478 µs | 591.1 µs | -19.13% |
| ❌ | Simulation | test_pickle_load[arm] |
1.6 ms | 1.8 ms | -11.82% |
| ❌ | Simulation | test_pickle_dump[arm] |
2.2 ms | 2.4 ms | -8.9% |
| ❌ | Simulation | test_decode_type[arm-msgpack-timedelta] |
2.8 ms | 3.1 ms | -7.49% |
| ❌ | Simulation | test_gc_times[arm-PointClassSlots] |
509.8 ms | 550.4 ms | -7.36% |
| ❌ | Simulation | test_encode[arm-json] |
1.4 ms | 1.5 ms | -6.75% |
| ❌ | Simulation | test_decode_type[arm-json-timedelta] |
3.1 ms | 3.3 ms | -6.2% |
| ❌ | Simulation | test_decode_type[arm-msgpack-dict] |
5.5 ms | 5.9 ms | -6.16% |
| ❌ | Simulation | test_encode_type[arm-json-bytes] |
78.7 µs | 83.6 µs | -5.8% |
| ❌ | Simulation | test_encode_type[arm-json-bytearray] |
79.1 µs | 83.5 µs | -5.32% |
| ❌ | Simulation | test_encode_type[arm-json-datetime] |
1.6 ms | 1.7 ms | -5.27% |
| ❌ | Simulation | test_encode_type[arm-json-memoryview] |
79.2 µs | 83.6 µs | -5.23% |
| ❌ | Simulation | test_import_time[arm-msgspec.toml] |
130.9 µs | 138.2 µs | -5.23% |
| ❌ | Simulation | test_import_time[arm-msgspec.json] |
130.1 µs | 137.1 µs | -5.09% |
| ❌ | Simulation | test_encode_type[arm-msgpack-None] |
323.5 µs | 340.6 µs | -5.02% |
| ❌ | Simulation | test_import_time[arm-msgspec.msgpack] |
130.5 µs | 137.2 µs | -4.89% |
| ❌ | Simulation | test_encode_type[arm-msgpack-datetime] |
1.7 ms | 1.8 ms | -4.85% |
| ❌ | Simulation | test_import_time[arm-msgspec.inspect] |
131 µs | 137.6 µs | -4.78% |
| ❌ | Simulation | test_import_time[arm-msgspec] |
131.5 µs | 138.1 µs | -4.78% |
| ❌ | Simulation | test_decode_convert_type_non_strict[arm-msgpack-float] |
2.1 ms | 2.2 ms | -4.77% |
| ... | ... | ... | ... | ... | ... |
ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing provinzkraut:bare-metal-codspeed (2260407) with main (fbfb76b)
Footnotes
-
135 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
|
@ofek ran this with the codspeed runners a couple of times now. The results seem to be stable, but the benchmarks run for 7 minutes now, which also isn't great |
|
I'm totally fine with that if it means more reliable measurements, especially since our build/test workflow is about the same anyway https://github.com/msgspec/msgspec/actions/runs/27905578518?pr=1079 |
|
Do you mind explaining what you mean by the results now being seemingly stable? I haven't looked. |
I just re-ran them a couple of times and checked if the results were about the same, compared to previous runs on the same branch. Probably not a super reliable way to test this, but it's the best I could some up with other than directly merging to
True. Just a bit longer turnaround time if there's something to address. What's more important is that only arm runners are available, so our main arch isn't being tested. Probably not super relevant for most parts, but especially in the hot paths there are some platform differences. |
|
I'm fine with merging and seeing what happens as it would be a clear improvement to what we have now! |
Alright, let's try it |
|
No but I found the issue, the docs say:
We were using 
|
No description provided.