fix(vm-runner): report compilation and cache metrics from all code paths#15580
Conversation
ssavenko-near
force-pushed
the
slavas/crt-fix-metrics
branch
from
f643ee3 to
affd119
Compare
There was a problem hiding this comment.
Pull request overview
This PR ensures VM-runner compilation and cache metrics are flushed to Prometheus from all compilation-triggering code paths (not only function calls), preserving shard_id and caller-context labeling. It also adds a dedicated counter to distinguish in-memory compiled-contract cache hits from disk cache hits.
Changes:
- Plumb a preformatted
shard_idintoReceiptPreparationPipelineand flush metrics from pipelining worker + main-thread fallback paths. - Flush VM-runner metrics after contract compilation triggered by deploys (including implicit account wallet precompile) and global contract distribution.
- Add
near_vm_compiled_contract_memory_cache_hits_totaland record memory-cache hits in both NearVM and Wasmtime runners.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| runtime/runtime/src/state_viewer/mod.rs | Passes shard id into the receipt preparation pipeline so pipelining metric flushes have correct labels in view execution. |
| runtime/runtime/src/pipelining.rs | Adds shard id to the pipeline and flushes VM-runner thread-local metrics on both worker and main-thread preparation paths. |
| runtime/runtime/src/lib.rs | Flushes metrics after DeployContract compilation; passes shard id into pipelining pipeline creation sites. |
| runtime/runtime/src/global_contracts.rs | Flushes metrics after global-contract distribution precompilation. |
| runtime/runtime/src/actions.rs | Flushes metrics after ETH-implicit wallet precompilation (deploy-like path). |
| runtime/near-vm-runner/src/wasmtime_runner/mod.rs | Records overall cache hit vs memory hit for compiled-contract lookups. |
| runtime/near-vm-runner/src/near_vm_runner/runner.rs | Records overall cache hit vs memory hit for compiled-contract lookups. |
| runtime/near-vm-runner/src/metrics.rs | Adds memory-cache-hit counter + thread-local accumulation and flush logic in report_metrics. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
ssavenko-near
force-pushed
the
slavas/crt-fix-metrics
branch
from
affd119 to
34be431
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #15580 +/- ##
==========================================
+ Coverage 69.27% 69.29% +0.02%
==========================================
Files 940 940
Lines 212069 212113 +44
Branches 212069 212113 +44
==========================================
+ Hits 146902 146990 +88
+ Misses 59329 59295 -34
+ Partials 5838 5828 -10
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
ssavenko-near
force-pushed
the
slavas/crt-fix-metrics
branch
from
34be431 to
931e0ee
Compare
…aths `report_metrics()`, which flushes vm-runner's thread-local metric accumulators to Prometheus, was only called on the function-call path. Compilation triggered by contract deployment, global contract distribution, or pipelining accumulated into thread-locals but never flushed — the data was silently lost. Add `report_metrics()`` calls at each of these call sites, where the shard context is available, so the existing near_vm_runner_compilation_seconds histogram and cache counters capture all compilations with proper shard_id and caller context labels. The simpler approach to observe directly in the `compile_uncached()` would loose the "shard_id" information. Also add near_vm_compiled_contract_memory_cache_hits_total to distinguish in-memory cache hits from disk hits. Disk hit rate can be derived as (total_hits - memory_hits) / (total_lookups - memory_hits).
ssavenko-near
force-pushed
the
slavas/crt-fix-metrics
branch
from
931e0ee to
3bf7a2e
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
darioush
left a comment
darioush
left a comment
There was a problem hiding this comment.
Seems fine. Only non-blocking concern is that we have to be careful to add the call to metrics whenever we make a new call to compiling contracts.
Yes. That bothers me as well. I did not find an acceptable way to handle this. But since I do not call |
|
@ssavenko-near BTW, please make changelog entry for metrics changes. Thanks. |
report_metrics(), which flushes vm-runner's thread-local metric accumulators to Prometheus, was only called on the function-call path. Compilation triggered by contract deployment, global contract distribution, or pipelining accumulated into thread-locals but never flushed — the data was silently lost.Add
report_metrics()`` calls at each of these call sites, where the shard context is available, so the existing near_vm_runner_compilation_seconds histogram and cache counters capture all compilations with proper shard_id and caller context labels. The simpler approach to observe directly in thecompile_uncached()` would loose the "shard_id" information.Also add near_vm_compiled_contract_memory_cache_hits_total to distinguish in-memory cache hits from disk hits. Disk hit rate can be derived as (total_hits - memory_hits) / (total_lookups - memory_hits).