You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 37ba40d
Browse filesBrowse the repository at this point in the historyBrowse files
Copy file name to clipboardExpand all lines: docs/source/user-guide/latest/metrics.md
+3-2Lines changed: 3 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -230,14 +230,15 @@ execution metrics. Counters accumulate per scan operator; they do not instrument
230
230
|`scan_io_object_store_response_bytes_read`| Response bytes actually consumed at that API, including bytes fetched between coalesced ranges. Not HTTP wire bytes. |
231
231
|`scan_io_metadata_cache_hits`| Successful, cache-eligible metadata opens requiring no storage reads. |
232
232
|`scan_io_metadata_cache_misses`| Successful, cache-eligible metadata opens requiring storage reads. Failed opens and encrypted opens, which bypass this shared cache, increment neither cache counter. |
233
-
|`scan_io_metadata_prefetches`|Metadata opens that used a load started when the previous file opened (`spark.comet.parquet.prefetchNextFileMetadata.enabled`). Each also counts as a cache miss.|
233
+
|`scan_io_metadata_prefetches`|Opens that used a load which read storage ahead of them, started when the previous file opened (`spark.comet.parquet.prefetchNextFileMetadata.enabled`). Each also counts as a cache miss. |
234
234
|`row_groups_pruned_dynamic_filter`| Row groups skipped by a runtime predicate whose value changes during execution, such as a local TopK threshold. |
235
235
236
236
Reader-level and object-store bytes are two views of the same reads; do not add them together.
237
237
Likewise, footer bytes are a subset of metadata bytes, not a third reader-level category. A warm
238
238
metadata-cache hit contributes no new metadata or footer I/O. A prefetched load's footer and
239
239
metadata bytes count when the prefetch reads them, so a prefetch that no open uses, because the
240
-
scan stopped early or pruned the file before opening it, adds bytes without a matching cache hit.
240
+
scan stopped early or pruned the file before opening it, adds bytes with no matching cache miss and
241
+
no `scan_io_metadata_prefetches`.
241
242
A valid plaintext footer followed by a page-index failure still contributes footer bytes; an
0 commit comments