Repository navigation
Conversation
|
bandaid guard rail fix for benchmarking as I dive deeper into mem/RAM guardrail is only faster bc it doesn't execute the whole path and limits things not a real fix imo |
fa6edbd to
4c1335f
Compare
|
| if (!stat) continue | ||
| changedFiles.push({ | ||
| path: filepath, | ||
| added: 0, |
There was a problem hiding this comment.
If we reintroduce line counts, large untracked files put us back on the crash path unless we cap by size. Which then we would have the perf fix + a guardrail but could be a compromise
There was a problem hiding this comment.
yeah, I imagine this is a "solved problem" there is prolly something we can do to get line counts for most cases and have the handling for the ones we don't right?
There was a problem hiding this comment.
Changed it to where we keep line counts for normal files and avoids the crash path for multi‑GB files.
This is the most optimal fix I saw from a deep dive and looking in stack overflow, we still also can:
- For large files, show size (from stat) so users still get useful metadata
- If we insist on line counts for huge files, add an opt‑in flag to run a streaming counter (e.g., wc -l) knowing it may be slow.
- configurable size threshold?
There was a problem hiding this comment.
Possible deep cache and higher perf approach but not sure it will improve perf much if at all.
There was a problem hiding this comment.
@rekram1-node
Built custom untracked-line caches that meets the “get line counts for most cases and handle the ones we don’t”:
- small/medium files keep counts
- large text eventually gets counts
- huge/binary stays safe.
- Hot path remains fast; the only “slow” work s the one-off background stream for ≤100 MB text.
Let me know if this recent commit or
tweak: count lines for small untracked files
is preferred
Summary
Benchmark (single 200MB untracked file)
1 MB text: first 13.5 ms, second 13.3 ms; lines=524,289 (inline + cached).
shortly after).
cache after.
What changed vs the prior guardrail we moved from:
counted.
Testing