Repository navigation
Commit 7f0f1ab
committed
fix(security): restore cooperative cancellation and bound the derived-view cache
Addresses both blocking findings on this PR. Both reproduced first; both were
real.
Cancellation. Seeking the next candidate jumps over the `offset % 4096`
checkpoints a character-by-character walk would hit, so a long scan could not be
cancelled. On the reviewer's reproducer -- `'aᅟa ' * 8192` with a callback
raising on its second call -- main aborts and this branch called the callback
once. `_check_skipped_checkpoints` now runs one check per checkpoint crossed,
reproducing the walk's cadence exactly: verified equal to `len // 4096` at 0,
4095, 4096, 10k, 32k and 100k characters. The whole-string early-return path had
the same gap and is covered too.
Memory. The derived-view cache was bounded by entry count, which bounds nothing
when normalization expands its input -- NFKC turns one U+FDFA into 18
characters, each carrying a four-byte offset. It is now bounded by stored
characters (4M budget) and declines to retain any single view larger than the
whole budget. Forty expanding inputs settle at 2.88M stored characters instead
of growing without limit.
Retention after a scan. clear_security_text_caches() releases the view cache and
the predicate caches, and is called from cleanup_result alongside the existing
clear_python_ast_cache. That also releases the text keys the predicate caches
hold, so a long-lived scanner process no longer keeps a scanned file's content
alive after the scan that produced it.
Performance is preserved. Full suite, 901 real skills, two alternating runs per
arm against upstream main: p95 -10.3%, p99 -12.7%, total scan time -2.6%.
Findings remain byte-identical (8,152 on both arms), as do coverage-ledger
outcomes (176) and error counts (0). The mean gain narrows from -4.1% to -2.6%,
which is the cost of the restored checkpoints and is the right trade.
Thirteen new tests cover the checkpoint cadence at six lengths, cancellation
across sparse in-word gaps and on text with no candidates, the size bound, the
oversized-single-view case, clearing, teardown via cleanup_result, and that an
evicted-then-rebuilt view is identical to the original. Full suite: 5821 passed.
Signed-off-by: Steven Moy <[email protected]>1 parent 9dc3c20 commit 7f0f1ab
3 files changed
Lines changed: 224 additions & 4 deletions
File tree
- src/skillspector
- tests/nodes/analyzers
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| 17 | + | |
17 | 18 | | |
18 | 19 | | |
19 | 20 | | |
| |||
1577 | 1578 | | |
1578 | 1579 | | |
1579 | 1580 | | |
| 1581 | + | |
| 1582 | + | |
| 1583 | + | |
| 1584 | + | |
| 1585 | + | |
| 1586 | + | |
| 1587 | + | |
| 1588 | + | |
| 1589 | + | |
| 1590 | + | |
| 1591 | + | |
| 1592 | + | |
| 1593 | + | |
| 1594 | + | |
| 1595 | + | |
1580 | 1596 | | |
1581 | 1597 | | |
1582 | 1598 | | |
| |||
1585 | 1601 | | |
1586 | 1602 | | |
1587 | 1603 | | |
| 1604 | + | |
| 1605 | + | |
1588 | 1606 | | |
1589 | 1607 | | |
1590 | 1608 | | |
1591 | 1609 | | |
1592 | 1610 | | |
1593 | 1611 | | |
1594 | 1612 | | |
| 1613 | + | |
| 1614 | + | |
1595 | 1615 | | |
| 1616 | + | |
| 1617 | + | |
1596 | 1618 | | |
1597 | 1619 | | |
1598 | 1620 | | |
| |||
1747 | 1769 | | |
1748 | 1770 | | |
1749 | 1771 | | |
1750 | | - | |
| 1772 | + | |
| 1773 | + | |
| 1774 | + | |
| 1775 | + | |
| 1776 | + | |
| 1777 | + | |
| 1778 | + | |
| 1779 | + | |
| 1780 | + | |
| 1781 | + | |
| 1782 | + | |
| 1783 | + | |
| 1784 | + | |
| 1785 | + | |
| 1786 | + | |
| 1787 | + | |
| 1788 | + | |
| 1789 | + | |
| 1790 | + | |
| 1791 | + | |
| 1792 | + | |
| 1793 | + | |
| 1794 | + | |
| 1795 | + | |
| 1796 | + | |
| 1797 | + | |
| 1798 | + | |
| 1799 | + | |
| 1800 | + | |
| 1801 | + | |
| 1802 | + | |
| 1803 | + | |
| 1804 | + | |
| 1805 | + | |
| 1806 | + | |
| 1807 | + | |
| 1808 | + | |
| 1809 | + | |
| 1810 | + | |
| 1811 | + | |
| 1812 | + | |
| 1813 | + | |
| 1814 | + | |
| 1815 | + | |
| 1816 | + | |
| 1817 | + | |
| 1818 | + | |
| 1819 | + | |
| 1820 | + | |
| 1821 | + | |
| 1822 | + | |
| 1823 | + | |
| 1824 | + | |
| 1825 | + | |
| 1826 | + | |
| 1827 | + | |
| 1828 | + | |
| 1829 | + | |
| 1830 | + | |
| 1831 | + | |
| 1832 | + | |
| 1833 | + | |
| 1834 | + | |
| 1835 | + | |
| 1836 | + | |
1751 | 1837 | | |
1752 | 1838 | | |
1753 | 1839 | | |
1754 | | - | |
1755 | | - | |
1756 | | - | |
| 1840 | + | |
| 1841 | + | |
| 1842 | + | |
| 1843 | + | |
| 1844 | + | |
| 1845 | + | |
| 1846 | + | |
| 1847 | + | |
| 1848 | + | |
1757 | 1849 | | |
| 1850 | + | |
| 1851 | + | |
| 1852 | + | |
| 1853 | + | |
| 1854 | + | |
| 1855 | + | |
| 1856 | + | |
| 1857 | + | |
| 1858 | + | |
1758 | 1859 | | |
1759 | 1860 | | |
1760 | 1861 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| 8 | + | |
8 | 9 | | |
9 | 10 | | |
10 | 11 | | |
11 | 12 | | |
12 | 13 | | |
| 14 | + | |
13 | 15 | | |
14 | 16 | | |
15 | 17 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
0 commit comments