Repository navigation
Commit fa9b4e2
fix(jsonl): dedup cumulative token totals by message.id (#8, sirmalloc#549)
Claude Code writes one JSONL entry per content block of a single API call,
all sharing one message.id. The cumulative token path summed every entry,
inflating Tokens Input/Output/Total by up to ~1.84x. Mirror the grouping
already used for Tokens Last Turn: consecutive counted entries sharing an
id contribute one usage (prompt side once, output as the running max).
- transcripts without duplicate-id entries are unchanged (existing tests)
- streaming partials resolve to the finalized value, counted once mid-stream
- sidechain/API-error rows keep their current treatment in sums and recency
Added vitest cases on a synthetic sirmalloc#549-shaped transcript asserting exact
cumulative totals (12 in / 292 out / 16660 total vs inflated 16/393/...),
streaming partials with ids, and a sidechain per-content-block group.
Co-Authored-By: Claude Code <[email protected]>1 parent 51a8819 commit fa9b4e2
2 files changed
Lines changed: 215 additions & 10 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
756 | 756 | | |
757 | 757 | | |
758 | 758 | | |
| 759 | + | |
| 760 | + | |
| 761 | + | |
| 762 | + | |
| 763 | + | |
| 764 | + | |
| 765 | + | |
| 766 | + | |
| 767 | + | |
| 768 | + | |
| 769 | + | |
| 770 | + | |
| 771 | + | |
| 772 | + | |
| 773 | + | |
| 774 | + | |
| 775 | + | |
| 776 | + | |
| 777 | + | |
| 778 | + | |
| 779 | + | |
| 780 | + | |
| 781 | + | |
| 782 | + | |
| 783 | + | |
| 784 | + | |
| 785 | + | |
| 786 | + | |
| 787 | + | |
| 788 | + | |
| 789 | + | |
| 790 | + | |
| 791 | + | |
| 792 | + | |
| 793 | + | |
| 794 | + | |
| 795 | + | |
| 796 | + | |
| 797 | + | |
| 798 | + | |
| 799 | + | |
| 800 | + | |
| 801 | + | |
| 802 | + | |
| 803 | + | |
| 804 | + | |
| 805 | + | |
| 806 | + | |
| 807 | + | |
| 808 | + | |
| 809 | + | |
| 810 | + | |
| 811 | + | |
| 812 | + | |
| 813 | + | |
| 814 | + | |
| 815 | + | |
| 816 | + | |
| 817 | + | |
| 818 | + | |
| 819 | + | |
| 820 | + | |
| 821 | + | |
| 822 | + | |
| 823 | + | |
| 824 | + | |
| 825 | + | |
| 826 | + | |
| 827 | + | |
| 828 | + | |
| 829 | + | |
| 830 | + | |
| 831 | + | |
| 832 | + | |
| 833 | + | |
| 834 | + | |
| 835 | + | |
| 836 | + | |
| 837 | + | |
| 838 | + | |
| 839 | + | |
| 840 | + | |
| 841 | + | |
| 842 | + | |
| 843 | + | |
| 844 | + | |
| 845 | + | |
| 846 | + | |
| 847 | + | |
| 848 | + | |
| 849 | + | |
| 850 | + | |
| 851 | + | |
| 852 | + | |
| 853 | + | |
| 854 | + | |
| 855 | + | |
| 856 | + | |
| 857 | + | |
| 858 | + | |
| 859 | + | |
| 860 | + | |
| 861 | + | |
| 862 | + | |
| 863 | + | |
| 864 | + | |
| 865 | + | |
| 866 | + | |
| 867 | + | |
| 868 | + | |
| 869 | + | |
| 870 | + | |
| 871 | + | |
| 872 | + | |
| 873 | + | |
| 874 | + | |
| 875 | + | |
| 876 | + | |
| 877 | + | |
| 878 | + | |
| 879 | + | |
| 880 | + | |
| 881 | + | |
| 882 | + | |
| 883 | + | |
| 884 | + | |
| 885 | + | |
| 886 | + | |
| 887 | + | |
| 888 | + | |
| 889 | + | |
| 890 | + | |
| 891 | + | |
| 892 | + | |
| 893 | + | |
| 894 | + | |
| 895 | + | |
| 896 | + | |
| 897 | + | |
| 898 | + | |
| 899 | + | |
| 900 | + | |
| 901 | + | |
| 902 | + | |
| 903 | + | |
| 904 | + | |
| 905 | + | |
| 906 | + | |
| 907 | + | |
| 908 | + | |
| 909 | + | |
| 910 | + | |
| 911 | + | |
| 912 | + | |
| 913 | + | |
| 914 | + | |
| 915 | + | |
| 916 | + | |
| 917 | + | |
| 918 | + | |
| 919 | + | |
| 920 | + | |
| 921 | + | |
| 922 | + | |
| 923 | + | |
| 924 | + | |
| 925 | + | |
| 926 | + | |
| 927 | + | |
| 928 | + | |
| 929 | + | |
| 930 | + | |
| 931 | + | |
| 932 | + | |
759 | 933 | | |
760 | 934 | | |
761 | 935 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
88 | 88 | | |
89 | 89 | | |
90 | 90 | | |
| 91 | + | |
91 | 92 | | |
92 | 93 | | |
93 | 94 | | |
| |||
104 | 105 | | |
105 | 106 | | |
106 | 107 | | |
| 108 | + | |
| 109 | + | |
107 | 110 | | |
108 | 111 | | |
109 | 112 | | |
| |||
173 | 176 | | |
174 | 177 | | |
175 | 178 | | |
| 179 | + | |
| 180 | + | |
176 | 181 | | |
177 | 182 | | |
178 | 183 | | |
| |||
195 | 200 | | |
196 | 201 | | |
197 | 202 | | |
198 | | - | |
199 | | - | |
200 | | - | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
201 | 229 | | |
202 | | - | |
203 | | - | |
| 230 | + | |
204 | 231 | | |
205 | 232 | | |
206 | 233 | | |
207 | 234 | | |
208 | 235 | | |
209 | 236 | | |
210 | 237 | | |
211 | | - | |
| 238 | + | |
212 | 239 | | |
213 | 240 | | |
214 | 241 | | |
| |||
224 | 251 | | |
225 | 252 | | |
226 | 253 | | |
| 254 | + | |
227 | 255 | | |
228 | 256 | | |
229 | 257 | | |
230 | 258 | | |
231 | | - | |
| 259 | + | |
| 260 | + | |
232 | 261 | | |
233 | 262 | | |
234 | 263 | | |
235 | 264 | | |
236 | 265 | | |
237 | 266 | | |
| 267 | + | |
| 268 | + | |
238 | 269 | | |
239 | 270 | | |
240 | | - | |
| 271 | + | |
241 | 272 | | |
242 | 273 | | |
243 | 274 | | |
244 | 275 | | |
245 | | - | |
| 276 | + | |
246 | 277 | | |
247 | 278 | | |
248 | 279 | | |
| |||
251 | 282 | | |
252 | 283 | | |
253 | 284 | | |
254 | | - | |
| 285 | + | |
255 | 286 | | |
256 | 287 | | |
257 | 288 | | |
| |||
0 commit comments