Repository navigation
Commit 1e28e62
authored
fix(devin): read usage from ModelUsageStats and classify cloud failures (#4419)
* fix(devin): read usage from ModelUsageStats and classify cloud failures
A cached Devin turn reported a bare token total with no cached subset, so
its log row looked like a smaller request than it was.
The decoder was reading GetChatMessageResponse field 28. Field 28 is
response_dimension_groups, the rows the IDE renders; field 7 is
ModelUsageStats, the per-turn accounting. The old path worked by accident:
ResponseDimension.uid is that message's field 5, which the entry walker
treats as a metric id, so cache numbers appeared only when the service
happened to render cache rows. Field 7 carries cache read and cache write
unconditionally.
Both fields arrive in the same message and the adapter keeps the last usage
event, so decoding both is not enough: field 7 now suppresses field 28
within a message and is yielded last, and it needs its own uint64 decoder
because the field-28 walker reads a fixed32 float out of a sub-message.
Whether Cognition's input_tokens already includes cache is unsettled, and
guessing inclusive is the expensive error: normalizeCostTokens only rejects
read + write > input, so an inflated input passes validation and bills
cached tokens at the uncached rate. The mapping is therefore derived from
the frame. Both branches agree on the 58k-prompt case that prompted this.
Two further classification defects. CloudChatError carried no HTTP status,
so inferHttpStatusFromAdapterMessage turned an upstream 429 into a 502 and
core's failover never rotated or backed off. And a cancelled turn said
"Devin turn was aborted.", which isClientClosedMessage does not recognise,
so a client hanging up was logged as an upstream failure; it now emits the
phrase the classifier knows, with status 499.
Usage frames are merged per field instead of replaced, because the counters
are cumulative and a later partial frame used to zero an earlier count.
* fix(devin): report field-7 usage ahead of finish and derive the merged total
Review follow-ups on the usage decode.
The authoritative ModelUsageStats event was yielded after the rest of the
frame, so a frame that also carried finish reported usage behind the turn's
end. It is now yielded first, which makes the order independent of where
the service places the field.
mergeDevinUsage took the max of two totals alongside the per-field maxima,
which can leave totalTokens different from input + output; the cost and log
paths read that total. The total is now derived from the merged counts.
Regression coverage for what the change is actually for: field 7 suppressing
the display rows within one frame and landing before finish, the display
rows still decoding when no field 7 is present, a partial frame not zeroing
an earlier count, and an HTTP status becoming a structured classification.
A Connect trailer still carries no HTTP status, so a cap delivered that way
keeps the older message-inference path. That is noted at the throw site as
a follow-up rather than silently left open.1 parent 7b3c4e9 commit 1e28e62
3 files changed
Lines changed: 287 additions & 9 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
16 | 63 | | |
17 | 64 | | |
18 | 65 | | |
| |||
210 | 257 | | |
211 | 258 | | |
212 | 259 | | |
213 | | - | |
| 260 | + | |
214 | 261 | | |
215 | 262 | | |
216 | 263 | | |
| |||
272 | 319 | | |
273 | 320 | | |
274 | 321 | | |
275 | | - | |
| 322 | + | |
276 | 323 | | |
277 | 324 | | |
278 | 325 | | |
| |||
305 | 352 | | |
306 | 353 | | |
307 | 354 | | |
308 | | - | |
| 355 | + | |
309 | 356 | | |
310 | 357 | | |
311 | 358 | | |
312 | 359 | | |
313 | 360 | | |
314 | 361 | | |
315 | 362 | | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
316 | 368 | | |
317 | 369 | | |
318 | 370 | | |
319 | 371 | | |
320 | 372 | | |
321 | | - | |
| 373 | + | |
322 | 374 | | |
323 | 375 | | |
324 | 376 | | |
325 | 377 | | |
326 | 378 | | |
327 | 379 | | |
328 | | - | |
| 380 | + | |
329 | 381 | | |
330 | 382 | | |
331 | 383 | | |
332 | 384 | | |
333 | 385 | | |
334 | 386 | | |
335 | 387 | | |
336 | | - | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
337 | 395 | | |
338 | 396 | | |
339 | 397 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
669 | 669 | | |
670 | 670 | | |
671 | 671 | | |
672 | | - | |
| 672 | + | |
| 673 | + | |
| 674 | + | |
| 675 | + | |
| 676 | + | |
| 677 | + | |
| 678 | + | |
| 679 | + | |
| 680 | + | |
| 681 | + | |
| 682 | + | |
| 683 | + | |
| 684 | + | |
| 685 | + | |
| 686 | + | |
| 687 | + | |
| 688 | + | |
| 689 | + | |
| 690 | + | |
| 691 | + | |
| 692 | + | |
| 693 | + | |
| 694 | + | |
| 695 | + | |
673 | 696 | | |
674 | 697 | | |
675 | 698 | | |
| |||
740 | 763 | | |
741 | 764 | | |
742 | 765 | | |
| 766 | + | |
743 | 767 | | |
744 | 768 | | |
745 | 769 | | |
| |||
834 | 858 | | |
835 | 859 | | |
836 | 860 | | |
| 861 | + | |
| 862 | + | |
| 863 | + | |
| 864 | + | |
| 865 | + | |
| 866 | + | |
| 867 | + | |
| 868 | + | |
| 869 | + | |
| 870 | + | |
| 871 | + | |
| 872 | + | |
| 873 | + | |
| 874 | + | |
| 875 | + | |
| 876 | + | |
| 877 | + | |
| 878 | + | |
| 879 | + | |
| 880 | + | |
| 881 | + | |
| 882 | + | |
| 883 | + | |
| 884 | + | |
| 885 | + | |
| 886 | + | |
| 887 | + | |
| 888 | + | |
| 889 | + | |
| 890 | + | |
| 891 | + | |
| 892 | + | |
| 893 | + | |
| 894 | + | |
| 895 | + | |
| 896 | + | |
| 897 | + | |
| 898 | + | |
| 899 | + | |
| 900 | + | |
| 901 | + | |
| 902 | + | |
| 903 | + | |
| 904 | + | |
| 905 | + | |
| 906 | + | |
| 907 | + | |
| 908 | + | |
| 909 | + | |
| 910 | + | |
| 911 | + | |
| 912 | + | |
| 913 | + | |
| 914 | + | |
| 915 | + | |
| 916 | + | |
| 917 | + | |
| 918 | + | |
| 919 | + | |
| 920 | + | |
| 921 | + | |
| 922 | + | |
837 | 923 | | |
838 | 924 | | |
839 | 925 | | |
| |||
864 | 950 | | |
865 | 951 | | |
866 | 952 | | |
867 | | - | |
| 953 | + | |
| 954 | + | |
| 955 | + | |
| 956 | + | |
| 957 | + | |
| 958 | + | |
| 959 | + | |
| 960 | + | |
| 961 | + | |
| 962 | + | |
| 963 | + | |
| 964 | + | |
868 | 965 | | |
869 | 966 | | |
870 | 967 | | |
| |||
987 | 1084 | | |
988 | 1085 | | |
989 | 1086 | | |
990 | | - | |
| 1087 | + | |
| 1088 | + | |
| 1089 | + | |
| 1090 | + | |
| 1091 | + | |
| 1092 | + | |
991 | 1093 | | |
992 | 1094 | | |
993 | 1095 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
8 | 11 | | |
9 | 12 | | |
10 | 13 | | |
| |||
280 | 283 | | |
281 | 284 | | |
282 | 285 | | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
0 commit comments