Repository navigation
fix(core): parse MiMo Code and fx on the local report path - #1300
Conversation
MiMoCode and Fx both declare `parse_local: true` in `define_clients!`, and `parse_all_messages_with_pricing_with_env_strategy` parses both, so their usage reaches submit and the TUI. `parse_local_clients` in `crates/tokscale-core/src/lib.rs` had no block for either client, so everything built on that function saw nothing. `tokscale report` fills its wiki DB from `populate_wiki_from_sessions`, which reads only `parse_local_clients`, so no MiMo Code or fx session ever reached the report table, and `tokscale clients` printed a zero message count for both even though it explicitly asks for every `parse_local()` client. This is the mirror of #986, where opencodereview was parsed locally but missing from the submit path. Read `scan_result.micode_dbs` after the OpenCode block, deduplicating on the payload message id the way the submit lane does, because MiMo writes channel-suffixed databases and one session can appear in both `mimocode.db` and `mimocode-<channel>.db`. Read `scan_result.get(ClientId::Fx)` after the Mux block. Both count through `summed_parsed_message_count`, so an fx row contributes its `request_count` rather than one. fx session titles are not applied here: `ParsedMessage` has no title field for the shared `sessions/index.json` lookup to write to. `test_parse_local_clients_includes_micode_and_fx` fails on main with zero rows for both clients.
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
…al lane PR #1300 added MiMo Code and fx to parse_local_clients, mirroring the submit lane. Two things the streaming loader does for fx did not make it across the ParsedMessage boundary. Date. parse_fx_file leaves `date` empty and relies on the loader's refresh_derived_fields to derive it from the timestamp. The local block converted straight to ParsedMessage, so every fx row reached filter_parsed_messages with date == "": `year` and `since` dropped all of them and `until` alone admitted rows after the cutoff. Default scanner settings pin no zone, so rebucket_parsed_days could not repair the key either. No shipped command reaches this today: `tokscale report` (report.rs:112) and `tokscale clients` (main.rs:4266) pass since/until/year as None, and `tokscale wrapped`, the one caller that filters (by year, wrapped.rs:228), feeds the result only to build_top_agents, which skips every row whose client is not opencode; fx is not in wrapped's default client list and its rows carry agent: None regardless. Everything else wrapped renders comes from generate_graph, whose streaming lane already parses fx and refreshes the date. The defect is on the public parse_local_clients API and any future caller that passes a date filter. The block now refreshes the derived date before conversion; the pinned-zone rebucket pass still runs afterwards. Reproduced by test_parse_local_clients_fx_rows_carry_a_derived_date, which failed with left: "" / right: "2026-05-29" before the refresh. fx is the only parser that leaves `date` empty; MiMo Code derives it in the parser, so its block did not share this. Cost. ParsedMessage carried tokens only, so `tokscale report` priced every row from its tokens. fx reports a positive total_cost for a snapshot with empty `models` and zero tokens (the synthetic fx-unknown row); pricing that from tokens recorded $0.00 in the wiki, and the wiki never rewrites a session it has recorded. ParsedMessage now carries `cost` and `cost_source`, and report's compute_msg_cost uses a ProviderReported figure as-is, the way the submit lane never reprices an authoritative cost. Reproduced by wiki_keeps_the_reported_cost_of_a_token_free_fx_session: with the precedence removed it fails with "wiki must keep the provider-reported cost, got 0", and compute_msg_cost_keeps_a_provider_reported_cost fails with 0.009 instead of 0.42. MiMo Code's embedded per-message cost now reaches the lane with the same provenance (test_parse_local_clients_keeps_provider_reported_cost). Constraint: parse_local_clients applies no pricing, so a non-authoritative row still has to be priced by the consumer from its tokens Rejected: deriving the date inside parse_fx_file | the streaming loader and cache already refresh it; the local block was the only caller that skipped that step Rejected: leaving the local lane token-only | an fx-unknown row has no tokens to price, so the wiki figure was wrong rather than approximate Directive: every parser that stamps CostSource::ProviderReported (cline, claudecode, commandcode, cursor, fx, gjc, junie, lmstudio, micode, opencode, unsloth) now drives report figures with that cost, authoritative zeros included; do not reprice a ProviderReported row Confidence: high Scope-risk: moderate Not-tested: wiki sessions recorded at $0 before this change keep that figure; `--rebuild` resets summaries only Not-tested: the local MiMo Code block keeps the first row per message id and does not upgrade its cost from a later channel db the way the submit lane does Not-tested: no CLI command exercises a date filter against the fx local block; the fix is covered at the parse_local_clients boundary only
junhoyeo
left a comment
There was a problem hiding this comment.
Your diagnosis is right: MiMoCode and Fx both declare parse_local: true, the submit path parses both, and parse_local_clients had a block for every other locally parsed client and none for these two — so tokscale report and tokscale clients showed nothing for either. The two new blocks are the correct fix.
An adversarial Codex pass found two integration defects in the new fx block, both reproduced and fixed in maintainer commit 8891eb3c on top of yours:
- Reported cost was dropped. The fx conversion discarded
costandcost_source, so an fx snapshot with a positivetotal_cost, empty models and zero tokens — a shape fx explicitly supports — reached the report as a provider-pricedfx-unknownrow at $0, and the wiki then kept that figure on later scans. The conversion now carries theProviderReportedprovenance through, matching the streaming path;wiki_keeps_the_reported_cost_of_a_token_free_fx_sessionandtest_parse_local_clients_keeps_provider_reported_costpin it. - fx rows carried an empty date.
parse_fx_filereturnsdateas an empty string (the only parser undersessions/that does), and the direct local path skipped the derived-field refresh the streaming loader performs, soyearandsincefilters would drop every fx row. The block now refreshes the derived date before conversion;test_parse_local_clients_fx_rows_carry_a_derived_datepins it. To be precise about impact: no shipped command exercises a date filter against this block today (report.rsandmain.rspasssince/until/yearasNone, andwrappedbuilds its graph from the streaming lane), so this hardens the publicparse_local_clientsAPI for future filtered callers rather than fixing a visible symptom.
The MiMoCode block was checked for both and has neither: micode.rs derives its own date, and its embedded per-message cost reaches the lane with ProviderReported provenance.
One nit, not blocking: parsed_to_unified, the public reverse of unified_to_parsed, still stamps CostSource::Unknown, so a ParsedMessage carrying a provider-reported cost loses its provenance on the round trip. Nothing shipped calls it beyond the round-trip test.
Gates on my machine: fmt, clippy with and without --all-targets, full workspace suite.
Summary
MiMo Code and fx usage never shows up in
tokscale report, andtokscale clientsprints a zero message count for both, even though both clients scan and submit fine.Cause
MiMoCodeandFxboth declareparse_local: trueindefine_clients!(crates/tokscale-core/src/clients.rs), and the submit path parses both: MiMo Code fromscan_result.micode_dbsatcrates/tokscale-core/src/lib.rs:2047, fx fromscan_result.get(ClientId::Fx)atcrates/tokscale-core/src/lib.rs:3133.parse_local_clients(same file, from line 5134) has a block for every other locally parsed client and none for these two, so it returns no rows and sets no count for either.Everything built on
parse_local_clientstherefore sees nothing.tokscale reportbuilds its whole table from the wiki DB, andpopulate_wiki_from_sessions(crates/tokscale-cli/src/commands/report.rs:114) fills that DB fromparse_local_clientsalone, so a MiMo Code or fx session is never recorded.run_clients_command(crates/tokscale-cli/src/main.rs:4266) asks for everyparse_local()client and then printsparsed.counts.get(client), so both rows read 0 no matter how much local data exists.This is the mirror of #986, where opencodereview was parsed locally but missing from the submit path.
Fix
MiMo Code goes in after the OpenCode block and reads
scan_result.micode_dbs, deduplicating on the payload message id through the existingshould_keep_deduped_messagehelper. That part matters: MiMo writes channel-suffixed databases, so one session can appear in bothmimocode.dbandmimocode-<channel>.db, and the submit lane already collapses them the same way.fx goes in after the Mux block and reads
scan_result.get(ClientId::Fx). Session titles are deliberately not applied here, becauseParsedMessagehas no title field for the sharedsessions/index.jsonlookup to write to.Both count through
summed_parsed_message_countrather than the row count, so an fx row contributes itsrequest_count.Verification
The new test writes a
~/.local/share/mimocode/mimocode.dbwith one assistant row and a~/.fx/sessions/sess-1/usage-v2.jsonsnapshot, then asserts both reachparse_local_clientswith their token buckets and client counts.With the test applied to an otherwise unmodified tree:
With the fix:
The same suite on
mainbefore the change was 2108 passed, 0 failed, so the only difference is the new test. Ran on macOS 15 with rustc 1.95.0. The test uses only tempdirs and fixed literals, with no clock, locale or timezone dependence, so it behaves the same on the Windows leg.Summary by cubic
Fixes the local report path so MiMo Code and fx usage now appears in
tokscale reportandtokscale clientsinstead of always showing zero, and preserves provider-reported dates and costs so the figures are correct.parse_local_clientsnow parses MiMo Code fromscan_result.micode_dbs, deduplicating by payload message id since one session can appear in bothmimocode.dbandmimocode-<channel>.db, and fx from per-session usage snapshots.year/since/untilfilters work on the local lane.ParsedMessagenow carriescostandcost_source;compute_msg_costuses a provider-reported cost as-is, so a token-free fx session records its snapshot cost instead of $0.00.request_count.Written for commit 8891eb3. Summary will update on new commits.