Skip to content

fix: order same-second human feedback rows newest first in the Python store - #1458

Merged
jayscambler merged 1 commit into
greyhaven-ai:mainfrom
MaxFreedomPollard:fix/human-feedback-id-tiebreak
Oct 23, 2026
Merged

jayscambler merged 1 commit into
greyhaven-ai:mainfrom
MaxFreedomPollard:fix/human-feedback-id-tiebreak

Conversation

@MaxFreedomPollard

Copy link
Copy Markdown
Contributor

Summary

  • get_human_feedback and get_calibration_examples in autocontext/src/autocontext/storage/sqlite_store_human_feedback.py now order by created_at DESC, id DESC instead of created_at DESC alone.
  • This is the Python half of fix: human feedback queries returned the oldest rows, not the newest #1360. That PR fixed the same two queries in the TypeScript store and recorded that "the Python side has no counterpart to this SQLite store". The Python store does have both queries, with the same ORDER BY created_at DESC LIMIT ? and the same one-second created_at default (autocontext/migrations/006_human_feedback.sql:8, and the pip bootstrap schema in storage/bootstrap_schema.py).

Reproduction on main: 12 rows inserted through insert_human_feedback in one second, all sharing one created_at:

get_human_feedback("s", limit=10)       -> ids [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
get_calibration_examples("s", limit=5)  -> ids [1, 2, 3, 4, 5]

With the tiebreak:

get_human_feedback("s", limit=10)       -> ids [12, 11, 10, 9, 8, 7, 6, 5, 4, 3]
get_calibration_examples("s", limit=5)  -> ids [12, 11, 10, 9, 8]

SQLite returns the tied rows in insert order, so the newest rows of a same-second burst come last, and once the burst reaches the LIMIT they are cut off: a caller asking for the most recent rows gets the oldest ones. In Python that affects:

  • the judge calibration anchors (limit=5) passed to evaluate_output and run_improvement_loop (MCP tools in mcp/agent_task_tools.py and mcp/knowledge_tools.py) and to live rubric calibration in execution/agent_task_completion.py
  • get_feedback (the MCP autocontext_get_feedback tool), and the lessons that export_agent_task_skill copies into an exported skill package (limit=10)

id is the AUTOINCREMENT rowid, so it is monotonic in insert order and breaks the tie the way DESC already intended.

Surfaces Touched

Verification

  • cd autocontext && uv run ruff check src tests: clean
  • cd autocontext && uv run mypy src: no issues in 950 source files
  • cd autocontext && uv run pytest: two new regression tests in tests/test_human_feedback.py. On main they fail with assert [1, 2, 3, 4, 5, 6, ...] == [12, 11, 10, 9, 8, 7, ...] and assert [1, 2, 3, 4, 5] == [8, 7, 6, 5, 4]; with the fix they pass. The tests pin every row's created_at to one value after inserting, so they reproduce a same-second burst without depending on timing. The storage, MCP and calibration test files also pass on Python 3.11, the CI version (227 passed, 1 skipped). Full suite with -m "not live" on Python 3.12: 11,339 passed, two more than on main (the new tests). The same 166 tests fail on main and on this branch in my sandboxed environment, mostly with IsolationUnavailableError: local child isolation is unavailable; none touch human feedback.
  • cd ts && npm run lint: not applicable, no TypeScript changes
  • cd ts && npm test: not applicable, no TypeScript changes
  • uv run python ../scripts/check_markdown_links.py: passed

Docs And Release Impact

  • no user-facing docs changes needed
  • updated relevant README/docs/examples
  • updated CHANGELOG.md (Unreleased, Fixed)
  • updated version metadata if this is part of a release

Notes

🤖 Generated with Claude Code

…newest

SQLiteHumanFeedbackStoreMixin.get_human_feedback and get_calibration_examples
order by `created_at DESC` with no tiebreak, and human_feedback.created_at
defaults to datetime('now'), which has one-second resolution
(migrations/006_human_feedback.sql:8, and the same default in
storage/bootstrap_schema.py). Rows written in the same second share one sort
key and SQLite returns them in insert order, so the newest rows of a
same-second burst come last, and once the burst reaches the LIMIT they are cut
off: a caller asking for the newest rows gets the oldest ones.

With 12 rows inserted in one second on main:

  get_human_feedback(limit=10)      -> ids 1..10, not 12..3
  get_calibration_examples(limit=5) -> ids 1..5, not 12..8

#1360 fixed the same two queries in the TypeScript store and recorded that the
Python side had no counterpart, but it has both queries with the same bug.
This adds the same `id DESC` tiebreak. id is the AUTOINCREMENT rowid, so it is
monotonic in insert order.

The regression tests pin every row's created_at to one value after inserting,
so they reproduce a same-second burst without depending on timing. Both fail
on main and pass with the fix.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

@jayscambler jayscambler left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head cc9c9ba, including both parameterized queries, schema/bootstrap compatibility, calibration callers and deterministic regression tests. The id DESC tiebreak matches newest-first semantics for tied timestamps, preserves scenario/calibration filters, and changes no dependency, workflow, API or schema. Independent validation: 130 tests passed across human feedback, MCP agent tasks, rubric calibration/drift, SQLite store and bootstrap; full Ruff passed; mypy passed for all 950 source files. GitHub dependency comparison against main reports zero added dependencies. Code review approved; merge remains subject to all required GitHub Actions checks, including dependency-security.

@jayscambler
jayscambler merged commit e720b29 into greyhaven-ai:main Oct 23, 2026
31 of 32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants