Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: agentevals-dev/agentevals
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v0.10.0
Choose a base ref
...
head repository: agentevals-dev/agentevals
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v0.11.0
Choose a head ref
  • 4 commits
  • 195 files changed
  • 3 contributors

Commits on Oct 13, 2026

  1. feat: v1alpha1 protobuf contract + native gRPC (#251)

    * chore: cleanup
    
    * feat(api): add the v1alpha1 protobuf contract with generated Go, Python and TypeScript code
    
    * feat(api): build session and turn views with token presence and coded gaps
    
    * feat(api): serve the v1alpha1 gRPC API with session watch, runs and evaluation, plus redaction and storage fixes
    
    * feat(ui): render the live view and annotation queue from server session views
    
    * chore(ui): make eslint clean and run it in CI
    
    * feat(chart): ship the gRPC API port in the image and chart, tag the Go module on release, and document the API
    
    * test(api): cover the v1alpha1 gRPC services, converters, views, redaction, self tracing, watchers and Postgres repos
    
    * fix(ui): commit the generated OTLP logs module hidden by the logs gitignore rule, and fail proto-check on ignored generated files
    krisztianfekete authored Oct 13, 2026
    Configuration menu
    Copy the full SHA
    bb723b9 View commit details
    Browse the repository at this point in the history
  2. Give rubric_based_* metrics a public rubric path (#132) (#229)

    * Give rubric_based_* metrics a public rubric path (#132)
    
    The rubric-based metrics have been reachable only through the internal
    build_eval_metric(..., rubrics=...) call: the metric builder accepted
    rubrics, the eval-set format documented them on cases and invocations,
    and nothing on the config, CLI or API path passed any of them through,
    so the metrics were listed as not working.
    
    Rubrics can now be declared on a builtin evaluator in the eval config,
    as {id, text} mappings or plain strings with positional ids, with
    duplicate ids refused. At run time the runner matches the eval case,
    not only its conversation, and the matched case's rubrics and each
    expected invocation's rubrics are attached to the actual invocations
    the way ADK's own eval service attaches them, with the metric's rubric
    type given to rubrics that have none, since ADK applies invocation
    rubrics by type. A rubric metric with no rubric from the config, the
    case or an invocation reports an error before any judge call rather
    than an empty evaluation. Its result carries per-rubric scores per
    invocation and overall in details. The API lists the rubric metrics as
    working.
    
    * docs: a case-level rubric is judged once per invocation
    
    * fix(rubrics): retain ADK parity on the public evaluation path
    
    * fix(rubrics): keep conversation metric unavailable until integrated
    
    * test(rubrics): cover clarification and completion through the public path
    
    * refactor(rubrics): one rubric metric constant, rubrics on the v1alpha1 contract
    
    ---------
    
    Co-authored-by: krisztianfekete <[email protected]>
    erauner12 and krisztianfekete authored Oct 13, 2026
    Configuration menu
    Copy the full SHA
    0e639bc View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    0d06413 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    38fdb9b View commit details
    Browse the repository at this point in the history
Loading