Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: agentevals-dev/agentevals
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v0.9.9
Choose a base ref
...
head repository: agentevals-dev/agentevals
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v0.9.10
Choose a head ref
  • 5 commits
  • 14 files changed
  • 3 contributors

Commits on Aug 31, 2026

  1. fix(otel): share one AnyValue decoder across all OTLP paths (#187)

    * fix(otel): share one AnyValue decoder across all OTLP paths
    
    Three AnyValue decoders had drifted apart. extraction's
    flatten_otlp_attributes silently dropped array/kvlist/bytes attributes,
    and the OTLP JSON loader json.dumps()'d the raw proto wrapper instead of
    decoding it. gen_ai.response.finish_reasons therefore surfaced as
    '{"values": [{"stringValue": "stop"}]}' on one path and vanished on
    another, where both should yield ["stop"].
    
    Move the already-correct recursive decoder out of api/otlp_processing.py
    into a dependency-free module and route all three call sites through it.
    The new module imports only the standard library, so extraction,
    loader.otlp and api.otlp_processing can all share it without creating an
    import cycle.
    
    bytesValue is returned unchanged: MessageToDict base64-encodes protobuf
    bytes fields, so callers already receive a str today. Decoding it here
    would change existing behaviour, which is out of scope for this fix.
    
    Adds coverage for array/kvlist/bytes attributes, which previously had
    none on either path.
    
    Fixes #173
    
    * fix(otel): decode AnyValue when promoting span event attributes
    
    Strands stores gen_ai.input.messages in span events, and newer GenAI
    
    semconv makes messages a complex array, so the stringValue-only promotion
    
    in the OTLP loader silently dropped anything that was not a plain string.
    
    Route it through the shared decoder like the other paths.
    
    * fix(otel): read session_name and eval_set_id as strings only
    
    flatten_otlp_attributes used to guarantee a scalar or nothing. Now that it
    
    delegates to the shared AnyValue decoder it can also return lists and dicts,
    
    which are unhashable and raise TypeError where these values are used as dict
    
    keys (_active_session_for_name in ws_server.py and otlp_processing.py).
    
    Add a string-only accessor and use it for both fields, matching what
    
    _extract_conversation_id already did; that function now shares the same
    
    implementation. The decoded value stays available in resource_attrs.
    
    * fix(otel): narrow spec-string attributes once at the decode boundary
    
    The per-field accessor only covered the two dict-key sites we had found;
    
    flatten_otlp_attributes still handed containers to every other consumer via
    
    span.tags, so the same unhashable hazard survived elsewhere.
    
    Move the guard into decode_attributes, keyed by SPEC_STRING_ATTRS in
    
    trace_attrs. Attributes the spec types as strings come back str-or-absent;
    
    a container in one of those slots is dropped with a warning rather than
    
    JSON-dumped, since dumping would put the literal blob back into user-visible
    
    output. Attributes the spec types as arrays or structured values keep their
    
    decoded containers, which is what #173 fixes.
    
    The registry is hand-maintained: opentelemetry-semantic-conventions exposes
    
    constants and prose docstrings only, so attribute value types are not
    
    machine-readable and the set cannot be derived from the package.
    
    _extract_string_attribute is gone and _extract_conversation_id is back to its
    
    original form. Tests now assert through extract_extended_model_info_from_attrs
    
    and extract_user_text_from_attrs, the layers #173's symptom actually names,
    
    instead of stopping at the decoder and span.tags.
    
    * fix(otel): decode containers on an opt-in allowlist instead of a denylist
    
    SPEC_STRING_ATTRS listed the attributes to narrow, so a missing entry let an
    
    unhashable value reach a dict key on an unauthenticated receiver port. It
    
    missed the whole gcp.vertex.agent.* namespace.
    
    Replace it with SPEC_CONTAINER_ATTRS: the seven keys the GenAI semconv types as
    
    arrays or structured values. Those decode to native containers; every other key
    
    decodes to a scalar or is dropped, which is what extraction.py did before the
    
    decoder was shared. A missing entry now drops one value rather than crashing
    
    ingestion, and attributes neither of us has thought about are safe by
    
    construction rather than by enumeration.
    
    decode_attribute() carries the rule so the OTLP-array path and the span-event
    
    promotion path share one implementation. bytesValue now survives as the base64
    
    string it already was, which #173 asks for and is hashable.
    
    The default for unlisted container values is tracked in #208; the nested-dict
    
    loader path that bypasses this decoder entirely is #207.
    Leroyyyyyyyyy authored Aug 31, 2026
    Configuration menu
    Copy the full SHA
    46ec211 View commit details
    Browse the repository at this point in the history

Commits on Sep 1, 2026

  1. chore(deps): bump the python group with 2 updates (#211)

    Bumps the python group with 2 updates: [click](https://github.com/pallets/click) and [langchain](https://github.com/langchain-ai/langchain).
    
    
    Updates `click` from 8.4.2 to 8.5.0
    - [Release notes](https://github.com/pallets/click/releases)
    - [Changelog](https://github.com/pallets/click/blob/main/CHANGES.md)
    - [Commits](pallets/click@8.4.2...8.5.0)
    
    Updates `langchain` from 1.3.16 to 1.3.17
    - [Release notes](https://github.com/langchain-ai/langchain/releases)
    - [Commits](langchain-ai/langchain@langchain==1.3.16...langchain==1.3.17)
    
    ---
    updated-dependencies:
    - dependency-name: click
      dependency-version: 8.5.0
      dependency-type: direct:production
      update-type: version-update:semver-minor
      dependency-group: python
    - dependency-name: langchain
      dependency-version: 1.3.17
      dependency-type: direct:development
      update-type: version-update:semver-patch
      dependency-group: python
    ...
    
    Signed-off-by: dependabot[bot] <[email protected]>
    Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
    dependabot[bot] authored Sep 1, 2026
    Configuration menu
    Copy the full SHA
    7451aed View commit details
    Browse the repository at this point in the history
  2. chore(deps): bump the npm group across 1 directory with 7 updates (#210)

    Bumps the npm group with 7 updates in the /ui directory:
    
    | Package | From | To |
    | --- | --- | --- |
    | [framer-motion](https://github.com/motiondivision/motion) | `13.1.0` | `13.1.1` |
    | [lucide-react](https://github.com/lucide-icons/lucide/tree/HEAD/packages/lucide-react) | `1.31.0` | `1.34.0` |
    | [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) | `26.2.0` | `26.4.0` |
    | [@types/react-dom](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/react-dom) | `19.2.4` | `19.2.5` |
    | [eslint](https://github.com/eslint/eslint) | `10.8.1` | `10.9.1` |
    | [eslint-plugin-react-refresh](https://github.com/ArnaudBarre/eslint-plugin-react-refresh) | `0.5.4` | `0.5.5` |
    | [typescript-eslint](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/typescript-eslint) | `8.67.0` | `8.68.0` |
    
    
    
    Updates `framer-motion` from 13.1.0 to 13.1.1
    - [Changelog](https://github.com/motiondivision/motion/blob/main/CHANGELOG.md)
    - [Commits](motiondivision/motion@v13.1.0...v13.1.1)
    
    Updates `lucide-react` from 1.31.0 to 1.34.0
    - [Release notes](https://github.com/lucide-icons/lucide/releases)
    - [Commits](https://github.com/lucide-icons/lucide/commits/1.34.0/packages/lucide-react)
    
    Updates `@types/node` from 26.2.0 to 26.4.0
    - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
    - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)
    
    Updates `@types/react-dom` from 19.2.4 to 19.2.5
    - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
    - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/react-dom)
    
    Updates `eslint` from 10.8.1 to 10.9.1
    - [Release notes](https://github.com/eslint/eslint/releases)
    - [Commits](eslint/eslint@v10.8.1...v10.9.1)
    
    Updates `eslint-plugin-react-refresh` from 0.5.4 to 0.5.5
    - [Release notes](https://github.com/ArnaudBarre/eslint-plugin-react-refresh/releases)
    - [Changelog](https://github.com/ArnaudBarre/eslint-plugin-react-refresh/blob/main/CHANGELOG.md)
    - [Commits](ArnaudBarre/eslint-plugin-react-refresh@v0.5.4...v0.5.5)
    
    Updates `typescript-eslint` from 8.67.0 to 8.68.0
    - [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
    - [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/typescript-eslint/CHANGELOG.md)
    - [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.68.0/packages/typescript-eslint)
    
    ---
    updated-dependencies:
    - dependency-name: "@types/node"
      dependency-version: 26.3.0
      dependency-type: direct:development
      update-type: version-update:semver-minor
      dependency-group: npm
    - dependency-name: "@types/react-dom"
      dependency-version: 19.2.5
      dependency-type: direct:development
      update-type: version-update:semver-patch
      dependency-group: npm
    - dependency-name: eslint
      dependency-version: 10.9.1
      dependency-type: direct:development
      update-type: version-update:semver-minor
      dependency-group: npm
    - dependency-name: eslint-plugin-react-refresh
      dependency-version: 0.5.5
      dependency-type: direct:development
      update-type: version-update:semver-patch
      dependency-group: npm
    - dependency-name: framer-motion
      dependency-version: 13.1.1
      dependency-type: direct:production
      update-type: version-update:semver-patch
      dependency-group: npm
    - dependency-name: lucide-react
      dependency-version: 1.34.0
      dependency-type: direct:production
      update-type: version-update:semver-minor
      dependency-group: npm
    - dependency-name: typescript-eslint
      dependency-version: 8.68.0
      dependency-type: direct:development
      update-type: version-update:semver-minor
      dependency-group: npm
    ...
    
    Signed-off-by: dependabot[bot] <[email protected]>
    Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
    dependabot[bot] authored Sep 1, 2026
    Configuration menu
    Copy the full SHA
    8fb3a58 View commit details
    Browse the repository at this point in the history
  3. chore(deps): bump node (#209)

    Bumps the docker-base-images group with 1 update in the / directory: node.
    
    
    Updates `node` from 25-bookworm-slim to 26-bookworm-slim
    
    ---
    updated-dependencies:
    - dependency-name: node
      dependency-version: 26-bookworm-slim
      dependency-type: direct:production
      dependency-group: docker-base-images
    ...
    
    Signed-off-by: dependabot[bot] <[email protected]>
    Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
    dependabot[bot] authored Sep 1, 2026
    Configuration menu
    Copy the full SHA
    cb84267 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    9047b96 View commit details
    Browse the repository at this point in the history
Loading