Skip to content

feat(cli): distinguish YAML parse vs AsyncAPI validation errors, add pluggable parser - #2858

Closed
SarthakB11 wants to merge 8 commits into
ag2ai:mainfrom
SarthakB11:fix/issue-2709
Closed

SarthakB11 wants to merge 8 commits into
ag2ai:mainfrom
SarthakB11:fix/issue-2709

Conversation

@SarthakB11

@SarthakB11 SarthakB11 commented May 6, 2026 •

Copy link
Copy Markdown
Contributor

Description

faststream docs serve currently fails with the same opaque "asyncapi.yaml not supported. Make sure that your schema is valid and schema version supported by FastStream" message regardless of whether the YAML failed to parse, the document validates against neither AsyncAPI version, or PyYAML produced a typed value the schema rejects (the unquoted-scalar quirk #2709 calls out).

This PR splits those into three distinct error paths and exposes a parser hook so users can swap the loader without touching FastStream.

$ faststream docs serve broken.yaml
`broken.yaml` not supported. Make sure that your schema is valid and schema version supported by FastStream

AsyncAPI 3.0.0 validation errors:
1 validation error for ApplicationSchema
servers.development.protocolVersion
  Input should be a valid string [type=string_type, input_value=3.2, input_type=float]

AsyncAPI 2.6.0 validation errors:
1 validation error for ApplicationSchema
asyncapi
  Input should be '2.6.0' [type=literal_error, input_value='3.0.0', input_type=str]

Hint: a string field received a number/bool. YAML parses bare `3.2` as float;
quote it as `'3.2'` or feed the JSON variant of the schema.
$ faststream docs serve totally-broken.yaml
Failed to parse `totally-broken.yaml` with `auto`: ScannerError: mapping values are not allowed here
$ faststream docs serve asyncapi.yaml --yaml-parser ruamel
# uses ruamel.yaml when installed
$ faststream docs serve asyncapi.yaml --yaml-parser my_module:my_loader
# any callable that accepts pathlib.Path and returns a dict

Changes

  • faststream/_internal/cli/utils/yaml_parsers.py (new) — get_yaml_parser registry exposing auto, pyyaml, ruamel, and module:callable resolution. Mirrors imports.py's import_from_string style and reuses it for the dotted-path case. Raises SetupError so callers stay decoupled from the CLI.
  • faststream/_internal/cli/docs.py — uses the registry on the YAML branch; captures ValidationError per AsyncAPI version and surfaces them after the existing "not supported" line; emits a hint when the validation errors look like a YAML-resolved scalar landing in a string field; catches parser-side exceptions with a clear "Failed to parse" message instead of an unhandled traceback.
  • faststream/_internal/cli/dto.py — adds the yaml_parser field on RunArgs so the value flows through the reload supervisor.
  • New --yaml-parser option on faststream docs serve (typer auto-renders this into --help and the docs site).

Note on the unquoted-scalar case

I dug into the specific ! '3.2' example in the issue body. Every standard YAML 1.2 mode (PyYAML default, ruamel.yaml(typ='safe'), ruamel.yaml(typ='rt')) resolves 3.2 to float; only typ='base' keeps it as a string, but that mode also strings every other value (booleans, ints) which the rest of the schema needs. So shipping a different default parser does not, on its own, make AsyncAPI Studio-style YAML validate. The hint in the first example tells the user to quote the value or use the JSON file Studio also exports. A schema-level coercion that accepts int|float|str for protocolVersion would be the bigger win and is worth a separate design discussion.

Fixes #2709.

Type of change

  • Documentation (typos, code examples, or any documentation updates)
  • Bug fix (a non-breaking change that resolves an issue)
  • New feature (a non-breaking change that adds functionality)
  • Breaking change (a fix or feature that would disrupt existing functionality)
  • This change requires a documentation update

Checklist

  • My code adheres to the style guidelines of this project (just lint equivalents — ruff check, ruff format, codespell — all clean on touched files)
  • I have conducted a self-review of my own code
  • I have made the necessary changes to the documentation (the new --yaml-parser flag is exposed via typer's auto-generated --help, which feeds the docs site)
  • My changes do not generate any new warnings
  • I have added tests to validate the effectiveness of my fix or the functionality of my new feature
  • Both new and existing unit tests pass on my local environment for the changed scope (12 new tests across tests/cli/test_yaml_parsers.py and tests/cli/test_asyncapi_docs.py — 6 unit, 5 in-process integration via CliRunner, 1 slow e2e via the faststream_cli subprocess fixture against a real docs server)
  • I have ensured that static analysis tests are passing on the touched files (just static-analysis equivalent — mypy — clean on the touched non-test files)
  • I have included code examples to illustrate the modifications

@Sehat1137

Copy link
Copy Markdown
Collaborator

Thank you for the contribution.

I do not think we should merge this PR in its current form.

The PR tries to address YAML parsing and validation diagnostics, but it also introduces a new public CLI option, a pluggable YAML parser registry, support for arbitrary module:callable parser references, and a change in the default parser selection logic. That is a much larger API and maintenance commitment than the underlying problem appears to require.

The most important concern is that auto prefers ruamel.yaml when it is installed and falls back to PyYAML otherwise. This means the same faststream docs serve <schema.yaml> command may parse the same file differently depending on optional packages present in the user environment. For a CLI command that loads user documentation, the default behavior should be deterministic and should not silently change because an unrelated dependency happens to be installed.

The module:callable extension point is also not something we should add without a broader design discussion. It exposes a new plugin-like execution path from a CLI argument, but the PR does not define a stable parser interface, compatibility guarantees, error model, documentation, or a clear use case that justifies making this part of the public contract.

There is also a mismatch between the stated intent and the implementation. The new module says that PyYAML is the default, while auto actually prefers ruamel.yaml if available. The unquoted-scalar hint is useful, but it is implemented as a broad heuristic over any Pydantic string_type error where the input is float, int, or bool; that can produce a YAML-specific hint for validation failures that are not necessarily caused by the documented protocolVersion: 3.2 case.

I think the useful part of this PR is the idea of improving diagnostics when a YAML schema parses successfully but fails AsyncAPI validation. That should be handled as a smaller, focused change: preserve the existing YAML parser behavior, surface clearer validation errors, add a regression test for the concrete failure case, and avoid introducing a new parser plugin API.

For now, I would prefer to close this PR rather than merge it. If we still want to improve the error message for invalid YAML-based AsyncAPI documents, I would be open to a smaller PR limited to that behavior, with deterministic parsing and regression tests.

Thanks again for working on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: PyYAML cannot correct parse asyncapi.yaml

2 participants