Skip to content

Rework how JSON schema refs get built/"simplified"#6402

Merged
dmontagu merged 3 commits intomainfrom
dmontagu/6369
Jul 4, 2023
Merged

Rework how JSON schema refs get built/"simplified"#6402
dmontagu merged 3 commits intomainfrom
dmontagu/6369

Conversation

@dmontagu
Copy link
Copy Markdown
Contributor

@dmontagu dmontagu commented Jul 3, 2023

This closes #6369, and also addresses various other issues:

  • Fixes a bug where names weren't being simplified fully in the presence of nested models (i.e., you'd get separate Input and Output schemas for an outer model even if the inner model had the same Input and Output schema, now you don't)
  • Fixes a bug where for certain $ref collisions, one model might be remapped and the other not (it was intended that both get renamed to include more detail in the $ref)
  • Makes the names of generated JSON schemas stable across runs even when multiple models have the exact same qualname. Previously we used id(model) to ensure the names were unique, but this would result in different schemas on every run. In this PR I have fixed this so that when there are qualname conflicts, we add __1 to the first encounter with that qualname, __2 to the second, etc., instead of using the id.

However, this PR currently involves some changes to the "ostensibly-public" APIs of GenerateJsonSchema.

@samuelcolvin: @adriangb and I discussed and I think we should make some of the "implementation detail" methods from GenerateJsonSchema private (which I at least partially do in this PR..), and publish something describing in more detail what we intend to treat as public and what we don't from the semver perspective.

In particular, after discovering the issues with the current approach from investigating #6369, I really think we should change the implementation a bit, which is done in the PR I'm about to open, but technically it involves replacing some public methods of GenerateJsonSchema. I think given how advanced this use case is we should probably make this change now.

Happy to discuss more before merging anything, and happy to put in more effort to rework this to not have any theoretically-breaking changes, but I think there's an argument to be made that we never intended to make these methods public and that we are now renaming them to make them private.

Comment thread pydantic/json_schema.py Outdated
Comment on lines +175 to +177
warnings.warn(
'Failed to simplify the JSON schema definitions, this may be a bug in Pydantic', PydanticJsonSchemaWarning
)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like we should just error here. I'd rather have someone report the error so we can fix it with a reproducible example than just warn.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Converted to an error

@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages Bot commented Jul 4, 2023

@dmontagu dmontagu merged commit 2a0f9e2 into main Jul 4, 2023
@dmontagu dmontagu deleted the dmontagu/6369 branch July 4, 2023 16:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Nested models created with create_model with same name results in key error

2 participants