Preserve Python types (Set, Mapping, Sequence) in --input-model by koxudaxi · Pull Request #2837 · koxudaxi/datamodel-code-generator

koxudaxi · 2025-12-28T14:34:11Z

Summary by CodeRabbit

New Features
- Generated models now preserve Python complex types (Set, FrozenSet, Mapping, Sequence) from JSON Schema definitions, improving code fidelity and preventing type information loss during conversion.
Tests
- Added comprehensive test coverage for type preservation across multiple output formats, nested models, and recursive structures.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

coderabbitai · 2025-12-28T14:34:18Z

Warning

Rate limit exceeded

@koxudaxi has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 9 minutes and 24 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between 632e736 and d01a0b1.

📒 Files selected for processing (6)

src/datamodel_code_generator/__main__.py
src/datamodel_code_generator/parser/jsonschema.py
src/datamodel_code_generator/types.py
tests/data/python/input_model/dataclass_models.py
tests/data/python/input_model/pydantic_models.py
tests/test_input_model.py

📝 Walkthrough

Walkthrough

This change introduces systematic preservation of Python typing semantics (Set, FrozenSet, Mapping, Sequence) across JSON Schema conversion by annotating schemas with x-python-type metadata during model loading, then propagating and applying these flags during code generation to restore accurate type hints in output models.

Changes

Cohort / File(s)	Summary
Core type preservation logic `src/datamodel_code_generator/__main__.py`	Adds utilities to serialize Python types into x-python-type strings and augments JSON Schema with preserved type metadata via `_serialize_python_type` and `_add_python_type_info` helpers when loading from Pydantic/dataclass/TypedDict sources
Schema parsing integration `src/datamodel_code_generator/parser/jsonschema.py`	Introduces `_get_python_type_flags` to extract container-type overrides from x-python-type metadata in JSON Schema; propagates parent_obj context through parsing methods to apply dynamic is_dict/is_list/is_set/is_mapping/is_sequence flags
Type system extensions `src/datamodel_code_generator/types.py`	Adds three new boolean flags (is_frozen_set, is_mapping, is_sequence) to Config and DataType; updates import generation and type_hint composition to prioritize frozen sets, sequences, and mappings in output
Test input models `tests/data/python/input_model/dataclass_models.py`, `tests/data/python/input_model/pydantic_models.py`	Introduces new test data classes and Pydantic models (DataclassWithPythonTypes, Tag, ModelWithPythonTypes, RecursiveNode) with fields exercising Set, FrozenSet, Mapping, Sequence, and nested/optional variants
Integration tests `tests/test_input_model.py`	Adds 9 test functions verifying x-python-type preservation and correct type generation for TypedDict, dataclass, and recursive model targets

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant __main__
    participant JsonSchema
    participant jsonschema parser
    participant types
    participant Output

    User->>__main__: Load model (Pydantic/dataclass)
    __main__->>__main__: Generate JSON Schema
    __main__->>__main__: _serialize_python_type() for each property
    __main__->>JsonSchema: Annotate with x-python-type metadata<br/>(Set→"set", Mapping→"Mapping", etc.)
    JsonSchema-->>__main__: Augmented schema with x-python-type
    
    __main__->>jsonschema parser: Parse augmented schema
    jsonschema parser->>jsonschema parser: _get_python_type_flags(parent_obj)<br/>Extract x-python-type from extras
    jsonschema parser->>jsonschema parser: Apply is_set/is_mapping/is_sequence<br/>to data_type construction
    jsonschema parser-->>types: Property data_type<br/>with preserved flags
    
    types->>types: type_hint(): Prefer FrozenSet,<br/>Mapping, Sequence based on flags
    types-->>Output: Generate code with<br/>correct Python types

    Output-->>User: Model with preserved types<br/>(Set[str], Mapping[str, int], etc.)

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Add --input-model option for Pydantic models and dicts #2804: Introduces foundational input-model plumbing in __main__.py and schema loading infrastructure that this PR extends to add x-python-type preservation and schema augmentation.

Suggested labels

breaking-change-analyzed

Poem

🐰 A rabbit hops through JSON schemes,
Preserving types—fulfilling dreams!
Set, Mapping, Sequence—no more lost,
x-python-type: worth every cost! ✨

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title directly and clearly describes the main objective of the pull request: preserving Python types (Set, Mapping, Sequence) when using the --input-model feature.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

github-actions · 2025-12-28T14:34:54Z

📚 Docs Preview: https://pr-2837.datamodel-code-generator.pages.dev

codspeed-hq · 2025-12-28T14:37:18Z

CodSpeed Performance Report

Merging #2837 will not alter performance

_{Comparing feature/preserve-python-types-in-input-model (d01a0b1) with main (fca31fe)}

⚠️

Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

Summary

✅ 11 untouched
⏩ 98 skipped¹

98 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

codecov · 2025-12-28T14:37:21Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.51%. Comparing base (e4394af) to head (d01a0b1).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2837      +/-   ##
==========================================
- Coverage   99.52%   99.51%   -0.01%     
==========================================
  Files          90       90              
  Lines       14092    14245     +153     
  Branches     1674     1697      +23     
==========================================
+ Hits        14025    14176     +151     
- Misses         36       37       +1     
- Partials       31       32       +1

Flag	Coverage Δ
unittests	`99.51% <100.00%> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

coderabbitai

Actionable comments posted: 0

🧹 Nitpick comments (3)

src/datamodel_code_generator/types.py (1)

336-347: New container flags are wired correctly, but base_type_hint ignores them

The additions of is_frozen_set, is_mapping, and is_sequence are cleanly integrated into imports and type_hint, so emitted hints and imports for FrozenSet, Mapping, and Sequence should be correct.

One behavioral gap: base_type_hint still only considers is_list, is_set, and is_dict. For types marked with the new flags, base_type_hint will effectively return just the inner type (e.g. int instead of Mapping[str, int] or str instead of Sequence[str]). If base_type_hint is ever used downstream for these preserved containers (beyond the current regex/RootModel use cases), this could be surprising.

Consider mirroring the type_hint container wrapping in base_type_hint for is_frozen_set, is_mapping, and is_sequence to keep the two accessors consistent, or explicitly documenting that base_type_hint may drop these container wrappers.

Also applies to: 489-533, 579-681

src/datamodel_code_generator/parser/jsonschema.py (1)

1226-1234: Clean up unused noqa directives flagged by Ruff

Ruff reports unused noqa directives here:

Line 1226: # noqa: PLR6301 on _get_python_type_flags

Line 2458: # noqa: PLR0912 on parse_property_names

Line 2462: # noqa: FBT001 on additional_properties parameter

Given these rules are not enabled in this project’s configuration, the noqa comments just generate RUF100 noise. Removing them keeps lint output clean without changing behavior.

Also applies to: 2458-2464

src/datamodel_code_generator/__main__.py (1)

603-608: Remove unused noqa pragmas on local imports and global mutation

Ruff’s RUF100 warnings about unused noqa directives apply here:

# noqa: PLC0415 on the local imports inside _init_preserved_type_origins, _serialize_python_type, _find_models_in_type, and _get_type_hints_safe.

# noqa: PLW0603 on the global _PRESERVED_TYPE_ORIGINS line.

Since those rules are not enabled, the noqa annotations are unnecessary and now themselves cause lint noise. Dropping these noqa comments will keep the linter quiet without affecting runtime behavior.

Also applies to: 624-624, 635-635, 690-690, 702-702

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e4394af and 632e736.

📒 Files selected for processing (6)

src/datamodel_code_generator/__main__.py
src/datamodel_code_generator/parser/jsonschema.py
src/datamodel_code_generator/types.py
tests/data/python/input_model/dataclass_models.py
tests/data/python/input_model/pydantic_models.py
tests/test_input_model.py

🧰 Additional context used

🧬 Code graph analysis (2)

src/datamodel_code_generator/parser/jsonschema.py (1)

src/datamodel_code_generator/types.py (1)

DataType (296-805)

src/datamodel_code_generator/types.py (7)

src/datamodel_code_generator/model/typed_dict.py (1)

imports (164-170)

src/datamodel_code_generator/model/pydantic/base_model.py (2)

imports (268-276)

field (92-106)

src/datamodel_code_generator/model/base.py (3)

imports (324-349)

imports (810-815)

field (403-405)

src/datamodel_code_generator/model/pydantic_v2/types.py (1)

imports (62-67)

src/datamodel_code_generator/model/dataclass.py (2)

imports (139-144)

field (147-152)

src/datamodel_code_generator/model/enum.py (1)

imports (119-121)

src/datamodel_code_generator/model/type_alias.py (1)

imports (28-34)

🪛 Ruff (0.14.10)

src/datamodel_code_generator/parser/jsonschema.py

1226-1226: Unused noqa directive (non-enabled: PLR6301)