Skip to content

Refactor generate tests to use assertion helper#3047

Merged
koxudaxi merged 7 commits intomainfrom
refactor/generate-tests-assertion-helper
Mar 10, 2026
Merged

Refactor generate tests to use assertion helper#3047
koxudaxi merged 7 commits intomainfrom
refactor/generate-tests-assertion-helper

Conversation

@koxudaxi
Copy link
Copy Markdown
Owner

@koxudaxi koxudaxi commented Mar 10, 2026

Summary

  • add a small helper for file-based generate() integration tests
  • use the helper in the JSON Schema generate() API tests to remove repeated assertion boilerplate
  • keep the existing relative-path coverage while centralizing the pattern

Testing

  • uv run pytest tests/main/jsonschema/test_main_jsonschema.py -k 'test_main_generate or test_main_generate_non_pydantic_output or test_main_generate_pydantic_v2'

Summary by CodeRabbit

  • Tests
    • Added a centralized test helper to standardize generation runs and content assertions, reducing boilerplate and improving consistency.
    • Updated many tests to use the new helper and a unified output fixture, streamlining test signatures and making generated-code validation more uniform across suites.
    • Maintained existing observable behaviors while consolidating assertion patterns for easier maintenance.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Mar 10, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Added a test helper run_generate_file_and_assert to tests/main/conftest.py (duplicate definition present), exported generate and InputFileType for tests, and refactored many tests to use the helper instead of calling generate(...) directly; several fixtures changed from tmp_path to output_file/output_path.

Changes

Cohort / File(s) Summary
Test Helper Infrastructure
tests/main/conftest.py
Added run_generate_file_and_assert(...) (duplicated definition), imported and re-exported generate and InputFileType, and added Any import for new signature types. Helper centralizes input/output handling, expected-file inference, optional transform, and passes through generate kwargs.
JSON Schema Tests
tests/main/jsonschema/test_main_jsonschema.py
Replaced numerous direct generate(...) calls with run_generate_file_and_assert(...); updated many test signatures from tmp_path: Path to output_file: Path; adjusted call arguments to input_path/output_path and unified assertion parameters (assert_func, expected_file, transform).
General Tests
tests/main/test_main_general.py
Switched tests (e.g., test_frozen_dataclasses, test_class_decorators) from direct generate(...) usage to run_generate_file_and_assert(...); updated fixtures from tmp_path to output_file and wired expected-file/assertion arguments.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Poem

🐇 I hopped through fixtures, tidy and fleet,

One helper calls, and tests all meet.
Inputs, outputs, neatly aligned,
Expected files now easy to find.
🥕

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: refactoring tests to use a new assertion helper function.
Docstring Coverage ✅ Passed Docstring coverage is 91.43% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch refactor/generate-tests-assertion-helper

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 10, 2026

@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented Mar 10, 2026

Merging this PR will not alter performance

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

✅ 11 untouched benchmarks
⏩ 98 skipped benchmarks1


Comparing refactor/generate-tests-assertion-helper (af16f88) with main (c48bdb2)

Open in CodSpeed

Footnotes

  1. 98 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@koxudaxi koxudaxi closed this Mar 10, 2026
@koxudaxi koxudaxi reopened this Mar 10, 2026
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
tests/main/conftest.py (1)

273-283: Extract expected-file inference into one private helper.

This block now duplicates the same caller-name inference already in run_main_and_assert() at Lines 438-449. Pulling it into a shared helper will keep the test_main_/test_ convention from drifting between the CLI and generate() test paths.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/main/conftest.py` around lines 273 - 283, Extract the caller-name
inference logic into a single private helper (e.g.
_infer_expected_file_from_caller) and replace the duplicated blocks in
tests/main/conftest.py: the current inline block and the similar code in
run_main_and_assert() (lines around 438-449). The helper should use
inspect.currentframe() to get the caller frame, assert frame and frame.f_back
are not None, read func_name = frame.f_back.f_code.co_name, del frame, strip the
prefixes ("test_main_", "test_") exactly as the current loop does, and return
f"{func_name}.py" so both call sites simply call
_infer_expected_file_from_caller() when expected_file is None to preserve
identical behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@tests/main/conftest.py`:
- Around line 273-283: Extract the caller-name inference logic into a single
private helper (e.g. _infer_expected_file_from_caller) and replace the
duplicated blocks in tests/main/conftest.py: the current inline block and the
similar code in run_main_and_assert() (lines around 438-449). The helper should
use inspect.currentframe() to get the caller frame, assert frame and
frame.f_back are not None, read func_name = frame.f_back.f_code.co_name, del
frame, strip the prefixes ("test_main_", "test_") exactly as the current loop
does, and return f"{func_name}.py" so both call sites simply call
_infer_expected_file_from_caller() when expected_file is None to preserve
identical behavior.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 89747c0e-07fb-4639-a82e-397505753a56

📥 Commits

Reviewing files that changed from the base of the PR and between f3f3912 and f15e364.

📒 Files selected for processing (2)
  • tests/main/conftest.py
  • tests/main/jsonschema/test_main_jsonschema.py

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@tests/main/jsonschema/test_main_jsonschema.py`:
- Around line 1654-1662: The test test_main_generate_relative_input_path fails
when pytest cwd is not an ancestor of JSON_SCHEMA_DATA_PATH because it uses
.relative_to(Path.cwd()); replace that with a cwd-independent relative path
computed via os.path.relpath, e.g. build the input_path with
Path(os.path.relpath(JSON_SCHEMA_DATA_PATH / "person.json")), and pass that into
run_generate_file_and_assert (references:
test_main_generate_relative_input_path, JSON_SCHEMA_DATA_PATH,
run_generate_file_and_assert).

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 83927274-25fa-46f8-81a3-4c97a8845f8b

📥 Commits

Reviewing files that changed from the base of the PR and between a8349bf and 5c95785.

📒 Files selected for processing (1)
  • tests/main/jsonschema/test_main_jsonschema.py

Comment thread tests/main/jsonschema/test_main_jsonschema.py
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
tests/main/conftest.py (1)

273-281: Extract expected-file inference into a shared helper.

run_generate_file_and_assert() now duplicates the caller-name inspection already used in run_main_and_assert(), and the prefix stripping has already drifted slightly between the two implementations. Pull this into one private helper so both code paths keep the same default expected-file convention.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/main/conftest.py` around lines 273 - 281, Extract the caller-name
inspection and prefix-stripping logic that builds expected_file into a single
private helper (e.g., _infer_expected_file_from_caller) and have both
run_generate_file_and_assert and run_main_and_assert call it; move the code that
uses inspect.currentframe(), checks f_back, reads f_code.co_name, deletes the
frame, and removes prefixes ("test_main_", "test_") into that helper so both
functions share identical behavior and avoid duplication or drift.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@tests/main/conftest.py`:
- Around line 266-270: The test currently forwards input_file_type
unconditionally into generate(), which overrides GenerateConfig.input_file_type
default (InputFileType.Auto) with None and breaks enum checks (e.g.,
InputFileType.Auto / InputFileType.GraphQL); change the call site in
tests/main/conftest.py so you build generate kwargs conditionally and only
include input_file_type when input_file_type is not None (keep passing input_
and output_path as before) so generate() can use its default when the test
omitted input_file_type.

In `@tests/main/jsonschema/test_main_jsonschema.py`:
- Around line 1666-1677: The test
test_main_generate_external_absolute_input_path must ensure the test file is
truly outside the repository root: instead of using the pytest tmp_path, create
an absolute temporary directory under the system temp (e.g. via
tempfile.mkdtemp() or Path(tempfile.gettempdir()) / unique name), write the
schema file there (using JSON_SCHEMA_DATA_PATH to copy contents), set input_path
to that external path, and add an explicit check that input_path.resolve() is
not inside Path.cwd() (e.g. assert not
input_path.resolve().is_relative_to(Path.cwd()) or compare parents) before
calling run_generate_file_and_assert; update references to input_path, tmp_path,
JSON_SCHEMA_DATA_PATH, and run_generate_file_and_assert accordingly.

---

Nitpick comments:
In `@tests/main/conftest.py`:
- Around line 273-281: Extract the caller-name inspection and prefix-stripping
logic that builds expected_file into a single private helper (e.g.,
_infer_expected_file_from_caller) and have both run_generate_file_and_assert and
run_main_and_assert call it; move the code that uses inspect.currentframe(),
checks f_back, reads f_code.co_name, deletes the frame, and removes prefixes
("test_main_", "test_") into that helper so both functions share identical
behavior and avoid duplication or drift.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 08154532-3ab0-4fd0-826c-dab76e2dedd7

📥 Commits

Reviewing files that changed from the base of the PR and between 5c95785 and 1c6a5d5.

📒 Files selected for processing (2)
  • tests/main/conftest.py
  • tests/main/jsonschema/test_main_jsonschema.py

Comment thread tests/main/conftest.py Outdated
Comment thread tests/main/jsonschema/test_main_jsonschema.py Outdated
@koxudaxi koxudaxi merged commit 3624533 into main Mar 10, 2026
35 checks passed
@koxudaxi koxudaxi deleted the refactor/generate-tests-assertion-helper branch March 10, 2026 20:30
@github-actions
Copy link
Copy Markdown
Contributor

Breaking Change Analysis

Result: No breaking changes detected

Reasoning: This PR only refactors internal test code by adding a helper function run_generate_file_and_assert() in tests/main/conftest.py and updating tests in tests/main/jsonschema/test_main_jsonschema.py and tests/main/test_main_general.py to use it. No production code, public APIs, CLI options, code generation output, templates, or default behaviors were changed. Users of datamodel-code-generator are not affected in any way by this internal test refactoring.


This analysis was performed by Claude Code Action

@github-actions
Copy link
Copy Markdown
Contributor

🎉 Released in 0.55.0

This PR is now available in the latest release. See the release notes for details.

@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 10, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (c48bdb2) to head (af16f88).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##              main     #3047   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           86        86           
  Lines        18091     18031   -60     
  Branches      2108      2112    +4     
=========================================
- Hits         18091     18031   -60     
Flag Coverage Δ
unittests 100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant