Skip to content

Export every payer transition referenced by exported claim transactions - #1734

Merged
hadleynet merged 1 commit into
synthetichealth:masterfrom
DG47:fix/payer-transitions-pre-1970-plans
Oct 19, 2026
Merged

hadleynet merged 1 commit into
synthetichealth:masterfrom
DG47:fix/payer-transitions-pre-1970-plans

Conversation

@DG47

@DG47 DG47 commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Fixes #1725

What was broken

claims_transactions.PATIENTINSURANCEID is documented as a foreign key to payer_transitions.MEMBERID, and since #1602 it holds the member id of the claim's plan record. Some of those ids have no row in payer_transitions.csv: in the official sample data every unresolved id belongs to a claim dated before 1970, and the earliest payer_transitions.START_DATE in that file is 1969-02-22.

Root cause

CSVExporter.export(Person, long) called exportPayerTransitions(person, 0L, time). The 0L was meant as "no lower bound" but it is the Unix epoch, so exportPayerTransitions only kept plans with getStopTime() >= 1970-01-01. Plans that ended before 1970 were never written, while the claims made under them were (when exporter.years_of_history keeps the full history), so the member ids in PATIENTINSURANCEID had nothing to join to.

What changed

  • exportPayerTransitions no longer takes a lower bound. It writes every plan that started on or before the end of the simulation, so the plan behind every exported claim transaction is present in payer_transitions.csv. The column set of both files is unchanged.
  • New test CSVExporterTest.testClaimTransactionsReferenceExportedPayerTransitions generates a seeded population aged 60-90 with exporter.years_of_history = 0, checks that the export contains claim transactions dated before 1970, and asserts that every non-empty PATIENTINSURANCEID matches a (PATIENT, MEMBERID) pair in payer_transitions.csv. It fails on master (hundreds of unresolved rows, all dated 1968) and passes with this change.

Testing

./gradlew test --tests '*CSVExporterTest*'
# 12 tests, 0 failures (new test fails before the fix, passes after)

./gradlew checkstyleMain checkstyleTest
# no violations in the changed files (one pre-existing LineLength warning in src/main/java/App.java:47)

exportPayerTransitions was called with a lower bound of 0L, which is the
Unix epoch rather than "no cutoff". Plans that ended before 1970 were
dropped from payer_transitions.csv while the claim transactions made
under them still carried their member ids in PATIENTINSURANCEID, so the
documented foreign key did not resolve for patients born before 1970
when exporter.years_of_history keeps the full history.

Drop the lower bound and write every plan that started before the end
of the simulation. Add a test that generates older patients with the
full history kept and checks that every PATIENTINSURANCEID in
claims_transactions.csv matches a MEMBERID for the same patient in
payer_transitions.csv.

Fixes synthetichealth#1725
@DG47

DG47 commented Oct 12, 2026

Copy link
Copy Markdown
Contributor Author

@hadleynet requesting review

@DG47

DG47 commented Oct 19, 2026

Copy link
Copy Markdown
Contributor Author

Friendly ping on this one. Without it, payer_transitions.csv can be missing transitions that exported claim transactions refer to, so joins from claims to payer history drop rows. #1736 and #1744 touch the same exporter, so if it's easier I can rebase any of them in whatever order you prefer to review.

@hadleynet
hadleynet merged commit 616e219 into synthetichealth:master Oct 19, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

claims_transactions.PATIENTINSURANCEID references plans missing from payer_transitions (plans ended before 1970)

2 participants