Skip to content

feat: Date validity - Calendar Expiration + Trip Coverage#1289

Merged
isabelle-dr merged 24 commits intoMobilityData:masterfrom
JarvusInnovations:issue/886/date-validity
Apr 12, 2023
Merged

feat: Date validity - Calendar Expiration + Trip Coverage#1289
isabelle-dr merged 24 commits intoMobilityData:masterfrom
JarvusInnovations:issue/886/date-validity

Conversation

@KClough
Copy link
Copy Markdown
Contributor

@KClough KClough commented Nov 7, 2022

Summary:

Resolves #886

Expected behavior:

Explain and/or show screenshots for how you expect the pull request to work in your testing (in case other devices exhibit different behavior).

Please make sure these boxes are checked before submitting your pull request - thanks!

Note: Marking as draft until I complete unit tests for the new validator.

@KClough KClough marked this pull request as draft November 7, 2022 14:56
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Dec 2, 2022

CLA assistant check
All committers have signed the CLA.

@CLAassistant
Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ KClough
❌ briandonahue
You have signed the CLA already but the status is still pending? Let us recheck it.

@briandonahue briandonahue force-pushed the issue/886/date-validity branch from 09c1755 to ff5ba70 Compare December 2, 2022 20:23
@briandonahue briandonahue self-assigned this Mar 3, 2023
@briandonahue briandonahue requested a review from bdferris-v2 March 9, 2023 17:17
@briandonahue briandonahue marked this pull request as ready for review March 9, 2023 17:28
@bdferris-v2
Copy link
Copy Markdown
Collaborator

Apologies for the slow delay. So looking at the logic for trip counting, I realize there is some additional complexity around handling things like frequency-based trips that the current code, as written, doesn't cover.

Digging into this on my side, I've gotten the go-ahead to open-source some utility code we have at Google that might help with this. @briandonahue if it's ok with you, can I send you a PR with that code and talk about how it might simplify this PR? Will send shortly.

@briandonahue
Copy link
Copy Markdown
Contributor

@bdferris-v2 this has now been updated to use TripCalendarUtil. Thanks for that!

@briandonahue briandonahue changed the title Feat: Date validity - Calendar Expiration + Trip Coverage feat: Date validity - Calendar Expiration + Trip Coverage Mar 16, 2023
Copy link
Copy Markdown
Contributor

@isabelle-dr isabelle-dr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the work here @briandonahue, and thank you for the review @bdferris-v2!
To follow our naming conventions, could you rename trip_data_should_be_valid_for_next7_days with trip_expiration7_days?

Also: what conditions are triggering the expired_calendar notice exactly? Do you have a test dataset for it?

@isabelle-dr isabelle-dr added this to the 4.1.0 milestone Mar 20, 2023
@bdferris-v2
Copy link
Copy Markdown
Collaborator

Naming conventions aside, I'm not sure trip_expiration7_days is a particularly good name for what this notice is validating.
Specifically, it's not trips that are expiring (whatever that might mean). Maybe something like majority_of_trip_coverage_not_active_for_next7_days would be more descriptive and still match naming conventions?

@briandonahue
Copy link
Copy Markdown
Contributor

Also: what conditions are triggering the expired_calendar notice exactly? Do you have a test dataset for it?

@isabelle-dr I didn't put a wide variety of test data into the tests because the new validators rely on CalendarUtil and TripCalendarUtil which are both tested for specific scenarios. So, my test data is fairly basic, just checking that it verifies when the start/end dates are not covering the 7 day range. For DateTripsValidator it's the same philosophy - I just verify that it validates the start/end dates of the majority service window as returned by TripCalendarUtil.

I haven't come across any examples of using mocks in the tests (very possible I've just missed them, if they exist) but testing this might be more clear if the utils we're mocked and just return specific window dates, rather than input test data and rely on the actual implementations of the Util classes in these tests.

@isabelle-dr
Copy link
Copy Markdown
Contributor

@briandonahue what I meant is more: what is the pseudo-code here?
Did you use the formula described in this comment? If so, can you update the documentation so users better understand what this notice means exactly and what needs to be changed in the data so that this notice is removed?

The name: I see Brian's point; for the purposes of making this name relatively short, I suggest: trip_coverage_not_active_for_next7_days

@briandonahue
Copy link
Copy Markdown
Contributor

@briandonahue what I meant is more: what is the pseudo-code here?

@isabelle-dr The TripCalendarUtil was imported by @bdferris-v2 from Google projects, as far as I understand, and merged in #1351 so that I could use it here. The specific method being used to calculate "Majority Trip Service Coverage" is here.

I can try to come up with an English translation for the documentation - it looks like it looks for consecutive days with > 75% of trips to calculate the service window. Open to suggestion for wording or any existing wording perhaps used by Google, @bdferris-v2?

Copy link
Copy Markdown
Contributor

@isabelle-dr isabelle-dr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @briandonahue I've added a suggestion in the doc to be a little more descriptive about how this role works.
Besides this small change, we are ready to merge!

Co-authored-by: isabelle-dr <[email protected]>
@isabelle-dr
Copy link
Copy Markdown
Contributor

@briandonahue can you update this branch with master? I will merge after

@briandonahue
Copy link
Copy Markdown
Contributor

@briandonahue can you update this branch with master? I will merge after

@isabelle-dr sorry, lost track of this! Done!

@isabelle-dr isabelle-dr merged commit f3a7eb9 into MobilityData:master Apr 12, 2023
bradyhunsaker pushed a commit to bradyhunsaker/gtfs-validator that referenced this pull request Apr 25, 2023
…ta#1289)

* feat: date validity

* Correct typo in test name

* fix: dates in notice

* test: basic tests for date trips validator

* Add documentation, incorporate feedback

* Include calendar date exceptions in expired calendar check

* refactor & cleanup

* formatting

* Update warning information.

* Add TripCalendarUtil, which provides methods for computing trip counts and service windows.

* Update to use TripCalendarUtil for date calculations

* remove unused code

* bug fix and var type clarification

* Add doc comments for new Notices

* Fix comment to match other examples

* formatting

* Rename notice, fix failing tests with notice fields

* Update RULES.md

Co-authored-by: isabelle-dr <[email protected]>

---------

Co-authored-by: Brian Donahue <[email protected]>
Co-authored-by: Brian Ferris <[email protected]>
Co-authored-by: isabelle-dr <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Check whether expired services (calendars) exist on schedule feed Best Practice: Date Validity (WARNING)

5 participants