v6.0.0

@praneethd7

Highlights

Partial validation support for Flex

GTFS Flex is a proposal that ensures riders can discover demand responsive services and how to book them. Flex was officially adopted into GTFS this March. With this validator release, we offer partial validation support for feeds that include Flex data. This includes:

Flex features in summary. If a feed has Flex data, you'll see its Flexible Services in the GTFS Features section of the validation report summary. Flex Features include Booking Rules, Predefined Routes with Deviation, Zone-Based Demand Responsive Services, and Fixed-Stops Demand Responsive Services.

No more false positives! Before, feeds with Flex features would trigger certain notices when the data was actually valid, such as missing_required_field for a missing stop_id in stop_times.txt when location_group_id or location_id was provided. Now, any error or warning generated for your Flex feed is legitimate and should be fixed.
Validation for booking_rules.txt. 10 new validation notices have been added to check data from booking_rules.txt
Validation for polygons in locations.geojson. Checks that each polygon in locations.geojson conforms to the OpenGIS Simple Features Specification, section 6.1.11, as required by GTFS.

We'll continue to add more Flex validation rules in future releases. You can see the full list of validation rules here.

Notice changes

🔴 missing_required_file: feed_info.txt is now conditionally required when translations.txt exists. Clarification adopted in google/transit#460.
🔴 missing_required_field for transfers.txt: from_stop_id and to_stop_id are no longer required for a transfer_type of 4 or 5. Clarification adopted in google/transit#455.
🔴 foreign_key_violation: Now checks if network_id exists in fare_leg_rules.txt but not in routes.txt OR networks.txt. Before, there was a bug that meant network_id had to be included in both routes.txt AND networks.txt.
🟡 missing_timepoint_value: Modified to clarify that timepoint values should be set explicitly to 0 or 1, not empty. Clarification adopted into GTFS in google/transit#474. 🟡 missing_recommended_column was also removed as a result of this clarification.

Flex notice changes

🔴 missing_required_file: stops.txt is now conditionally required. If locations.geojson is not present, stops.txt is required.
🔴 missing_required_field: stop_id is conditionally required in stop_times.txt, in cases where the record does not include location_id or location_group_id.
🔴 missing_trip_edge: Only checks stop times records if they do not include start_pickup_dropoff_window or end_pickup_drop_off_window.
🔴 decreasing_or_equal_stop_time_distance: Skips stop times records with a location_id or location_group_id instead of a stop_id. Cases where shape_dist_traveled is provided with a location_id or location_group_id are triggered in the new 🔴 forbidden_shape_dist_traveled error.
🟡 stop_without_stop_time: Skips stops that are a part of a location group referenced in stop_times.txt.
🟡 stop_too_far_from_shape: Only checks stop times records with a stop_id.

New notices

🔴 bidirectional_exit_gate: Triggered when exit gates (pathway_mode=7 in pathways.txt) are bidirectional.
🔴 invalid_character: Triggered when text contains invalid characters, such as the replacement character ("�"). Check that text was properly encoded in UTF-8 as required by GTFS.
🟡 single_shape_point: Triggered when a shape in shapes.txt only includes one shape point.

Flex new notices

🔴 forbidden_shape_dist_traveled: Triggered when shape_dist_traveled is provided with a location_id or location_group_id in stop_times.txt.
🔴 forbidden_geography_id: Triggered when a stop_times.txt includes more than one geography id. Only one of stop_id, location_group_id or location_id can be defined for a given entry.

locations.geojson notices

🔴 malformed_json: Triggered when locations.geojson is malformed.
🔴 invalid_geometry: Triggered when polygon in locations.geojson is unparsable or invalid. Each polygon must be valid by the definition of the OpenGIS Simple Features Specification, section 6.1.11.
🔴 missing_required_element: Triggered when an element that's required in the geoJSON file is missing.
🔴 unsupported_geo_json_type: Triggered when geoJSON type is a value other than "FeatureCollection".
🔴 unsupported_feature_type: Triggered when feature type is a value other than "Feature".
🔴 unsupported_geometry_type: Triggered when the geometry type is a value other than "Polygon" or "MultiPolygon".

booking_rules.txt notices

🔴 forbidden_prior_day_booking_field_value: Triggered when a forbidden field value is present for a prior-day booking rule.
🔴 forbidden_prior_notice_start_day: prior_notice_start_day value is forbidden when prior_notice_duration_max is set.
🔴 forbidden_prior_notice_start_time: prior_notice_start_time value is forbidden when prior_notice_start_day value is not set
🔴 forbidden_real_time_booking_field_value: A forbidden field value is present for a real-time booking rule.
🔴 forbidden_same_day_booking_field_value: A forbidden field value is present for a same-day booking rule.
🔴 invalid_prior_notice_duration_min: The prior_notice_duration_max field value needs to be greater or equal to the prior_notice_duration_min field value.
🔴 missing_prior_notice_duration_min: prior_notice_duration_min value is required for same day booking_type
🔴 missing_prior_day_booking_field_value: prior_notice_last_day and prior_notice_last_time values are required for prior day booking_type.
🔴 missing_prior_notice_start_time: prior_notice_start_time value is required when prior_notice_start_day value is set
🔴 prior_notice_last_day_after_start_day: prior_notice_last_day should not be greater than the prior_notice_start_day.

Impacts on existing data

You can find the impact of this release on all existing datasets we have in the Mobility Database in this spreadsheet.

Other notable improvements

New Contributors

@praneethd7 made their first contribution in #1753

PR List

feat: Changed notice migration generation. Added zip file to release. by @jcpitre in #1736
feat: documented the changes made in PR 1633 by @qcdyx in #1748
Feat: 1639 release process for minor versions by @jcpitre in #1657
feat: notify mobility-feeds-api when the web validator is updated by @cka-y in #1760
fix: set transfer from and to stop ids as conditional required fields by @davidgamez in #1772
feat: add performance assessment to acceptance tests by @cka-y in #1771
feat: New rule - SingleShapePointValidator for flagging shapes with a single shape point by @praneethd7 in #1753
Docs: Fix typo in FEATURES.md by @isabelle-dr in #1777
fix: Display empty values for feed_start_date and feed_end_date if there's… by @qcdyx in #1798
fix: Count appears as 1 for block even when no blocks exist by @qcdyx in #1802
fix: Added a validator for bad networkId foreign key in fareLegRules by @jcpitre in #1804
feat: added new field for rule priority in GtfsFareLegRuleSchema by @qcdyx in #1806
feat: 1791 feed infotxt should be added to missing required file when translationstxt exists by @qcdyx in #1803
feat: 1786 exit gates pathway mode7 must not be bidirectional by @qcdyx in #1808
feat: 1776 add flex features to list by @qcdyx in #1780
feat: Flex - Removed required field false positive for stop_id + added Foreign key violation for location_groups_id by @jcpitre in #1834
feat: add feed contact email to json and html report by @davidgamez in #1836
feat: update missing_trip_edge for flex feed by @cka-y in #1843
feat: flex - forbidden_real_time_booking_field_value validation notice by @cka-y in #1845
feat: 1534 service window in summary report by @qcdyx in #1837
feat: add feedServiceWindow to JSON report by @qcdyx in #1852
fix: only display service window on HTML report, not start/end date by @qcdyx in #1855
feat: flex - added forbidden_same_day_booking_field_value notice by @cka-y in #1847
feat: changes + added documentation to features by @cka-y in #1844
feat: 1829 flex invalid prior notice day range by @qcdyx in #1858
fix: keep the notice description as is by @qcdyx in #1863
feat: added forbidden_prior_day_booking_field_value, invalid_prior_notice_duration_min and forbidden_prior_notice_start_day notices by @cka-y in #1860
locations.geojson POC phase2: End-to-end partial support of json data by @jcpitre in #1810
feat: added support notice for flex feeds by @cka-y in #1861
feat: new booking_rules.txt validation notices by @cka-y in #1866
fix: Predefined Routes with Deviation trigger + test by @cka-y in #1862
feat: modify stops.txt to be conditionally required by @cka-y in #1868
docs: Update FEATURES.md by @emmambd in #1812
docs: Update doc links in features.md by @emmambd in #1877
docs: fix color typos in FEATURES.md by @emmambd in #1878
feat: 1795 timepoint clarification by @qcdyx in #1867
chore: add memory usage records to the JSON report by @davidgamez in #1857
feat: modified stop_too_far_from_shape so it only checks stop times record with a stop_id by @qcdyx in #1884
docs: Clarified below threshold language in notices by @emmambd in #1886
feat: Do not call validators for missing columns by @jcpitre in #1875
feat: add memory metrics to acceptance tests by @davidgamez in #1874
feat: Changed Fares Transfers to Fare Transfers by @qcdyx in #1880
feat: 1883 flex modify stop without stop time to exclude stops associated with a used location group by @qcdyx in #1887
feat: Added a validator for forbidden shape_distance by @jcpitre in #1896
feat:locations.geojson file parsing and geometry validation by @cka-y in #1879
fix: add exception for decreasing_or_equal_stop_time_distance by @davidgamez in #1895
feat: 1840 invalid characters by @qcdyx in #1892
feat: included network_id as part of the primary key for fare_leg_rules.txt by @qcdyx in #1894
chore(deps): bump urllib3 from 1.26.6 to 1.26.19 in /scripts/mobility-database-harvester in the pip group across 1 directory by @dependabot in #1899
chore(deps): bump the pip group across 2 directories with 4 updates by @dependabot in #1900
ci: aling matrix dimensions to max concurrent jobs by @davidgamez in #1904
chore(deps): bump requests from 2.32.2 to 2.32.3 in /scripts/mobility-database-harvester in the pip group across 1 directory by @dependabot in #1903

Full Changelog: v5.0.1...v6.0.0

Provide feedback

Saved searches

Use saved searches to filter your results more quickly