Proposal to update GTFS Schedule to RFC 2119 by sccmcca · Pull Request #277 · google/transit

sccmcca · 2021-06-03T19:37:16Z

As discussed in #273, this PR contains a proposal to update GTFS Schedule to RFC 2119 in order to disambiguate interpretations and make the spec easier for producing, consuming, and creating products (i.e., validators) around GTFS. Additionally, these changes aim to be useful when articulating obligations in contract requirements.

Note that BCP 14 (containing RFC 2119 and RFC 8174) has not been implemented in this initial proposal. Thus, I have not CAPITALIZED the keywords as seen in GBFS. This has the purpose of making the initial proposal easier to review with pre-existing instances of lowercase RFC 2119 keywords, but we can decide if the CAPITALIZATION is useful and if this proposal should be updated to BCP 14, or alternative styling as seen in IMDF (i.e., only capitalizing "MUST"s, "SHOULD"s, and "MAY"s).

The PR contains separate commits for each applicable section of the GTFS Schedule reference to ease review (the changes are mostly light).

See all the most up-to-date changes in the "Files changed" tab. Feel free to provide in-line comments and any actionable feedback if any sections have mistakenly altered interpretations or otherwise.

Thanks!

Edits

2021-08-03: Round of revisions based on feedback from @isabelle-dr and @aababilov, in commit 3875914.
2021-06-22: Round of revisions based on feedback from @abyrd, in commit fba6020.
2021-06-15: Minor "may vs can" corrections based on feedback, in commit 630bd5d.
2021-06-09: Round of revisions based on community feedback, in commit 73bc764 .
2021-06-03: Commit fb61255 catches some instances where "can" is suitable instead of "may". Generally, "can" is used to denote optional usage by a data consumer or producer, and "may" is used to denote optional behaviour of the specification.

Changed heading to "File Conventions" from "File Requirements" since, by RFC 2119 definition, this section also contains recommendation.

- Corrected TOC heading - Corrected tabs, carriage returns or new lines to "must not" - Typo fix in "Date" field type - Differentiated "may" and "can"

timMillet

1 note, otherwise LGTM!

gtfs/spec/en/reference.md

lionel-nj

Thanks @scmcca for working on that! I added a few comments that actually may be addressed in a separate PR according to your workflow :)

gtfs/spec/en/reference.md

isabelle-dr · 2021-06-25T15:10:22Z

I think the two sections below could be part of this review as well.
"For times occurring after midnight, enter the time as a value greater than 24:00:00" in Field Types under Time.
"Timezone names never contain the space character but may contain an underscore." in Field Types under Timezone.

gtfs/spec/en/reference.md

sccmcca · 2021-07-02T14:48:03Z

@isabelle-dr

I think the two sections below could be part of this review as well.
"For times occurring after midnight, enter the time as a value greater than 24:00:00" in Field Types under Time.

I can think of a clarification to make this sentence less instructional and more descriptive:

"Values greater than 24:00:00 are allowed to define times occurring after midnight for the service day on which the trip schedule begins."

What is also missing from the "Time" definition are invalid values. There is no maximum time value described. perhaps after some value like +12 or +24 hours above 24:00:00 a warning is issued in the validator? Any other suggestions?

"Timezone names never contain the space character but may contain an underscore." in Field Types under Timezone.

I'd like to reduce the "Timezone" description more generally to:

"TZ timezone as described in https://www.iana.org/time-zones. Refer to http://en.wikipedia.org/wiki/List_of_tz_zones for a list of valid values.
Example: Asia/Tokyo, America/Los_Angeles or Africa/Cairo."

isabelle-dr · 2021-07-06T15:05:10Z

@scmcca

What is also missing from the "Time" definition are invalid values. There is no maximum time value described. perhaps after some value like +12 or +24 hours above 24:00:00 a warning is issued in the validator? Any other suggestions?

Currently, the Canonical GTFS Validator accepts values in formats H:MM:SS, HH:MM:SS, as well as HHH:MM:SS, and emits an ERROR otherwise.
There have been discussions on having a separate WARNING if a time field contains a value above a certain threshold. Since this would be an addition to the specification, it could happen in a separate PR or added to the Best Practices if we see a need for such a threshold.

I think those two suggestions work perfectly in the context of this PR!

gtfs/spec/en/reference.md

As per isabelle-dr and aababilov feedback.

abyrd · 2021-08-17T13:53:28Z

@isabelle-dr

I think the two sections below could be part of this review as well.
"For times occurring after midnight, enter the time as a value greater than 24:00:00" in Field Types under Time.

I can think of a clarification to make this sentence less instructional and more descriptive:

"Values greater than 24:00:00 are allowed to define times occurring after midnight for the service day on which the trip schedule begins."

As I understand it, trips may begin after 24:00:00, i.e. times over 24:00:00 are not restricted to trips beginning before 24:00:00. This is because some agencies internally define an overnight service break (when no service runs, maintenance can be done, etc.) and the next service day only begins after that overnight service break. If the service break is 3-5AM for example, a final trip of the day from 2-2:30 AM could reasonably be coded on the previous day using 26:00:00-26:30:00. This pull request is described as being focused on the use of certain specific terms, so I think it would be best to avoid any potential changes to the meaning of the specification.

What is also missing from the "Time" definition are invalid values. There is no maximum time value described. perhaps after some value like +12 or +24 hours above 24:00:00 a warning is issued in the validator? Any other suggestions?

I agree it is reasonable to issue warnings on strange values of the time field. However there are services that run for several days and will exceed these limits - notably some ferries in Norway that are known to be represented in GTFS data. The values that are considered "unusual" are somewhat subjective and might be different in different validation contexts. To me this seems like good material for the "best practices" guide, with the specification addressing only more objective limits.

I'd like to reduce the "Timezone" description more generally to:

"TZ timezone as described in https://www.iana.org/time-zones. Refer to http://en.wikipedia.org/wiki/List_of_tz_zones for a list of valid values.
Example: Asia/Tokyo, America/Los_Angeles or Africa/Cairo."

This wording seems good to me, but this change seems out of scope for the current pull request. Given that some people may decide to read, comment, and vote on the PR based on its title and initial description, it might be better to make such changes in another PR.

sccmcca · 2021-08-17T14:19:43Z

@abyrd @skinkie

Fair points. I'll revert the changes made to the 24:00:00+ and TZ Timezone descriptions.

After that do you think we're good to go?

Revert stop_name clarification for separate PR.

sccmcca · 2021-08-19T19:24:35Z

Thanks to everyone involved in the revisions. This was a lot of work! We seem to have reached a stable consensus.

As per the Specification Amendment Process, I am calling a vote to update GTFS Schedule to RFC 2119 as outlined in this PR.

Voting will end on 2021-08-26 at 23:59:59 UTC.

skinkie · 2021-08-19T19:57:18Z

+1 (OpenGeo)

westontrillium · 2021-08-19T20:43:02Z

+1 from Trillium

abyrd · 2021-08-20T03:34:48Z

+1 (Conveyal)

mgilligan · 2021-08-20T14:29:48Z

+1, TriMet

e-lo · 2021-08-20T19:46:47Z

+1 Cal-ITP

cc: @mcplanner, @thekaveman, @hunterowens

flocsy · 2021-08-22T13:36:53Z

+1 Moovit

sccmcca · 2021-08-27T12:03:15Z

The vote ended on 2021-08-26 at 23:59:59 UTC with 6 votes in favor and 0 votes opposed. As per the Specification Amendment Process, this proposal passes!

Big thanks to everyone involved in bringing RFC 2119 to GTFS Schedule.

e-lo · 2021-08-27T22:58:21Z

A BIG GIANT thanks to @scmcca for leading the process to get to an RFC 2119 compliant spec 🍾 !

scmcca added 16 commits June 3, 2021 12:13

Establish RFC 2119

0b526ff

Field types

08395b1

File conventions

1436d98

Changed heading to "File Conventions" from "File Requirements" since, by RFC 2119 definition, this section also contains recommendation.

agency.txt

0c020d0

stops.txt

08e1ceb

routes.txt

62d872d

trips.txt

d5bde8c

stop_times.txt

d5da8a9

calendar.txt

4c67cb2

calendar_dates.txt

ce1e980

shapes.txt

4659d27

frequencies.txt

e613bce

translations.txt

70d120c

feed_info.txt

82a6351

attributions.txt

bfe0dae

revisions-may-and-can

fb61255

- Corrected TOC heading - Corrected tabs, carriage returns or new lines to "must not" - Typo fix in "Date" field type - Differentiated "may" and "can"

google-cla bot added the cla: yes label Jun 3, 2021

timMillet approved these changes Jun 3, 2021

View reviewed changes