Skip to content

Commit 0ff991a

Browse files
bdferris-v2bdferris-v2
authored
Merge 1e7493b into df8f221 
2 parents df8f221 + 1e7493b commit 0ff991a

File tree

10 files changed

+189
-80
lines changed

10 files changed

+189
-80
lines changed

‎.github/workflows/test_pack_doc.yml‎

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,8 @@ name: Test Package Document
33
on:
44
push:
55
branches: [ master ]
6-
paths-ignore:
7-
- '**.md'
86
pull_request:
97
branches: [ master ]
10-
paths-ignore:
11-
- '**.md'
128
release:
139
types: [ prereleased, released ]
1410

‎RULES.md‎

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -104,7 +104,7 @@ Each Notice is associated with a severity: `INFO`, `WARNING`, `ERROR`.
104104
| [`fast_travel_between_far_stops`](#fast_travel_between_far_stops) | A transit vehicle moves too fast between two far stops. |
105105
| [`feed_expiration_date7_days`](#feed_expiration_date7_days) | Dataset should be valid for at least the next 7 days. |
106106
| [`feed_expiration_date30_days`](#feed_expiration_date30_days) | Dataset should cover at least the next 30 days of service. |
107-
| [`feed_info_lang_and_agency_mismatch`](#feed_info_lang_and_agency_mismatch) | Mismatching feed and agency language fields. |
107+
| [`feed_info_lang_and_agency_lang_mismatch`](#feed_info_lang_and_agency_lang_mismatch) | Mismatching feed and agency language fields. |
108108
| [`inconsistent_agency_lang`](#inconsistent_agency_lang) | Inconsistent language among agencies. |
109109
| [`leading_or_trailing_whitespaces`](#leading_or_trailing_whitespaces) | The value in CSV file has leading or trailing whitespaces. |
110110
| [`missing_feed_info_date`](#missing_feed_info_date) | `feed_end_date` should be provided if `feed_start_date` is provided. `feed_start_date` should be provided if `feed_end_date` is provided. |
@@ -409,7 +409,7 @@ A row from GTFS file `fare_transfer_rules.txt` has a defined `duration_limit_typ
409409

410410
<a name="FareTransferRuleDurationLimitWithoutTypeNotice"/>
411411

412-
### fare_transfer_rule_duration_limit_without_type
412+
### fare_transfer_rule_duration_limit_without_type
413413

414414
A row from GTFS file `fare_transfer_rules.txt` has a defined `duration_limit` field but no `duration_limit_type` specified.
415415

@@ -1893,7 +1893,7 @@ At any time, the GTFS dataset should cover at least the next 30 days of service,
18931893

18941894
<a name="FeedInfoLangAndAgencyLangMismatchNotice"/>
18951895

1896-
### feed_info_lang_and_agency_mismatch
1896+
### feed_info_lang_and_agency_lang_mismatch
18971897
1. Files `agency.txt` and `feed_info.txt` should define matching `agency.agency_lang` and `feed_info.feed_lang`.
18981898
The default language may be multilingual for datasets with the original text in multiple languages. In such cases, the feed_lang field should contain the language code mul defined by the norm ISO 639-2.
18991899
* If `feed_lang` is not `mul` and does not match with `agency_lang`, that's an error
@@ -1987,7 +1987,7 @@ Even though `feed_info.start_date` and `feed_info.end_date` are optional, if one
19871987

19881988
</details>
19891989

1990-
<a name="MissingTimepointColumnNotice"/>
1990+
<a name="MissingRecommendedFileNotice"/>
19911991

19921992
### missing_recommended_file
19931993

@@ -2007,6 +2007,8 @@ A recommended file is missing.
20072007

20082008
</details>
20092009

2010+
<a name="MissingRecommendedFieldNotice"/>
2011+
20102012
### missing_recommended_field
20112013

20122014
The given field has no value in some input row, even though values are recommended.
@@ -2027,6 +2029,8 @@ The given field has no value in some input row, even though values are recommend
20272029

20282030
</details>
20292031

2032+
<a name="MissingTimepointColumnNotice"/>
2033+
20302034
### missing_timepoint_column
20312035

20322036
The `timepoint` column should be provided.
@@ -2182,7 +2186,7 @@ A platform has no `parent_station` field set.
21822186

21832187
<a name="RouteColorContrastNotice"/>
21842188

2185-
#### route_color_contrast
2189+
### route_color_contrast
21862190

21872191
A route's color and `route_text_color` should be contrasting.
21882192

@@ -2699,6 +2703,7 @@ Error in IO operation.
26992703
| `exception` | The name of the exception. | String |
27002704
| `message` | The error message that explains the reason for the exception. | String |
27012705
</details>
2706+
27022707
<a name="RuntimeExceptionInLoaderError"/>
27032708

27042709
### runtime_exception_in_loader_error

‎cli/src/main/java/org/mobilitydata/gtfsvalidator/cli/Main.java‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@
2828
import org.mobilitydata.gtfsvalidator.notice.NoticeSchemaGenerator;
2929
import org.mobilitydata.gtfsvalidator.runner.ValidationRunner;
3030
import org.mobilitydata.gtfsvalidator.util.VersionResolver;
31+
import org.mobilitydata.gtfsvalidator.validator.ClassGraphDiscovery;
3132

3233
/** The main entry point for GTFS Validator CLI. */
3334
public class Main {
@@ -85,7 +86,7 @@ private static void exportNoticeSchema(final Arguments args) {
8586
Paths.get(args.getOutputBase(), NOTICE_SCHEMA_JSON),
8687
gson.toJson(
8788
NoticeSchemaGenerator.jsonSchemaForPackages(
88-
NoticeSchemaGenerator.DEFAULT_NOTICE_PACKAGES))
89+
ClassGraphDiscovery.DEFAULT_NOTICE_PACKAGES))
8990
.getBytes(StandardCharsets.UTF_8));
9091
} catch (IOException e) {
9192
logger.atSevere().withCause(e).log("Cannot store notice schema file");

‎core/src/main/java/org/mobilitydata/gtfsvalidator/notice/Notice.java‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -68,17 +68,18 @@ public SeverityLevel getSeverityLevel() {
6868
* @return notice code, e.g., "foreign_key_violation".
6969
*/
7070
public String getCode() {
71-
return getCode(getClass().getSimpleName());
71+
return getCode(getClass());
7272
}
7373

7474
/**
75-
* Returns a descriptive type-specific name for this notice class simple name.
75+
* Returns a descriptive type-specific name for this notice class.
7676
*
7777
* @return notice code, e.g., "foreign_key_violation".
7878
*/
79-
static String getCode(String className) {
79+
public static String getCode(Class<?> noticeClass) {
8080
return CaseFormat.UPPER_CAMEL.to(
81-
CaseFormat.LOWER_UNDERSCORE, StringUtils.removeEnd(className, NOTICE_SUFFIX));
81+
CaseFormat.LOWER_UNDERSCORE,
82+
StringUtils.removeEnd(noticeClass.getSimpleName(), NOTICE_SUFFIX));
8283
}
8384

8485
/**

‎core/src/main/java/org/mobilitydata/gtfsvalidator/notice/NoticeSchemaGenerator.java‎

Lines changed: 11 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -17,31 +17,23 @@
1717
package org.mobilitydata.gtfsvalidator.notice;
1818

1919
import com.google.common.annotations.VisibleForTesting;
20-
import com.google.common.collect.ImmutableList;
2120
import com.google.common.geometry.S2LatLng;
2221
import com.google.gson.JsonArray;
2322
import com.google.gson.JsonElement;
2423
import com.google.gson.JsonObject;
2524
import com.google.gson.JsonPrimitive;
26-
import io.github.classgraph.ClassGraph;
27-
import io.github.classgraph.ClassInfo;
28-
import io.github.classgraph.ScanResult;
2925
import java.io.IOException;
3026
import java.lang.reflect.Field;
31-
import java.util.ArrayList;
3227
import java.util.List;
3328
import java.util.Map;
3429
import java.util.TreeMap;
3530
import org.mobilitydata.gtfsvalidator.type.GtfsColor;
3631
import org.mobilitydata.gtfsvalidator.type.GtfsDate;
3732
import org.mobilitydata.gtfsvalidator.type.GtfsTime;
33+
import org.mobilitydata.gtfsvalidator.validator.ClassGraphDiscovery;
3834

3935
/** Exports schema describing all possible notices and their contexts. */
4036
public class NoticeSchemaGenerator {
41-
/** Default packages to find notices in open-source validator. */
42-
public static final ImmutableList<String> DEFAULT_NOTICE_PACKAGES =
43-
ImmutableList.of(
44-
"org.mobilitydata.gtfsvalidator.notice", "org.mobilitydata.gtfsvalidator.validator");
4537

4638
/**
4739
* Exports JSON schema for all notices in given packages. This includes notices that are declared
@@ -86,16 +78,15 @@ public class NoticeSchemaGenerator {
8678
* }</pre>
8779
*
8880
* @param packages List of packages where notices are declared. Use {@link
89-
* #DEFAULT_NOTICE_PACKAGES} and add your custom Java packages here, if any
81+
* ClassGraphDiscovery#DEFAULT_NOTICE_PACKAGES} and add your custom Java packages here, if any
9082
* @return a {@link JsonObject} describing all notices in given packages (see above)
9183
* @throws IOException
9284
*/
9385
public static JsonObject jsonSchemaForPackages(List<String> packages) throws IOException {
9486
JsonObject schema = new JsonObject();
9587
for (Map.Entry<String, Map<String, Class<?>>> entry :
9688
contextFieldsInPackages(packages).entrySet()) {
97-
schema.add(
98-
Notice.getCode(entry.getKey()), jsonSchemaForNotice(entry.getKey(), entry.getValue()));
89+
schema.add(entry.getKey(), jsonSchemaForNotice(entry.getKey(), entry.getValue()));
9990
}
10091
return schema;
10192
}
@@ -107,27 +98,29 @@ public static JsonObject jsonSchemaForPackages(List<String> packages) throws IOE
10798
*
10899
* <pre>{@code
109100
* {
110-
* "AttributionWithoutRoleNotice": {
101+
* "attribution_without_role": {
111102
* "attributionId": String.class,
112103
* "csvRowNumber": Long.class,
113104
* }
114105
* }
115106
* }</pre>
116107
*
117108
* @param packages List of packages where notices are declared
118-
* @return a map describing all notices in given packages (see above)
109+
* @return a map describing all notices in given packages, keyed by notice code (see above)
119110
* @throws IOException
120111
*/
121112
@VisibleForTesting
122113
static Map<String, Map<String, Class<?>>> contextFieldsInPackages(List<String> packages)
123114
throws IOException {
124115
// Return a sorted TreeMap for stable results.
125-
Map<String, Map<String, Class<?>>> contextFieldsByNotice = new TreeMap<>();
126-
for (Class<Notice> noticeClass : findNoticeSubclasses(packages)) {
127-
contextFieldsByNotice.put(noticeClass.getSimpleName(), contextFieldsForNotice(noticeClass));
116+
Map<String, Map<String, Class<?>>> contextFieldsByNoticeCode = new TreeMap<>();
117+
118+
for (Class<Notice> noticeClass : ClassGraphDiscovery.discoverNoticeSubclasses(packages)) {
119+
contextFieldsByNoticeCode.put(
120+
Notice.getCode(noticeClass), contextFieldsForNotice(noticeClass));
128121
}
129122

130-
return contextFieldsByNotice;
123+
return contextFieldsByNoticeCode;
131124
}
132125

133126
@VisibleForTesting
@@ -144,33 +137,6 @@ private static boolean isSubclassOf(Class<?> parent, Class<?> child) {
144137
return !child.equals(parent) && parent.isAssignableFrom(child);
145138
}
146139

147-
/**
148-
* Finds all subclasses of {@link Notice} that belong to the given packages.
149-
*
150-
* <p>This function also dives into validator classes that may contain inner notice classes.
151-
*/
152-
@VisibleForTesting
153-
static List<Class<Notice>> findNoticeSubclasses(List<String> packages) throws IOException {
154-
String[] packagesAsArray = packages.toArray(new String[] {});
155-
List<Class<Notice>> notices = new ArrayList<>();
156-
ClassGraph classGraph =
157-
new ClassGraph()
158-
.enableClassInfo()
159-
.acceptPackages(packagesAsArray)
160-
.ignoreClassVisibility()
161-
.verbose();
162-
try (ScanResult scanResult = classGraph.scan()) {
163-
for (ClassInfo classInfo : scanResult.getSubclasses(Notice.class)) {
164-
if (classInfo.isAbstract()) {
165-
continue;
166-
}
167-
Class<?> clazz = classInfo.loadClass();
168-
notices.add((Class<Notice>) clazz);
169-
}
170-
}
171-
return notices;
172-
}
173-
174140
private static final class JsonTypes {
175141

176142
private static final String NUMBER = "number";

‎core/src/main/java/org/mobilitydata/gtfsvalidator/validator/ClassGraphDiscovery.java‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,14 +4,20 @@
44
import io.github.classgraph.ClassGraph;
55
import io.github.classgraph.ClassInfo;
66
import io.github.classgraph.ScanResult;
7+
import java.util.List;
78
import org.mobilitydata.gtfsvalidator.annotation.GtfsValidator;
9+
import org.mobilitydata.gtfsvalidator.notice.Notice;
810
import org.mobilitydata.gtfsvalidator.table.GtfsTableDescriptor;
911

1012
/** Discovers GTFS table descriptor and validator classes in the given Java packages. */
1113
public class ClassGraphDiscovery {
1214

1315
public static final String DEFAULT_VALIDATOR_PACKAGE = "org.mobilitydata.gtfsvalidator.validator";
1416
public static final String DEFAULT_TABLE_PACKAGE = "org.mobilitydata.gtfsvalidator.table";
17+
/** Default packages to find notices in open-source validator. */
18+
public static final ImmutableList<String> DEFAULT_NOTICE_PACKAGES =
19+
ImmutableList.of(
20+
"org.mobilitydata.gtfsvalidator.notice", "org.mobilitydata.gtfsvalidator.validator");
1521

1622
private ClassGraphDiscovery() {}
1723

@@ -64,4 +70,30 @@ public static ImmutableList<Class<?>> discoverValidators(
6470
}
6571
return validatorClasses.build();
6672
}
73+
74+
/**
75+
* Finds all subclasses of {@link Notice} that belong to the given packages.
76+
*
77+
* <p>This function also dives into validator classes that may contain inner notice classes.
78+
*/
79+
public static ImmutableList<Class<Notice>> discoverNoticeSubclasses(List<String> packages) {
80+
String[] packagesAsArray = packages.toArray(new String[] {});
81+
ImmutableList.Builder<Class<Notice>> notices = ImmutableList.builder();
82+
ClassGraph classGraph =
83+
new ClassGraph()
84+
.enableClassInfo()
85+
.acceptPackages(packagesAsArray)
86+
.ignoreClassVisibility()
87+
.verbose();
88+
try (ScanResult scanResult = classGraph.scan()) {
89+
for (ClassInfo classInfo : scanResult.getSubclasses(Notice.class)) {
90+
if (classInfo.isAbstract()) {
91+
continue;
92+
}
93+
Class<?> clazz = classInfo.loadClass();
94+
notices.add((Class<Notice>) clazz);
95+
}
96+
}
97+
return notices.build();
98+
}
6799
}

‎core/src/test/java/org/mobilitydata/gtfsvalidator/notice/NoticeSchemaGeneratorTest.java‎

Lines changed: 5 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -25,11 +25,7 @@
2525
import com.google.gson.JsonElement;
2626
import java.io.IOException;
2727
import org.junit.Test;
28-
import org.mobilitydata.gtfsvalidator.notice.testnotices.DoubleFieldNotice;
29-
import org.mobilitydata.gtfsvalidator.notice.testnotices.GtfsTypesValidationNotice;
3028
import org.mobilitydata.gtfsvalidator.notice.testnotices.S2LatLngNotice;
31-
import org.mobilitydata.gtfsvalidator.notice.testnotices.StringFieldNotice;
32-
import org.mobilitydata.gtfsvalidator.notice.testnotices.TestValidator.TestInnerNotice;
3329
import org.mobilitydata.gtfsvalidator.type.GtfsColor;
3430
import org.mobilitydata.gtfsvalidator.type.GtfsDate;
3531
import org.mobilitydata.gtfsvalidator.type.GtfsTime;
@@ -39,17 +35,6 @@ public class NoticeSchemaGeneratorTest {
3935
private static final String TEST_NOTICES_PACKAGE =
4036
"org.mobilitydata.gtfsvalidator.notice.testnotices";
4137

42-
@Test
43-
public void findNoticeSubclasses() throws IOException {
44-
assertThat(NoticeSchemaGenerator.findNoticeSubclasses(ImmutableList.of(TEST_NOTICES_PACKAGE)))
45-
.containsExactly(
46-
DoubleFieldNotice.class,
47-
TestInnerNotice.class,
48-
GtfsTypesValidationNotice.class,
49-
S2LatLngNotice.class,
50-
StringFieldNotice.class);
51-
}
52-
5338
@Test
5439
public void jsonSchemaForPackages_succeeds() throws IOException {
5540
assertThat(
@@ -64,16 +49,16 @@ public void contextFieldsInPackages_testNotices() throws IOException {
6449
NoticeSchemaGenerator.contextFieldsInPackages(ImmutableList.of(TEST_NOTICES_PACKAGE)))
6550
.isEqualTo(
6651
ImmutableMap.of(
67-
"DoubleFieldNotice",
52+
"double_field",
6853
ImmutableMap.of("doubleField", double.class),
69-
"TestInnerNotice",
54+
"test_inner",
7055
ImmutableMap.of("intField", int.class),
71-
"GtfsTypesValidationNotice",
56+
"gtfs_types_validation",
7257
ImmutableMap.of(
7358
"color", GtfsColor.class, "date", GtfsDate.class, "time", GtfsTime.class),
74-
"S2LatLngNotice",
59+
"s2_lat_lng",
7560
ImmutableMap.of("point", S2LatLng.class),
76-
"StringFieldNotice",
61+
"string_field",
7762
ImmutableMap.of("someField", String.class)));
7863
}
7964

‎core/src/test/java/org/mobilitydata/gtfsvalidator/validator/ClassGraphDiscoveryTest.java‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
package org.mobilitydata.gtfsvalidator.validator;
2+
3+
import static com.google.common.truth.Truth.assertThat;
4+
5+
import com.google.common.collect.ImmutableList;
6+
import org.junit.Test;
7+
import org.mobilitydata.gtfsvalidator.notice.testnotices.DoubleFieldNotice;
8+
import org.mobilitydata.gtfsvalidator.notice.testnotices.GtfsTypesValidationNotice;
9+
import org.mobilitydata.gtfsvalidator.notice.testnotices.S2LatLngNotice;
10+
import org.mobilitydata.gtfsvalidator.notice.testnotices.StringFieldNotice;
11+
import org.mobilitydata.gtfsvalidator.notice.testnotices.TestValidator.TestInnerNotice;
12+
13+
public class ClassGraphDiscoveryTest {
14+
private static final String TEST_NOTICES_PACKAGE =
15+
"org.mobilitydata.gtfsvalidator.notice.testnotices";
16+
17+
@Test
18+
public void discoverNoticeSubclasses() {
19+
assertThat(ClassGraphDiscovery.discoverNoticeSubclasses(ImmutableList.of(TEST_NOTICES_PACKAGE)))
20+
.containsExactly(
21+
DoubleFieldNotice.class,
22+
TestInnerNotice.class,
23+
GtfsTypesValidationNotice.class,
24+
S2LatLngNotice.class,
25+
StringFieldNotice.class);
26+
}
27+
}

‎main/build.gradle‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,9 +58,16 @@ dependencies {
5858
testImplementation 'org.mockito:mockito-core:4.5.1'
5959
}
6060

61+
// A custom task to copy RULES.md into the test resource directory so that we can reference it in
62+
// unit tests. See NoticeDocumentationTest for more details.
63+
tasks.register('copyRulesMarkdown', Copy) {
64+
from "$rootDir/RULES.md"
65+
into "$projectDir/build/resources/test"
66+
}
67+
6168
test {
6269
// Always run tests, even when nothing changed.
63-
dependsOn 'cleanTest'
70+
dependsOn 'cleanTest', 'copyRulesMarkdown'
6471

6572
// Show test results.
6673
testLogging {

0 commit comments

Comments
 (0)