Skip to content

Simplify logic surrounding building of Java & Kotlin parsers.#665

Merged
timtebeek merged 1 commit intomainfrom
avoid-building-parsers
Nov 28, 2023
Merged

Simplify logic surrounding building of Java & Kotlin parsers.#665
timtebeek merged 1 commit intomainfrom
avoid-building-parsers

Conversation

@timtebeek
Copy link
Copy Markdown
Member

@timtebeek timtebeek commented Nov 25, 2023

What's changed?

What's your motivation?

Readability suffered a bit when mentally trying to parse what we had before.

Stream<? extends SourceFile> cus = mainJavaSources.isEmpty() ? Stream.empty()
        : Stream.of(javaParserBuilder).map(JavaParser.Builder::build)
                .flatMap(parser -> parser.parse(mainJavaSources, baseDir, ctx));

Then further improvements from there on as spotted.

Have you considered any alternatives or workarounds?

Could have merely gone for javaParserBuilder.build() in ternary, but that didn't yet help readability much.

@timtebeek
Avoid building Java & Kotlin parsers when possible
0b8fc33 
Also simplify logic related to log statements, markers & return value
@timtebeek timtebeek self-assigned this Nov 25, 2023
@timtebeek timtebeek added the enhancement New feature or request label Nov 25, 2023
knutwannheden
knutwannheden reviewed Nov 26, 2023
View reviewed changes
Comment thread src/main/java/org/openrewrite/maven/MavenMojoProjectParser.java
Path buildDirectory = baseDir.relativize(Paths.get(mavenProject.getBuild().getDirectory()));
Stream<SourceFile> sourceFiles = Stream.concat(parsedJava, parsedKotlin).filter(s -> !s.getSourcePath().startsWith(buildDirectory));
Stream<SourceFile> sourceFiles = Stream.concat(parsedJava, parsedKotlin)
.filter(s -> !s.getSourcePath().startsWith(buildDirectory))
Copy link
Copy Markdown
Contributor

@knutwannheden knutwannheden Nov 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like it would be even better if we could filter out these generated sources before parsing them.

Copy link
Copy Markdown
Member Author

@timtebeek timtebeek Nov 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had thought we parse them such that we have their types available for attribution, as indicated on line 302 above. What improvement would you make here then?

Copy link
Copy Markdown
Contributor

@knutwannheden knutwannheden Nov 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose we have to test this, but these sources should still not need to be parsed, because the type attribution in other sources is via class files.

Copy link
Copy Markdown
Member Author

@timtebeek timtebeek Nov 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it boils down to the question if getCompileClasspathElements() returns the classes compiled fom generated sources as well, which could depend on a number of different factors.

rewrite-maven-plugin/src/main/java/org/openrewrite/maven/MavenMojoProjectParser.java

Lines 317 to 324 in 0b8fc33

logInfo(mavenProject, "Parsing source files");
List<Path> dependencies = mavenProject.getCompileClasspathElements().stream()
.distinct()
.map(Paths::get)
.collect(toList());
JavaTypeCache typeCache = new JavaTypeCache();
javaParserBuilder.classpath(dependencies).typeCache(typeCache);
kotlinParserBuilder.classpath(dependencies).typeCache(new JavaTypeCache());

Right now we define generated sources as anything we find in the target/ directory, and parse those as source rather than compiled classes.

rewrite-maven-plugin/src/main/java/org/openrewrite/maven/MavenMojoProjectParser.java

Lines 302 to 308 in 0b8fc33

// Some annotation processors output generated sources to the /target directory. These are added for parsing but
// should be filtered out of the final SourceFile list.
List<Path> generatedSourcePaths = listJavaSources(mavenProject.getBasedir().toPath().resolve(mavenProject.getBuild().getDirectory()));
List<Path> mainJavaSources = Stream.concat(
generatedSourcePaths.stream(),
listJavaSources(mavenProject.getBasedir().toPath().resolve(mavenProject.getBuild().getSourceDirectory())).stream()
).collect(toList());

While I agree it might be interesting to not parse those generated sources as source files but rather only as classpath entries, I'm not entirely sure that would work in all cases, and verifying that would take more time than I currently have available. Are you ok merging this PR as-is to at least simplify the logic?

@timtebeek timtebeek merged commit 9a9428f into main Nov 28, 2023
@timtebeek timtebeek deleted the avoid-building-parsers branch November 28, 2023 10:29
@timtebeek timtebeek mentioned this pull request Nov 28, 2023
Closed
@timtebeek timtebeek mentioned this pull request Apr 11, 2026
Merged
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@knutwannheden knutwannheden knutwannheden approved these changes

Assignees

@timtebeek timtebeek

Labels

enhancement New feature or request

Projects

OpenRewrite
Archived in project

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@timtebeek @knutwannheden