Skip to content

Conversation

@matvj250
Copy link
Contributor

@matvj250 matvj250 commented Apr 30, 2025

Implementing japicmp in the pinot-spi module to fail builds (i.e. running mvn clean) when incompatible SPI changes are made, such as removing methods, changing method return types, etc. Compatible SPI changes, such as adding new methods, will not fail builds. I have tested that this works as intended.

Some additional information about the package:

  • japicmp works by comparing two versions (.jar files) of a repository. I am pushing a .jar representing apache/pinot in its current state, which will be compared against whatever SPI changes a user makes.
  • japicmp will generate a .diff and .md file with in-depth information on changes made, and whether they are compatible or not.
  • The compatibility of a few of the checks japicmp performs were changed; for example, annotation deletions will be treated as incompatible.

Every commit except the last is irrelevant to japicmp, and relates to an old implementation I was working on for this. None of that old code is reflected in this pull request.

matvj250 and others added 30 commits April 21, 2025 13:34
…fChecker, added functionality for displaying line number, and reverted temporary change to TableConfig.java
…g sh file so that "No incorrect..." message only displayed once, and adding blank line check to GitDiffChecker
This reverts commit 270937f.
…onditional to reflect return type change. java: switch from returning line number to string of code, as previous logic did not work if multiple "chunks" of code were changed, and added annotation logic that excluded json-related annotations.
…message and put text file in my module. java: slightly altered regex after rethinking it, removed System.out.println, i think it's not necessary
…turns nothing, there isn't an accidental test passing
…ecking. Switched to the japicmp plugin, with some modifications to the compatibility of checks that japicmp performs. With this, contributors will be able to see if they made incompatible SPI changes when running mvn clean, rather than waiting until they make a PR. Adding a .jar of pinot-spi that japicmp will use for comparisons.
Copy link

@UOETianleZhang UOETianleZhang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making the change!

…justification for using a baseline jar for comparison, and that we need to eventually transition away from it.
@xiangfu0
Copy link
Contributor

xiangfu0 commented May 1, 2025

For the jar file to compare, can we download it as part of the CI or use the previous release version?

@codecov-commenter
Copy link

codecov-commenter commented May 1, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 62.65%. Comparing base (1a476de) to head (cd93b6b).
Report is 17 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff              @@
##             master   #15684      +/-   ##
============================================
- Coverage     62.90%   62.65%   -0.25%     
  Complexity     1386     1386              
============================================
  Files          2867     2872       +5     
  Lines        163354   164004     +650     
  Branches      24952    25097     +145     
============================================
+ Hits         102755   102760       +5     
- Misses        52847    53510     +663     
+ Partials       7752     7734      -18     
Flag Coverage Δ
custom-integration1 100.00% <ø> (ø)
integration 100.00% <ø> (ø)
integration1 100.00% <ø> (ø)
integration2 0.00% <ø> (ø)
java-11 62.63% <ø> (-0.24%) ⬇️
java-21 62.61% <ø> (-0.21%) ⬇️
skip-bytebuffers-false ?
skip-bytebuffers-true ?
temurin 62.65% <ø> (-0.25%) ⬇️
unittests 62.65% <ø> (-0.25%) ⬇️
unittests1 55.62% <ø> (-0.20%) ⬇️
unittests2 33.38% <ø> (-0.19%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@UOETianleZhang
Copy link

UOETianleZhang commented May 1, 2025

For the jar file to compare, can we download it as part of the CI or use the previous release version?

Hi @xiangfu0, could you elaborate a little bit more on the idea of downloading as CI?

or use the previous release version?

This is also what @matvj250 and I wanted to achieve. In theory, japicmp should compare the current commit with the last stable version (1.3.0 in this case) to make sure no back-incompatible interface changes.

However, this is not possible because there are already back-incompatible changes in 1.4.0-SNAPSHOT. This is why we have the jar of the current commit as the baseline (temporarily). After 1.4.0 is released, we can delete the jar and update the base version to 1.4.0 in the plugin config.

…ed pom file so that all pinot-spi files are checked
Copy link

@UOETianleZhang UOETianleZhang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for making the change!

@siddharthteotia siddharthteotia merged commit c36d88f into apache:master May 2, 2025
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants