Skip to content

Update Caffeine benchmark#1342

Merged
msridhar merged 5 commits intomasterfrom
update-caffeine-benchmark
Nov 14, 2025
Merged

Update Caffeine benchmark#1342
msridhar merged 5 commits intomasterfrom
update-caffeine-benchmark

Conversation

@msridhar
Copy link
Copy Markdown
Collaborator

@msridhar msridhar commented Nov 14, 2025 •
edited by coderabbitai Bot
Loading

Fixes #1108

We update to the latest version of Caffeine, and also update the source file list and NullAway options used.
This is our first benchmark that uses JSpecify mode.

Also did a couple other drive-by cleanups.

Summary by CodeRabbit

  • Chores
    • Upgraded Caffeine library dependency to the latest stable version
    • Modernized build system task declarations and classpath handling
  • Enhancements
    • Expanded null-safety analysis with additional configuration options
    • Broadened the set of sources/packages included in analysis
  • Refactor
    • Made internal compiler-related fields immutable for stability
  • Tests
    • Re-enabled a previously skipped benchmark test to run normally

@msridhar msridhar requested a review from yuxincs November 14, 2025 01:28
@codecov
Copy link
Copy Markdown

codecov Bot commented Nov 14, 2025 •
edited
Loading

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.40%. Comparing base (6a5e72a) to head (16b94ea).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files 
@@            Coverage Diff            @@
##             master    #1342   +/-   ##
=========================================
  Coverage     88.40%   88.40%           
  Complexity     2578     2578           
=========================================
  Files            97       97           
  Lines          8654     8654           
  Branches       1719     1719           
=========================================
  Hits           7651     7651           
  Misses          504      504           
  Partials        499      499

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Nov 14, 2025 •
edited
Loading

Walkthrough

The PR updates JMH benchmarking to use Caffeine 3.2.3 (was 3.0.2), adjusts the caffeine classpath filter, and modernizes three Gradle task declarations to tasks.register(...). CaffeineCompiler's source file set and annotated package scope are expanded and additional NullAway options (JSpecifyMode, CheckOptionalEmptiness, SuggestSuppressions, CheckContracts) are enabled. Five private fields in NullawayJavac are made final. The @Ignore annotation was removed from the testCaffeine test so it now runs.

Suggested reviewers

  • lazaroclapp
  • yuxincs

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Update Caffeine benchmark' is concise and clearly describes the main change—updating the Caffeine dependency and related benchmark configuration.
Linked Issues check ✅ Passed The PR successfully addresses issue #1108 by updating Caffeine from 3.0.2 to 3.2.3, modernizing Gradle task declarations, updating the source file list, and introducing JSpecify mode support.
Out of Scope Changes check ✅ Passed All changes are within scope: Caffeine version updates, Gradle modernization, source file list adjustments, and NullAway options related to the benchmark are all directly tied to updating the Caffeine benchmark.
✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch update-caffeine-benchmark
📜 Recent review details

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 82b6ce8 and 16b94ea.

📒 Files selected for processing (1)
  • jmh/build.gradle (4 hunks)
🧰 Additional context used
🧠 Learnings (1)
📓 Common learnings 
Learnt from: msridhar
Repo: uber/NullAway PR: 1245
File: guava-recent-unit-tests/src/test/java/com/uber/nullaway/guava/NullAwayGuavaParametricNullnessTests.java:101-102
Timestamp: 2025-08-14T18:50:06.159Z
Learning: In NullAway JSpecify tests, when JDK version requirements exist due to bytecode annotation reading capabilities, prefer failing tests over skipping them on unsupported versions to ensure CI catches regressions and enforces proper JDK version usage for developers.
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: Build and test on macos-latest
  • GitHub Check: Build and test on windows-latest
  • GitHub Check: Build and test on ubuntu-latest
🔇 Additional comments (2)
jmh/build.gradle (2)

48-48: Version updates are consistent and correctly applied across all Caffeine references.

The migration from Caffeine 3.0.2 to 3.2.3 is properly reflected in:

  • Sources dependency (line 48)
  • Runtime dependency (line 58)
  • Classpath filter pattern (line 101)

The filter on line 101 correctly excludes the main Caffeine JAR from the classpath to isolate only its transitive dependencies for benchmarking.

Also applies to: 58-58, 101-101

77-77: Gradle task declarations properly modernized to tasks.register() API.

All three task declarations have been correctly migrated from the legacy Groovy DSL (task foo(type: Copy) { ... }) to the modern Gradle API (tasks.register('foo', Copy) { ... }). The syntax, closure bodies, and task dependencies are all correct.

Also applies to: 82-82, 87-87

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Nov 14, 2025

Main Branch:

Benchmark                          Mode  Cnt   Score   Error  Units
AutodisposeBenchmark.compile      thrpt   25   9.966 ± 0.104  ops/s
CaffeineBenchmark.compile         thrpt   25   3.902 ± 0.023  ops/s
DFlowMicroBenchmark.compile       thrpt   25  35.086 ± 0.386  ops/s
NullawayReleaseBenchmark.compile  thrpt   25   1.547 ± 0.054  ops/s

With This PR:

Benchmark                          Mode  Cnt   Score   Error  Units
AutodisposeBenchmark.compile      thrpt   25  10.117 ± 0.053  ops/s
CaffeineBenchmark.compile         thrpt   25   2.070 ± 0.012  ops/s
DFlowMicroBenchmark.compile       thrpt   25  35.174 ± 0.146  ops/s
NullawayReleaseBenchmark.compile  thrpt   25   1.563 ± 0.026  ops/s

@msridhar msridhar merged commit 82784a2 into master Nov 14, 2025
12 checks passed
@msridhar msridhar deleted the update-caffeine-benchmark branch November 14, 2025 03:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yuxincs yuxincs yuxincs approved these changes

Assignees

No one assigned

Labels

run-benchmarks

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Update Caffeine version used for JMH benchmarking

2 participants

@msridhar @yuxincs