Skip to content

Comments

ci(transformer): enable unfinished plugins in benchmark#7040

Merged
graphite-app[bot] merged 1 commit intomainfrom
10-31-ci_transformer_enable_unfinished_plugins_in_benchmark
Nov 2, 2024
Merged

ci(transformer): enable unfinished plugins in benchmark#7040
graphite-app[bot] merged 1 commit intomainfrom
10-31-ci_transformer_enable_unfinished_plugins_in_benchmark

Conversation

@Dunqing
Copy link
Member

@Dunqing Dunqing commented Oct 31, 2024

Even the plugins are unfinished, we still want to enable all of them to track the performance changes during the development.

@graphite-app
Copy link
Contributor

graphite-app bot commented Oct 31, 2024

Your org has enabled the Graphite merge queue for merging into main

Add the label “0-merge” to the PR and Graphite will automatically add it to the merge queue when it’s ready to merge. Or use the label “hotfix” to add to the merge queue as a hot fix.

You must have a Graphite account and log in to Graphite in order to use the merge queue. Sign up using this link.

@github-actions github-actions bot added the A-transformer Area - Transformer / Transpiler label Oct 31, 2024
Copy link
Member Author

Dunqing commented Oct 31, 2024

@Dunqing Dunqing force-pushed the 10-31-ci_transformer_enable_unfinished_plugins_in_benchmark branch from aae39c9 to 3aacc67 Compare October 31, 2024 15:58
@codspeed-hq
Copy link

codspeed-hq bot commented Oct 31, 2024

CodSpeed Performance Report

Merging #7040 will degrade performances by 17.14%

Comparing 10-31-ci_transformer_enable_unfinished_plugins_in_benchmark (70e2582) with main (001058a)

Summary

❌ 4 regressions
✅ 26 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark main 10-31-ci_transformer_enable_unfinished_plugins_in_benchmark Change
transformer[antd.js] 41.1 ms 43.9 ms -6.4%
transformer[cal.com.tsx] 29.7 ms 33.2 ms -10.7%
transformer[checker.ts] 15.6 ms 17.1 ms -8.87%
transformer[pdf.mjs] 5.2 ms 6.3 ms -17.14%

@overlookmotel
Copy link
Member

I agree that it's good to measure all our transforms, even the unfinished ones, in benchmarks.

However, enable_all is a public API (like Babel's "force all transforms" option), so in my opinion it seems a bit odd to offer an "include unfinished plugins" option, as that's a bit like offering a "break all your code" option!

I think it'd be preferable to leave enable_all as it is, and manually enable the unfinished transforms in tasks/benchmark/benches/transformer.rs.

Or, if you want to locate this logic in oxc_transformer without exposing it to the user:

impl EnvOptions {
    pub fn enable_all() -> Self {
        Self::enable_all_impl(false)
    }

    /// For internal use only
    #[doc(hidden)]
    pub fn enable_all_impl(include_unfinished_plugins: bool) -> Self {
        // Same as impl in this PR
    }
}

Benchmarks can use enable_all_impl, but we don't "officially" expose ability to run our shonky half-finished transforms to the user.

@Dunqing
Copy link
Member Author

Dunqing commented Nov 1, 2024

However, enable_all is a public API (like Babel's "force all transforms" option), so in my opinion it seems a bit odd to offer an "include unfinished plugins" option, as that's a bit like offering a "break all your code" option!

I think it'd be preferable to leave enable_all as it is, and manually enable the unfinished transforms in tasks/benchmark/benches/transformer.rs.

We tend to forget to enable unfinished plugins in the benchmark easily. So I am not a fan of this way

Or, if you want to locate this logic in oxc_transformer without exposing it to the user:

impl EnvOptions {
    pub fn enable_all() -> Self {
        Self::enable_all_impl(false)
    }

    /// For internal use only
    #[doc(hidden)]
    pub fn enable_all_impl(include_unfinished_plugins: bool) -> Self {
        // Same as impl in this PR
    }
}

Benchmarks can use enable_all_impl, but we don't "officially" expose ability to run our shonky half-finished transforms to the user.

IMO, enable_all is also an internal-only API. So we can add

/// For internal use only
#[doc(hidden)]

to enable_all method. Tell me what's your thought about this

@overlookmotel
Copy link
Member

overlookmotel commented Nov 1, 2024

@Boshen Is enable_all meant to be a public method or not? I thought maybe it is (like Babel's "force all transforms" option).

@Boshen
Copy link
Member

Boshen commented Nov 1, 2024

@Boshen Is enable_all meant to be a public method or not? I thought maybe it is (like Babel's "force all transforms" option).

This is not a public API, but I need this in monitor-oxc.

@overlookmotel
Copy link
Member

OK great. @Dunqing I had misunderstood. I had thought enable_all was a public API.

Since it's not, this PR with this added to enable_all (as you suggested), seems good to me:

/// For internal use only
#[doc(hidden)]

Sorry I wasted time!

@Boshen Boshen added the 0-merge Merge with Graphite Merge Queue label Nov 2, 2024
Copy link
Member

Boshen commented Nov 2, 2024

Merge activity

  • Nov 2, 2:17 AM EDT: The merge label '0-merge' was detected. This PR will be added to the Graphite merge queue once it meets the requirements.
  • Nov 2, 2:18 AM EDT: Graphite disabled "merge when ready" on this PR due to: a merge conflict with the target branch; resolve the conflict and try again..
  • Nov 2, 2:18 AM EDT: The merge label '0-merge' was detected. This PR will be added to the Graphite merge queue once it meets the requirements.
  • Nov 2, 2:26 AM EDT: A user added this pull request to the Graphite merge queue.
  • Nov 2, 2:27 AM EDT: The Graphite merge queue couldn't merge this PR because it had conflicts with the trunk branch.
  • Nov 2, 11:16 AM EDT: The merge label '0-merge' was detected. This PR will be added to the Graphite merge queue once it meets the requirements.
  • Nov 2, 11:17 AM EDT: A user added this pull request to the Graphite merge queue.
  • Nov 2, 11:22 AM EDT: A user merged this pull request with the Graphite merge queue.

@graphite-app graphite-app bot removed the 0-merge Merge with Graphite Merge Queue label Nov 2, 2024
@Dunqing Dunqing force-pushed the 10-31-ci_transformer_enable_unfinished_plugins_in_benchmark branch 2 times, most recently from d2b9343 to fe6818b Compare November 2, 2024 15:07
@Dunqing Dunqing added the 0-merge Merge with Graphite Merge Queue label Nov 2, 2024
Even the plugins are unfinished, we still want to enable all of them to track the performance changes during the development.
@Dunqing Dunqing force-pushed the 10-31-ci_transformer_enable_unfinished_plugins_in_benchmark branch from fe6818b to 70e2582 Compare November 2, 2024 15:17
@graphite-app graphite-app bot merged commit 70e2582 into main Nov 2, 2024
@graphite-app graphite-app bot deleted the 10-31-ci_transformer_enable_unfinished_plugins_in_benchmark branch November 2, 2024 15:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

0-merge Merge with Graphite Merge Queue A-transformer Area - Transformer / Transpiler

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants