RUM-14050: Reduce CI pipeline time via parallelism and Gradle consolidation#3559
Conversation
The analysis:detekt-custom job runs custom Detekt rules by invoking ./gradlew customDetektRules, which sequentially executes 31 independent JavaExec tasks — one JVM per module. Because the tasks share no state and declare no inter-dependencies, they are safe to run concurrently. Adding --parallel --max-workers=4 reduces step 6 of the job from ~155s to ~69s (2.2× speedup) on a local benchmark of all 31 modules. Each worker JVM is capped at maxHeapSize=2g to bound heap usage. The job's Kubernetes memory limit is increased to 32Gi (matching datadog-android) to accommodate 4 concurrent JVMs: 4 × ~3.25Gi (heap + metaspace) + 4Gi Gradle daemon ≈ 18Gi, well within the new limit. # Conflicts: # ci/pipelines/default-pipeline.yml
assembleLibrariesRelease is the dominant step in analysis:detekt-custom, taking ~8m 17s (56% of the job) in CI. Adding --parallel enables Gradle to compile independent library modules concurrently within the same daemon process (in-process workers sharing the 4Gi heap) rather than sequentially. Unlike customDetektRules --parallel which spawns separate JVMs per task, --parallel on assembleLibrariesRelease uses Gradle's built-in worker infrastructure — no additional memory per concurrent task. The module dependency graph has a natural fan-out: dd-sdk-android-internal and dd-sdk-android-core must compile first, then 25+ feature and integration modules can compile in parallel.
Move KUBERNETES_MEMORY_LIMIT from 13Gi to 32Gi in the root .gitlab-ci.yml, matching datadog-android which uses the same limit on the same runner pool. The higher limit is needed for jobs that run parallel JVM workers (analysis:detekt-custom, test:debug). Setting it globally is simpler than per-job overrides and has no scheduling cost since KUBERNETES_MEMORY_REQUEST remains at 8Gi — the scheduler still only needs 8Gi free on the node. Actual memory consumed per job is unchanged for jobs that don't use the extra headroom. Removes the now-redundant KUBERNETES_MEMORY_LIMIT: 32Gi override from analysis:detekt-custom.
…--parallel test:debug ran 5 separate ./gradlew --no-daemon invocations sequentially. Each call pays the full Gradle startup and project configuration overhead (~10-15s in CI × 4 extra calls = ~40-60s wasted). Consolidating into a single invocation eliminates that redundancy. --parallel --max-workers=4 lets Gradle run up to 4 modules' test tasks concurrently. Each module forks its own test JVM (~768MB); 4 concurrent test JVMs + 3Gi Gradle JVM ≈ 6Gi total — well within the 32Gi limit now set globally. Note: unlike assembleLibrariesRelease --parallel (in-process workers, shared heap), test tasks fork separate JVMs. --max-workers=4 acts as the memory safety valve, the same role it plays for customDetektRules.
…--parallel test:kover ran 4 separate ./gradlew --no-daemon invocations for coverage generation, paying ~33s of Gradle startup overhead across 3 extra calls. Consolidating into a single invocation using the >- YAML folded scalar eliminates that overhead (same fix applied to test:debug). --parallel --max-workers=4 lets Gradle run up to 4 modules' koverXmlReport tasks concurrently. koverXmlReportRelease forks a test JVM per module for instrumented coverage tracking (~1.5-2GB each); 4 workers stay well within the 32Gi global limit set in the previous commit. test:kover is the current critical path of the test stage (~17m 55s avg) now that test:debug has been reduced to ~7m 57s.
|
@codex review |
|
Codex Review: Didn't find any major issues. Breezy! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
kikoveiga
left a comment
There was a problem hiding this comment.
Very nice! Is it also possible to add some of these --parallel optimizations to local_ci.sh?
Ideally this should live in |
hamorillo
left a comment
There was a problem hiding this comment.
Very nice! Is it also possible to add some of these --parallel optimizations to local_ci.sh?
Ideally this should live in gradle.properties that is commited to Git. However on CI we get gradle.properties from secrets that complicates things a bit.
Not sure if I understand what you mean with the gradle.properties. I would say we can leverage a similar approach in the local-ci.sh for some gradlew tasks. (In fact, that was something I wanted to add in following PRs).
I mean doing essentially the same thing as |
I see what you mean 👍 |
we can think about it in future PRs current PR LGTM |
…analysis-optimization RUM-14050: Reduce CI pipeline time via parallelism and Gradle consolidation
What does this PR do?
Optimises the
analysis:detekt-customandteststage jobs by introducing Gradle parallel execution and consolidating redundant sequential./gradlew --no-daemoninvocations. No changes to what is compiled or tested.Motivation
analysis:detekt-customran its assembly and custom rules sequentially.test:debugandtest:kovereach paid full Gradle JVM startup and project-configuration overhead across 5 and 4 separate invocations respectively, adding unnecessary wall-clock time to every PR pipeline.Changes
.gitlab-ci.yml—KUBERNETES_MEMORY_LIMIT: 13Gi → 32Giglobally (matchingdatadog-androidon the same runner pool; no infrastructure request needed).DetektCustomConfig.kt—maxHeapSize = "2g"oncustomDetektRulesto bound per-worker JVM heap for safe parallel execution.ci/pipelines/default-pipeline.yml:analysis:detekt-customassembleLibrariesRelease --parallel(in-process workers, shared daemon heap)analysis:detekt-customcustomDetektRules --parallel --max-workers=4(31 JVMs sequential → 4 concurrent, heap capped at 2g each)test:debug./gradlew --no-daemoncalls → 1 consolidated with--parallel --max-workers=4test:kover./gradlew --no-daemoncalls → 1 consolidated with--parallel --max-workers=4Note:
>-YAML folded scalar is used for multiline commands — shell\continuation breaks in GitLab CI YAML because the backslash is preserved literally and becomes an escaped space, causing Gradle to reject task paths.Measured results
Baseline: 27 recent PRs on
develop(excluding this branch).analysisstageanalysis:detekt-customteststagetest:debugtest:koverTotal pipeline wall-clock
Review checklist (to be filled by reviewers)