perf(cli): ограничить число живых документов в analyze обычным пулом#4263
perf(cli): ограничить число живых документов в analyze обычным пулом#4263nixel2007 wants to merge 2 commits into
Conversation
Batch analyze ran files on cliExecutor (a ForkJoinPool) via parallelStream, while each file's getDiagnostics blocks on CompletableFuture.join() to diagnosticComputerExecutor. join() inside a ForkJoinPool worker is managed blocking, so the pool spawns compensation threads to keep parallelism — and each extra worker materializes another DocumentContext. The natural "≈ cores" concurrency bound is lost: live documents (and threads) grow far past the core count (63 threads on a 4-core box; the issue reports 65 on many cores and 16-22 GiB RSS). Give analyze its own bounded executor: a fixed-size ThreadPoolExecutor (analyzeExecutor) wrapped in ContextPropagatingExecutorService, and submit one task per file instead of parallelStream. On a plain pool, a blocked worker does not compensate, so the number of simultaneously materialized documents is hard-capped at the pool size. cliExecutor (used by FormatCommand, no blocking join) and diagnosticComputerExecutor stay ForkJoinPool. Measured on SSL 3.2 (2184 files, 4 cores, warm cache), analyze --reporter sarif, identical output (72160 diagnostics before and after): - peak threads 62 -> 49; threads created over the run (JFR) 89 -> 74 - peak RSS 3081 MB -> 2607 MB (-15%) - wall time unchanged (98s) The effect scales with core count / blocking degree. Co-Authored-By: Claude Opus 4.8 <[email protected]> Claude-Session: https://claude.ai/code/session_01BSiRGLm633B4EmvG3vkk4V
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (2)
📝 WalkthroughWalkthroughAnalysis now uses a dedicated workspace-scoped bounded executor. ChangesAnalysis execution
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant AnalyzeCommand
participant analyzeExecutor
participant FileAnalysisTask
participant ProgressBar
AnalyzeCommand->>analyzeExecutor: submit one task per file
analyzeExecutor->>FileAnalysisTask: execute file analysis
FileAnalysisTask->>ProgressBar: advance when provided
analyzeExecutor-->>AnalyzeCommand: return futures
AnalyzeCommand->>AnalyzeCommand: collect FileInfo results
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
src/main/java/com/github/_1c_syntax/bsl/languageserver/infrastructure/ExecutorConfiguration.java (1)
160-188: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDuplicate workspace-resolution logic across factory methods.
createWorkspaceForkJoinPool(Lines 161-166) andcreateWorkspaceBoundedPool(Lines 173-178) repeat identicalWorkspaceContextHolder.get()/null-check/getName()resolution logic. Extract a shared private helper (e.g.resolveWorkspaceContext()) returning the URI/name pair to avoid drift between the two implementations.♻️ Suggested extraction
+ private record WorkspaceContext(URI uri, String name) {} + + private static WorkspaceContext resolveWorkspaceContext(String errorMessage) { + var workspaceUri = WorkspaceContextHolder.get(); + if (workspaceUri == null) { + throw new IllegalStateException(errorMessage); + } + var workspaceName = Optional.ofNullable(WorkspaceContextHolder.getName()) + .orElse("default"); + return new WorkspaceContext(workspaceUri, workspaceName); + } + private static ExecutorService createWorkspaceForkJoinPool(String prefix) { - var workspaceUri = WorkspaceContextHolder.get(); - if (workspaceUri == null) { - throw new IllegalStateException("Workspace context is not set when creating ForkJoinPool"); - } - var workspaceName = Optional.ofNullable(WorkspaceContextHolder.getName()) - .orElse("default"); - var factory = new WorkspaceAwareFJWTFactory(workspaceUri, workspaceName, prefix); + var ctx = resolveWorkspaceContext("Workspace context is not set when creating ForkJoinPool"); + var factory = new WorkspaceAwareFJWTFactory(ctx.uri(), ctx.name(), prefix); var pool = new ForkJoinPool(ForkJoinPool.getCommonPoolParallelism(), factory, null, true); return new ContextPropagatingExecutorService(pool); } private static ExecutorService createWorkspaceBoundedPool(String prefix) { - var workspaceUri = WorkspaceContextHolder.get(); - if (workspaceUri == null) { - throw new IllegalStateException("Workspace context is not set when creating executor"); - } - var workspaceName = Optional.ofNullable(WorkspaceContextHolder.getName()) - .orElse("default"); + var ctx = resolveWorkspaceContext("Workspace context is not set when creating executor"); var parallelism = ForkJoinPool.getCommonPoolParallelism(); - var factory = new NamedThreadFactory(prefix + workspaceName + "-"); + var factory = new NamedThreadFactory(prefix + ctx.name() + "-");🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/main/java/com/github/_1c_syntax/bsl/languageserver/infrastructure/ExecutorConfiguration.java` around lines 160 - 188, Extract the duplicated workspace URI and name resolution from createWorkspaceForkJoinPool and createWorkspaceBoundedPool into a shared private helper such as resolveWorkspaceContext(), returning both values and preserving the existing null validation and “default” name fallback. Update both factory methods to reuse that helper when constructing their thread factories.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@src/main/java/com/github/_1c_syntax/bsl/languageserver/cli/AnalyzeCommand.java`:
- Around line 215-235: Update analyzeFiles so any failure from future.get()
cancels all outstanding futures before propagating the exception. Preserve
completed results, ensure cancellation covers both running and queued tasks, and
keep the existing successful return behavior unchanged.
---
Nitpick comments:
In
`@src/main/java/com/github/_1c_syntax/bsl/languageserver/infrastructure/ExecutorConfiguration.java`:
- Around line 160-188: Extract the duplicated workspace URI and name resolution
from createWorkspaceForkJoinPool and createWorkspaceBoundedPool into a shared
private helper such as resolveWorkspaceContext(), returning both values and
preserving the existing null validation and “default” name fallback. Update both
factory methods to reuse that helper when constructing their thread factories.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 45d7d960-12c7-40ab-8809-94d6f18009c6
📒 Files selected for processing (2)
src/main/java/com/github/_1c_syntax/bsl/languageserver/cli/AnalyzeCommand.javasrc/main/java/com/github/_1c_syntax/bsl/languageserver/infrastructure/ExecutorConfiguration.java
…e resolution Address review feedback: - analyzeFiles now cancels not-yet-started futures (cancel(false)) when a future.get() fails, so the batch stops materializing documents once it can no longer succeed; running tasks are left alone (document processing is not guaranteed interruption-safe). - Extract the duplicated WorkspaceContextHolder resolution shared by createWorkspaceForkJoinPool/createWorkspaceBoundedPool into resolveWorkspaceContext. Co-Authored-By: Claude Opus 4.8 <[email protected]> Claude-Session: https://claude.ai/code/session_01BSiRGLm633B4EmvG3vkk4V
|
|
Закрываю в пользу #4264. #4264 чинит ту же первопричину (§4.2 #4248) на уровне Generated by Claude Code |



Описание
Batch-
analyzeгоняет файлы черезcliExecutor(ForkJoinPool) внутриparallelStream, аgetDiagnosticsкаждого файла блокируется наCompletableFuture.join()кdiagnosticComputerExecutor.join()внутри воркераForkJoinPool— это managed blocking: пул, чтобы держать номинальный параллелизм, поднимает компенсирующие потоки, и каждый лишний воркер материализует ещё одинDocumentContext. Естественный потолок «≈ число ядер» теряется — живых документов (и потоков) становится сильно больше, чем ядер.Это ровно претензия §4.2 из #4248 (в дампе —
65cli-потоков при13номинала и16–22 ГиБRSS). До ввода отдельных пулов (ExecutorConfiguration)analyzeиспользовал общийForkJoinPoolбез вложенного блокирующегоsubmit, и потолок «≈ ядра» держался сам собой.Фикс
Дать
analyzeсобственный ограниченный исполнитель: фиксированныйThreadPoolExecutor(analyzeExecutor) в обёрткеContextPropagatingExecutorService, и сабмитить по задаче на файл вместоparallelStream. На обычном пуле заблокированный воркер потоки не плодит, поэтому число одновременно материализованныхDocumentContextжёстко ограничено размером пула.cliExecutor(используетсяFormatCommand, без блокирующегоjoin) иdiagnosticComputerExecutorостаютсяForkJoinPool.ContextPropagatingExecutorService(per-task snapshot через micrometerThreadLocalAccessor), поэтому per-threadonStart-хукForkJoinPoolтут не нужен.Замеры (JFR)
SSL 3.2 (2184 файла, 4 ядра, тёплый кэш),
analyze --reporter sarif, вывод идентичен (72160 диагностик до и после):ForkJoinPool)jdk.ThreadStart)Wall без изменений (первый «183→98s» был целиком эффектом прогрева page-cache — привожу честно). Эффект тем больше, чем больше ядер/степень блокировок: на многоядерном ARA2 из #4248 (
65потоков,22 ГиБ) разница между «≈ ядра» и «65 живых документов» кратно крупнее.Связанные задачи
Пункт §4.2 архитектурного разбора #4248. Продолжение серии #4249–#4258.
Closes
Чеклист
Общие
AnalyzeCommandTest,FormatCommandTest— зелёные; вывод SARIF идентичен на реальном прогоне SSL)Дополнительно
Освобождение AST per-document уже делается (
getFileInfoFromFile→tryClearDocument), поэтому bounded-пул автоматически ограничивает и живой AST размером пула — доп. действий не требуется. Размер пула сейчас =commonPoolParallelism; при желании легко вынести в CLI-опцию/конфиг.Generated by Claude Code
Summary by CodeRabbit