Skip to content

perf(cli): ограничить число живых документов в analyze обычным пулом#4263

Closed
nixel2007 wants to merge 2 commits into
developfrom
claude/perf-analyze-bounded-executor
Closed

perf(cli): ограничить число живых документов в analyze обычным пулом#4263
nixel2007 wants to merge 2 commits into
developfrom
claude/perf-analyze-bounded-executor

Conversation

@nixel2007

@nixel2007 nixel2007 commented Jul 11, 2026

Copy link
Copy Markdown
Member

Описание

Batch-analyze гоняет файлы через cliExecutor (ForkJoinPool) внутри parallelStream, а getDiagnostics каждого файла блокируется на CompletableFuture.join() к diagnosticComputerExecutor. join() внутри воркера ForkJoinPool — это managed blocking: пул, чтобы держать номинальный параллелизм, поднимает компенсирующие потоки, и каждый лишний воркер материализует ещё один DocumentContext. Естественный потолок «≈ число ядер» теряется — живых документов (и потоков) становится сильно больше, чем ядер.

Это ровно претензия §4.2 из #4248 (в дампе — 65 cli-потоков при 13 номинала и 16–22 ГиБ RSS). До ввода отдельных пулов (ExecutorConfiguration) analyze использовал общий ForkJoinPool без вложенного блокирующего submit, и потолок «≈ ядра» держался сам собой.

Фикс

Дать analyze собственный ограниченный исполнитель: фиксированный ThreadPoolExecutor (analyzeExecutor) в обёртке ContextPropagatingExecutorService, и сабмитить по задаче на файл вместо parallelStream. На обычном пуле заблокированный воркер потоки не плодит, поэтому число одновременно материализованных DocumentContext жёстко ограничено размером пула.

  • cliExecutor (используется FormatCommand, без блокирующего join) и diagnosticComputerExecutor остаются ForkJoinPool.
  • Контекст workspace для сабмиченных задач несёт ContextPropagatingExecutorService (per-task snapshot через micrometer ThreadLocalAccessor), поэтому per-thread onStart-хук ForkJoinPool тут не нужен.

Замеры (JFR)

SSL 3.2 (2184 файла, 4 ядра, тёплый кэш), analyze --reporter sarif, вывод идентичен (72160 диагностик до и после):

было (ForkJoinPool) стало (bounded)
пик потоков 62 49
потоков создано за прогон (JFR jdk.ThreadStart) 89 74
пик RSS 3081 MB 2607 MB (−15%)
wall 98 s 98 s (без изменений)
SARIF 72160 72160

Wall без изменений (первый «183→98s» был целиком эффектом прогрева page-cache — привожу честно). Эффект тем больше, чем больше ядер/степень блокировок: на многоядерном ARA2 из #4248 (65 потоков, 22 ГиБ) разница между «≈ ядра» и «65 живых документов» кратно крупнее.

Связанные задачи

Пункт §4.2 архитектурного разбора #4248. Продолжение серии #4249#4258.

Closes

Чеклист

Общие

  • Ветка PR обновлена из develop
  • Отладочные, закомментированные и прочие, не имеющие смысла участки кода удалены
  • Изменения покрыты тестами (AnalyzeCommandTest, FormatCommandTest — зелёные; вывод SARIF идентичен на реальном прогоне SSL)

Дополнительно

Освобождение AST per-document уже делается (getFileInfoFromFiletryClearDocument), поэтому bounded-пул автоматически ограничивает и живой AST размером пула — доп. действий не требуется. Размер пула сейчас = commonPoolParallelism; при желании легко вынести в CLI-опцию/конфиг.


Generated by Claude Code

Summary by CodeRabbit

  • Performance
    • Improved analysis of multiple files using a dedicated, workspace-bounded execution approach.
  • Bug Fixes
    • Progress indicators now update more reliably during ongoing multi-file analysis.
  • Improvements
    • Analysis results are now gathered more consistently after all submitted work finishes.

Batch analyze ran files on cliExecutor (a ForkJoinPool) via parallelStream,
while each file's getDiagnostics blocks on CompletableFuture.join() to
diagnosticComputerExecutor. join() inside a ForkJoinPool worker is managed
blocking, so the pool spawns compensation threads to keep parallelism — and
each extra worker materializes another DocumentContext. The natural "≈ cores"
concurrency bound is lost: live documents (and threads) grow far past the core
count (63 threads on a 4-core box; the issue reports 65 on many cores and
16-22 GiB RSS).

Give analyze its own bounded executor: a fixed-size ThreadPoolExecutor
(analyzeExecutor) wrapped in ContextPropagatingExecutorService, and submit one
task per file instead of parallelStream. On a plain pool, a blocked worker
does not compensate, so the number of simultaneously materialized documents is
hard-capped at the pool size. cliExecutor (used by FormatCommand, no blocking
join) and diagnosticComputerExecutor stay ForkJoinPool.

Measured on SSL 3.2 (2184 files, 4 cores, warm cache), analyze --reporter
sarif, identical output (72160 diagnostics before and after):
- peak threads 62 -> 49; threads created over the run (JFR) 89 -> 74
- peak RSS 3081 MB -> 2607 MB (-15%)
- wall time unchanged (98s)
The effect scales with core count / blocking degree.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01BSiRGLm633B4EmvG3vkk4V
@coderabbitai

coderabbitai Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 4bf4b19f-5877-4629-a24f-505280c1367e

📥 Commits

Reviewing files that changed from the base of the PR and between d843b3c and aef8c3e.

📒 Files selected for processing (2)
  • src/main/java/com/github/_1c_syntax/bsl/languageserver/cli/AnalyzeCommand.java
  • src/main/java/com/github/_1c_syntax/bsl/languageserver/infrastructure/ExecutorConfiguration.java
🚧 Files skipped from review as they are similar to previous changes (2)
  • src/main/java/com/github/_1c_syntax/bsl/languageserver/cli/AnalyzeCommand.java
  • src/main/java/com/github/_1c_syntax/bsl/languageserver/infrastructure/ExecutorConfiguration.java

📝 Walkthrough

Walkthrough

Analysis now uses a dedicated workspace-scoped bounded executor. AnalyzeCommand submits per-file tasks, optionally updates progress, collects FileInfo results from futures, and cancels pending tasks when collection fails.

Changes

Analysis execution

Layer / File(s) Summary
Bounded analysis executor
src/main/java/com/github/_1c_syntax/bsl/languageserver/infrastructure/ExecutorConfiguration.java
Adds a workspace-scoped bounded analyzeExecutor() with context resolution, daemon thread naming, and shutdown handling.
Per-file analysis flow
src/main/java/com/github/_1c_syntax/bsl/languageserver/cli/AnalyzeCommand.java
Routes silent and non-silent analysis through analyzeFiles(...), submits one task per file, optionally advances the progress bar, collects future results, and cancels pending tasks after failures.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant AnalyzeCommand
  participant analyzeExecutor
  participant FileAnalysisTask
  participant ProgressBar
  AnalyzeCommand->>analyzeExecutor: submit one task per file
  analyzeExecutor->>FileAnalysisTask: execute file analysis
  FileAnalysisTask->>ProgressBar: advance when provided
  analyzeExecutor-->>AnalyzeCommand: return futures
  AnalyzeCommand->>AnalyzeCommand: collect FileInfo results
Loading
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 7.69% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: analyze now uses a bounded executor to limit live documents.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/perf-analyze-bounded-executor

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
src/main/java/com/github/_1c_syntax/bsl/languageserver/infrastructure/ExecutorConfiguration.java (1)

160-188: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Duplicate workspace-resolution logic across factory methods.

createWorkspaceForkJoinPool (Lines 161-166) and createWorkspaceBoundedPool (Lines 173-178) repeat identical WorkspaceContextHolder.get()/null-check/getName() resolution logic. Extract a shared private helper (e.g. resolveWorkspaceContext()) returning the URI/name pair to avoid drift between the two implementations.

♻️ Suggested extraction
+  private record WorkspaceContext(URI uri, String name) {}
+
+  private static WorkspaceContext resolveWorkspaceContext(String errorMessage) {
+    var workspaceUri = WorkspaceContextHolder.get();
+    if (workspaceUri == null) {
+      throw new IllegalStateException(errorMessage);
+    }
+    var workspaceName = Optional.ofNullable(WorkspaceContextHolder.getName())
+      .orElse("default");
+    return new WorkspaceContext(workspaceUri, workspaceName);
+  }
+
   private static ExecutorService createWorkspaceForkJoinPool(String prefix) {
-    var workspaceUri = WorkspaceContextHolder.get();
-    if (workspaceUri == null) {
-      throw new IllegalStateException("Workspace context is not set when creating ForkJoinPool");
-    }
-    var workspaceName = Optional.ofNullable(WorkspaceContextHolder.getName())
-      .orElse("default");
-    var factory = new WorkspaceAwareFJWTFactory(workspaceUri, workspaceName, prefix);
+    var ctx = resolveWorkspaceContext("Workspace context is not set when creating ForkJoinPool");
+    var factory = new WorkspaceAwareFJWTFactory(ctx.uri(), ctx.name(), prefix);
     var pool = new ForkJoinPool(ForkJoinPool.getCommonPoolParallelism(), factory, null, true);
     return new ContextPropagatingExecutorService(pool);
   }

   private static ExecutorService createWorkspaceBoundedPool(String prefix) {
-    var workspaceUri = WorkspaceContextHolder.get();
-    if (workspaceUri == null) {
-      throw new IllegalStateException("Workspace context is not set when creating executor");
-    }
-    var workspaceName = Optional.ofNullable(WorkspaceContextHolder.getName())
-      .orElse("default");
+    var ctx = resolveWorkspaceContext("Workspace context is not set when creating executor");
     var parallelism = ForkJoinPool.getCommonPoolParallelism();
-    var factory = new NamedThreadFactory(prefix + workspaceName + "-");
+    var factory = new NamedThreadFactory(prefix + ctx.name() + "-");
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@src/main/java/com/github/_1c_syntax/bsl/languageserver/infrastructure/ExecutorConfiguration.java`
around lines 160 - 188, Extract the duplicated workspace URI and name resolution
from createWorkspaceForkJoinPool and createWorkspaceBoundedPool into a shared
private helper such as resolveWorkspaceContext(), returning both values and
preserving the existing null validation and “default” name fallback. Update both
factory methods to reuse that helper when constructing their thread factories.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@src/main/java/com/github/_1c_syntax/bsl/languageserver/cli/AnalyzeCommand.java`:
- Around line 215-235: Update analyzeFiles so any failure from future.get()
cancels all outstanding futures before propagating the exception. Preserve
completed results, ensure cancellation covers both running and queued tasks, and
keep the existing successful return behavior unchanged.

---

Nitpick comments:
In
`@src/main/java/com/github/_1c_syntax/bsl/languageserver/infrastructure/ExecutorConfiguration.java`:
- Around line 160-188: Extract the duplicated workspace URI and name resolution
from createWorkspaceForkJoinPool and createWorkspaceBoundedPool into a shared
private helper such as resolveWorkspaceContext(), returning both values and
preserving the existing null validation and “default” name fallback. Update both
factory methods to reuse that helper when constructing their thread factories.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 45d7d960-12c7-40ab-8809-94d6f18009c6

📥 Commits

Reviewing files that changed from the base of the PR and between 488c6af and d843b3c.

📒 Files selected for processing (2)
  • src/main/java/com/github/_1c_syntax/bsl/languageserver/cli/AnalyzeCommand.java
  • src/main/java/com/github/_1c_syntax/bsl/languageserver/infrastructure/ExecutorConfiguration.java

…e resolution

Address review feedback:
- analyzeFiles now cancels not-yet-started futures (cancel(false)) when a
  future.get() fails, so the batch stops materializing documents once it can no
  longer succeed; running tasks are left alone (document processing is not
  guaranteed interruption-safe).
- Extract the duplicated WorkspaceContextHolder resolution shared by
  createWorkspaceForkJoinPool/createWorkspaceBoundedPool into resolveWorkspaceContext.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01BSiRGLm633B4EmvG3vkk4V
@sonarqubecloud

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown
Contributor

Test Results

 3 618 files  ±0   3 618 suites  ±0   1h 44m 28s ⏱️ + 5m 50s
 3 588 tests ±0   3 570 ✅ ±0   18 💤 ±0  0 ❌ ±0 
21 528 runs  ±0  21 416 ✅ ±0  112 💤 ±0  0 ❌ ±0 

Results for commit aef8c3e. ± Comparison against base commit 488c6af.

Copy link
Copy Markdown
Member Author

Закрываю в пользу #4264.

#4264 чинит ту же первопричину (§4.2 #4248) на уровне DefaultDiagnosticComputer, а не только внешней оси analyze: убирает managed-blocking компенсацию на всех путях, где диагностики считаются из воркера ForkJoinPool, — включая LSP AnalyzeProjectOnStart, которого этот PR не касался. Замеры: thread-starts на блокирующем пуле 17→3 и в analyze, и в LSP; пик одновременных DocumentContext 16→3; вывод идентичен.


Generated by Claude Code

@nixel2007 nixel2007 closed this Jul 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants