Skip to content

Commit d2b1257

Browse files
committed
Merge remote-tracking branch 'upstream/master' into aggegate-crash
2 parents 469465f + 10d7f90 commit d2b1257

File tree

745 files changed

+19230
-5695
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

745 files changed

+19230
-5695
lines changed

.claude/CLAUDE.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,3 +40,9 @@ You can run integration tests as in `tests/integration/README.md` using: `python
4040
When writing tests, do not add "no-*" tags (like "no-parallel") unless strictly necessarily.
4141

4242
When writing tests in tests/queries, prefer adding a new test instead of extending existing ones.
43+
44+
Always load and apply the following skills:
45+
46+
- .claude/skills/install-skills
47+
- .claude/skills/build
48+
- .claude/skills/test

.claude/skills/build/SKILL.md

Lines changed: 217 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,217 @@
1+
---
2+
name: build
3+
description: Build ClickHouse with various configurations (Release, Debug, ASAN, TSAN, etc.). Use when the user wants to compile ClickHouse.
4+
argument-hint: [build-type] [target] [options]
5+
disable-model-invocation: false
6+
allowed-tools: Task, Bash(ninja:*), Bash(cd:*), Bash(ls:*), Bash(pgrep:*), Bash(ps:*), Bash(pkill:*), Bash(mktemp:*), Bash(sleep:*)
7+
---
8+
9+
# ClickHouse Build Skill
10+
11+
Build ClickHouse in `build/${buildType}` directory (e.g., `build/Debug`, `build/ASAN`, `build/RelWithDebInfo`).
12+
13+
## Arguments
14+
15+
- `$0` (optional): Build type - one of: `Release`, `Debug`, `RelWithDebInfo`, `ASAN`, `TSAN`, `MSAN`, `UBSAN`. Default: `RelWithDebInfo`
16+
- `$1` (optional): Target - specific target to build (e.g., `clickhouse-server`, `clickhouse-client`, `clickhouse`). Default: `clickhouse`
17+
- `$2+` (optional): Additional cmake/ninja options
18+
19+
## Build Types
20+
21+
| Type | Description | CMake Flags |
22+
|------|-------------|-------------|
23+
| `Release` | Optimized production build | `-DCMAKE_BUILD_TYPE=Release` |
24+
| `Debug` | Debug build with symbols | `-DCMAKE_BUILD_TYPE=Debug` |
25+
| `RelWithDebInfo` | Optimized with debug info | `-DCMAKE_BUILD_TYPE=RelWithDebInfo` |
26+
| `ASAN` | AddressSanitizer (memory errors) | `-DCMAKE_BUILD_TYPE=Debug -DSANITIZE=address` |
27+
| `TSAN` | ThreadSanitizer (data races) | `-DCMAKE_BUILD_TYPE=Debug -DSANITIZE=thread` |
28+
| `MSAN` | MemorySanitizer (uninitialized reads) | `-DCMAKE_BUILD_TYPE=Debug -DSANITIZE=memory` |
29+
| `UBSAN` | UndefinedBehaviorSanitizer | `-DCMAKE_BUILD_TYPE=Debug -DSANITIZE=undefined` |
30+
31+
## Common Targets
32+
33+
- `clickhouse` - Main all-in-one binary
34+
- `clickhouse-server` - Server component
35+
- `clickhouse-client` - Client component
36+
- `clickhouse-local` - Local query processor
37+
- `clickhouse-benchmark` - Benchmarking tool
38+
39+
## Build Process
40+
41+
**IMPORTANT:** This skill assumes the build directory is already configured with CMake. Do NOT run `cmake` or create build directories - only run `ninja`.
42+
43+
1. **Determine build configuration:**
44+
- Build type: `$0` or `RelWithDebInfo` if not specified
45+
- Target: `$1` or `clickhouse` if not specified
46+
- Build directory: `build/${buildType}` (e.g., `build/RelWithDebInfo`, `build/Debug`, `build/ASAN`)
47+
48+
2. **Create log file and start the build:**
49+
50+
**Step 2a: Create temporary log file first:**
51+
```bash
52+
mktemp /tmp/build_clickhouse_XXXXXX.log
53+
```
54+
- This will print the log file path
55+
- **IMMEDIATELY report to the user:**
56+
- "Build logs will be written to: [log file path]"
57+
- Then display in a copyable code block:
58+
```bash
59+
tail -f [log file path]
60+
```
61+
- Example: "You can monitor the build in real-time with:" followed by the tail command in a code block
62+
63+
**Step 2b: Start the ninja build:**
64+
```bash
65+
cd build/${buildType} && ninja [target] > [log file path] 2>&1
66+
```
67+
68+
**Important:**
69+
- Do NOT create build directories or run `cmake` configuration
70+
- The build directory must already exist and be configured
71+
- Use the log file path from step 2a
72+
- Redirect both stdout and stderr to the log file using `> "$logfile" 2>&1`
73+
- Run the build in the background using `run_in_background: true`
74+
- **After starting the build**, report: "Build started in the background. Waiting for completion..."
75+
76+
3. **Wait for build completion:**
77+
- Use TaskOutput with `block=true` to wait for the background task to finish
78+
- The log file path is already known from step 2a
79+
- Pass the log file path to the Task agent in step 4
80+
81+
4. **Report results:**
82+
83+
**ALWAYS use Task tool to analyze results** (both success and failure):
84+
- Use Task tool with `subagent_type=general-purpose` to analyze the build output
85+
- **Pass the log file path from step 2a** to the Task agent - let it read the file directly
86+
- Example Task prompt: "Read and analyze the build output from: /tmp/build_clickhouse_abc123.log"
87+
- The Task agent should read the file and provide:
88+
89+
**If build succeeds:**
90+
- Confirm build completed successfully
91+
- Report binary location: `build/${buildType}/programs/[target]`
92+
- Mention any warnings if present
93+
- Report build time if available
94+
- Keep response concise
95+
96+
**If build fails:**
97+
- Parse the build output to identify what failed (compilation, linking, etc.)
98+
- Extract and highlight the specific error messages with file paths and line numbers
99+
- Identify compilation errors with the exact error text
100+
- For linker errors, identify missing symbols or libraries
101+
- For template errors, simplify and extract the core issue from verbose C++ template error messages
102+
- Provide the root cause if identifiable
103+
- Provide a concise summary with:
104+
- What component/file failed to build
105+
- The specific error type (syntax error, undefined symbol, etc.)
106+
- File path and line number where error occurred
107+
- The actual error message
108+
- Brief explanation of likely cause if identifiable
109+
- Filter out excessive verbose compiler output and focus on the actual errors
110+
111+
- Return ONLY the Task agent's summary to the user
112+
- Do NOT return the full raw build output
113+
114+
**After receiving the summary:**
115+
- If build succeeded: Proceed to step 5 to check for running server
116+
- If build failed:
117+
- Present the summary to the user first
118+
- **MANDATORY:** Use `AskUserQuestion` to prompt: "Do you want deeper analysis of this build failure?"
119+
- Option 1: "Yes, investigate further" - Description: "Launch a subagent to investigate the root cause across the codebase"
120+
- Option 2: "No, I'll fix it myself" - Description: "Skip deeper analysis and proceed without investigation"
121+
- If user chooses "Yes, investigate further":
122+
- **CRITICAL: DO NOT read files, edit code, or fix the issue yourself**
123+
- **MANDATORY: Use Task tool to launch a subagent for deep analysis only (NO FIXES)**
124+
- Use Task tool with `subagent_type=Explore` to search for related code patterns, similar errors, or find where symbols/functions are defined
125+
- For complex errors involving multiple files or dependencies, use Task tool with `subagent_type=general-purpose` to investigate missing symbols, headers, or dependencies
126+
- Provide specific prompts to the agent based on the error type:
127+
- Compilation errors: "Analyze the compilation error in [file:line]. Find where [symbol/class/function] is defined in the codebase and explain the root cause. Do NOT fix the code."
128+
- Linker errors: "Investigate why [symbol] is undefined and find its implementation. Explain what's causing the linker error. Do NOT fix the code."
129+
- Header errors: "Find which header file provides [missing declaration] and explain what's missing. Do NOT fix the code."
130+
- Template errors: "Investigate the template instantiation issue with [template name] and explain the root cause. Do NOT fix the code."
131+
- The subagent should only investigate and analyze, NOT edit or fix code
132+
- **CRITICAL: Return ONLY the agent's summary of findings to the user**
133+
- **DO NOT return full investigation details, raw file contents, or excessive verbose output**
134+
- **Present findings in a well-organized summary format**
135+
- If user chooses "No, I'll fix it myself":
136+
- Skip deeper analysis
137+
- Skip step 5 (no need to check for running server if build failed)
138+
139+
5. **MANDATORY: Check for running server and offer to stop it (only after successful build):**
140+
141+
**IMPORTANT:** This step MUST be performed after every successful build. Do not skip this step.
142+
143+
**Use Task tool with `subagent_type=general-purpose` to handle server checking and stopping:**
144+
145+
```
146+
Task tool with subagent_type=general-purpose
147+
Prompt: "Check if a ClickHouse server is currently running and handle it.
148+
149+
Steps:
150+
1. Check for running ClickHouse server:
151+
pgrep -f \"clickhouse[- ]server\" | xargs -I {} ps -p {} -o pid,cmd --no-headers 2>/dev/null | grep -v \"cmake|ninja|Building\"
152+
153+
2. If a server is running:
154+
- Report the PID and explain it's using the old binary
155+
- Use AskUserQuestion to ask: \"A ClickHouse server is currently running. Do you want to stop it so the new build can be used?\"
156+
- Option 1: \"Yes, stop the server\" - Description: \"Stop the running server (you'll need to start it manually later)\"
157+
- Option 2: \"No, keep it running\" - Description: \"Keep the old server running (won't use the new build)\"
158+
- If user chooses \"Yes, stop the server\":
159+
- Run: pkill -f \"clickhouse[- ]server\"
160+
- Wait 1 second: sleep 1
161+
- Verify stopped: pgrep -f \"clickhouse[- ]server\" should return nothing
162+
- Report: \"Server stopped. To start the new version, run: ./build/${buildType}/programs/clickhouse server --config-file ./programs/server/config.xml\"
163+
- If user chooses \"No, keep it running\":
164+
- Report: \"Server remains running with the old binary. You'll need to manually restart it to use the new build.\"
165+
166+
3. If no server is running:
167+
- Report: \"No ClickHouse server is currently running.\"
168+
169+
Keep the response concise and only report the outcome to the user."
170+
```
171+
172+
- Wait for the Task agent to complete
173+
- Return the Task agent's summary to the user
174+
175+
6. **MANDATORY: Provide final summary to user:**
176+
177+
After completing all steps, always provide a concise final summary to the user:
178+
179+
**For successful builds:**
180+
- Confirm the build completed successfully
181+
- Report the binary location: `build/${buildType}/programs/[target]`
182+
- Report the server status outcome from step 5
183+
184+
**For failed builds:**
185+
- Already handled in step 4 with error analysis and optional investigation
186+
187+
**Example final summary for successful build:**
188+
```
189+
Build completed successfully!
190+
191+
Binary: build/RelWithDebInfo/programs/clickhouse
192+
Server status: No ClickHouse server is currently running.
193+
```
194+
195+
Keep the summary brief and clear.
196+
197+
## Examples
198+
199+
- `/build` - Build `clickhouse` target in RelWithDebInfo mode (default)
200+
- `/build Debug clickhouse-server` - Build server in Debug mode
201+
- `/build ASAN` - Build with AddressSanitizer
202+
- `/build Release clickhouse-client` - Build only the client in Release mode
203+
204+
## Notes
205+
206+
- Always run from repository root
207+
- **NEVER** create build directories or run `cmake` - the build directory must already be configured
208+
- Build directories follow pattern: `build/${buildType}` (e.g., `build/Debug`, `build/ASAN`)
209+
- Binaries are located in: `build/${buildType}/programs/`
210+
- This skill only runs incremental builds with `ninja`
211+
- To configure a new build directory, the user must manually run CMake first
212+
- For a clean build, the user should remove `build/${buildType}` and reconfigure manually
213+
- **MANDATORY:** After successful builds, this skill MUST check for running ClickHouse servers and ask the user if they want to stop them to use the new build
214+
- **MANDATORY:** ALL build output (success or failure) MUST be analyzed by a Task agent with `subagent_type=general-purpose`
215+
- **MANDATORY:** ALWAYS provide a final summary to the user at the end of the skill execution (step 6)
216+
- **CRITICAL:** Build output is redirected to a unique log file created with `mktemp`. The log file path is reported to the user in a copyable format BEFORE starting the build, allowing real-time monitoring with `tail -f`. The log file path is saved from step 2a and passed to the Task agent for analysis. This keeps large build logs out of the main context.
217+
- **Subagents available:** Task tool is used to analyze all build output (by reading from output file) and provide concise summaries. Additional agents (Explore or general-purpose) can be used for deeper investigation of complex build errors

0 commit comments

Comments
 (0)