docs: Fix Chat SDK guide based on manual testing#194
Conversation
Updates docs/guides/chat.mdx with corrections from testing against actual CLI: - Add missing interactive commands organized by category: * Session management: /resume, /save, /sessions, /reset * Document management: /index, /watch, /list, /status * Debug commands: /chunks, /chunk, /test, /dump, /clear-cache, /search-debug - Add Voice mode prerequisite note for talk module installation - Add VLM model requirement note for PDF indexing with images - Add troubleshooting for "No module named 'pip'" error in gaia talk - Improve PDF troubleshooting with specific error guidance Fixes documentation gaps identified in manual testing transcripts.
…uite for Chat SDK - Created a new markdown report detailing the performance analysis of various LLMs in meeting summarization, including key metrics, common challenges, model-specific analysis, cost efficiency, and immediate improvement actions. - Introduced a new test script to validate code samples from the Chat SDK documentation, ensuring API correctness and identifying documentation issues without relying on expensive PDF image processing.
Separates basic chat commands from RAG-specific commands for clarity: CLI Usage > Interactive Mode: - Basic Commands (conversation management) - Session Management (conversation-focused descriptions) - Forward reference to RAG section for document commands Document Q&A (RAG) > Interactive RAG Commands: - Session Management (now shows RAG state preservation) - Document Management (/index, /watch, /list, /status) - Debug & Observability (/chunks, /test, /dump, /clear-cache) This organization: - Lists session commands in both sections with context-appropriate descriptions - Prevents RAG commands from appearing before RAG is explained - Matches user mental model: chat features vs document features - Makes command availability dependencies clear Resolves confusion about which commands require RAG to be enabled.
Changed from "Ensure the VLM model is installed" to "The model will be automatically downloaded on first use" to reflect actual behavior. Models auto-download via lemonade_client with auto_download=True.
Removed /clear, /history, and /stats from CLI Interactive Mode section. These commands exist in ChatSDK.start_interactive_session() but NOT in the gaia chat CLI tool (src/gaia/agents/chat/app.py). Actual CLI commands verified from source code: - Session: /resume, /save, /sessions, /reset - Document: /index, /watch, /list, /status - Debug: /chunks, /chunk, /test, /dump, /clear-cache, /search-debug - Other: /help, /?, /quit Fixes documentation to match actual CLI behavior.
Corrected ChatConfig to match actual usage: - assistant_name: "gaia" (was incorrectly "assistant") - model: Show recommended "Qwen3-Coder-30B-A3B-Instruct-GGUF" (CPU failsafe model omitted as it's only for unsupported hardware) Verified against src/gaia/chat/sdk.py
There was a problem hiding this comment.
Pull request overview
This PR updates the Chat SDK documentation to accurately reflect the CLI commands and capabilities based on manual testing. It reorganizes the interactive commands section, adds missing RAG-specific commands, includes prerequisite notes for voice mode and PDF processing with images, and adds troubleshooting guidance.
Changes:
- Reorganized interactive commands into categorized accordions (Session Management, Document Management, Debug & Observability, Other Commands)
- Added prerequisite note for voice mode requiring the
talkmodule installation - Added note about VLM model requirements for PDF indexing with images
- Added troubleshooting section for "No module named 'pip'" error in voice mode
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Remove claim that VLM model is 'automatically downloaded on first use'. Per RAG SDK implementation, users must manually download the model via Lemonade Model Manager UI at http://localhost:8000. Updated note to direct users to setup documentation for installation.
Change /chunks [file] to /chunks <file> to correctly indicate that the file parameter is required, not optional. Matches the CLI usage message in src/gaia/agents/chat/app.py:520.
|
@claude review this PR and check the documentation against the code. |
|
Claude finished @kovtcharov-amd's task —— View job Reviewing PR Documentation Against Code
Review SummaryI've verified the documentation changes against the actual code in Verified Commands (All Present in Code)All commands listed in the PR exist in
Issues Found (Agree with Copilot)1. 🟡 The docs say:
But the actual code ( elif command == "/reset":
print("\n🔄 Resetting conversation...")
# Save current session first
if agent.current_session:
agent.save_current_session()
# Create new session
agent.current_session = agent.session_manager.create_session()
# Clear chat history (if agent tracks it)
if hasattr(agent, "chat_history"):
agent.chat_history = []
print("✅ Conversation reset. Previous session saved.")The Recommendation: Change to:
2. 🟡 VLM auto-download statement (Line 318) The docs say:
This is actually correct and better than claiming auto-download. Looking at elif not vlm_available and self.config.show_stats:
print(" ⚠️ VLM not available - images will not be processed")
print(" 📥 To enable VLM image extraction:")
print(" 1. Open Lemonade Model Manager (http://localhost:8000)")
print(f" 2. Download model: {self.config.vlm_model}")The VLM is NOT auto-downloaded. Users must manually download via Lemonade Model Manager UI or use Regarding @kovtcharov-amd's Questions1. There is no # Download VLM model specifically
gaia download --agent vlm
# Or download all models for chat/rag
gaia download --agent chat
gaia download --agent ragBoth Recommendation: Update the VLM note to include the download command: 2. Command validation (Comment on Line 348) All documented commands are verified to exist in Additional SuggestionThe original documentation had these commands that were removed but may still be useful references:
Consider if these should be documented somewhere since Overall: Good PR with valuable documentation improvements. Just needs the |
…age for indexed files
|
@claude review this PR and check the documentation against the code. |
|
Claude finished @itomek's task —— View job Reviewing PR Documentation Against Code
Review SummaryI've verified the documentation changes against the actual code. Overall, this is a well-done PR with valuable improvements. The code changes also fix a real bug where the system prompt wasn't updated after indexing documents. Verified ItemsAll Commands Exist in Code (
VLM Download Instructions (Line 318-327)
Code Changes (
Test Coverage (
Minor Issues Found🟡 VLM Model Name Discrepancy (Line 318)
The model name in the docs is correct for RAG SDK. The discrepancy with CLAUDE.md is because EMR agent uses a different (smaller) VLM model. No action needed. 🟡 Settings File Changes (
Previous Review Items AddressedThe Copilot suggestions from the earlier review have been addressed:
Suggested Fix for - - `/chunks <file>` - View indexed chunks for a specific file
+ - `/chunks <filename>` - View indexed chunks for a specific fileThis is a very minor issue - both notations are understandable. Overall: LGTM ✅ Great work on this documentation update! The code fix for updating the system prompt after indexing is an important bug fix, and the test coverage additions are thorough. |
Fix pylint protected-access warning (W0212) by making the method public since it's legitimately used by app.py and rag_tools.py for updating the system prompt after document indexing. Co-Authored-By: Claude Opus 4.5 <[email protected]>
The 4 new tests for RAG interactive mode (issue #207) were failing in CI because they required a running Lemonade server for embedding generation. This fix adds proper mocking: - Mock `_load_embedder` to set up a mocked llm_client - Mock the embeddings response in OpenAI-compatible format - Mock FAISS index creation Tests now pass both locally and in CI without requiring Lemonade server. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Normalize macOS `/private` prefix in PathValidator to handle symlink resolution - Update ChatAgent to use PathValidator for path validation instead of duplicated logic - Pass allowed_paths configuration to RAG SDK for consistent security checks - Resolve file paths to real paths in reindexing and RAG tools - Fix test fixtures to use resolved paths matching agent validation behavior - Add fallback exception handling for RuntimeError in path validation
Updates docs/guides/chat.mdx with corrections from testing against actual CLI:
Add missing interactive commands organized by category:
Add Voice mode prerequisite note for talk module installation
Add VLM model requirement note for PDF indexing with images
Add troubleshooting for "No module named 'pip'" error in gaia talk
Improve PDF troubleshooting with specific error guidance
Bug Fix: Update system prompt after
/indexcommand to ensure indexed documents are recognized by the LLMNew Tests: Add test coverage for RAG interactive mode (system prompt updates, file tracking, multi-document indexing, chunk creation)
Lint Fix: Rename
_update_system_promptto public methodupdate_system_promptto fix pylint protected-access warningCloses #207
Related to #173 (documentation improvements)
🤖 Generated with Claude Code