Skip to content

docs: Fix Chat SDK guide based on manual testing#194

Merged
itomek merged 26 commits intoamd:mainfrom
itomek:tmi/docs-00
Jan 22, 2026
Merged

docs: Fix Chat SDK guide based on manual testing#194
itomek merged 26 commits intoamd:mainfrom
itomek:tmi/docs-00

Conversation

@itomek
Copy link
Collaborator

@itomek itomek commented Jan 16, 2026

Updates docs/guides/chat.mdx with corrections from testing against actual CLI:

  • Add missing interactive commands organized by category:

    • Session management: /resume, /save, /sessions, /reset
    • Document management: /index, /watch, /list, /status
    • Debug commands: /chunks, /chunk, /test, /dump, /clear-cache, /search-debug
  • Add Voice mode prerequisite note for talk module installation

  • Add VLM model requirement note for PDF indexing with images

  • Add troubleshooting for "No module named 'pip'" error in gaia talk

  • Improve PDF troubleshooting with specific error guidance

  • Bug Fix: Update system prompt after /index command to ensure indexed documents are recognized by the LLM

  • New Tests: Add test coverage for RAG interactive mode (system prompt updates, file tracking, multi-document indexing, chunk creation)

  • Lint Fix: Rename _update_system_prompt to public method update_system_prompt to fix pylint protected-access warning

Closes #207

Related to #173 (documentation improvements)


🤖 Generated with Claude Code

Updates docs/guides/chat.mdx with corrections from testing against actual CLI:

- Add missing interactive commands organized by category:
  * Session management: /resume, /save, /sessions, /reset
  * Document management: /index, /watch, /list, /status
  * Debug commands: /chunks, /chunk, /test, /dump, /clear-cache, /search-debug

- Add Voice mode prerequisite note for talk module installation

- Add VLM model requirement note for PDF indexing with images

- Add troubleshooting for "No module named 'pip'" error in gaia talk

- Improve PDF troubleshooting with specific error guidance

Fixes documentation gaps identified in manual testing transcripts.
@itomek itomek self-assigned this Jan 16, 2026
@github-actions github-actions bot added the documentation Documentation changes label Jan 16, 2026
itomek and others added 7 commits January 16, 2026 15:04
…uite for Chat SDK

- Created a new markdown report detailing the performance analysis of various LLMs in meeting summarization, including key metrics, common challenges, model-specific analysis, cost efficiency, and immediate improvement actions.
- Introduced a new test script to validate code samples from the Chat SDK documentation, ensuring API correctness and identifying documentation issues without relying on expensive PDF image processing.
Separates basic chat commands from RAG-specific commands for clarity:

CLI Usage > Interactive Mode:
- Basic Commands (conversation management)
- Session Management (conversation-focused descriptions)
- Forward reference to RAG section for document commands

Document Q&A (RAG) > Interactive RAG Commands:
- Session Management (now shows RAG state preservation)
- Document Management (/index, /watch, /list, /status)
- Debug & Observability (/chunks, /test, /dump, /clear-cache)

This organization:
- Lists session commands in both sections with context-appropriate descriptions
- Prevents RAG commands from appearing before RAG is explained
- Matches user mental model: chat features vs document features
- Makes command availability dependencies clear

Resolves confusion about which commands require RAG to be enabled.
Changed from "Ensure the VLM model is installed" to "The model will be
automatically downloaded on first use" to reflect actual behavior.

Models auto-download via lemonade_client with auto_download=True.
Removed /clear, /history, and /stats from CLI Interactive Mode section.
These commands exist in ChatSDK.start_interactive_session() but NOT in
the gaia chat CLI tool (src/gaia/agents/chat/app.py).

Actual CLI commands verified from source code:
- Session: /resume, /save, /sessions, /reset
- Document: /index, /watch, /list, /status
- Debug: /chunks, /chunk, /test, /dump, /clear-cache, /search-debug
- Other: /help, /?, /quit

Fixes documentation to match actual CLI behavior.
Corrected ChatConfig to match actual usage:
- assistant_name: "gaia" (was incorrectly "assistant")
- model: Show recommended "Qwen3-Coder-30B-A3B-Instruct-GGUF"
  (CPU failsafe model omitted as it's only for unsupported hardware)

Verified against src/gaia/chat/sdk.py
@itomek itomek marked this pull request as ready for review January 16, 2026 21:04
Copilot AI review requested due to automatic review settings January 16, 2026 21:04
@itomek itomek linked an issue Jan 16, 2026 that may be closed by this pull request
26 tasks
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Chat SDK documentation to accurately reflect the CLI commands and capabilities based on manual testing. It reorganizes the interactive commands section, adds missing RAG-specific commands, includes prerequisite notes for voice mode and PDF processing with images, and adds troubleshooting guidance.

Changes:

  • Reorganized interactive commands into categorized accordions (Session Management, Document Management, Debug & Observability, Other Commands)
  • Added prerequisite note for voice mode requiring the talk module installation
  • Added note about VLM model requirements for PDF indexing with images
  • Added troubleshooting section for "No module named 'pip'" error in voice mode

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

itomek and others added 3 commits January 16, 2026 16:38
Remove claim that VLM model is 'automatically downloaded on first use'.
Per RAG SDK implementation, users must manually download the model via
Lemonade Model Manager UI at http://localhost:8000.

Updated note to direct users to setup documentation for installation.
Change /chunks [file] to /chunks <file> to correctly indicate that
the file parameter is required, not optional. Matches the CLI usage
message in src/gaia/agents/chat/app.py:520.
@kovtcharov-amd
Copy link
Collaborator

@claude review this PR and check the documentation against the code.

@github-actions
Copy link

github-actions bot commented Jan 16, 2026

Claude finished @kovtcharov-amd's task —— View job


Reviewing PR Documentation Against Code

  • Read the changed file (docs/guides/chat.mdx)
  • Review existing PR comments and suggestions
  • Verify commands against actual code implementations
  • Check gaia init for VLM model download capability
  • Validate all added commands exist in the codebase
  • Provide comprehensive review feedback

Review Summary

I've verified the documentation changes against the actual code in src/gaia/agents/chat/app.py and related files. Overall the PR makes valuable improvements to the documentation. Here are my findings:

Verified Commands (All Present in Code)

All commands listed in the PR exist in src/gaia/agents/chat/app.py:

  • Session Management: /resume, /save, /sessions, /reset - Lines 253-324
  • Document Management: /index, /watch, /list, /status - Lines 326-516
  • Debug & Observability: /chunks, /chunk, /test, /dump, /clear-cache, /search-debug - Lines 518-916

Issues Found (Agree with Copilot)

1. 🟡 /reset command description is misleading (Line 332)

The docs say:

/reset - Clear conversation and remove all indexed documents

But the actual code (src/gaia/agents/chat/app.py:313-324) shows:

elif command == "/reset":
    print("\n🔄 Resetting conversation...")
    # Save current session first
    if agent.current_session:
        agent.save_current_session()
    # Create new session
    agent.current_session = agent.session_manager.create_session()
    # Clear chat history (if agent tracks it)
    if hasattr(agent, "chat_history"):
        agent.chat_history = []
    print("✅ Conversation reset. Previous session saved.")

The /reset command does NOT remove indexed documents - it only clears conversation history and starts a new session. The indexed documents remain in the RAG system.

Recommendation: Change to:

  • /reset - Clear conversation and start fresh (indexed documents are preserved)

2. 🟡 VLM auto-download statement (Line 318)

The docs say:

Ensure the VLM model is installed via [Lemonade Server]...

This is actually correct and better than claiming auto-download. Looking at src/gaia/rag/sdk.py:448-452:

elif not vlm_available and self.config.show_stats:
    print("  ⚠️  VLM not available - images will not be processed")
    print("  📥 To enable VLM image extraction:")
    print("     1. Open Lemonade Model Manager (http://localhost:8000)")
    print(f"     2. Download model: {self.config.vlm_model}")

The VLM is NOT auto-downloaded. Users must manually download via Lemonade Model Manager UI or use gaia download --agent vlm.

Regarding @kovtcharov-amd's Questions

1. gaia init command for VLM (Comment on Line 318)

There is no gaia init command. However, there is gaia download which can download VLM models:

# Download VLM model specifically
gaia download --agent vlm

# Or download all models for chat/rag
gaia download --agent chat
gaia download --agent rag

Both chat and rag agent profiles include the VLM model (qwen2.5-vl-7b) in their model requirements (see src/gaia/llm/lemonade_client.py:179-207).

Recommendation: Update the VLM note to include the download command:

**PDF Indexing Requirements:** Processing PDFs with images requires a Vision Language Model (VLM). 
GAIA uses `Qwen3-VL-4B-Instruct-GGUF` by default. To download the VLM model:
- Run `gaia download --agent vlm`, or
- Download via [Lemonade Model Manager](http://localhost:8000)

2. Command validation (Comment on Line 348)

All documented commands are verified to exist in src/gaia/agents/chat/app.py. The commands work as documented with one exception (the /reset behavior noted above).

Additional Suggestion

The original documentation had these commands that were removed but may still be useful references:

  • /clear - Clear conversation history (now handled by /reset)
  • /history - Show conversation history
  • /stats - Show performance statistics

Consider if these should be documented somewhere since /reset saves the session before clearing, while a simple /clear (if it existed) might be more direct for just clearing history.


Overall: Good PR with valuable documentation improvements. Just needs the /reset description fixed.

@github-actions github-actions bot added agents Agent system changes tests Test changes labels Jan 20, 2026
@itomek
Copy link
Collaborator Author

itomek commented Jan 20, 2026

@claude review this PR and check the documentation against the code.

@github-actions
Copy link

github-actions bot commented Jan 20, 2026

Claude finished @itomek's task —— View job


Reviewing PR Documentation Against Code

  • Read pr-diff.txt and pr-files.txt for changed files
  • Read the changed documentation file (docs/guides/chat.mdx)
  • Read the changed code file (src/gaia/agents/chat/app.py)
  • Read the changed test file (tests/test_chat_agent.py)
  • Verify all documented commands exist in code
  • Check VLM download instructions (gaia download command)
  • Validate /reset command behavior
  • Review test coverage additions
  • Provide comprehensive review feedback

Review Summary

I've verified the documentation changes against the actual code. Overall, this is a well-done PR with valuable improvements. The code changes also fix a real bug where the system prompt wasn't updated after indexing documents.

Verified Items

All Commands Exist in Code (src/gaia/agents/chat/app.py)

  • Session Management (Lines 253-324): /resume, /save, /sessions, /reset
  • Document Management (Lines 326-524): /index, /watch, /list, /status
  • Debug Commands (Lines 525-924): /chunks, /chunk, /test, /dump, /clear-cache, /search-debug

VLM Download Instructions (Line 318-327)

  • gaia download --agent chat correctly documented
  • ✅ The chat agent profile includes qwen2.5-vl-7b VLM model (verified in src/gaia/llm/lemonade_client.py:183)
  • gaia download --list documented for viewing model requirements

/reset Command Behavior (Line 341)

  • ✅ The description now correctly says "indexed documents are preserved" - this matches the code at app.py:313-324 which only clears conversation history and creates a new session

Code Changes (src/gaia/agents/chat/app.py:370-376, 421-426)

  • ✅ Fixes a real bug: agent._update_system_prompt() is now called after indexing documents
  • This ensures the LLM knows about newly indexed documents for RAG queries

Test Coverage (tests/test_chat_agent.py)

  • ✅ Excellent new tests added (Lines 92-213):
    • test_system_prompt_updated_after_index - validates the bug fix
    • test_indexed_files_tracked_after_index - verifies file tracking
    • test_multiple_documents_indexed - multi-document support
    • test_rag_chunks_created_after_index - chunk creation verification

Minor Issues Found

🟡 VLM Model Name Discrepancy (Line 318)

  • Docs say: Qwen2.5-VL-7B-Instruct-GGUF
  • But src/gaia/rag/sdk.py:73 uses: Qwen2.5-VL-7B-Instruct-GGUF (same)
  • CLAUDE.md says: Qwen3-VL-4B-Instruct-GGUF for vision tasks

The model name in the docs is correct for RAG SDK. The discrepancy with CLAUDE.md is because EMR agent uses a different (smaller) VLM model. No action needed.

🟡 Settings File Changes (.claude/settings.local.json)

  • Duplicate entries added for Bash(git add:*) and Bash(git commit:*) (lines 8-11)
  • This is harmless but could be cleaned up

Previous Review Items Addressed

The Copilot suggestions from the earlier review have been addressed:

  • /reset description fixed (now says "indexed documents are preserved")
  • ✅ VLM auto-download claim removed (now correctly uses gaia download)
  • 🟡 /chunks <file> parameter notation still uses [file] (optional syntax) but code requires the argument

Suggested Fix for /chunks (Line 351):

-    - `/chunks <file>` - View indexed chunks for a specific file
+    - `/chunks <filename>` - View indexed chunks for a specific file

This is a very minor issue - both notations are understandable.


Overall: LGTM

Great work on this documentation update! The code fix for updating the system prompt after indexing is an important bug fix, and the test coverage additions are thorough.

itomek and others added 2 commits January 20, 2026 14:33
Fix pylint protected-access warning (W0212) by making the method public
since it's legitimately used by app.py and rag_tools.py for updating
the system prompt after document indexing.

Co-Authored-By: Claude Opus 4.5 <[email protected]>
@github-actions github-actions bot added the devops DevOps/infrastructure changes label Jan 20, 2026
The 4 new tests for RAG interactive mode (issue #207) were failing in CI
because they required a running Lemonade server for embedding generation.

This fix adds proper mocking:
- Mock `_load_embedder` to set up a mocked llm_client
- Mock the embeddings response in OpenAI-compatible format
- Mock FAISS index creation

Tests now pass both locally and in CI without requiring Lemonade server.

Co-Authored-By: Claude Opus 4.5 <[email protected]>
@kovtcharov-amd kovtcharov-amd added this to the v0.15.3 milestone Jan 20, 2026
@itomek itomek modified the milestones: v0.15.3, v0.15.2 Jan 20, 2026
@itomek itomek enabled auto-merge January 21, 2026 19:32
kovtcharov-amd and others added 3 commits January 21, 2026 18:11
- Normalize macOS `/private` prefix in PathValidator to handle symlink resolution
- Update ChatAgent to use PathValidator for path validation instead of duplicated logic
- Pass allowed_paths configuration to RAG SDK for consistent security checks
- Resolve file paths to real paths in reindexing and RAG tools
- Fix test fixtures to use resolved paths matching agent validation behavior
- Add fallback exception handling for RuntimeError in path validation
@github-actions github-actions bot added the security Security-sensitive changes label Jan 22, 2026
@itomek itomek added this pull request to the merge queue Jan 22, 2026
Merged via the queue into amd:main with commit c25d798 Jan 22, 2026
1 check passed
@itomek itomek deleted the tmi/docs-00 branch January 22, 2026 19:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent system changes devops DevOps/infrastructure changes documentation Documentation changes security Security-sensitive changes tests Test changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Documentation Review Tracking [Bug] Chat command doesn't find indexed documents + documentation gaps

4 participants