See discord thread here

Summary

When using Claude Sonnet models (claude-3.7-sonnet, claude-sonnet-4) through the GitHub Copilot provider, conversations that involve tool usage stop responding after the initial "Let me start..." message. GPT models work perfectly with the same setup.

Environment

• OS: Windows 11 (Enterprise environment)
• Provider: GitHub Copilot (only available provider in enterprise setup)
• Working Models: gpt-4o, gpt-4o-mini
• Broken Models: claude-3.7-sonnet, claude-sonnet-4
• Context: Same Claude models work perfectly with tools in other applications (e.g., Emacs with gptel)

Expected Behavior

Claude Sonnet models should continue the conversation and execute tool calls just like GPT models do.

Actual Behavior

1. User asks a question requiring tool usage
2. Claude responds with "Let me start [something]..."
3. Conversation stops completely - no tool calls are executed
4. No error messages are shown to the user

Root Cause Analysis

After investigating the codebase, the issue is in /crates/goose/src/providers/githubcopilot.rs:

1. Forced Streaming for Claude Models

// Lines 32-33
pub const GITHUB_COPILOT_STREAM_MODELS: &[&str] = 
    &["gpt-4.1", "claude-3.7-sonnet", "claude-sonnet-4"];

// Lines 122-127 - Forces streaming mode for Claude
let stream_only_model = GITHUB_COPILOT_STREAM_MODELS
    .iter()
    .any(|prefix| model_name.starts_with(prefix));
if stream_only_model {
    payload.as_object_mut().unwrap()
        .insert("stream".to_string(), serde_json::Value::Bool(true));
}

2. Silent Error Handling in Stream Parser

// Lines 137-158 - Silently ignores parsing errors
match serde_json::from_str::<OAIStreamChunk>(payload) {
    Ok(ch) => collector.add_chunk(&ch),
    Err(_) => continue,  // ⚠️ SILENTLY IGNORES ERRORS!
}

3. The Problem

• GitHub Copilot's Claude streaming format differs slightly from OpenAI's format
• When stream parsing fails, errors are silently ignored
• Tool calls get lost in the parsing failure
• User sees the conversation "stop" with no indication of what went wrong

Why GPT Models Work

GPT models through GitHub Copilot use OpenAI-compatible streaming format, so parsing succeeds.

Why Other Tools Work

Tools like Emacs/gptel likely use non-streaming requests or have different parsing logic that handles GitHub Copilot's Claude format correctly.

Proposed Fix

1. Add logging to reveal the silent failures:

   Err(e) => {
       tracing::warn!("Failed to parse streaming chunk for {}: {} | payload: {}", model_name, e, payload);
       continue;
   }

2. Add fallback to non-streaming mode when streaming fails for Claude models

3. Consider making Claude models non-streaming by default until GitHub Copilot's Claude streaming format is fully compatible

Reproduction Steps

1. Set up Goose with GitHub Copilot provider in enterprise environment
2. Use any Claude Sonnet model (claude-3.7-sonnet or claude-sonnet-4)
3. Ask a question that requires tool usage (e.g., "What files are in the current directory?")
4. Observe that the conversation stops after the initial response

Workaround

Use GPT models (gpt-4o, gpt-4o-mini) instead of Claude models when tool usage is required.

Impact

• Blocks enterprise users from using Claude models with Goose
• Silent failure provides no debugging information
• Reduces model choice for users in GitHub Copilot-only environments