Skip to content

fix(sleep): make --bare conditional on ANTHROPIC_API_KEY (#68) - #73

Merged
Yifan Yang (Yif-Yang) merged 2 commits into
microsoft:mainfrom
Yif-Yang:fix/bare-subscription-auth
Jun 20, 2026
Merged

Yifan Yang (Yif-Yang) merged 2 commits into
microsoft:mainfrom
Yif-Yang:fix/bare-subscription-auth

Conversation

@Yif-Yang

Copy link
Copy Markdown
Contributor

Summary

Fixes #68. ClaudeCliBackend hardcoded --bare in both _call() and attempt_with_tools(), which broke subscription-token auth — every model call silently returned "Not logged in" and scored 0.

Changes

  • Conditional --bare: only passed when ANTHROPIC_API_KEY is set. Without it, the remaining isolation flags (--disable-slash-commands, --disallowedTools, --exclude-dynamic-system-prompt-sections, clean temp cwd) already provide the needed isolation.
  • _detect_cli_error(): logs a warning when CLI output matches known auth error patterns ("Not logged in", "Invalid API key", etc.), so auth failures surface loudly instead of deflating every score to 0.
  • Narrowed markers: error detection only checks stderr + short stdout (<300 chars) to avoid false positives on legitimate model responses mentioning auth-related terms.

Verified by

  • Codex + GPT-5.5 review (2 rounds — round 1 caught overly broad "API key" marker, fixed; round 2 clean pass)
  • Codex actually ran the real claude CLI to confirm:
    • --bare with no API key → "Not logged in · Please run /login" ✅ detected
    • --bare with invalid API key → "Invalid API key · Fix external API key" ✅ detected
  • 139 tests pass (3 new: --bare included/omitted, error detection)

Test plan

  • test_bare_included_when_api_key_set — --bare appears when ANTHROPIC_API_KEY is set
  • test_bare_omitted_without_api_key — --bare absent without API key
  • test_cli_error_detected_and_logged — auth error string recorded in last_call_error
  • Full suite: 139 passed

🤖 Generated with Claude Code

ClaudeCliBackend._call() and attempt_with_tools() hardcoded --bare,
which skips Claude CLI's credential resolution. This broke subscription-
token auth: every model call silently returned "Not logged in" and
scored 0 — the user saw "baseline 0.0 → candidate 0.0, gate reject"
with no indication of an auth failure.

Fix: only pass --bare when ANTHROPIC_API_KEY is set. The remaining
isolation flags (--disable-slash-commands, --disallowedTools,
--exclude-dynamic-system-prompt-sections, clean temp cwd) already
provide the needed isolation without --bare.

Also adds _detect_cli_error() to log a warning when CLI output matches
known auth error patterns, so auth failures surface loudly instead of
deflating every score to 0.

Co-Authored-By: Claude Fable 5 <[email protected]>
Address codex review: "API key" was too generic — a model response
about configuring API keys would trigger a false auth warning. Now:
- Use specific phrases ("Invalid API key", "Unauthorized: invalid x-api-key")
- Only check short stdout (<300 chars) to skip real model responses
- Still check stderr unconditionally

Co-Authored-By: Claude Fable 5 <[email protected]>
@Yif-Yang
Yifan Yang (Yif-Yang) merged commit b5a1c2b into microsoft:main Jun 20, 2026
1 check was pending
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ClaudeCliBackend hardcodes --bare, breaking subscription-token auth (silent 0-score)

2 participants