Skip to content

Add tracer web app pipeline evidence and LLM hypotheses#10

Merged
davincios merged 23 commits intomainfrom
feature/generate-hypothesis
Jan 24, 2026
Merged

Add tracer web app pipeline evidence and LLM hypotheses#10
davincios merged 23 commits intomainfrom
feature/generate-hypothesis

Conversation

@davincios
Copy link
Copy Markdown
Contributor

Introduce a tracer web app client, LLM-based hypothesis planning, and web run evidence collection with end-to-end tests.

Introduce a tracer web app client, LLM-based hypothesis planning, and web run evidence collection with end-to-end tests.
…text directory

- Fix: Restore executed_hypotheses variable definition in generate_hypotheses.py
- Refactor: Remove agent/context directory, move files to appropriate node directories
  - context_building.py → nodes/hypothesis_execution/
  - service_graph.py → nodes/frame_problem/
  - utils.py → nodes/hypothesis_execution/
- Update all imports to use new file locations
- Fix linting issues (whitespace, formatting)
- Rename: graph_routing.py → routing.py
- Update import in graph_pipeline.py
- Fix linting issues:
  - Use contextlib.suppress instead of try/except/pass (SIM105)
  - Combine nested if statements (SIM102)
  - Fix whitespace issues (W293)
  - Fix formatting with ruff format
  - Fix E402 import issue in tests/run_demo.py
- Fix: Check if existing evidence exists before skipping evidence gathering
- Previously: When plan_sources was empty, hypothesis_execution would skip
  gathering evidence even if no evidence existed
- Now: Only skip if we have both executed sources AND existing evidence
- If no evidence exists, always attempt to gather from context
- This fixes the 'No evidence sources were successfully checked' error
- Remove 'tracer' source type (duplicate of 'tracer_web')
  - Both use the same TracerClient (tracer_cloud_app)
  - Keep only 'tracer_web' for consistency
- Remove build_context_tracer() function (unused)
- Update EvidenceSource type to exclude 'tracer'
- Update all references to use only 'tracer_web'
- Fix confusing message: 'gathering evidence from context' →
  'gathering evidence from available metadata' (clearer terminology)
- Create clients/ folder for service client modules:
  - cloudwatch_client.py
  - s3_client.py
  - tracer_client/ (entire module)
- Create tool_actions/ folder for service-specific tool actions:
  - s3_actions.py (S3 tool actions)
  - cloudwatch_actions.py (CloudWatch tool actions)
  - tracer_actions.py (Tracer tool actions)
- Update all imports across codebase to use new structure
- Remove old tools.py (split into service-specific files)
- Update __init__.py files to export from new locations
- Fix internal imports in tracer_client module
…d add LangSmith tracking

- Rename rca_report_publishing to publish_findings (simpler naming)
- Combine validate_analysis into diagnose_root_cause:
  - Simplified diagnose_root_cause significantly (removed complex investigation logic)
  - Integrated validation directly into diagnosis step
  - Removed separate validate_analysis node
- Add @Traceable decorators to all nodes for LangSmith tracking:
  - node_frame_problem
  - node_generate_hypotheses
  - node_hypothesis_execution
  - node_diagnose_root_cause
  - node_publish_findings
- Update graph_pipeline to remove validate_analysis node
- LLM calls are automatically tracked via LangChain (if LANGSMITH_API_KEY is set)
- Track node latency in diagnose_root_cause
- Remove incorrect call to check_evidence_sources with empty dict
- This was causing error message to be printed even when evidence exists
- Now only checks if web_run.get('found') is True directly
- Remove whitespace from blank line in llm.py
- Auto-format all files with ruff format
- Ensure all files pass ruff check and format
- Update frame_problem_test to expect superfluid_prod_pipeline instead of events_fact
- This matches the actual data in grafana_alert.json fixture
- Add _extract_org_id_from_jwt function to tracer_client
- Auto-extract org_id from JWT if TRACER_ORG_ID env var not set
- Update tests to not require TRACER_ORG_ID (extracted from JWT)
- Use DEFAULT_ORG_ID = org_33W1pou1nUzYoYPZj3OCQ3jslB2
- Remove JWT extraction logic (simpler)
- Still allows env var override if needed
- Tracer client already has default URL (staging.tracer.cloud)
- Tests now only require ANTHROPIC_API_KEY and JWT_TOKEN
- All 4 tests pass
- Add DEFAULT_BASE_URL = https://staging.tracer.cloud
- Fix context_building.py default (was localhost:3000)
- Only JWT_TOKEN is required now
- Use 'or' instead of default param (handles empty string)
- Auto-add https:// if protocol missing
- Remove redundant warning message in hypothesis_execution
- Remove all os.getenv calls for these values
- DEFAULT_ORG_ID = org_33W1pou1nUzYoYPZj3OCQ3jslB2
- DEFAULT_BASE_URL = https://staging.tracer.cloud
- Only JWT_TOKEN is required from environment
- Remove unused _extract_org_id_from_jwt function
@davincios davincios merged commit 5c4cfad into main Jan 24, 2026
1 of 2 checks passed
@davincios davincios deleted the feature/generate-hypothesis branch January 24, 2026 23:03
VaibhavUpreti pushed a commit that referenced this pull request Mar 13, 2026
Add tracer web app pipeline evidence and LLM hypotheses
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant