Add tracer web app pipeline evidence and LLM hypotheses#10
Merged
Conversation
Introduce a tracer web app client, LLM-based hypothesis planning, and web run evidence collection with end-to-end tests.
…text directory - Fix: Restore executed_hypotheses variable definition in generate_hypotheses.py - Refactor: Remove agent/context directory, move files to appropriate node directories - context_building.py → nodes/hypothesis_execution/ - service_graph.py → nodes/frame_problem/ - utils.py → nodes/hypothesis_execution/ - Update all imports to use new file locations - Fix linting issues (whitespace, formatting)
- Rename: graph_routing.py → routing.py - Update import in graph_pipeline.py - Fix linting issues: - Use contextlib.suppress instead of try/except/pass (SIM105) - Combine nested if statements (SIM102) - Fix whitespace issues (W293) - Fix formatting with ruff format - Fix E402 import issue in tests/run_demo.py
- Fix: Check if existing evidence exists before skipping evidence gathering - Previously: When plan_sources was empty, hypothesis_execution would skip gathering evidence even if no evidence existed - Now: Only skip if we have both executed sources AND existing evidence - If no evidence exists, always attempt to gather from context - This fixes the 'No evidence sources were successfully checked' error
- Remove 'tracer' source type (duplicate of 'tracer_web') - Both use the same TracerClient (tracer_cloud_app) - Keep only 'tracer_web' for consistency - Remove build_context_tracer() function (unused) - Update EvidenceSource type to exclude 'tracer' - Update all references to use only 'tracer_web' - Fix confusing message: 'gathering evidence from context' → 'gathering evidence from available metadata' (clearer terminology)
- Create clients/ folder for service client modules: - cloudwatch_client.py - s3_client.py - tracer_client/ (entire module) - Create tool_actions/ folder for service-specific tool actions: - s3_actions.py (S3 tool actions) - cloudwatch_actions.py (CloudWatch tool actions) - tracer_actions.py (Tracer tool actions) - Update all imports across codebase to use new structure - Remove old tools.py (split into service-specific files) - Update __init__.py files to export from new locations - Fix internal imports in tracer_client module
…d add LangSmith tracking - Rename rca_report_publishing to publish_findings (simpler naming) - Combine validate_analysis into diagnose_root_cause: - Simplified diagnose_root_cause significantly (removed complex investigation logic) - Integrated validation directly into diagnosis step - Removed separate validate_analysis node - Add @Traceable decorators to all nodes for LangSmith tracking: - node_frame_problem - node_generate_hypotheses - node_hypothesis_execution - node_diagnose_root_cause - node_publish_findings - Update graph_pipeline to remove validate_analysis node - LLM calls are automatically tracked via LangChain (if LANGSMITH_API_KEY is set) - Track node latency in diagnose_root_cause
- Remove incorrect call to check_evidence_sources with empty dict
- This was causing error message to be printed even when evidence exists
- Now only checks if web_run.get('found') is True directly
- Remove whitespace from blank line in llm.py - Auto-format all files with ruff format - Ensure all files pass ruff check and format
- Update frame_problem_test to expect superfluid_prod_pipeline instead of events_fact - This matches the actual data in grafana_alert.json fixture
- Add _extract_org_id_from_jwt function to tracer_client - Auto-extract org_id from JWT if TRACER_ORG_ID env var not set - Update tests to not require TRACER_ORG_ID (extracted from JWT)
- Use DEFAULT_ORG_ID = org_33W1pou1nUzYoYPZj3OCQ3jslB2 - Remove JWT extraction logic (simpler) - Still allows env var override if needed
- Tracer client already has default URL (staging.tracer.cloud) - Tests now only require ANTHROPIC_API_KEY and JWT_TOKEN - All 4 tests pass
- Add DEFAULT_BASE_URL = https://staging.tracer.cloud - Fix context_building.py default (was localhost:3000) - Only JWT_TOKEN is required now
- Use 'or' instead of default param (handles empty string) - Auto-add https:// if protocol missing - Remove redundant warning message in hypothesis_execution
- Remove all os.getenv calls for these values - DEFAULT_ORG_ID = org_33W1pou1nUzYoYPZj3OCQ3jslB2 - DEFAULT_BASE_URL = https://staging.tracer.cloud - Only JWT_TOKEN is required from environment - Remove unused _extract_org_id_from_jwt function
VaibhavUpreti
pushed a commit
that referenced
this pull request
Mar 13, 2026
Add tracer web app pipeline evidence and LLM hypotheses
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduce a tracer web app client, LLM-based hypothesis planning, and web run evidence collection with end-to-end tests.