Initiate support for agent and tracing frameworks#66
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR introduces integration support for agent and tracing frameworks including Langchain, Weave, AgentOps, and MLFlow. The changes expand the POML library's tracing capabilities from local-only to supporting multiple external tracing backends.
- Enhanced
set_trace()to support multiple backends ("local", "weave", "agentops", "mlflow") - Added framework-specific integration modules for logging POML calls
- Introduced new output formats including OpenAI chat and Langchain message formats
Reviewed Changes
Copilot reviewed 17 out of 18 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| python/poml/api.py | Core API changes for multi-backend tracing support and new output formats |
| python/poml/integration/weave.py | Weave integration for logging POML calls |
| python/poml/integration/mlflow.py | MLflow integration with prompt registration |
| python/poml/integration/langchain.py | Langchain template integration with POML |
| python/poml/integration/agentops.py | AgentOps integration for operation tracking |
| python/tests/manual/*.py | Manual test examples for each integration |
| python/tests/manual/example_poml.poml | POML template file for testing |
| pyproject.toml | Added pydantic dependency |
| python/.gitignore | Added common integration artifacts to ignore |
Comments suppressed due to low confidence (1)
python/tests/manual/example_mlflow_poml.py:29
- The model 'gpt-4.1-mini' does not exist. This should likely be 'gpt-4o-mini' to match the model used in the previous call on line 24.
model="gpt-4.1-mini",
| if enabled or "local" in enabled: | ||
| # When enabled is non-empty, we always enable local tracing. |
There was a problem hiding this comment.
This condition will always be True when enabled is a non-empty list, even if 'local' is not in the list. This should be 'if enabled and (isinstance(enabled, list) and "local" in enabled or enabled == ["local"])' or similar logic to properly check for local tracing.
| if enabled or "local" in enabled: | |
| # When enabled is non-empty, we always enable local tracing. | |
| if enabled and (isinstance(enabled, list) and "local" in enabled or enabled == ["local"]): | |
| # Enable local tracing only when explicitly requested. |
| if "weave" in enabled: | ||
| _weave_enabled = True | ||
| else: | ||
| _trace_dir = None | ||
| _weave_enabled = False | ||
|
|
||
| if "agentops" in enabled: | ||
| _agentops_enabled = True | ||
| else: | ||
| _agentops_enabled = False | ||
|
|
||
| if "mlflow" in enabled: |
There was a problem hiding this comment.
This will raise a TypeError if enabled is True or False (bool values don't support 'in' operator). Add a type check: 'if isinstance(enabled, list) and "weave" in enabled:'
| if "weave" in enabled: | |
| _weave_enabled = True | |
| else: | |
| _trace_dir = None | |
| _weave_enabled = False | |
| if "agentops" in enabled: | |
| _agentops_enabled = True | |
| else: | |
| _agentops_enabled = False | |
| if "mlflow" in enabled: | |
| if isinstance(enabled, list) and "weave" in enabled: | |
| _weave_enabled = True | |
| else: | |
| _weave_enabled = False | |
| if isinstance(enabled, list) and "agentops" in enabled: | |
| _agentops_enabled = True | |
| else: | |
| _agentops_enabled = False | |
| if isinstance(enabled, list) and "mlflow" in enabled: |
| if "weave" in enabled: | ||
| _weave_enabled = True | ||
| else: | ||
| _trace_dir = None | ||
| _weave_enabled = False | ||
|
|
||
| if "agentops" in enabled: | ||
| _agentops_enabled = True | ||
| else: | ||
| _agentops_enabled = False | ||
|
|
||
| if "mlflow" in enabled: |
There was a problem hiding this comment.
This will raise a TypeError if enabled is True or False (bool values don't support 'in' operator). Add a type check: 'if isinstance(enabled, list) and "agentops" in enabled:'
| if "weave" in enabled: | |
| _weave_enabled = True | |
| else: | |
| _trace_dir = None | |
| _weave_enabled = False | |
| if "agentops" in enabled: | |
| _agentops_enabled = True | |
| else: | |
| _agentops_enabled = False | |
| if "mlflow" in enabled: | |
| if isinstance(enabled, list) and "weave" in enabled: | |
| _weave_enabled = True | |
| else: | |
| _weave_enabled = False | |
| if isinstance(enabled, list) and "agentops" in enabled: | |
| _agentops_enabled = True | |
| else: | |
| _agentops_enabled = False | |
| if isinstance(enabled, list) and "mlflow" in enabled: |
| if "weave" in enabled: | ||
| _weave_enabled = True | ||
| else: | ||
| _trace_dir = None | ||
| _weave_enabled = False | ||
|
|
||
| if "agentops" in enabled: | ||
| _agentops_enabled = True | ||
| else: | ||
| _agentops_enabled = False | ||
|
|
||
| if "mlflow" in enabled: |
There was a problem hiding this comment.
This will raise a TypeError if enabled is True or False (bool values don't support 'in' operator). Add a type check: 'if isinstance(enabled, list) and "mlflow" in enabled:'
| if "weave" in enabled: | |
| _weave_enabled = True | |
| else: | |
| _trace_dir = None | |
| _weave_enabled = False | |
| if "agentops" in enabled: | |
| _agentops_enabled = True | |
| else: | |
| _agentops_enabled = False | |
| if "mlflow" in enabled: | |
| if isinstance(enabled, list) and "weave" in enabled: | |
| _weave_enabled = True | |
| else: | |
| _weave_enabled = False | |
| if isinstance(enabled, list) and "agentops" in enabled: | |
| _agentops_enabled = True | |
| else: | |
| _agentops_enabled = False | |
| if isinstance(enabled, list) and "mlflow" in enabled: |
First batch: