Skip to content

Send step and project context to monitor for metric tagging#3093

Merged
talsperre merged 2 commits into
masterfrom
monitor-step-context
May 14, 2026
Merged

Send step and project context to monitor for metric tagging#3093
talsperre merged 2 commits into
masterfrom
monitor-step-context

Conversation

@talsperre

Copy link
Copy Markdown
Collaborator

Summary

  • Sends step_name, project_name, branch_name, is_production, and is_user_branch as a MUST_SEND context update to the monitor at the start of run_step
  • Enables monitor implementations to include per-step and per-project tags on all metrics emitted during step execution
  • Filters out None values to avoid invalid tags

Motivation

The monitor currently only receives flow-level context (flow_name, username, metaflow_version, etc.) at init time. Per-step context like step_name and project metadata are available in run_step but never propagated to the monitor. This makes it impossible to slice metrics by step or project without custom workarounds.

Test plan

  • Verify existing monitor tests pass
  • Verify NullMonitor.send() (base class) forwards the message to the sidecar without error
  • Verify custom monitor implementations that override send() receive the new tags

🤖 Generated with Claude Code

The monitor receives flow-level tags at init time but has no
visibility into per-step context (step_name, project_name,
branch_name, is_production, is_user_branch). This sends those
as a MUST_SEND context update at the start of run_step so that
monitor implementations can include them as metric tags.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
@greptile-apps

greptile-apps Bot commented Apr 13, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR propagates step-level and project-level context (step_name, project_name, branch_name, is_production, is_user_branch) to the monitor at the start of run_step via a MUST_SEND message, enabling per-step and per-project metric tagging. A follow-up commit addressed the previously flagged boolean serialization issue by converting all tag values to str before sending.

Confidence Score: 5/5

Safe to merge — the change is well-scoped, both previously flagged concerns are resolved, and no new issues are present.

The boolean serialization bug raised in the prior review thread was fixed in the follow-up commit via explicit str() conversion. The is_active guard asymmetry is not a correctness bug (confirmed by reading Sidecar.send() which guards internally). The None-filtering, MUST_SEND choice, and post-current._set_env() placement are all correct. No remaining P0 or P1 findings.

No files require special attention.

Important Files Changed

Filename Overview
metaflow/task.py Adds monitor_tags dict with step/project fields (str-coerced, None-filtered) sent as MUST_SEND after current._set_env() ensures project metadata is available; logic is correct and well-placed.

Reviews (2): Last reviewed commit: "Fix boolean tag values being dropped by ..." | Re-trigger Greptile

Comment thread metaflow/task.py
Comment thread metaflow/task.py
}.items()
if v is not None
}
self.monitor.send(Message(MessageTypes.MUST_SEND, monitor_tags))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 send() skips the is_active guard present in count() / measure()

NullMonitor.count() and NullMonitor.measure() wrap their payload construction in if self._sidecar.is_active:, avoiding the overhead of building and serialising a message when no sidecar is running. NullMonitor.send() delegates directly to Sidecar.send(), which does guard internally, so this is not a correctness bug — but the inconsistency means a no-op path still serialises monitor_tags into a Message object before the check fires.

If the pattern is intentional (because send() is caller-constructed), a short comment on NullMonitor.send() documenting that the is_active check is deferred to Sidecar.send() would prevent future confusion.

spectator's validate_tags only accepts string values — booleans from
current.get("is_production") and current.get("is_user_branch") were
silently stripped. Apply str(v) in the comprehension so booleans become
"True"/"False" strings while None values are still filtered out.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
@saikonen
saikonen self-requested a review May 14, 2026 17:22
@talsperre
talsperre merged commit 4871fb1 into master May 14, 2026
46 checks passed
@talsperre
talsperre deleted the monitor-step-context branch May 14, 2026 18:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants