Repository navigation
feat: add datahub-ml-impact skill - #66
cnpierrepapi wants to merge 2 commits into
Conversation
|
Bumping, though I can see there are over a hundred open PRs here at the moment, so this is a queue problem rather than anyone ignoring it. One thing that may make it easier to place. This skill answers "what breaks if I change this, and did anything protected reach a deployed model", which the existing five do not cover. The failure modes written into it are not hypothetical: column edges stop at the feature table so a column-only walk reports that nothing reaches a deployed model, tags land on the dbt sibling rather than the warehouse entity so a naive check reports all clear, and the graph index is asynchronous for roughly forty seconds after an ingest so a check that runs immediately reports all clear against an empty graph. Each one returns a confident wrong answer rather than an error, which is the reason they are in the skill body. Related upstream, in case it is useful context for whoever picks this up: datahub-project/datahub#18999 and #18685 came out of the same work, and I have taken on closing the Happy to rebase or adjust to house style on request. |
e854486 to
30b6f63
Compare
The routing was one way. using-datahub sends ML questions to the new skill, but datahub-lineage still claimed "what breaks if I change X?" without a carve out, so anyone landing on it directly never got redirected.
Adds a sixth catalog interaction skill, for the case where a model is on one end
of a lineage question, and registers it in the
using-datahubrouting table.Why a separate skill rather than more of
/datahub-lineage/datahub-lineageis good and general, and the moment a model is involved ageneric walk starts producing confident wrong answers. Four things about ML
lineage are different, and each one fails silently rather than raising:
Tags live on the sibling you did not walk. dbt holds the governance tags,
the warehouse entity holds none, and traversal returns the warehouse one. Ask
it whether a column is protected and the answer is no. Nothing errors, and a
check written that way reports all clear forever.
Filtering a column walk by column name drops the origin. A warehouse
renames as it cleans. In my case the column was
public_coverage_flagin themarts and
pubcovin the raw extract, and the name filter turned a seven hopanswer into a five hop one that read as complete.
Column edges stop at the feature table. dbt emits column level edges. ML
platforms emit a table level edge into the model, because a training run
consumes a table. A walk that does not cross that boundary reports the column
reaches no model, which is the exact question that was asked.
The environment in a URN is not a deployment status.
PRODis the fabricthe entity lives in. Read the stage from the registry, or every archived
experiment counts as live.
All four are written into the skill as explicit traps with the workaround, so an
agent following it gets the answer rather than the plausible version.
Scope
The workflow uses only the DataHub CLI and the MCP server, so it stands alone
and pulls in no external dependency. It declares its own boundary: if nothing
downstream is a model it routes back to
/datahub-lineage, which is simpler andcorrect.
Changes
skills/datahub-ml-impact/SKILL.md, newskills/using-datahub/SKILL.md, routing row, a Lineage vs. ML Impactdisambiguation rule, count 5 to 6, and the CLI attribution example
prettierandmarkdownlint-cli2both pass against the repo config.Found while building a lineage-grounded governance tool on DataHub Core
(GMS v1.5.0.6, MLflow, dbt, Postgres). Happy to adjust tone or structure to
match the house style more closely.