Proposal: Data-aware, versioned prompts in Feast #7110

Summary

LLM applications put user data into their prompts: a customer's balance, their recent orders, their location. Feast already stores and serves that data, but it knows nothing about the prompt around it.

We'd like Feast to also hold the prompt: defined next to the features it uses, versioned like other Feast definitions, and filled with live features when an application asks for it. Feast would only build the prompt. Calling the model and running agents stay in the application, in any framework.

Problem

Today every team writes its own glue between features and prompts:

row = store.get_online_features(features=[...], entity_rows=[{"customer_id": 42}]).to_dict()
prompt = f"You are a loan advisor for {row['name'][0]}. Balance: {row['balance'][0]} ..."

• No versions. Nobody can say which prompt produced last week's answer.
• Drift. Training and evaluation code rebuilds the prompt differently from the live service.
• No size control. A customer with a long history overflows the model's context window.

Who hits this: an ML engineer who already uses Feast for classic models and now builds an LLM feature on the same data, often in a regulated company such as a bank or an insurer.

Why not just application code? The hard part is the part Feast already owns: rebuilding exactly what the model saw at a past time. A fine-tuning or evaluation dataset needs each example's prompt as it would have been at that moment, which takes point-in-time features and the prompt version used then. Application code can't reproduce that reliably; Feast's registry and offline store can.

Other tools solve only half of this. LangChain, LlamaIndex and CrewAI have prompt templates, but each is tied to its framework and doesn't know the data. The MLflow Prompt Registry versions the wording, but the application still fetches and fills in every value. Feast has the data but no prompt.

The idea

A prompt is a function over feature views, stored in the Feast registry:

@prompt(sources=[customer_profile, active_loans, question], max_tokens=1500)
def loan_advisor(profile, loans, question):
    return (
        f"You are a loan advisor for {profile['first_name']}. "
        f"Credit score: {profile['credit_score']}, active loans: {loans['count']}.\n"
        f"Customer asks: {question['text']}"
    )

prompt = store.render_prompt("loan_advisor", [{"customer_id": 42, "text": "Should I take another loan?"}])

• Versioned: feast apply saves each change as a new version, and loan_advisor@v2 pins one.
• Filled on request: Feast reads the features and returns the text, tagged with its version. Nothing about the prompt is stored in the online store.
• Same prompt for training: the same function runs on point-in-time features from get_historical_features.
• Fits a token budget: when there is too much data, the least important parts are cut first, and important ones like the instructions are never cut.
• Works with MLflow: the wording can live in the MLflow Prompt Registry (template="prompts:/loan_advisor@production"), with Feast supplying the data.
• Any framework, any language: the result is plain text, from the Python SDK or the feature server over HTTP.

Example: bank loan assistant

A bank's assistant explains a customer's loan history and whether to take another loan. Its prompt needs current loans and credit score, the last 20 payments, and preferences from earlier conversations. With this idea:

• the bank can answer "what did the assistant know when it gave this advice?" from the prompt version and point-in-time features;
• the assistant only ever sees this customer's features, through Feast's access control, and never touches core banking systems;
• prompt size stays bounded and predictable.

Questions for the community

1. Should a prompt be written as a Python function, a template, or both?
2. Should Feast have its own version aliases (loan_advisor@production), or leave rollout to MLflow?
3. Should a prompt render chat messages (system, user) as well as plain text?
4. Should prompts be their own permission resource, separate from the features they read?

References

• #6824 — Token budgets for prompts (in review)