Attach prompts, content items, and visual editor elements to AI Assistant Context by bryantgillespie · Pull Request #26512 · directus/directus

bryantgillespie · 2026-01-22T16:24:03Z

Scope

What's changed:

Add context system for the AI Assistant
Ability to attach content items, attach visual editor elements, and reuse MCP AI System Prompts within AI Assistant conversations
New AiContextCard component to display context attachment
Updated AI endpoint to support context
Support for new page context

Potential Risks / Drawbacks

TBD

Tested Scenarios

Tested in Chrome and Firefox

Review Notes / Questions

TBD

Checklist

Added or updated tests
Documentation PR created here

This pull request introduces a new AI Assistant Context system, allowing prompts, content items, and visual editor elements to be attached as context for AI interactions. The changes span API, backend models, prompt formatting, and frontend UI, making context-aware AI responses possible and improving the user experience in the visual editor and chat. The update also refines how item updates are handled in AI tools.

AI Assistant Context Attachments and Context-Aware Prompts:

Added new types (ContextAttachment, ChatContext, etc.) to represent prompts, items, and visual elements as context attachments in chat-request.ts. The chat request schema now accepts a context field.
Implemented formatContextForSystemPrompt utility to format these attachments and page context for inclusion in the AI system prompt, ensuring the AI receives structured, prioritized context.
Updated the AI chat handler and UI stream creation to accept and pass the context field, dynamically appending formatted context to the system prompt for each AI step. [1] [2] [3] [4]

Frontend Visual Improvements:

Added a new ai-context-card.vue component for displaying context attachments (items, prompts, visual elements) in the UI, with icons, display text, and removal support.

AI Tools and Prompt Documentation:

Updated the items AI tool to allow data to be either an object or array, supporting both single and batch updates, and clarified this in the prompt documentation. [1] [2] [3]

Changelog:

Added a changeset entry summarizing the new context attachment functionality for the AI Assistant.

Copilot

Pull request overview

Adds support for attaching additional context (prompts, content items, and visual editor selections) to AI Chat requests, and exposes new UI to stage/manage that context from the App and Visual Editor.

Changes:

Introduces “context attachments” plumbing end-to-end (App → API request schema → system-prompt formatting).
Adds UI for staging prompts/items/visual elements, plus AI sidebar integration in the visual editor and live preview split panel.
Refactors AI tooling state into a dedicated store and improves related typings/tests.

Reviewed changes

Copilot reviewed 46 out of 47 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
pnpm-lock.yaml	Bumps `webidl-conversions` to `8.0.1`.
packages/ai/src/types.ts	Adds shared context attachment types for AI package consumers.
app/src/views/private/components/live-preview.vue	Adds optional split-panel sidebar support + new slots/events for sidebar and notifications.
app/src/views/private/components/file-preview.vue	Allows overriding preview `src` instead of always using asset URL.
app/src/views/private/components/file-lightbox.vue	Passes through optional `src` to support non-asset image previews.
app/src/modules/visual/types/index.ts	Extends visual editor message protocol for context add + element highlight + stronger types.
app/src/modules/visual/routes/visual-editor.vue	Adds AI sidebar/drawer in visual editor, responsive behavior, and header button integration.
app/src/modules/visual/components/editing-layer.vue	Adds support for staging visual elements into AI context and highlighting elements from AI UI.
app/src/modules/content/routes/item.vue	Refreshes previews on non-read item tool calls and supports staging visual elements from popup.
app/src/lang/translations/en-US.yaml	Adds new i18n keys for live preview + AI context menu/staging.
app/src/components/v-form/composables/use-ai-tools.ts	Improves tool descriptions to clarify UI-only vs database mutations.
app/src/ai/types/prompts.ts	Adds MCP prompt type used by prompt staging.
app/src/ai/types/index.ts	Re-exports new AI app types.
app/src/ai/types/context.ts	Defines pending context items + type guards for context UI/store.
app/src/ai/stores/use-ai.ts	Adds context snapshotting + page context, integrates new tools/context stores, adds highlight/focus hooks.
app/src/ai/stores/use-ai.test.ts	Updates tests for refactors/typing changes in AI store.
app/src/ai/stores/use-ai-tools.ts	New store to manage system tools, local tools, and approval modes.
app/src/ai/stores/use-ai-tools.test.ts	Adds tests for the new AI tools store.
app/src/ai/stores/use-ai-context.ts	New store for pending context + snapshotting attachments (incl. fetching current visual element data).
app/src/ai/stores/use-ai-context.test.ts	Adds tests for context store behaviors and snapshotting.
app/src/ai/composables/use-search-filter.ts	Adds reusable search filtering composable for context menus.
app/src/ai/composables/use-prompts.ts	Adds prompt fetching, template variable extraction, rendering, and conversion to chat messages.
app/src/ai/composables/use-prompts.test.ts	Adds tests for prompt fetching/rendering/variable extraction.
app/src/ai/composables/use-context-staging.ts	Adds staging logic for prompts/items/visual elements into pending context with notifications.
app/src/ai/composables/use-context-staging.test.ts	Adds tests for context staging behavior.
app/src/ai/composables/define-tool.ts	Moves local tool registration from AI store to tools store.
app/src/ai/composables/define-tool.test.ts	Updates mocks/tests to reflect tools store usage.
app/src/ai/components/parts/ai-message-tool.vue	Updates typing for tool message parts.
app/src/ai/components/parts/ai-message-file.vue	Adds lightbox support for image attachments in AI messages.
app/src/ai/components/ai-settings-menu.vue	Adds fallback icon handling for approval preview.
app/src/ai/components/ai-prompt-variables-modal.vue	New modal to collect template variables when inserting prompts.
app/src/ai/components/ai-pending-context.vue	New UI component to display/removal/hover-highlighting of pending context items.
app/src/ai/components/ai-message.vue	Renders context attachments from user message metadata + hover highlight integration.
app/src/ai/components/ai-input.vue	Adds context menu + pending context strip and focus hook wiring.
app/src/ai/components/ai-conversation.vue	Minor layout updates for conversation container.
app/src/ai/components/ai-context-menu/list-view.vue	New reusable list view wrapper for context menu sections.
app/src/ai/components/ai-context-menu/empty-state.vue	New empty state list item for context menu lists.
app/src/ai/components/ai-context-menu/context-menu-item.vue	New menu item component used across context menu lists.
app/src/ai/components/ai-context-menu.vue	New context picker menu for staging prompts and items into AI context.
app/src/ai/components/ai-context-card.vue	New pill/card UI for context items/attachments (pending + snapshotted).
api/src/ai/tools/items/prompt.md	Updates items tool documentation examples for update payload shape.
api/src/ai/tools/items/index.ts	Allows `data` to be object or array to support both keyed and batch updates.
api/src/ai/chat/utils/format-context.ts	New formatter to append attachments/page context to the system prompt.
api/src/ai/chat/models/chat-request.ts	Extends request schema with validated `context` + attachment types.
api/src/ai/chat/lib/create-ui-stream.ts	Passes `context` into prompt assembly per step (via formatter).
api/src/ai/chat/controllers/chat.post.ts	Wires `context` from request into UI stream creation.
.changeset/fresh-doodles-relax.md	Adds changeset for minor bumps across ai/api/app packages.

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

app/src/ai/stores/use-ai-context.ts

app/src/ai/components/ai-prompt-variables-modal.vue

app/src/ai/components/parts/ai-message-file.vue

api/src/ai/chat/lib/create-ui-stream.ts

api/src/ai/chat/utils/format-context.ts

app/src/ai/composables/use-search-filter.ts

app/src/ai/stores/use-ai-context.ts

Co-authored-by: Copilot <[email protected]>

… ve package

Co-authored-by: Florian C. Wachmann <[email protected]>

api/src/ai/chat/utils/format-context.ts

* refactor: replace useVisualEditorAi with useAiSidebar for clarity * fix: add type annotation for ComputedRef in useAiSidebar function * fix: add type annotations for sidebarSize and sidebarCollapsed emit events * add comment * remove comment (from html) * changeset: add usage notice for visual editing library

bryantgillespie · 2026-02-04T14:11:53Z

Code Review

I found one issue that needs attention:

Context not included in initial AI system prompt File: api/src/ai/chat/lib/create-ui-stream.ts (lines 45-70)

The code computes fullSystemPrompt (which includes context) but uses baseSystemPrompt (without context) on line 52. The prepareStep callback returns fullSystemPrompt, but the comment on lines 61-62 reveals confusion about when context should be applied.

Fix: Change line 52 from system: baseSystemPrompt to system: fullSystemPrompt

This ensures context attachments are properly included from the first AI step.

Reference:

directus/api/src/ai/chat/lib/create-ui-stream.ts

Lines 45 to 70 in c079eb9

const baseSystemPrompt = systemPrompt || SYSTEM_PROMPT;

const contextBlock = context ? formatContextForSystemPrompt(context) : null;

const providerOptions = getProviderOptions(provider, model, aiSettings);

// Compute the full system prompt once to avoid re-computing on each step

const fullSystemPrompt = contextBlock ? baseSystemPrompt + contextBlock : baseSystemPrompt;

const stream = streamText({

system: baseSystemPrompt,

model: registry.languageModel(`${provider}:${model}`),

messages: await convertToModelMessages(messages),

stopWhen: [stepCountIs(10)],

providerOptions,

tools,

/**

* prepareStep is called before each AI step to prepare the system prompt.

* When context exists, we override the system prompt to include context attachments.

* This allows the initial system prompt to be simple while ensuring all steps

* (including tool continuation steps) receive the full context.

*/

prepareStep: () => {

if (contextBlock) {

return { system: fullSystemPrompt };

}

return {};

},

this can be ignored - due to the AI SDK we need to make sure we modify the system prompt in the onPrepareStep everytime so that it gets the updated context.

formfcw

Well done @bryantgillespie ❤️

LGTM 🎉

AlexGaillard

…tant Context (#26512) * wip * cleanup * fix: Hide button when AI_ENABLED = false * wip: add chat to visual editor module * fix: menu tweaks * cleanup more * fix: only refresh if mutation * merge menu components * use new menu component * cleanup more * add changeset * cleanup more * add some tests * fix: make resize handles visible again * extract format context to separate util * fix: items tool prompt causing issues * fix visual editing notifications route * refactor out visual elements tool in favor of context plus items tool * refactor to use visual editor config to handle highligthing * wip: context for prompts * Update format-context.ts Co-authored-by: Copilot <[email protected]> * Update use-search-filter.ts Co-authored-by: Copilot <[email protected]> * Update ai-prompt-variables-modal.vue Co-authored-by: Copilot <[email protected]> * remove logging * more fixes * formatter * add some lightweight xml prompt injection protection * failed to add to context notification * remove unneeded prop * appease ai overlords again * fix tests * actually fix tests * changeset past tense * part of claude code review * no proxy stores * more of claudes fixes * fix claudes composable * more of claudes fixes * fix focus * restore staged context if send message throws * simplify because nested field updates weren't working * add v-textoverflow * move display value to core responsiblity instead of being passed from ve package * fix test * show errors on submit * fix escaping * Update app/src/lang/translations/en-US.yaml Co-authored-by: Florian C. Wachmann <[email protected]> * Update app/src/ai/components/ai-context-menu.vue Co-authored-by: Florian C. Wachmann <[email protected]> * Update app/src/modules/content/routes/item.vue Co-authored-by: Florian C. Wachmann <[email protected]> * latest round of feedback * change id to key to match conventions and fix tool calling issues * make codebot happy * change it to test * fix test * fix translation * one more * veai ← cleanup (#26577) * refactor: replace useVisualEditorAi with useAiSidebar for clarity * fix: add type annotation for ComputedRef in useAiSidebar function * fix: add type annotations for sidebarSize and sidebarCollapsed emit events * add comment * remove comment (from html) * changeset: add usage notice for visual editing library --------- Co-authored-by: Copilot <[email protected]> Co-authored-by: Florian C. Wachmann <[email protected]>

bryantgillespie added 20 commits January 21, 2026 19:55

wip

8925f97

cleanup

bc8f39f

fix: Hide button when AI_ENABLED = false

8fc3735

wip: add chat to visual editor module

35220bc

fix: menu tweaks

31e1c56

cleanup more

e89da8d

fix: only refresh if mutation

9e11da6

merge menu components

320ec64

use new menu component

2bf2ae2

cleanup more

5aca82c

add changeset

ef0fef9

cleanup more

f567888

add some tests

db64aea

fix: make resize handles visible again

a24209d

extract format context to separate util

0587aa8

fix: items tool prompt causing issues

f1d9312

fix visual editing notifications route

7fcd414

refactor out visual elements tool in favor of context plus items tool

1bdd40e

refactor to use visual editor config to handle highligthing

56c32ed

wip: context for prompts

57f88e4

bryantgillespie requested a review from Copilot January 23, 2026 18:58

Copilot started reviewing on behalf of bryantgillespie January 23, 2026 18:58 View session

Copilot AI reviewed Jan 23, 2026

View reviewed changes

bryantgillespie and others added 5 commits January 23, 2026 14:11

Update format-context.ts

e4ef4a5

Co-authored-by: Copilot <[email protected]>

Update use-search-filter.ts

5dfd708

Co-authored-by: Copilot <[email protected]>

Update ai-prompt-variables-modal.vue

d60639d

Co-authored-by: Copilot <[email protected]>

Merge branch 'main' into bry/ai-visual-editor-context

0d9481f

remove logging

ae40e21

bryantgillespie mentioned this pull request Jan 26, 2026

Attach visual editor elements to AI Assistant Context directus/visual-editing#24

Merged

more fixes

3c33193

bryantgillespie and others added 10 commits February 2, 2026 14:25

move display value to core responsiblity instead of being passed from…

53a855e

… ve package

fix test

c8ceac0

show errors on submit

3f95912

fix escaping

d1cfdbb

Update app/src/lang/translations/en-US.yaml

eb72acd

Co-authored-by: Florian C. Wachmann <[email protected]>

Update app/src/ai/components/ai-context-menu.vue

776039b

Co-authored-by: Florian C. Wachmann <[email protected]>

Update app/src/modules/content/routes/item.vue

39abe6f

Co-authored-by: Florian C. Wachmann <[email protected]>

latest round of feedback

e79f79c

Merge branch 'main' into bry/ai-visual-editor-context

6b66e94

change id to key to match conventions and fix tool calling issues

eb14dc9

github-advanced-security bot found potential problems Feb 3, 2026

View reviewed changes

api/src/ai/chat/utils/format-context.ts Fixed Show fixed Hide fixed

This comment was marked as resolved.

Sign in to view

bryantgillespie added 2 commits February 2, 2026 21:28

make codebot happy

1ad618c

change it to test

d444786

This comment was marked as resolved.

Sign in to view

bryantgillespie requested a review from formfcw February 3, 2026 02:35

bryantgillespie and others added 4 commits February 2, 2026 21:37

fix test

bd210c8

fix translation

36485d5

one more

2fc8f8d

This comment was marked as resolved.

Sign in to view

formfcw approved these changes Feb 4, 2026

View reviewed changes

AlexGaillard approved these changes Feb 4, 2026

View reviewed changes

AlexGaillard merged commit 6a05cd7 into main Feb 4, 2026
72 checks passed

AlexGaillard deleted the bry/ai-visual-editor-context branch February 4, 2026 14:25

github-actions bot added this to the Next Release milestone Feb 4, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Attach prompts, content items, and visual editor elements to AI Assistant Context#26512

Attach prompts, content items, and visual editor elements to AI Assistant Context#26512
AlexGaillard merged 63 commits intomainfrom
bry/ai-visual-editor-context

bryantgillespie commented Jan 22, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

bryantgillespie commented Feb 4, 2026 •

edited

Loading

Code Review

Uh oh!

formfcw left a comment

Uh oh!

AlexGaillard left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

bryantgillespie commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Scope

Potential Risks / Drawbacks

Tested Scenarios

Review Notes / Questions

Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

bryantgillespie commented Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review

Uh oh!

formfcw left a comment

Choose a reason for hiding this comment

Uh oh!

AlexGaillard left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

bryantgillespie commented Jan 22, 2026 •

edited

Loading

bryantgillespie commented Feb 4, 2026 •

edited

Loading