Skip to content

Commit 20d9159

Browse files
feat(studio): ask for feedback when a render ends, not on a session counter (#3205)
## What Studio's feedback prompt now fires when a render finishes or fails, instead of on a session counter, and the reports it collects carry enough context to act on. - Replaces the 32px inline bar with a card in the existing toast stack - Adds `studio_feedback_shown` / `studio_feedback_dismissed` / `studio_feedback_interview_click`, so the funnel is visible - A failed export and a crash skip the 0-10 score and ask what happened - Adds a crash prompt to the error boundary - Attaches a breadcrumb trail, render settings and outcome, and how the project was created ## Before / after <img width="1500" alt="Before: a 32px feedback strip pinned under the preview. After: the feedback card in the toast stack, with one-tap answers, and the red variant for a failed export." src="https://github.com/user-attachments/assets/29e2373f-ee96-42c4-b006-ab9a0e56a60a" /> The old bar's rating numbers are `neutral-600` on `neutral-900/80`, which is why it reads as a disabled row rather than a control. **In the running Studio** — bottom-right, sharing the toast stack, hovering a chip explains it on the line above the input: <img width="1500" alt="The feedback card in the bottom-right of the running Studio, with the rotated follow-up question and a hovered chip explained inline." src="https://github.com/user-attachments/assets/3969da50-cc7a-466a-bbf9-b151a1bc1d5d" /> **On the crash screen** — the prompt the error boundary renders, asking what the user was doing rather than what went wrong, since the stack trace already covers the latter: <img width="1200" alt="The Studio crash screen with the feedback card below the Try again and Reload Studio buttons." src="https://github.com/user-attachments/assets/4d287011-9c19-4b94-99d0-07a0388fb39c" /> | | Before | After | |---|---|---| | **Trigger** | Every 10th session | A render finishing, failing, or a crash | | **Placement** | 32px inline bar, pushes preview up | Card in the toast stack, no layout shift | | **Rating targets** | 11 bare buttons, `neutral-600` | Native radios, resting fill, `neutral-400` | | **Follow-up** | Free text or nothing | One rotated question, four to seven one-tap answers | | **Option help** | None | Inline hint on hover and focus | | **Press feedback** | None | `active:scale-[0.97]`, 150ms ease-out | | **Keyboard** | No exit path | Escape closes, Enter sends, arrows move the rating | | **Auto-dismiss** | 20s, always | 30s, cancelled the moment you interact | | **Failure case** | Same NPS question | Its own question, no score, error quoted back | | **Crash case** | Nothing | Prompt on the crash screen | | **Visibility** | Submissions only | Shown / dismissed (with reason) / submitted / interview click | | **Report content** | Rating, comment | Plus breadcrumbs, render settings and outcome, project provenance | ## Why The old bar fired on a session count, so it interrupted at a moment with no subject: nothing the user had just done, nothing to have an opinion about. It also emitted nothing when it appeared or when it was dismissed, which made the collection rate impossible to diagnose. A prompt nobody answers and a prompt that never renders looked identical from the outside. Visually it read as a disabled row: 11 buttons at 11px in `neutral-600` on a dark strip, with no resting affordance. And appearing mid-task pushed the whole preview stack up, which the old code carried a comment apologising for. Separately, the reports it did collect were not actionable. A comment says what went wrong; it almost never says how to get there. ## How **One trigger, one owner.** `feedbackTrigger` owns eligibility and nothing else does: once per tab, thirty days after an answer, seven after a dismissal, never when telemetry is off (prompting someone whose response we would then drop wastes their attention). `VITE_HYPERFRAMES_NO_FEEDBACK=1` still disables it entirely. **One hook, every failure path.** The trigger watches the render job list rather than each of the four places a render can finish (server rejection, unreachable server, SSE terminal event, SSE connection drop), so paths added later are covered without touching the trigger. Renders loaded from disk history never fire it. **Reuses what exists.** The card wears `StudioToast`'s glass treatment and joins its stack, so there is no second visual language and no new CSS. The rating row is native radios, which gives arrow-key navigation, grouping and labels for free. **One question each, rotated across users.** A corner card that asks three things gets answered by nobody. Each person gets one follow-up with one-tap answers, explained on a reserved line rather than a floating tooltip (the card is 340px in a corner; a bubble above the chips lands on the question, below lands on the input). Detractors are never given a rotated question, because they already have a specific complaint. Every option was checked against the code: an option naming a feature Studio already has would collect taps meaning "I could not find it", which is indistinguishable afterwards from "it does not exist". **Breadcrumbs cost one line.** Every studio event already flows through `trackEvent`, so recording the trail there needs no new instrumentation and stays correct as events are added. **Provenance lives outside React.** A crash unmounts the tree, so it is captured when the project loads and read from module scope when the crash prompt renders. ### Privacy Breadcrumbs and provenance carry names, enums and counts only. Values are copied from a fixed allowlist of short keys, and anything longer than a slug is dropped rather than truncated, so comments, file paths, stack traces and project titles cannot reach them even if a future event carries one. Tests assert this. ### Where these responses land Studio feedback goes to PostHog and nowhere else, which is what it did before this change too. Worth stating because the CLI behaves differently: `hyperframes feedback` also forwards to the backend feedback endpoint via `submitFeedback`, on top of its PostHog event. Studio has never used that path, before or after this PR, so if you read CLI feedback anywhere other than PostHog, Studio responses will not show up there. Nothing here changes that either way. Whether the two surfaces should share a delivery path is a product question, not a defect in this change, and closing it would need a field on the backend DTO: it is shaped around `cli_version`, and Studio reports from a crash or a failed export deliberately carry no rating. ## Test plan - [x] Unit tests added/updated - [x] Manual testing performed - [ ] Documentation updated (if applicable) **Unit** — 39 new tests: trigger eligibility and cooldowns, the detractor override, rotation, preset shape and the no-brands rule, breadcrumb rolling and privacy, provenance parsing and its failure modes, and the crash boundary rendering the prompt with no rating input. **Live** — both render paths driven end to end against a running Studio on a production bundle, with real renders. Every PostHog request was intercepted and dropped, so nothing reached the project. Verified the emitted payload for a finished render, a failed export, the rotated follow-ups, each chip's hint, and the interview link. **Not covered** — no live capture of a spontaneous crash. Three attempts to force one failed because Studio's guards held and it kept rendering, so the crash path is verified by component tests rather than by driving it. Touch devices see chip labels without hints, since the hint is revealed on hover and focus.
1 parent dc43831 commit 20d9159

19 files changed

Lines changed: 1503 additions & 244 deletions

‎packages/studio/src/components/EditorShell.tsx‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,6 @@ import {
88
} from "./nle/useTimelineEditCallbacks";
99
import { NLEProvider, useNLEContext } from "./nle/NLEContext";
1010
import { CaptionTimeline } from "../captions/components/CaptionTimeline";
11-
import { StudioFeedbackBar } from "./StudioFeedbackBar";
1211
import { useStudioPlaybackContext, useStudioShellContext } from "../contexts/StudioContext";
1312
import { useDomEditActionsContext, useDomEditSelectionContext } from "../contexts/DomEditContext";
1413
import { TimelineEditProvider } from "../contexts/TimelineEditContext";
@@ -189,7 +188,6 @@ export function EditorShell({
189188
/>
190189
</NLEProvider>
191190
</TimelineEditProvider>
192-
<StudioFeedbackBar />
193191
</div>
194192
);
195193
}
Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
// @vitest-environment happy-dom
2+
3+
import React, { act } from "react";
4+
import { createRoot } from "react-dom/client";
5+
import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
6+
7+
vi.mock("../utils/studioTelemetry", () => ({ trackStudioEvent: vi.fn() }));
8+
vi.mock("../telemetry/policy", () => ({ browserTelemetryAllowed: () => true }));
9+
vi.mock("../telemetry/events", () => ({
10+
trackStudioFeedback: vi.fn(),
11+
trackStudioFeedbackShown: vi.fn(),
12+
trackStudioFeedbackDismissed: vi.fn(),
13+
trackStudioFeedbackInterviewClick: vi.fn(),
14+
}));
15+
16+
const { StudioErrorBoundary } = await import("./StudioErrorBoundary");
17+
const { trackStudioFeedbackShown } = await import("../telemetry/events");
18+
19+
function Boom(): React.ReactElement {
20+
throw new Error("timeline exploded");
21+
}
22+
23+
let container: HTMLDivElement;
24+
let root: ReturnType<typeof createRoot>;
25+
let consoleError: ReturnType<typeof vi.spyOn>;
26+
27+
beforeEach(() => {
28+
localStorage.clear();
29+
sessionStorage.clear();
30+
vi.clearAllMocks();
31+
container = document.createElement("div");
32+
document.body.appendChild(container);
33+
root = createRoot(container);
34+
// React logs the caught error itself; the boundary is the thing under test.
35+
consoleError = vi.spyOn(console, "error").mockImplementation(() => {});
36+
});
37+
38+
afterEach(() => {
39+
act(() => root.unmount());
40+
container.remove();
41+
consoleError.mockRestore();
42+
});
43+
44+
const renderCrashed = () =>
45+
act(() => {
46+
root.render(
47+
<StudioErrorBoundary>
48+
<Boom />
49+
</StudioErrorBoundary>,
50+
);
51+
});
52+
53+
describe("StudioErrorBoundary", () => {
54+
it("shows the crash screen with the error message and both recovery paths", () => {
55+
renderCrashed();
56+
expect(container.textContent).toContain("Something went wrong");
57+
expect(container.textContent).toContain("timeline exploded");
58+
expect(container.textContent).toContain("Try again");
59+
expect(container.textContent).toContain("Reload Studio");
60+
});
61+
62+
it("asks what the user was doing, since the stack trace cannot say", () => {
63+
renderCrashed();
64+
const card = container.querySelector('[aria-label="Send feedback to the HyperFrames team"]');
65+
expect(card).not.toBeNull();
66+
expect(card?.textContent).toContain("Studio crashed. What were you doing?");
67+
});
68+
69+
it("offers one-tap answers instead of a 0-10 score", () => {
70+
renderCrashed();
71+
const card = container.querySelector('[aria-label="Send feedback to the HyperFrames team"]');
72+
const chips = [...(card?.querySelectorAll("button") ?? [])]
73+
.map((b) => b.textContent?.trim())
74+
.filter((t) => t && t !== "Send");
75+
expect(chips).toContain("Editing the timeline");
76+
expect(chips).toContain("Just opened it");
77+
// Rating a crash is a question with no useful answer.
78+
expect(card?.querySelector('input[name="hf-studio-feedback-rating"]')).toBeNull();
79+
});
80+
81+
it("reports the crash prompt to telemetry so the funnel is visible", () => {
82+
renderCrashed();
83+
expect(trackStudioFeedbackShown).toHaveBeenCalledWith(
84+
expect.objectContaining({ reason: "crash" }),
85+
);
86+
});
87+
88+
it("stays quiet when the user already gave feedback recently", () => {
89+
localStorage.setItem("hyperframes-studio:feedbackAnsweredAt", String(Date.now()));
90+
renderCrashed();
91+
// The crash screen itself still works; only the ask is suppressed.
92+
expect(container.textContent).toContain("Something went wrong");
93+
expect(
94+
container.querySelector('[aria-label="Send feedback to the HyperFrames team"]'),
95+
).toBeNull();
96+
});
97+
});

‎packages/studio/src/components/StudioErrorBoundary.tsx‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
import { Component, type ErrorInfo, type ReactNode } from "react";
22
import { trackStudioEvent } from "../utils/studioTelemetry";
3+
import { CrashFeedbackPrompt } from "./feedback/CrashFeedbackPrompt";
34

45
interface Props {
56
children: ReactNode;
@@ -51,6 +52,12 @@ export class StudioErrorBoundary extends Component<Props, State> {
5152
Reload Studio
5253
</button>
5354
</div>
55+
{/* The crash report tells us what broke; only the user can tell us what
56+
they were doing when it did. This is also the one screen where they
57+
have nothing else to get on with. */}
58+
<div className="mt-6">
59+
<CrashFeedbackPrompt />
60+
</div>
5461
</div>
5562
);
5663
}

‎packages/studio/src/components/StudioFeedbackBar.tsx‎

Lines changed: 0 additions & 217 deletions
This file was deleted.

‎packages/studio/src/components/StudioOverlays.tsx‎

Lines changed: 15 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ import { LintModal } from "./LintModal";
33
import { AskAgentModal } from "./AskAgentModal";
44
import { StudioGlobalDragOverlay } from "./StudioGlobalDragOverlay";
55
import { StudioToast } from "./StudioToast";
6+
import { StudioFeedbackCard } from "./feedback/StudioFeedbackCard";
67
import { buildAgentContextPreview } from "./editor/domEditingAgentPrompt";
78
import type { useDomEditSession } from "../hooks/useDomEditSession";
89
import type { useToast } from "../hooks/useToast";
@@ -78,19 +79,20 @@ export function StudioOverlays({
7879
/>
7980
)}
8081
{dragOverlayActive && <StudioGlobalDragOverlay />}
81-
{toasts.length > 0 && (
82-
<div className="absolute bottom-6 right-6 z-[91] flex flex-col items-end gap-2">
83-
{toasts.map((toast) => (
84-
<StudioToast
85-
key={toast.id}
86-
message={toast.message}
87-
tone={toast.tone}
88-
leaving={toast.leaving}
89-
onDismiss={() => dismissToast(toast.id)}
90-
/>
91-
))}
92-
</div>
93-
)}
82+
{/* One bottom-right stack so the feedback card and toasts queue instead
83+
of covering each other. Empty when nothing is showing. */}
84+
<div className="absolute bottom-6 right-6 z-[91] flex flex-col items-end gap-2">
85+
{toasts.map((toast) => (
86+
<StudioToast
87+
key={toast.id}
88+
message={toast.message}
89+
tone={toast.tone}
90+
leaving={toast.leaving}
91+
onDismiss={() => dismissToast(toast.id)}
92+
/>
93+
))}
94+
<StudioFeedbackCard />
95+
</div>
9496
</>
9597
);
9698
}

0 commit comments

Comments
 (0)