Timeline: optional unified history for dictation, meetings and captures #1638

Request

Could Hex offer an optional Dayflow-style timeline that brings together the dictations, meetings, screenshots and Scrawl/Annotate recordings it already stores?

The visual reference is Dayflow, particularly its timeline animation. A day view with time-scaled activity cards and a source inspector would make it easier to reconstruct what I worked on and open the original evidence. It should work with existing Hex history first; passive screen sampling should be a separate optional capability.

This complements the Time Turner capture/recall proposal. Neither background capture nor AI summaries should be required to use the basic timeline.

Existing Hex pieces to reuse

Pinned source: dev a65a5e3. This is source inspection, not verification that the current build has shipped or every flow works on hardware.

• PR #1371 already adds a Captures pane, persistent cards, recording and Restore Last Capture. Extend these ideas rather than replacing them with a second capture library.
• CaptureStore has IDs, timestamps, kind, paths, duration, narration and timed words. Capture bundles provide another source, so deduplicate exports and their original Annotate bundles.
• Dictation history has timestamp, app, raw/final text, duration, processing metadata and transcript versions.
• MeetingManifest has start/end, duration, app, calendar association, notes, chapters and media paths. meeting/query.rs and read-only MCP demonstrate shared CLI/UI/MCP queries.
• Meeting completion events are hints, not a durable event log. Follow subscribe-first, list/reconcile, paginate and deduplicate. #1264 already documents this recovery pattern.
• Hex's capture_host/timeline.rs handles audio alignment; an activity timeline should be named/scoped distinctly.

Dayflow implementation references

Source pinned to 4b9064b, inspected November 9, 2026.

• Layout and card selection: Layout+Panels.swift, CanvasTimelineDataView.swift, and ActivityCard.swift implement native timeline panels, time-scaled/category-colored activity cards and selection. The inspector exposes title/category editing, summary/detail and screenshot playback.
• Capture policy: RecordingPreferences.swift makes capture configurable. Current defaults are 10-second screenshots at 1080p; interval choices include 1/5/10/20/30/60 seconds. Raw storage has a 10 GB default capacity limit. Those are reference defaults, not recommended Hex defaults.
• Frame storage: FrameStore.swift is the segmented HEVC/frame-index implementation. Reuse the bounded-storage pattern, not an assumption that every image is an independent file.
• Scheduling and grouping: AnalysisManager.swift runs a minute scheduler, groups approximately 15-minute batches and splits at gaps over two minutes. LLMService.swift handles analysis using a 45-minute context. These are useful reference parameters, not an exact Hex requirement.
• Transactional publication: StorageManager+TimelineCards.swift is the timeline-card persistence/replacement seam. Study serialized atomic replacement of the relevant time range and late-result handling before implementing live updates.
• Cleanup: StorageManager+Maintenance.swift handles storage maintenance. Activity-card deletion and raw-capture lifetime are distinct; summaries can outlive raw screenshots. Hex should make that distinction visible and intentional.

Useful commits to inspect:

• ef23f565: HEVC storage
• a81e6373: capture settings
• f99dc503: replacement spans
• 240c8713: late-batch handling

Privacy boundary: Dayflow's local/BYO and Pro routes differ. Pro sends screenshots to Dayflow's server; claims about post-analysis server deletion are policy claims, not behavior independently established from the public client source. “Local AI” also does not imply zero network traffic: analytics and favicon retrieval need their own settings/audit. Hex should describe capture storage, provider egress, telemetry, icon fetching and deletion separately rather than inherit a blanket privacy claim.

Proposed implementation

1. Start with factual source events

Create a normalized, read-only event query layer over the existing stores. Each event should carry source kind/ID, UTC start/end, optional app, media/text references, source revision and availability. Keep each native store authoritative; make any index rebuildable.

Show events even with AI and passive screenshots disabled. A dictation belongs at its actual timestamp, a meeting spans its known duration, and a screenshot is a point observation. Do not infer continuous app usage from a few sampled images.

2. Build a native day view and inspector

• Date navigation and a time-scaled vertical timeline with readable overlapping events.
• Distinct source types and optional user-editable categories.
• Click a card to inspect its summary, original transcript or narration, and media availability.
• Scrub available screenshots/recordings and jump to the exact source timestamp.
• Display real gaps, expired media and failed analysis explicitly.
• Filter/search by source, app, date and text without loading all historical media.
• Preserve original evidence when users edit a generated title/category.
• Add week navigation only after the day view and accessibility behavior are solid.

Use the linked Dayflow animation as interaction reference, adapted to Hex's UI rather than copying a complete visual identity.

3. Add optional derived activity cards

An optional, bounded background processor can group source events into activities. Keep factual event rows separate from inferred titles, categories, summaries and durations. Record the model/provider and evidence IDs behind each derived card.

Use explicit local/remote selection, consistent with Hex meeting summaries. No automatic remote fallback. Pass untrusted screen/transcript text as evidence, never as instructions. Show which data classes will be sent to a provider before enabling remote analysis.

Generation should be serialized or revision-checked: snapshot inputs, compute off-thread, then atomically replace only the matching source interval/revision. Stale work must not overwrite newer user edits or resurrect deleted evidence.

4. Keep capture optional and bounded

If Time Turner is enabled later, its observation IDs become another source adapter. Offer a conservative 60-second sampling choice and explicit cleanup controls. Dayflow currently defaults to 10-second screenshots, with a 60-second option; it is not simply a “take one picture every minute and immediately delete it” implementation.

A timeline should remain useful if the user only wants manual captures, dictation and meetings. Disabled background capabilities must do no background capture or model work. Dictation should retain priority over indexing and summarization.

5. Define retention and deletion across the view

Current Hex retention policies differ by source. Captures pane index expiry does not delete exported screenshots/MP4s. Annotate retention has separate heavy-media/bundle lifetimes; meetings have their own policy and optional audio expiry.

The timeline must respect source deletion and expiry rather than becoming a hidden permanent transcript copy. Define separately:

• Hide a card.
• Delete a derived card/summary.
• Delete the underlying source evidence.
• Expire images while intentionally retaining text.
• Remove exported copies outside the managed store.

Cancel or invalidate in-flight analysis when sources disappear. Evict search entries, thumbnails and derived cards as appropriate. Do not claim secure/forensic erasure or encryption from ordinary local storage.

Acceptance criteria

• Timeline works using existing dictation/meeting/capture history with background recording and AI off.
• Source IDs are stable; replaying ingestion or reconciling after restart creates no duplicates.
• Annotate bundle and exported recording are not counted twice.
• Time zones, DST, overnight meetings, equal timestamps, overlaps and missing samples render correctly.
• Every generated card can show its source evidence and uncertainty; users can correct labels without altering transcripts.
• Deletion/expiry propagates to indexes and derived content, including races with queued or running generation.
• Analysis cannot overwrite newer edits or republish forgotten material.
• Off/local/remote modes have tested network behavior, including thumbnails/icons and telemetry.
• Large histories use paginated/bounded queries, lazy media and bounded caches.
• Keyboard navigation, screen-reader labels and non-color category indicators work.

Related: #1388 OCR is separate open design work; #1067 separate meeting audio retention is completed. #1401 local Annotate destination is relevant to exporting bundles, not a requirement for this internal view.

This is an implementation proposal based on source inspection. Review licenses before adapting code and validate the behavior with runtime tests.