Skip to content

perf(subtitles): decouple AI smart context summary from translation#1195

Merged
taiiiyang merged 3 commits intomainfrom
fix/1148-subtitles-ai-smart-context-latency
Mar 28, 2026
Merged

perf(subtitles): decouple AI smart context summary from translation#1195
taiiiyang merged 3 commits intomainfrom
fix/1148-subtitles-ai-smart-context-latency

Conversation

@taiiiyang
Copy link
Copy Markdown
Collaborator

@taiiiyang taiiiyang commented Mar 23, 2026

Type of Changes

  • ✨ New feature (feat)
  • 🐛 Bug fix (fix)
  • 📝 Documentation change (docs)
  • 💄 UI/style change (style)
  • ♻️ Code refactoring (refactor)
  • ⚡ Performance improvement (perf)
  • ✅ Test related (test)
  • 🔧 Build or dependencies update (build)
  • 🔄 CI/CD related (ci)
  • 🌐 Internationalization (i18n)
  • 🧠 AI model related (ai)
  • 🔄 Revert a previous commit (revert)
  • 📦 Other changes that do not modify src or test files (chore)

Description

This PR decouples AI Smart Context summary generation from the subtitle translation hot path.

Subtitle translation no longer waits synchronously for summary generation before sending translation requests. Instead, subtitle summary is fetched separately, then injected into later subtitle requests once available. This keeps subtitle translation responsive while still allowing AI Smart Context to improve later batches.

It also normalizes subtitle summary responses to empty strings instead of undefined, and keeps subtitle summary context hashing aligned with the existing cleanText() default length limit rather than adding an extra 1000-character cutoff.

Related Issue

Closes #1148

How Has This Been Tested?

  • Added unit tests
  • Verified through manual testing

Screenshots

Checklist

  • I have tested these changes locally
  • I have updated the documentation accordingly if necessary
  • My code follows the code style of this project
  • My changes do not break existing functionality
  • If my code was generated by AI, I have proofread and improved it as necessary.

Additional Information

Subtitle summary is now fetched through a dedicated background message and reused by later subtitle translation requests once it becomes available.


Summary by cubic

Decouples Smart Context summary generation from the subtitle translation path to reduce blocking and latency. Translation starts immediately; summaries are fetched in the background and applied to later batches (addresses #1148).

  • Performance

    • Added background handler getSubtitlesSummary; enqueueSubtitlesTranslateRequest now accepts summary (removed subtitlesContext) and passes it through the batch queue.
    • Updated hashing to encode summary ready/missing and include content when present; the universal adapter computes a cleaned-transcript context hash via buildSubtitlesSummaryContextHash, fetches via fetchSubtitlesSummary, and injects the summary only when the hash matches.
  • Bug Fixes

    • Normalize missing summaries to an empty string and deduplicate concurrent summary requests.
    • Discard stale summaries when the context hash changes; align summary context hashing with cleanText()’s default limit for stable cache keys.

Written for commit 4202db3. Summary will update on new commits.

@changeset-bot
Copy link
Copy Markdown

changeset-bot bot commented Mar 23, 2026

🦋 Changeset detected

Latest commit: 4202db3

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@read-frog/extension Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions github-actions bot added the perf label Mar 23, 2026
Copy link
Copy Markdown
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 7 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/entrypoints/subtitles.content/universal-adapter.ts">

<violation number="1" location="src/entrypoints/subtitles.content/universal-adapter.ts:323">
P2: Handle rejection from the background summary request to avoid unhandled promise rejections.</violation>
</file>

<file name="src/utils/subtitles/processor/translator.ts">

<violation number="1" location="src/utils/subtitles/processor/translator.ts:112">
P2: Handle `sendMessage` failures in `fetchSubtitlesSummary`; it currently rejects and can create unhandled promise rejections in the caller.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

Comment thread src/entrypoints/subtitles.content/universal-adapter.ts
Comment thread src/utils/subtitles/processor/translator.ts
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1c9d64ed26

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/entrypoints/subtitles.content/universal-adapter.ts
@dosubot dosubot bot added the size:XXL This PR changes 1000+ lines, ignoring generated files. label Mar 23, 2026
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c59e613d55

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/entrypoints/subtitles.content/universal-adapter.ts
@taiiiyang taiiiyang requested a review from mengxi-ream March 27, 2026 09:18
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Mar 27, 2026
@taiiiyang taiiiyang merged commit ce61cc9 into main Mar 28, 2026
7 checks passed
@taiiiyang taiiiyang deleted the fix/1148-subtitles-ai-smart-context-latency branch March 28, 2026 06:36
@dosubot
Copy link
Copy Markdown

dosubot bot commented Mar 28, 2026

Documentation Updates

1 document(s) were updated by changes in this PR:

how translation cache works
View Changes
@@ -11,7 +11,9 @@
 When a translation request is made, the system first checks the cache for an existing translation using a hash key. If a cached translation is found, it is returned immediately, avoiding an external translation API call. If the cache does not contain the requested translation, the system performs the translation using the selected provider (Google, Microsoft, or an AI model). After a successful translation, the result is stored in the cache with the hash key and a timestamp.
 
 ### Cache Key Generation
-The cache key is generated as a hash of the following values:
+
+#### Article Translation
+The cache key for article translation is generated as a hash of the following values:
 - The original text to be translated
 - The provider configuration
 - The configured source language code (as set by the user or 'auto'; detected language is not included)
@@ -20,6 +22,26 @@
 - **For LLM (AI) providers with AI Content Aware enabled:** Article context (title and content)
 
 **Note:** The detected language code (`detectedCode`) is intentionally excluded from the cache key. This ensures that cache lookups remain stable even if language detection changes after translation. For LLM providers, the prompt used for translation is included in the cache key. This means that if you change the translation prompt (for example, by editing or selecting a different custom prompt), the cache will be invalidated for that prompt, and new translations will be fetched and cached accordingly. For non-LLM providers, the prompt is not included in the cache key.
+
+#### Subtitle Translation
+Subtitle translation uses separate cache key generation logic from article translation. The cache key for subtitle translation includes all the standard components (text, provider, languages, prompt for LLM providers) plus additional subtitle-specific context:
+
+**When AI Content Aware is enabled:**
+- **Summary status flag:** Either `subtitleSummary=ready` or `subtitleSummary=missing`
+- **Summary content:** The actual summary text (when available), prepended with `summary:`
+
+This means that subtitle translations are cached separately based on whether AI Smart Context summary was available at translation time. The same subtitle text will have different cache entries when:
+1. Translated with AI Smart Context summary available vs. missing
+2. Translated with different summary content (e.g., different videos or context)
+
+**Implementation Details:**
+The subtitle summary context hash is computed using the `buildSubtitlesSummaryContextHash()` function in `translator.ts`, which hashes the cleaned subtitle transcript content and provider configuration. The summary is fetched asynchronously in the background via the `getSubtitlesSummary` message handler and injected into translation requests once available. If the context hash changes (e.g., video navigation), stale summaries are discarded to ensure cache consistency.
+
+**Impact on Cache Behavior:**
+- Subtitle translations without a summary are cached separately from those with a summary
+- This prevents cache mismatches when AI Smart Context becomes available
+- Cache invalidation occurs when summary content or availability changes
+- The decoupled summary fetch allows translation to start immediately without waiting for summary generation
 
 #### Article Context in Cache Keys (LLM Providers with AI Content Aware)
 
@@ -83,7 +105,9 @@
 
 **Summary of Key Points:**
 - The cache key includes the resolved prompt for LLM providers; changing the prompt invalidates the cache for those translations.
-- For LLM providers with AI Content Aware enabled, the cache key also includes article context (title and first 1000 characters of content).
+- For article translation with LLM providers and AI Content Aware enabled, the cache key includes article context (title and first 1000 characters of content).
+- For subtitle translation with AI Content Aware enabled, the cache key includes a summary status flag and summary content (when available).
+- Subtitle translations are cached separately based on whether AI Smart Context summary was available at translation time.
 - Only custom prompts are stored in user config; the default prompt is a code constant.
 - Cache entries older than 7 days are automatically deleted.
 - Users can manually clear the cache from the Options page.

How did I do? Any feedback?  Join Discord

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lgtm This PR has been approved by a maintainer perf size:XXL This PR changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf: investigate AI Smart Context slowing subtitle translation

3 participants