feat(sync): three way (kobo, koreader and bookorbit) progress and annotation sync#338
Merged
Conversation
|
Important Review skippedToo many files! This PR contains 291 files, which is 141 over the limit of 150. To get a review, narrow the scope: Upgrade to a paid plan to raise the limit. ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (291)
You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
added
ci
neonsolstice
force-pushed
the
BO-337-three-way-sync
branch
from
June 14, 2026 17:58
f5a4fbe to
4d36998
Compare
neonsolstice
changed the title
neonsolstice
changed the title
neonsolstice
force-pushed
the
BO-337-three-way-sync
branch
2 times, most recently
from
June 14, 2026 18:55
bb08621 to
6f64644
Compare
neonsolstice
force-pushed
the
BO-337-three-way-sync
branch
4 times, most recently
from
June 21, 2026 06:48
69b91aa to
fe6b2c4
Compare
neonsolstice
force-pushed
the
BO-337-three-way-sync
branch
5 times, most recently
from
June 25, 2026 04:14
c608ef4 to
0fb5ca0
Compare
Introduces a custom KOReader plugin to synchronize reading progress, stats, and annotations. Adds backend controllers, services, and database schema updates to ingest plugin data. Updates the frontend to display KOReader annotations alongside built-in reader highlights.
Zips the KOReader plugin alongside a generated provision.lua file that contains the user's sync credentials and server URL. This allows users to download a ready-to-use plugin zip directly from their settings page, bypassing the need for manual configuration on the e-reader.
Introduces bidirectional annotation synchronization between the KOReader plugin and the web reader. - Implements a hub-and-spoke sync engine for annotations across devices - Adds a position converter to seamlessly translate KOReader XPointer coordinates to EPUB CFI paths - Includes migrations to update annotation schemas for sync tracking - Adds an Annotations Hub UI on the client to manage cross-device notes and highlights - Updates the KOReader Lua plugin to push and pull annotation changes
…ploads KOReader datetimes carry no timezone. Minting them from UTC server time often lands in the device's future, freezing the device-side upload watermark. This adds deviceTime to the API to compute a clock offset and bring minted datetimes into the device's frame. Also configures the Lua JSON encoder to emit empty arrays instead of objects for empty lists, fixing server-side validation failures, and scrubs JSON nulls from responses to avoid truthiness bugs.
Consolidate KOReader annotations sync and remove legacy frontend components and database migrations.
- Move jumpFileId and pageno to base AnnotationItem - Add originBreakdown to AnnotationStats - Update backend services and repositories to provide these fields - Update frontend components to utilize the new fields
Add backend queries and frontend charts to break down reading time and sessions by source (web, koreader, kobo, manual). Includes a new SourceDistributionChart for user statistics and a ReadingLogSourceSplit for book-level stats.
Add an interactive progress toast to the KOReader plugin during sync sweeps to keep the user informed of the current phase without flooding the e-ink screen. Update the plugin's sync status label to display matched and unmatched book counts, and refine strategy labels. Additionally, widen the Kobo and KOReader settings containers in the web client to max-w-4xl for better readability.
Scale the book facet list by fetching only the top books by recent activity and adding a typeahead search combobox to filter by title or author. Pinned book selections are always preserved in the result set to prevent the selected filter from disappearing.
…ion management Extracted complex state logic from useAnnotationsHub and useBookHighlights into smaller, focused composables (useAnnotationMutations, useAnnotationSelection). Created distinct components for toolbar, bulk operations, pagination, and filter panels to improve maintainability. Improved URL sync logic for annotation filters and fixed a test mock in statistics.
Main added 0019_add_lubimyczytac_provider_id while this branch independently used 0019-0023 for the koreader/kobo/reading-session migrations. Keep main's migration at 0019 and renumber this branch's to 0020-0024, rebuild the journal, repoint the snapshot prevId chain, and fold the lubimyczytac_id column into the affected snapshots. Verified: drizzle-kit generate reports no schema changes against the merged schema, and all 25 migrations apply cleanly in order on a fresh database.
Main's 0020 (achievements devices + reading_sessions.source) and the
branch's migrations both claimed slot 0020 when diverging from the same
base. Renumber the branch migrations 0020-0024 to 0021-0025 so main's
0020 lands first, and rebuild the journal plus the snapshot prevId
chain to match.
Main's 0020 already adds reading_sessions.source as NOT NULL DEFAULT
'web', so rewrite 0023 to drop the default and NOT NULL and widen the
source check to the union ('web','koreader','manual','kobo') instead of
re-adding the column. Adjust the 0024 backfill to re-attribute kor:%
sessions to 'koreader', since main's default already set every existing
row to 'web' and the previous IS NULL guard would no-op.
Drop a duplicate source assertion in the repository spec left by the
merge.
Wrap the ECharts containers as relative and position the charts absolute inset-0 so their rendered size cannot feed back into the grid row height, preventing the autoresize growth loop in the reading log.
Rebased BO-337 onto main, which added 0021_add_book_series_memberships and 0022_add_aladin_id. Renumbered the feature migrations 0021-0025 to 0023-0027, restored main's 0021/0022 snapshots, rebuilt the journal, and re-pointed the first feature snapshot's prevId to main's aladin snapshot. Regenerated the final 0027 snapshot to capture the merged schema. Validated: drizzle-kit generate reports no schema changes, and all 28 migrations apply cleanly in order on a fresh database.
Keep the rebased client code on the installed icon package and update server utility imports for the dependency versions on main.
Expose JSON catalog endpoints for KOReader credentials and wire the plugin to browse, search, download, and link files to sync. Ref #337
Merge separate BookOrbit sync and browse menu items into a single unified menu. Improve catalog navigation by adding first/last page support and properly cleaning up thumbnail jobs on menu return or close. Ref #337
Extend the OPDS catalog endpoints to support filtering by read status, file format, and explicit book IDs. Add new sort orders for ascending recent/updated dates and recently read status.
Move the search option from a dedicated row in the root catalog view to the top-left title bar button for better accessibility across all non-book contexts. Reorder the root sections to prioritize 'Continue reading' and 'Recently added' at the top of the list. Ref #337
Adds support for browsing distinct authors and series in the native KOReader catalog, complete with pagination and progress tracking. This updates the backend OPDS/KOReader services to return paginated lists of authors and series, and introduces new Lua widgets in the plugin to render them.
Normalize provider identifier prefixes during OPF parsing so malformed scheme values do not exceed database column limits. Add accessible labels to reader color buttons.
Pace vs Goal chart hardcoded goalBooks=12 regardless of the user's saved reading goal. The API call now reads from dashboardConfig.readingGoal and re-fetches reactively via a new extraWatchSources option on useStatisticsQuery. Search inside a Smart Scope returned unfiltered results because getBooksPage early-returned from getBooksBySmartScope before the q clause was applied. The q filter is now forwarded and AND-ed with the scope's where clause. The reading log journey chart grew infinitely in multi-column layout. The h-full + flex-1 + echarts autoresize combination created a grid row-height feedback loop. Adding lg:items-start to the grid container prevents row stretching and breaks the loop. KOReader plugin: the old bookorbit-koplugin stored settings under the same bookorbit key with a different schema. The new plugin read those stale values without resetting behavior keys (sync_forward, sync_backward, catalog settings), causing the sync prompt to hang and crash on second tap. Adds settings_version detection and migrates old settings to defaults while preserving credentials and provision_fingerprint. Ref #337
Calling downloadmgr:chooseDir while a ButtonDialog was still open caused a crash on Android when the user tapped a folder in the chooser. The fix closes the download dialog first, then reopens it via nextTick once the new download directory has been saved. Any filename customisation made before picking a folder is reset on reopen, which is an acceptable tradeoff against the crash. Ref #337
Daily reading statistics were previously computed using UTC for date boundaries, which led to incorrect daily values for users in other timezones. This updates the aggregation logic, the reading session daily stats service, and the backfill script to respect the user's configured timezone when bucketting sessions into days. Ref #337
…lank Date Started and Date Finished remained null after KOReader sync or progress-based auto-update, silently breaking achievements and dashboard widgets that filter by these dates. Fall back to MIN(startedAt) and MAX(endedAt) from reading sessions when the status fields would otherwise be null. Explicit clears, unread/want_to_read statuses, and finishedAt values that would precede startedAt are all guarded.
Adds a version check and one-tap update flow to the KOReader plugin. Server: two new KOReader-auth endpoints - GET /koreader/plugin/version returns the latest plugin and server versions; GET /koreader/plugin/package serves a code-only zip (no provision file, so credentials are never overwritten). Plugin: new 'Plugin: vX.Y.Z' label and 'Check for update' button in the menu. Update uses an atomic backup/stage/rename swap instead of in-place extraction, with POSIX single-quote shell escaping and unknown-version error detection.
Preserve main's Hardcover sync migration at 0023 and shift the annotation sync migration series to 0024 through 0028 after rebasing onto main.
Prompt users to update their KOReader plugin when a new version is released to ensure they have the latest sync fixes and features. Ref #337
neonsolstice
force-pushed
the
BO-337-three-way-sync
branch
from
June 26, 2026 02:00
a880296 to
4313938
Compare
|
🎉 This PR is included in version 2.0.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
github-actions
Bot
added
the
released
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Introduces a complete three-way synchronization system for reading progress, annotations, and reading sessions across KOReader, Kobo, and BookOrbit.
Closes #337
How did you test this?
Screenshots
Anything non-obvious in the diff?
There is significant logic in bridging the position formats (e.g., KOReader progress percentages vs. Kobo chapter spans) through the
position-convertermodule to ensure accuracy across the ecosystems.Checklist
.env, personal configs)