Skip to content

[GROUT-134] Save Sync Redux#150

Merged
BrandonKowalski merged 44 commits intov4.7.0.0from
GROUT-134
Mar 6, 2026
Merged

[GROUT-134] Save Sync Redux#150
BrandonKowalski merged 44 commits intov4.7.0.0from
GROUT-134

Conversation

@BrandonKowalski
Copy link
Copy Markdown
Member

Closes #134 #135

malkavi and others added 30 commits February 9, 2026 08:08
There are some work actually to port grout on arm32, and this code was merged before it was ready. Revert back to the old launch.sh for spruce
Installed on Anbernix RG34XX without any issues.

Saw instance and tested download of single game as well.
Bump gabagool to v2.9.5 (table sections), sqlite to v1.46.1.
Add save_sync_history table in schema v10 with indexes on
rom_id and device_id.
- cache/save_sync.go: sync history recording, querying, GetSyncedRomIDs
- cache/games.go: add GetRomByFSLookup for resolving ROMs by fs_slug
- internal/config.go: add SlotPreferences, GetSlotPreference,
  SetSlotPreference, GetDirectoryMapping
- romm/saves.go: switch device tracking endpoints from query to body
  params, add SaveDeviceBody type
- sync/models.go: action types and sync plan models
- sync/flow.go: core sync flow with conflict detection
- sync/flow_test.go: tests for sync action determination
- sync/roms.go: rewrite to resolve local ROMs against cache
- cfw/roms.go: local ROM filesystem scanning (moved from sync)
- cfw/saves.go: add GetSaveDirectory helper
- tools/save-sync-dry-run/: dry-run testing tool
- ui/sync_menu.go: sync hub with Sync Now, Synced Games, History
- ui/save_sync.go: sync execution with progress display
- ui/save_conflict.go: conflict resolution (keep local/remote)
- ui/synced_games.go: platform-grouped view from local sync history
- ui/sync_history.go: table-based history with action icons and
  platform slugs, grouped by date then action
- ui/device_registration.go: device name input for registration
- ui/game_options.go: add save slot selection when multiple slots exist
- ui/actions.go: add action types for new screens
- app/screens.go: register new screen constants
- app/router.go: register screen handlers
- app/transitions.go: add transition functions for sync menu,
  synced games, history, and save sync settings
- ui/platform_selection.go: add Y button for sync menu access
- ui/settings.go: add Save Sync entry in settings menu
- ui/rebuild_cache.go: use cache Clear() instead of folder delete
Add locale keys for sync menu, synced games, sync history,
save conflicts, device registration, and save slot options
across all supported languages (en, de, es, fr, it, ja, pt, ru).
Use outline cloud icons for upload/download actions, add cloud
outline as the action column header. Convert UTC timestamps to
local time for display. Sort rows by action group then alphabetically.
Load all sync history records instead of capping at 50.
Performance tested with 2000+ records without issue.
fix(muos): handle grout icons for muos jacaranda and prior
# Conflicts:
#	cfw/saves.go
Previously, FetchRemoteSaves and DiscoverRemoteSaves silently
continued on HTTP failures, causing local saves to appear as
new uploads and potentially overwriting newer remote data.
Now errors are propagated and the UI shows a connection error
instead of proceeding with a misleading sync.

Closes #144
…cleanup dead code

- Add concurrent fetching (8 workers) for FetchRemoteSaves and DiscoverRemoteSaves
- Guard against empty emulator string on upload
- Add backup retention setting (5/10/15/No Limit) with cleanup logic
- Show last synced relative time in sync menu
- Add health check before starting save sync
- Remove unused API methods (TrackSave, UntrackSave, GetSaveIdentifiers)
- Remove dead ScreenSaveConflict router registration
- Set with_filter_values and with_char_index to false on ROM queries
- Add missing i18n keys to all locale files
@BrandonKowalski BrandonKowalski added this to the v4.7.0.0 milestone Mar 5, 2026
@greptile-apps
Copy link
Copy Markdown

greptile-apps bot commented Mar 5, 2026

Greptile Summary

This PR rewrites the Save Sync feature for grout, replacing the previous auto-sync model with an explicit, user-driven Redux-style flow. The new architecture introduces a clean resolve → conflict-resolution → execute pipeline (sync/flow.go), a Sync Menu hub screen, a Synced Games browser with per-game slot management, a Sync History screen backed by a new save_sync_history SQLite table, and device registration moved to a dedicated UI. The schema is bumped from v7 to v10, dropping legacy save_mappings/filename_mappings/failed_lookups tables and adding save_sync_history and platform_sync_status. Slot preferences are migrated out of config.json into a separate save_slots.json file. A companion test suite (sync/flow_test.go) covers the core determineAction and selectSaveForSync logic.

Key findings:

  • transitionSaveSyncSettings discards SaveConfig error (app/transitions.go:237): device ID, device name, and backup-limit changes survive the current session but are silently lost on restart if the filesystem write fails.
  • GetLastSyncTime silently swallows timestamp parse errors (cache/save_sync.go:119–131) and is never called anywhere in the codebase — inconsistent error handling compared to GetSaveSyncHistory and likely dead code.
  • save_sync_history missing composite (device_id, synced_at DESC) index (cache/schema.go:368–376): the primary read query filters by device_id and sorts by synced_at DESC; a separate single-column index on device_id forces a sort pass that a composite index would eliminate.

Confidence Score: 4/5

  • Safe to merge with one moderate fix — SaveConfig error must be handled in transitionSaveSyncSettings before the feature ships.
  • The core sync logic is well-tested and the conflict-resolution flow is structurally sound. Previously flagged issues (rows.Err(), time.Parse logging, SaveSlotPreferences error handling, Hosts slice bounds check) have all been addressed. The one remaining moderate issue is the silently discarded SaveConfig error that could silently lose device settings. Two minor issues (GetLastSyncTime dead code, missing composite index) are not blocking.
  • app/transitions.go (SaveConfig error handling), cache/save_sync.go (GetLastSyncTime), cache/schema.go (composite index for save_sync_history)

Important Files Changed

Filename Overview
sync/flow.go New core save-sync orchestrator: scans local saves, fetches remote saves concurrently, determines upload/download/conflict/skip actions, and executes them. Logic is well-structured and tests cover the key decision branches. One outstanding concern from previous review (single-ROM failure aborts entire sync) is still present.
app/transitions.go New transition functions for SyncMenu, SyncedGames, SaveSync, and SaveConflict screens added. The conflict-resolution flow and cancel-path handling are correct. However, internal.SaveConfig error is silently discarded in transitionSaveSyncSettings (line 237), which can silently lose device ID and backup-limit settings on filesystem failures.
cache/schema.go Schema bumped from v7 to v10; drops legacy save_mappings/filename_mappings/failed_lookups tables, adds save_sync_history and platform_sync_status. Migration guard uses < 10 correctly and the new tables are created via CREATE TABLE IF NOT EXISTS in createTables. Missing a composite (device_id, synced_at) index for the primary history query pattern.
cache/save_sync.go New file providing CRUD for save_sync_history. GetSaveSyncHistory and GetSyncedRomIDs correctly check rows.Err(). GetLastSyncTime silently swallows a timestamp parse error (unlike GetSaveSyncHistory which logs a Warn) and is never called anywhere in the codebase — potential dead code.
ui/save_sync.go Reworked save-sync screen: split into resolve and execute phases, added multi-slot selection for first-time downloads, and wired up conflict resolution handoff. SaveSlotPreferences errors are now logged (previous concern addressed). Logic is clean and handles edge cases (empty items, cancelled slot selection) correctly.
ui/synced_games.go New Synced Games screen grouping previously-synced ROMs by platform. Slot preference changes trigger targeted uploads back through the router. SaveSlotPreferences error is logged. Navigation loops are well-guarded against ErrCancelled.
ui/save_conflict.go New conflict-resolution screen presenting Keep Local / Keep Remote options per conflicting game. applyResolutions correctly mutates the conflicts slice in-place via pointer receivers. Cancellation returns SaveConflictActionCancel, which causes the router to pop back cleanly.
sync/flow_test.go New test suite for the core sync decision logic (determineAction, selectSaveForSync, DetermineActions, cleanupBackups). Coverage is good across conflict, upload, download, and skip scenarios.
internal/config.go SaveSyncMode and GameSaveOverrides removed from config. New SlotPreferences field backed by save_slots.json. LoadSlotPreferences/SaveSlotPreferences helpers added. Migration path from old config keys is handled via the json:"-" tag so old fields are ignored on load.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[User: Tap Sync Now] --> B[SaveSyncScreen.Execute]
    B --> C{ResolvedItems provided?}
    C -- Yes --> G
    C -- No --> D[Health check: GetHeartbeat]
    D -- Fail --> E[Show error message\nReturn empty output]
    D -- OK --> F[ResolveSaveSync\nScan local saves\nFetch remote saves concurrently\nDiscoverRemoteSaves for ROMs without local saves]
    F --> F1{Multiple slots\non first-time downloads?}
    F1 -- Yes --> F2[Show slot picker\nUpdate config.SlotPreferences\nSave to save_slots.json]
    F1 -- No --> F3
    F2 --> F3{Any conflicts?}
    F3 -- Yes --> H[Return NeedsConflictResolution=true\nto router]
    H --> I[SaveConflictScreen\nKeep Local / Keep Remote per game]
    I -- Cancelled --> J[popOrExit: back to Sync Menu]
    I -- Resolved --> K[transitionSaveConflict\nMerge resolved items back into AllItems]
    K --> B
    F3 -- No --> G[executeSyncPhase]
    G --> L[ExecuteSaveSync\nFor each item: upload / download / skip / conflict]
    L --> L1[upload: POST save file\nMarkDeviceSynced\nChtimes to match server mtime]
    L --> L2[download: backup existing file\nGET save content\nwrite file\nMarkDeviceSynced]
    L --> M[RecordSaveSync for successful items\nINSERT INTO save_sync_history]
    M --> N[showReport: Uploaded / Downloaded / Errors / Conflicts]
Loading

Last reviewed commit: 86d120b

@rommapp rommapp deleted a comment from greptile-apps bot Mar 6, 2026
@BrandonKowalski BrandonKowalski merged commit 422bff9 into v4.7.0.0 Mar 6, 2026
1 check passed
@BrandonKowalski BrandonKowalski deleted the GROUT-134 branch March 6, 2026 03:10
} else {
ctx.state.Config.Hosts = []romm.Host{ctx.state.Host}
}
internal.SaveConfig(ctx.state.Config)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SaveConfig error silently discarded

internal.SaveConfig returns an error, but it is ignored here. If the write fails (e.g., read-only filesystem, disk full on the handheld), the device registration (DeviceID, DeviceName) and backup-limit changes are held in memory for the current session but will be permanently lost on the next restart with no feedback to the user.

Suggested change
internal.SaveConfig(ctx.state.Config)
if err := internal.SaveConfig(ctx.state.Config); err != nil {
gaba.GetLogger().Error("Failed to save config after save-sync settings update", "error", err)
}

Comment on lines +119 to +131
err := cm.db.QueryRow(`
SELECT synced_at FROM save_sync_history
WHERE device_id = ?
ORDER BY synced_at DESC
LIMIT 1
`, deviceID).Scan(&syncedAt)
if err != nil {
return nil
}

t, err := time.Parse(time.RFC3339, syncedAt)
if err != nil {
return nil
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GetLastSyncTime silently swallows parse error and is never called

Two separate concerns here:

  1. When time.Parse fails, the function returns nil without logging anything. GetSaveSyncHistory (line 60–65 of the same file) logs a Warn for the identical scenario — this is an inconsistency that could make silent DB corruption harder to diagnose.

  2. GetLastSyncTime is not called anywhere in the codebase. If it is intended for future use, keeping it around without callers means it will drift out of sync with the schema changes around it.

Consider adding the same Warn log that GetSaveSyncHistory uses, and either adding a caller or removing the function until it is needed.

Suggested change
err := cm.db.QueryRow(`
SELECT synced_at FROM save_sync_history
WHERE device_id = ?
ORDER BY synced_at DESC
LIMIT 1
`, deviceID).Scan(&syncedAt)
if err != nil {
return nil
}
t, err := time.Parse(time.RFC3339, syncedAt)
if err != nil {
return nil
t, err := time.Parse(time.RFC3339, syncedAt)
if err != nil {
gaba.GetLogger().Warn("Failed to parse last sync timestamp", "value", syncedAt, "error", err)
return nil
}

Comment on lines +368 to 376
_, err = tx.Exec(`CREATE INDEX IF NOT EXISTS idx_save_sync_history_rom_id ON save_sync_history(rom_id)`)
if err != nil {
return err
}

// Track per-platform game sync status
_, err = tx.Exec(`
CREATE TABLE IF NOT EXISTS platform_sync_status (
platform_id INTEGER PRIMARY KEY,
last_successful_sync TEXT,
last_attempt TEXT,
games_synced INTEGER DEFAULT 0,
status TEXT DEFAULT 'pending'
)
`)
_, err = tx.Exec(`CREATE INDEX IF NOT EXISTS idx_save_sync_history_device_id ON save_sync_history(device_id)`)
if err != nil {
return err
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

save_sync_history missing composite index for primary query pattern

The main read path in GetSaveSyncHistory is:

SELECTFROM save_sync_history
WHERE device_id = ?
ORDER BY synced_at DESC

There are separate indexes on rom_id and device_id, but no composite (device_id, synced_at) index. SQLite will use the device_id index to satisfy the WHERE, but must then do a separate sort pass for ORDER BY synced_at DESC. On a handheld device with many sync entries this in-memory sort adds unnecessary overhead.

Adding a composite index eliminates the sort:

CREATE INDEX IF NOT EXISTS idx_save_sync_history_device_synced
    ON save_sync_history(device_id, synced_at DESC)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants