feat(secret-sync): add Infisical-to-Infisical secret sync#5743
feat(secret-sync): add Infisical-to-Infisical secret sync#5743
Conversation
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
c00b612 to
700a088
Compare
Introduce External Infisical as a new app connection type with complete service implementation, including credential validation, enums, type definitions, and schema mappings.
Register External Infisical connection router and integrate service into the app connection factory. Add identityUaDAL dependency to support credential management.
Implement External Infisical as a new secret sync destination with complete service implementation, schemas, queue handlers, and API routes. Register sync router and integrate into secret sync factory.
Add frontend components and forms for External Infisical app connection and secret sync configuration, including type definitions, API hooks, form validation schemas, and integration logo. Update default on-prem license features to enable audit logs for self-hosted deployments.
700a088 to
6d9670b
Compare
|
@greptile review this please! |
Greptile SummaryThis PR implements a new External Infisical Secret Sync — allowing secrets to be synced between Infisical instances (e.g., cloud → self-hosted) using Machine Identity (Universal Auth). It includes a new app connection type, a dedicated secret sync destination, a daily-retry mechanism for failed syncs via the The architecture is well-structured and consistent with how other sync destinations are implemented in the codebase. Key concerns from earlier review rounds (SSRF via Notable issues found:
Confidence Score: 3/5
Important Files Changed
Last reviewed commit: "fix(ui): add Externa..." |
backend/src/services/app-connection/external-infisical/external-infisical-connection-fns.ts
Show resolved
Hide resolved
backend/src/services/secret-sync/external-infisical/external-infisical-sync-fns.ts
Outdated
Show resolved
Hide resolved
backend/src/services/app-connection/external-infisical/external-infisical-connection-fns.ts
Show resolved
Hide resolved
… and add daily retry destinations Configure ExternalInfisical sync type to use default retry behavior and register it as a daily retry destination, enabling automatic re-attempts of failed syncs.
…l syncs Implement a new daily retry queue job that processes failed External Infisical syncs. This job retrieves syncs that need retry and re-executes them to improve reliability for external Infisical integrations.
…leanup job Wire the new secretSync daily retry queue into the daily resource cleanup job, ensuring failed syncs are retried as part of the system's regular cleanup cadence.
…ntation Add comprehensive documentation for the External Infisical app connection and sync integration, including setup instructions and configuration details.
6d9670b to
c91144c
Compare
…yncSecretsById call sites
…instead of per batch call
backend/src/services/secret-sync/external-infisical/external-infisical-sync-fns.ts
Outdated
Show resolved
Hide resolved
… job Add isDailyResourceCleanUpDevelopmentMode flag support to run the daily secret sync retry job every 5 minutes in development mode instead of only at midnight. This enables faster testing and iteration during development.
Add comprehensive API documentation for the new Infisical-to-Infisical secret sync integration (External Infisical). Includes 9 endpoint references (list, get-by-id, get-by-name, create, update, delete, sync-secrets, import-secrets, remove-secrets) organized under the Infisical nav group in docs.json.
Update the Project field tooltip to clarify that the machine identity must be added as a member to target projects for the sync to work correctly.
…type only Only expose Secret Manager projects when listing remote Infisical projects for sync destination, preventing selection of non-compatible project types.
… API query param Move the filtering logic from client-side to the remote Infisical API layer by passing `type=secret-manager` as a query parameter instead of filtering after the response arrives. This removes the need for the `type` field in the response schema and simplifies the mapping logic. No behavioral change — results are identical but the API call is more efficient.
…deletion Add canRemoveSecretsOnDeletion property to External Infisical sync configuration. This allows users to opt into automatic secret deletion when the sync destination is removed, providing safer cleanup of synced secrets.
…wsing Replace the conditional FilterableSelect + Input fallback pattern with a unified SecretPathInput component that derives child folder names dynamically from the remote Infisical folder tree. This simplifies the component logic and improves the UX for navigating destination folder paths. Changes: - Add folderNames prop to SecretPathInput to accept external folder suggestions, allowing callers to override the internal hook-based folder discovery - Fix bug where selecting a subfolder didn't show its children by appending trailing "/" on selection - Simplify ExternalInfisicalSyncFields by replacing dual-path logic with a single SecretPathInput, computing childFolderNamesAtPath from the current browsed path in the remote folder tree - Remove unused type field from TRemoteInfisicalProject (matches backend fix)
…sync error messages In withExternalInfisicalErrorHandling, the message field was set to parseSyncErrorMessage(err), which returns a pre-serialized JSON string. When the queue later called parseSyncErrorMessage on the outer SecretSyncError object, it resulted in re-serialization of that string, causing double-nested JSON in the audit log. Now we extract the plain human-readable error message directly: err.response?.data?.message ?? err.message. This ensures the message stored in the audit log is a readable string, not nested JSON. Also removes the now-unused parseSyncErrorMessage import from the external-infisical-sync-fns module.
…eSyncErrorMessage Previously, when handling a SecretSyncError, the error field was set to parseSyncErrorMessage(err.error) which returns a string. When this string was passed to JSON.stringify, it was escaped into a nested JSON string, resulting in double-serialization. The fix extracts the inner error directly as an object: - For AxiosError: extract response.data (object) - For other errors: extract .message (string) This ensures JSON.stringify serializes the error field cleanly without intermediate stringification.
…eSyncErrorMessage Previously, when handling a SecretSyncError, the error field was set to parseSyncErrorMessage(err.error) which returns a string. When this string was passed to JSON.stringify, it was escaped into a nested JSON string, resulting in double-serialization. The fix extracts the inner error directly as an object: - For AxiosError: extract response.data (object) - For other errors: extract .message (string) This ensures JSON.stringify serializes the error field cleanly without intermediate stringification.
…t-sync' of https://github.com/Infisical/infisical into igor/eng-4703-implement-infisical-to-in-infisical-secret-sync
…c with key schema When an Infisical-to-Infisical sync had a key schema configured, applying the schema would prefix the secret key. On the destination Infisical instance, the synced secret with the prefixed key would match the sync rule again, triggering another sync cycle that would apply the schema again, creating an infinite loop. The fix disables key schema support for Infisical-to-Infisical syncs entirely: - Pass raw secretMap (not schemaSecretMap) to ExternalInfisicalSyncFns - Set supportsKeySchema: false in sync configuration and schemas - This prevents the prefixed key from triggering downstream syncs
…ng for unsupported destinations
Context
Implements a new External Infisical secret sync that allows syncing secrets from one Infisical instance to another (e.g., cloud → self-hosted, or between two self-hosted deployments).
This includes:
instanceUrl+clientId+clientSecretDailyResourceCleanUpqueue, since remote instances may be temporarily unreachable for extended periods (longer than the normal BullMQ retry window). New providers can opt into this behavior by adding toSECRET_SYNC_DAILY_RETRY_DESTINATIONSinsecret-sync-maps.tsCloses ENG-4703
Screenshots
Steps to verify the change
DAILY_RESOURCE_CLEAN_UP_DEVELOPMENT_MODE=true, force a sync failure, and confirm it gets re-enqueued within 5 minutesType
Checklist