Skip to content

Commit 04cc5a9

Browse files
BlaiseAntonyarmru
andauthored
fix(bootstrap): set up metrics-exporter role in a separate transaction (#10749)
The bootstrap step on the primary used to run `streaming_replica` creation, pg_rewind grants, and `SetupMetricsExporterRole` in a single transaction. A failure during metrics-exporter setup (commonly a duplicate-key race with the controller reconciler) rolled the whole thing back, dropping `streaming_replica` and wedging replica joins until pod restart. Split the bootstrap into two transactions so the metrics-exporter step can fail in isolation without touching the replication setup. Closes #10748 Signed-off-by: Blaise Antony <[email protected]> Signed-off-by: Armando Ruocco <[email protected]> Co-authored-by: Armando Ruocco <[email protected]>
1 parent 6b885c8 commit 04cc5a9

1 file changed

Lines changed: 13 additions & 1 deletion

File tree

  • internal/cmd/manager/instance/run/lifecycle

‎internal/cmd/manager/instance/run/lifecycle/run.go‎

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -178,7 +178,6 @@ func configureInstancePermissions(ctx context.Context, instance *postgres.Instan
178178

179179
contextLogger.Debug("Validating DB configuration")
180180

181-
// A transaction is required to temporarily disable synchronous replication
182181
tx, err := db.BeginTx(ctx, nil)
183182
if err != nil {
184183
return fmt.Errorf("creating a new transaction to setup the instance: %w", err)
@@ -194,6 +193,19 @@ func configureInstancePermissions(ctx context.Context, instance *postgres.Instan
194193
return err
195194
}
196195

196+
if err = tx.Commit(); err != nil {
197+
return err
198+
}
199+
200+
// Set up the metrics-exporter role in a separate transaction: the
201+
// InstanceReconciler also creates this role on the primary, so a
202+
// duplicate-key race here must not roll back the replication-user setup
203+
// committed above (see issue #10748).
204+
tx, err = db.BeginTx(ctx, nil)
205+
if err != nil {
206+
return fmt.Errorf("creating a new transaction to set up the metrics exporter role: %w", err)
207+
}
208+
197209
if err = postgres.SetupMetricsExporterRole(ctx, tx); err != nil {
198210
_ = tx.Rollback()
199211
return err

0 commit comments

Comments
 (0)