Skip to content

[FIx #55866] Fix SQLite3 data loss or corruption when altering tables inside a transaction#57128

Open
emirn wants to merge 3 commits intorails:mainfrom
emirn:fix-sqlite3-cascade-data-loss-inside-transaction
Open

[FIx #55866] Fix SQLite3 data loss or corruption when altering tables inside a transaction#57128
emirn wants to merge 3 commits intorails:mainfrom
emirn:fix-sqlite3-cascade-data-loss-inside-transaction

Conversation

@emirn
Copy link
Copy Markdown

@emirn emirn commented Apr 2, 2026

Motivation:

Addition to the #55907 (created because of #55866) to additionally support case when table alteration is wrapped by ActiveRecord::Base.connection.transaction.

Update 1: This issue is critical for projects using SQLIte with current Rails edge (as of April 3, 2026) and release versions 8.1.3 (see test-full-rails-app.sh below used for testing) silently removing children records data on renaming or removing columns in a parent table.

Detail

#55907 fixed silent data loss during table alterations in SQLite but deletion still happened when table alteration was wrapped by ActiveRecord::Base.connection.transaction. This fix checks if there is an outer transaction (single joinable transaction, as in db:migrate) and the table is referenced by other tables with ON DELETE CASCADE or ON DELETE SET NULL foreign keys. In such case the fix temporarily commits the transaction before disable_referential_integrity runs so the PRAGMA takes effect.

**Update 2**: **test-full-rails-app.sh** below reproduces locally by creating a new Ruby on Rails apps, adding a migration with User and User's Payments (with FK and set for ON DELETE CASCADE) and demonstrating that renaming a column in the parent table causing a removal of all children records in Payments table:
#!/bin/bash
set -e

# Rails versions to test: "edge" for main branch, or a version like "8.1.3"
RAILS_VERSIONS=("edge" "8.1.3")
RAILS_REPO="rails/rails"
RAILS_BRANCH="main"

APP_BASE="cascade_test"

SETUP_MIGRATION='
class CreateTables < ActiveRecord::Migration[7.2]
  def change
    create_table :users do |t|
      t.string :name
      t.string :email
    end

    create_table :payments do |t|
      t.integer :amount
      t.references :user, null: false, foreign_key: { on_delete: :cascade }
    end
  end
end
'

RENAME_MIGRATION='
class RenameNameInUsers < ActiveRecord::Migration[7.2]
  def change
    rename_column :users, :name, :name2
  end
end
'

SEED_DATA='
user = User.create!(name: "Alice", email: "[email protected]")
Payment.create!(amount: 100, user: user)
Payment.create!(amount: 200, user: user)
puts "Seeded: #{User.count} users, #{Payment.count} payments"
'

SETUP_MIGRATION_FILE="db/migrate/00000000000001_create_tables.rb"
RENAME_MIGRATION_FILE="db/migrate/00000000000002_rename_name_in_users.rb"

PRINT_COUNTS='
users    = User.count
payments = Payment.count
puts "  Users: #{users}"
puts "  Payments: #{payments}"
puts ""
puts "  Payment detail:"
Payment.all.each { |p| puts "    [#{p.id}] amount=#{p.amount} user_id=#{p.user_id}" }
'

LOG_FILE="$(pwd)/log.txt"
exec > >(tee "$LOG_FILE") 2>&1

if ! command -v rails &>/dev/null; then
  echo "Installing rails gem..."
  gem install rails --no-document
fi

START_DIR=$(pwd)

# clean up all previous app folders
for VERSION in "${RAILS_VERSIONS[@]}"; do
  rm -rf "$START_DIR/${APP_BASE}_${VERSION}"
done

# results array: "version:BUG" or "version:OK"
RESULTS=()

for VERSION in "${RAILS_VERSIONS[@]}"; do
  APP_NAME="${APP_BASE}_${VERSION}"

  echo ""
  echo "############################################"
  echo "# Testing Rails $VERSION"
  echo "############################################"

  echo "[$VERSION] Creating app"
  rails new $APP_NAME --database=sqlite3 --skip-git --skip-test --skip-bundle --skip-action-mailer --skip-action-mailbox --skip-action-text --skip-active-storage --skip-action-cable --skip-asset-pipeline --skip-javascript --skip-hotwire --skip-bootsnap --quiet
  cd "$START_DIR/$APP_NAME"

  if [ "$VERSION" = "edge" ]; then
    sed -i '' "s|^gem \"rails\".*|gem \"rails\", github: \"$RAILS_REPO\", branch: \"$RAILS_BRANCH\"|" Gemfile
  else
    sed -i '' "s|^gem \"rails\".*|gem \"rails\", \"$VERSION\"|" Gemfile
    # fix load_defaults to match the target version (rails new may generate a newer one)
    MAJOR_MINOR=$(echo "$VERSION" | cut -d. -f1,2)
    sed -i '' "s|load_defaults .*|load_defaults $MAJOR_MINOR|" config/application.rb
  fi

  echo "[$VERSION] Installing bundle"
  bundle install

  echo "[$VERSION] Generating models"
  rails generate model User name:string email:string --no-timestamps --skip-migration --quiet
  rails generate model Payment amount:integer user:references --no-timestamps --skip-migration --quiet

  echo "[$VERSION] Writing setup migration"
  mkdir -p db/migrate
  echo "$SETUP_MIGRATION" > "$SETUP_MIGRATION_FILE"

  echo "[$VERSION] Running initial migrations"
  rails db:migrate 2>&1 | grep -v "^$"

  echo "[$VERSION] Seeding data"
  echo "$SEED_DATA" > db/seeds.rb
  rails db:seed

  echo ""
  echo "[$VERSION] BEFORE column rename migration"
  rails runner "$PRINT_COUNTS"

  echo "[$VERSION] Writing rename migration (forces SQLite table recreate)"
  echo "$RENAME_MIGRATION" > "$RENAME_MIGRATION_FILE"

  echo "[$VERSION] Running rename migration"
  rails db:migrate 2>&1 | grep -v "^$"

  echo ""
  echo "[$VERSION] AFTER column rename migration"
  rails runner "$PRINT_COUNTS"

  PAYMENT_COUNT=$(rails runner "print Payment.count")
  if [ "$PAYMENT_COUNT" = "0" ]; then
    echo "[$VERSION] ISSUE CONFIRMED: all child records were CASCADE-DELETEd!"
    RESULTS+=("$VERSION:BUG")
  else
    echo "[$VERSION] OK: child records survived."
    RESULTS+=("$VERSION:OK")
  fi

  echo ""
  echo "[$VERSION] Versions"
  rails runner "puts \"  Rails:  #{Rails.version}\""
  ruby -e "require 'sqlite3'; puts \"  SQLite: #{SQLite3::VERSION} (lib: #{SQLite3::SQLITE_VERSION})\""
  ruby --version | sed 's/^/  Ruby:   /'

  cd "$START_DIR"
done

echo ""
echo "############################################"
echo "# Summary"
echo "############################################"
for R in "${RESULTS[@]}"; do
  V="${R%%:*}"
  S="${R##*:}"
  if [ "$S" = "BUG" ]; then
    echo "  $V — BUG CONFIRMED (cascade data loss)"
  else
    echo "  $V — OK (no data loss)"
  fi
done

Output (log.txt):

############################################
# Testing Rails edge
############################################
[edge] Creating app
[edge] Installing bundle
Fetching https://github.com/rails/rails.git
Fetching gem metadata from https://rubygems.org/...........
Resolving dependencies...
Bundle complete! 14 Gemfile dependencies, 104 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
[edge] Generating models
[edge] Writing setup migration
[edge] Running initial migrations
== 1 CreateTables: migrating ==================================================
-- create_table(:users)
   -> 0.0006s
-- create_table(:payments)
   -> 0.0004s
== 1 CreateTables: migrated (0.0010s) =========================================
[edge] Seeding data
Seeded: 1 users, 2 payments

[edge] BEFORE column rename migration
  Users: 1
  Payments: 2

  Payment detail:
    [1] amount=100 user_id=1
    [2] amount=200 user_id=1
[edge] Writing rename migration (forces SQLite table recreate)
[edge] Running rename migration
== 2 RenameNameInUsers: migrating =============================================
-- rename_column(:users, :name, :name2)
   -> 0.0041s
== 2 RenameNameInUsers: migrated (0.0042s) ====================================

[edge] AFTER column rename migration
  Users: 1
  Payments: 0

  Payment detail:
[edge] ISSUE CONFIRMED: all child records were CASCADE-DELETEd!

[edge] Versions
  Rails:  8.2.0.alpha
  SQLite: 2.9.2 (lib: 3.51.3)
  Ruby:   ruby 3.4.5 (2025-07-16 revision 20cda200d3) +PRISM [arm64-darwin24]

############################################
# Testing Rails 8.1.3
############################################
[8.1.3] Creating app
[8.1.3] Installing bundle
Fetching gem metadata from https://rubygems.org/..........
Resolving dependencies...
Bundle complete! 14 Gemfile dependencies, 104 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
[8.1.3] Generating models
[8.1.3] Writing setup migration
[8.1.3] Running initial migrations
== 1 CreateTables: migrating ==================================================
-- create_table(:users)
   -> 0.0008s
-- create_table(:payments)
   -> 0.0008s
== 1 CreateTables: migrated (0.0016s) =========================================
[8.1.3] Seeding data
Seeded: 1 users, 2 payments

[8.1.3] BEFORE column rename migration
  Users: 1
  Payments: 2

  Payment detail:
    [1] amount=100 user_id=1
    [2] amount=200 user_id=1
[8.1.3] Writing rename migration (forces SQLite table recreate)
[8.1.3] Running rename migration
== 2 RenameNameInUsers: migrating =============================================
-- rename_column(:users, :name, :name2)
   -> 0.0052s
== 2 RenameNameInUsers: migrated (0.0052s) ====================================

[8.1.3] AFTER column rename migration
  Users: 1
  Payments: 0

  Payment detail:
[8.1.3] ISSUE CONFIRMED: all child records were CASCADE-DELETEd!

[8.1.3] Versions
  Rails:  8.1.3
  SQLite: 2.9.2 (lib: 3.51.3)
  Ruby:   ruby 3.4.5 (2025-07-16 revision 20cda200d3) +PRISM [arm64-darwin24]

############################################
# Summary
############################################
  edge — BUG CONFIRMED (cascade data loss)
  8.1.3 — BUG CONFIRMED (cascade data loss)

Additional information

1. **test-fk-cascade.rb** to confirm data loss for `ON DELETE CASCADE`:
# frozen_string_literal: true

require "bundler/inline"

gemfile(true) do
  source "https://rubygems.org"
  gem "rails", github: "rails/rails", branch: "main"
  gem "sqlite3"
end

require "active_record/railtie"
require "minitest/autorun"

ENV["DATABASE_URL"] = "sqlite3::memory:"

class TestApp < Rails::Application
  config.load_defaults Rails::VERSION::STRING.to_f
  config.eager_load = false
  config.logger = Logger.new($stdout)
  config.secret_key_base = "secret_key_base"
end
Rails.application.initialize!

ActiveRecord::Schema.define do
  create_table :users, force: true do |t|
    t.string :email
    t.string :name
  end
  create_table :payments, force: true do |t|
    t.references :user, null: false, foreign_key: { on_delete: :cascade }
    t.integer :amount, default: 0, null: false
  end
end

class User < ActiveRecord::Base; end
class Payment < ActiveRecord::Base; end

User.create!(email: "[email protected]", name: "Alice")
Payment.create!(user_id: User.first.id, amount: 100)
Payment.create!(user_id: User.first.id, amount: 200)

class RenameColumnMigration < ActiveRecord::Migration::Current
  def change
    rename_column :users, :name, :name2
  end
end

class CascadeTest < ActiveSupport::TestCase
  def test_cascade_fk_data_preserved_after_alter_inside_transaction
    ActiveRecord::Base.connection.transaction do
      RenameColumnMigration.migrate(:up)
    end
    assert_equal 2, Payment.count, "Payments were silently CASCADE-deleted during table alteration inside transaction"
  end
end
2. **test-fk-nullify.rb** to test data corruption with `ON DELETE SET NULL`:
# frozen_string_literal: true

require "bundler/inline"

gemfile(true) do
  source "https://rubygems.org"
  gem "rails", github: "rails/rails", branch: "main"
  gem "sqlite3"
end

require "active_record/railtie"
require "minitest/autorun"

ENV["DATABASE_URL"] = "sqlite3::memory:"

class TestApp < Rails::Application
  config.load_defaults Rails::VERSION::STRING.to_f
  config.eager_load = false
  config.logger = Logger.new($stdout)
  config.secret_key_base = "secret_key_base"
end
Rails.application.initialize!

ActiveRecord::Schema.define do
  create_table :users, force: true do |t|
    t.string :email
    t.string :name
  end
  create_table :payments, force: true do |t|
    t.integer :user_id
    t.integer :amount, default: 0, null: false
  end
  add_foreign_key :payments, :users, on_delete: :nullify
end

class User < ActiveRecord::Base; end
class Payment < ActiveRecord::Base; end

User.create!(email: "[email protected]", name: "Alice")
Payment.create!(user_id: User.first.id, amount: 100)

class RenameColumnMigration < ActiveRecord::Migration::Current
  def change
    rename_column :users, :name, :name2
  end
end

class SetNullTest < ActiveSupport::TestCase
  def test_set_null_fk_data_not_corrupted_after_alter_inside_transaction
    ActiveRecord::Base.connection.transaction do
      RenameColumnMigration.migrate(:up)
    end
    p = Payment.first
    assert_equal 1, p.user_id, "Payment.first.user_id was silently set to NULL during table alteration inside transaction"
  end
end

Checklist

Before submitting the PR make sure the following are checked:

  • This Pull Request is related to one change. Unrelated changes should be opened in separate PRs.
  • Commit message has a detailed description of what changed and why. If this PR fixes a related issue include it in the commit message. Ex: [Fix #issue-number]
  • Tests are added or updated if you fix a bug or add a feature.
  • CHANGELOG files are updated for the changed libraries if there is a behavior change or additional feature. Minor bug fixes and documentation changes should not be included.

@emirn emirn force-pushed the fix-sqlite3-cascade-data-loss-inside-transaction branch 5 times, most recently from 64a3209 to 24250bb Compare April 2, 2026 16:21
@emirn emirn marked this pull request as draft April 2, 2026 17:44
@emirn emirn force-pushed the fix-sqlite3-cascade-data-loss-inside-transaction branch from 24250bb to 290732e Compare April 2, 2026 20:33
@emirn emirn changed the title [#55866] Fix SQLite3 CASCADE data loss when altering tables inside a transaction [#55866] Fix SQLite3 data loss when altering tables inside a transaction Apr 2, 2026
@emirn emirn force-pushed the fix-sqlite3-cascade-data-loss-inside-transaction branch from 290732e to 60d0b10 Compare April 2, 2026 20:57
@emirn emirn marked this pull request as ready for review April 2, 2026 20:58
… foreign keys inside `ActiveRecord::Base.connection.transaction`.

Silent cascade removal of child records was fixed in rails#55907 but when table alteration code was wrapped by `ActiveRecord::Base.connection.transaction`, SQLite3 ignored `PRAGMA foreign_keys = OFF`.

This fix ensures that the transaction (if exists) is now temporarily committed before `disable_referential_integrity` runs for the `PRAGMA` to take effect.

Fixes rails#55866
@emirn emirn force-pushed the fix-sqlite3-cascade-data-loss-inside-transaction branch from 60d0b10 to dfec2ed Compare April 2, 2026 21:07
@emirn emirn changed the title [#55866] Fix SQLite3 data loss when altering tables inside a transaction [#55866] Fix SQLite3 data loss or corruption when altering tables inside a transaction Apr 2, 2026
@emirn emirn changed the title [#55866] Fix SQLite3 data loss or corruption when altering tables inside a transaction [FIx #55866] Fix SQLite3 data loss or corruption when altering tables inside a transaction Apr 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant