Skip to content

Unexpected behaviour of Retry feature using some RDBMSs versions #8352

@rubens21

Description

@rubens21

What are you doing?

The Retry feature fails to identify a rolled back transaction caused by a deadlock (or there is a lack of documentation).

The default behaviour of some MySQL and MariaDb versions is rolling back a transaction if a dead lock is detected (https://dev.mysql.com/doc/refman/5.7/en/innodb-deadlock-detection.html).
Although it may be changed by configuration option (MySQL
https://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html#sysvar_innodb_deadlock_detect, and MariaDb >= 10.2.26 https://mariadb.com/kb/en/library/xtradbinnodb-server-system-variables/#innodb_deadlock_detect), the default action is rolling back the whole transaction, not only the last query.

Due this behaviour, the Retry option may hide an error because the Transaction Promise chain is not broken even after the transaction in the database has been rolled back.

In summary, the Sequelize Transaction promise chain may finish successfully even when all data has been lost.

const Promise = require('bluebird');
const Sequelize = require('sequelize');

let sequelize = new Sequelize('maydb', 'myuser', 'a-secure-password',
  {
    host: 'localhost',
    dialect: 'mysql',
    retry: {
      match: /Deadlock/i,
      max: 5
    },
    logging: console.log
  });

/*
    CREATE TABLE foo (col1 TEXT);
    CREATE TABLE tableA (col1 INT);
    INSERT INTO tableA VALUES (0);
    CREATE TABLE tableB (col1 INT);
    INSERT INTO  tableB VALUES (0);
     */

let update = (t, table, value, delay) => {
  return new Promise((resolve, reject) => {
    setTimeout(() => {
      console.log(`${table} -> ${value};`);
      sequelize
      .query(`UPDATE ${table} SET col1 = ${value};`, {transaction: t})
      .then(resolve)
      .catch(reject);
    }, delay);
  });
};

let process1 = () => {
  return sequelize.transaction((t1) => {
    return writeInFoo("Process1 has started", t1).then(()=> {
      return update(t1, 'tableA', 1, 0);//first update: timer to keep the right order to force a dead lock
    }).then(() => {
      return writeInFoo("Process1 changed tableA", t1);
    }).then(() => {
      return update(t1, 'tableB', 1, 10);//fourth update: timer to keep the right order to force a dead lock
    }).then(() => {
      return writeInFoo("Process1 changed tableB", t1);
    });
  }).then(() => {
    return true;
  }).catch((err) => {
    console.error(err);
    return false;
  });
};

let process2 = () => {
  return sequelize.transaction((t2) => {
    return writeInFoo("Process2 has started", t2).then(()=> {
      return update(t2, 'tableB', 2, 5);//second update: timer to keep the right order to force a dead lock
    }).then(() => {
      return writeInFoo("Process2 changed tableB", t2);
    }).then(() => {
      return update(t2, 'tableA', 2, 0);//third to keep the right order to force a dead lock
    }).then(() => {
      return writeInFoo("Process2 changed tableA", t2);
    });
  }).then(() => {
    return true;
  }).catch((err) => {
    console.error(err);
    return false;
  });
};

let writeInFoo = (message, t) => {
  return sequelize.query(`INSERT INTO foo VALUES (:msg);`, {
    replacements: {msg: message},
    type: sequelize.QueryTypes.INSERT,
    transaction: t
  });
};

Promise.all([process1(), process2()]).then((r) => {
  console.log(r);
  process.exit()
});

What do you expect to happen?

Both process should be atomic.

  1. The table foo should have 3 or 6 entries. (all success or only one success)
  2. The tables tableA and tableB should have all entries with the same value (all 1 or all 2).
  3. If one of the process has been failed, the final result should have one FALSE value.
  4. The MySQL/MariaDb general log should not have a COMMIT query in a thread which lost the transaction.

What is actually happening?

  1. The table foo has 4 entries (3 from the succeeded process, and the last one of the failed process).
    image

  2. The tables tableA and tableB has different values because the last update was made by the failed process.

  3. Both process returns TRUE

Script output:
image

  1. Both treads has all query executed, even after one of them has been rolled back.
    Note that there are two UPDATE tableB SET col1 = 1 been executed! The second one is the second try done by Sequelize, and the transaction of the thread 3231 has already been rolled back at this moment.

image

Dialect: mysql
Dialect version: 2
Database version: 10.0.31-MariaDB-1~trusty-wsrep mariadb.org binary distribution, wsrep_25.19.rc3fc46e
Sequelize version: 4.9.0
Tested with master branch: Yes

Metadata

Metadata

Assignees

No one assigned

    Labels

    dialect: mariadbFor issues and PRs. Things that involve MariaDB (and do not involve all dialects).dialect: mysqlFor issues and PRs. Things that involve MySQL (and do not involve all dialects).type: bugDEPRECATED: replace with the "bug" issue type

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions