-
-
Notifications
You must be signed in to change notification settings - Fork 6.5k
Description
Issue description
Properly handle TransactionRetryWithProtoRefreshError in CockroachDB
Expected Behavior
The client should either retry or simply fail and let application handle the error. CockroachDB describes how to handle the error here: https://www.cockroachlabs.com/docs/v22.2/transaction-retry-error-reference.html
Actual Behavior
The app crashes, and a full restart is needed
Steps to reproduce
It's difficult for me to provoke a TransactionRetryWithProtoRefreshError error, but it may happen if theres many updates to the same row
My Environment
| Dependency | Version |
|---|---|
| Operating System | |
| Node.js version | 16.14.2 |
| Typescript version | 4.7.4 |
| TypeORM version | 0.3.6 |
| NestJS | 9.3.9 |
Additional Context
We have a cronjob that runs a query that sometimes will result in a TransactionRetryWithProtoRefreshError error. The TypeORM query is wrapped in a try/catch, but despite this, the whole cornjob will end with the following error:
/node_modules/typeorm/driver/cockroachdb/CockroachQueryRunner.js:225
throw new QueryFailedError_1.QueryFailedError(query, parameters, err);
^
QueryFailedError: restart transaction: TransactionRetryWithProtoRefreshError: WriteTooOldError: write for key /Table/6691/1/"@"/"\x17r\xb0\x80 |LW\xa5\x99\a\x03ED\xbf>"/"\x90\xf6\xf4\xbfNQMO\xb8`\x19\xa8\x00<\xa8o"/"L\xaa!<\x9e\xa7N\xb7\xac\x8b\x13W`V\xe2\xeb"/"ǫ\xd6Ϲ@D\x8c\x8d#\x9f\xac\r\xf4\xa8y"/0 at timestamp 1682440869.268046258,0 too old; wrote at 1682440896.583088232,2: "sql txn" meta={id=c1097e86 key=/Table/6687/1/"@"/"\x13R6s\xfa\"M\x83\xabVť\x04\xa7Z\x8b"/0 pri=0.00328254 epo=0 ts=1682440896.731858898,2 min=1682440869.268046258,0 seq=45} lock=true stat=PENDING rts=1682440869.268046258,0 wto=false gul=1682440869.518046258,0
at CockroachQueryRunner.query (/node_modules/typeorm/driver/cockroachdb/CockroachQueryRunner.js:225:19)
at runMicrotasks (<anonymous>)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async InsertQueryBuilder.execute (/node_modules/typeorm/query-builder/InsertQueryBuilder.js:106:33)
at async /dist/match-making/service/match-making.service.js:1099:29
at async EntityManager.transaction (/node_modules/typeorm/entity-manager/EntityManager.js:72:28)
at async MatchMakingService.createMatchesFromEntities (/dist/match-making/service/match-making.service.js:1078:9)
at async MatchMakingService.createMatches (/dist/match-making/service/match-making.service.js:1068:26)
at async MatchMakingService.createMatchesForTeammates (/dist/match-making/service/match-making.service.js:1134:38)
at async EntityManager.transaction (/node_modules/typeorm/entity-manager/EntityManager.js:72:28) {
query: 'INSERT INTO "match"("id", "user_region", "user_id", "match_id", "team_mate_id", "match_team_mate_id", "team_chat_id", "_chat_id", "algorithm", "trigger", "created_at") VALUES ($1, $2, $3, $4, $5, $6, $7, $8, DEFAULT, $9, DEFAULT), ($10, $11, $12, $13, $14, $15, $16, $17, DEFAULT, $18, DEFAULT), ($19, $20, $21, $22, $23, $24, $25, $26, DEFAULT, $27, DEFAULT), ($28, $29, $30, $31, $32, $33, $34, $35, DEFAULT, $36, DEFAULT) ON CONFLICT DO NOTHING RETURNING id',
parameters: [
'3a28db58-2fce-4869-9b3b-4ca8c0689a6d',
'EUROPE_WEST',
'4caa213c-9ea7-4eb7-ac8b-13576056e2eb',
'1772b080-207c-4c57-a599-07034544bf3e',
'c7abd6cf-b940-448c-8d23-9fac0df4a879',
'90f6f4bf-4e51-4d4f-b860-19a8003ca86f',
'13523673-fa22-4d83-ab56-c5a504a75a8b',
'd3019a8f-2751-492a-b88c-df511f059672',
'REACTIVATION',
'3a28db58-2fce-4869-9b3b-4ca8c0689a6d',
'EUROPE_WEST',
'1772b080-207c-4c57-a599-07034544bf3e',
'4caa213c-9ea7-4eb7-ac8b-13576056e2eb',
'90f6f4bf-4e51-4d4f-b860-19a8003ca86f',
'c7abd6cf-b940-448c-8d23-9fac0df4a879',
'af7bb747-dd6a-4cf4-93f2-bbaef2123c1c',
'd3019a8f-2751-492a-b88c-df511f059672',
'REACTIVATION',
'3a28db58-2fce-4869-9b3b-4ca8c0689a6d',
'EUROPE_WEST',
'c7abd6cf-b940-448c-8d23-9fac0df4a879',
'90f6f4bf-4e51-4d4f-b860-19a8003ca86f',
'4caa213c-9ea7-4eb7-ac8b-13576056e2eb',
'1772b080-207c-4c57-a599-07034544bf3e',
'13523673-fa22-4d83-ab56-c5a504a75a8b',
'd3019a8f-2751-492a-b88c-df511f059672',
'REACTIVATION',
'3a28db58-2fce-4869-9b3b-4ca8c0689a6d',
'EUROPE_WEST',
'90f6f4bf-4e51-4d4f-b860-19a8003ca86f',
'c7abd6cf-b940-448c-8d23-9fac0df4a879',
'1772b080-207c-4c57-a599-07034544bf3e',
'4caa213c-9ea7-4eb7-ac8b-13576056e2eb',
'af7bb747-dd6a-4cf4-93f2-bbaef2123c1c',
'd3019a8f-2751-492a-b88c-df511f059672',
'REACTIVATION'
],
driverError: error: restart transaction: TransactionRetryWithProtoRefreshError: WriteTooOldError: write for key /Table/6691/1/"@"/"\x17r\xb0\x80 |LW\xa5\x99\a\x03ED\xbf>"/"\x90\xf6\xf4\xbfNQMO\xb8`\x19\xa8\x00<\xa8o"/"L\xaa!<\x9e\xa7N\xb7\xac\x8b\x13W`V\xe2\xeb"/"ǫ\xd6Ϲ@D\x8c\x8d#\x9f\xac\r\xf4\xa8y"/0 at timestamp 1682440869.268046258,0 too old; wrote at 1682440896.583088232,2: "sql txn" meta={id=c1097e86 key=/Table/6687/1/"@"/"\x13R6s\xfa\"M\x83\xabVť\x04\xa7Z\x8b"/0 pri=0.00328254 epo=0 ts=1682440896.731858898,2 min=1682440869.268046258,0 seq=45} lock=true stat=PENDING rts=1682440869.268046258,0 wto=false gul=1682440869.518046258,0
at Parser.parseErrorMessage (/node_modules/pg-protocol/dist/parser.js:287:98)
at Parser.handlePacket (/node_modules/pg-protocol/dist/parser.js:126:29)
at Parser.parse (/node_modules/pg-protocol/dist/parser.js:39:38)
at TLSSocket.<anonymous> (/node_modules/pg-protocol/dist/index.js:11:42)
at TLSSocket.emit (node:events:513:28)
at TLSSocket.emit (node:domain:489:12)
at addChunk (node:internal/streams/readable:315:12)
at readableAddChunk (node:internal/streams/readable:289:9)
at TLSSocket.Readable.push (node:internal/streams/readable:228:10)
at TLSWrap.onStreamRead (node:internal/stream_base_commons:190:23) {
length: 739,
severity: 'ERROR',
code: '40001',
detail: undefined,
hint: 'See: https://www.cockroachlabs.com/docs/v22.2/transaction-retry-error-reference.html',
position: undefined,
internalPosition: undefined,
internalQuery: undefined,
where: undefined,
schema: undefined,
table: undefined,
column: undefined,
dataType: undefined,
constraint: undefined,
file: undefined,
line: undefined,
routine: undefined
},
length: 739,
severity: 'ERROR',
code: '40001',
detail: undefined,
hint: 'See: https://www.cockroachlabs.com/docs/v22.2/transaction-retry-error-reference.html',
position: undefined,
internalPosition: undefined,
internalQuery: undefined,
where: undefined,
schema: undefined,
table: undefined,
column: undefined,
dataType: undefined,
constraint: undefined,
file: undefined,
line: undefined,
routine: undefined
}
Relevant Database Driver(s)
- aurora-mysql
- aurora-postgres
- better-sqlite3
- cockroachdb
- cordova
- expo
- mongodb
- mysql
- nativescript
- oracle
- postgres
- react-native
- sap
- spanner
- sqlite
- sqlite-abstract
- sqljs
- sqlserver
Are you willing to resolve this issue by submitting a Pull Request?
Yes, I have the time, but I don't know how to start. I would need guidance.