Skip to content

Comments

Fix sqlite connection closed issue#6815

Merged
prathyushpv merged 9 commits intomainfrom
ppv/sqliteConnFix
Nov 14, 2024
Merged

Fix sqlite connection closed issue#6815
prathyushpv merged 9 commits intomainfrom
ppv/sqliteConnFix

Conversation

@prathyushpv
Copy link
Contributor

What changed?

Use an sqlite connection that has ResetSession() and IsValid() implemented.

Why?

If these methods are not implemented, sql library will close the connection when the context is cancelled during a transaction. We will lose the database if this happens.
IsValid() will always return true in this case. We should always have a valid database connection, otherwise we would have already lost the database. Similarly ResetSession() does nothing.

How did you test it?

Tried to reproduce the issue by cancelling a transaction context.

Potential risks

None

Documentation

Is hotfix candidate?

No

@prathyushpv prathyushpv requested a review from a team as a code owner November 14, 2024 02:39
Copy link
Member

@yiminc yiminc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you add a unit test to verify this fix?

@prathyushpv
Copy link
Contributor Author

Added a unit test which will fail if we don't have this fix.


const (
goSqlDriverName = "sqlite"
goSqlDriverName = "sqlite_temporal"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering why this change is required. What's the implication of changing this string?

Copy link
Contributor Author

@prathyushpv prathyushpv Nov 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

golang's sql library keeps a list of registered drivers adressed by the driver name. modernc.org/sqlite registers a driver using the name sqlite. So we have to register this new driver using a different name and use this name when opening connections.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, as long as this doesn't have other implications, this works for me.
You should also add a TODO to remove this once the sqlite driver fix is adopted.

Comment on lines 69 to 77
// ResetSession does nothing.
func (c *conn) ResetSession(ctx context.Context) error {
return nil
}

// IsValid always returns true. We only have one connection to the sqlite db. It should always be valid. Otherwise, we would have lost the database.
func (c *conn) IsValid() bool {
return true
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder what the implications are for always returning valid and ignoring ResetSession apart from preventing the connection from being closed on canceled context.

Copy link
Contributor Author

@prathyushpv prathyushpv Nov 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the connection is not valid for some reason, we must have lost the database already. and if the connection is not valid, there is nothing we can do to create a new connection. Because of this I think it is safe to return valid in all cases. We can let the next database operation fail if connection is not valid.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Put this in a comment in the code please.

@prathyushpv prathyushpv requested a review from bergundy November 14, 2024 19:27
Copy link
Member

@bergundy bergundy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved with comments. Thanks!


const (
goSqlDriverName = "sqlite"
goSqlDriverName = "sqlite_temporal"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, as long as this doesn't have other implications, this works for me.
You should also add a TODO to remove this once the sqlite driver fix is adopted.

Comment on lines 69 to 77
// ResetSession does nothing.
func (c *conn) ResetSession(ctx context.Context) error {
return nil
}

// IsValid always returns true. We only have one connection to the sqlite db. It should always be valid. Otherwise, we would have lost the database.
func (c *conn) IsValid() bool {
return true
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Put this in a comment in the code please.

@prathyushpv
Copy link
Contributor Author

Thank you! Added comments.

@prathyushpv prathyushpv enabled auto-merge (squash) November 14, 2024 21:14
@prathyushpv prathyushpv merged commit 1f3ebb5 into main Nov 14, 2024
@prathyushpv prathyushpv deleted the ppv/sqliteConnFix branch November 14, 2024 21:31
prathyushpv added a commit that referenced this pull request Nov 18, 2024
## What changed?
Use the latest release of modernc.org/sqlite library which has our fix
for the connection cancellation issue:
https://gitlab.com/cznic/sqlite/-/merge_requests/74.
We had a temporary fix merged in #6815. Reverting that as well.

## Why?
Sqlite library did not implement functions ResetSession() and IsValid()
in the database connection. sql library was discarding a database
connection when a context was canceled during a transaction. The new
version of modernc.org/sqlite has these two methods implemented.

## How did you test it?
Existing unit test to verify the connection is not lost.

## Potential risks
None

## Documentation

## Is hotfix candidate?
No
stephanos added a commit to stephanos/temporal that referenced this pull request Dec 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants