Skip to content

Comments

SQL performance improvements#6378

Merged
WithoutPants merged 4 commits intostashapp:developfrom
WithoutPants:sql-optimisations
Dec 7, 2025
Merged

SQL performance improvements#6378
WithoutPants merged 4 commits intostashapp:developfrom
WithoutPants:sql-optimisations

Conversation

@WithoutPants
Copy link
Collaborator

I fell down a bit of a rabbit hole while exploring for performance improvements for #6370.

I made the following improvements:

  • changed queryStruct to use tx.Get instead of queryFunc
  • fixed a latent bug in the table joining code that would add the join's args to the args list even if the join was not added
  • omitted joining tables in the get aggregate query that were necessary only for ordering. Minor increase in performance when getting object counts, total file sizes, etc.
  • made all calls to the sqlite interface pass the context

The last item was the most important one. This fixes what I believe is a pretty nasty latent bug, where queries would not be canceled when the applicable http request was canceled.

An an example, on a database with 32m images - where each image request takes around a minute - I flicked through pages, changed the sort order and direction, and the page sizes all before any results came back. I had to force kill the process because it was cooking my server 😬

A nice side-effect of this is that you can now kill the stash instance while a query is running if you cancel the request via navigating to another page in the UI (assuming the request came from the UI). Previously it would hang until the queries were complete - unless you kill -9 it.

Using queryFunc meant that the performance logging was inaccurate due to the query actually being executed during the call to Scan.
Should provide some marginal improvement on systems with a lot of items.
This means that long queries can be cancelled by navigating to another page. Previously the query would continue to run, impacting on future queries.
@WithoutPants WithoutPants added this to the Version 0.30.0 milestone Dec 5, 2025
@WithoutPants WithoutPants added the improvement Something needed tweaking. label Dec 5, 2025
@smith113-p
Copy link
Contributor

The last item was the most important one. This fixes what I believe is a pretty nasty latent bug, where queries would not be canceled when the applicable http request was canceled.

I wonder if this is what #2272 was about

@WithoutPants
Copy link
Collaborator Author

I wonder if this is what #2272 was about

I think almost certainly.

@WithoutPants WithoutPants merged commit 0fd7a2a into stashapp:develop Dec 7, 2025
2 checks passed
@discourse-stashapp
Copy link

This pull request has been mentioned on Stash. There might be relevant details there:

https://discourse.stashapp.cc/t/stash-v0-30-release-development-thread/4484/1

WithoutPants added a commit to WithoutPants/stash that referenced this pull request Dec 15, 2025
WithoutPants added a commit to WithoutPants/stash that referenced this pull request Dec 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement Something needed tweaking.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants