Janitor optimization use requested at index#3115
Conversation
|
|
Codecov Report
@@ Coverage Diff @@
## master #3115 +/- ##
==========================================
- Coverage 79.67% 79.48% -0.19%
==========================================
Files 112 112
Lines 7932 7864 -68
==========================================
- Hits 6320 6251 -69
Misses 1214 1214
- Partials 398 399 +1
Continue to review full report at Codecov.
|
aeneasr
left a comment
There was a problem hiding this comment.
Thank you very much! We are in the process of refactoring the storage layer. Could you please:
- Confirm this issue exists on the v2.x branch
- If it still exists, make your changes against that branch
Thank you very much!
|
Hello, I had a look on v2.x branch. It seems the code changed a lot. However, the main issue still arise on refresh_token table even if the query changed (https://github.com/ory/hydra/blob/v2.x/persistence/sql/persister_oauth2.go#L430) :
By the way, I think it would be better if the existing index on access_token table would be on (nid, requested_at), rather than on (requested_at, nid). With composite index, it's always better to put more selective index at first, that is, equality first, and then comparison. I've made tests we a small amount of data, and forcing usage of indexes on PostgreSQL: Here the actual query plan : With index on (nid, requested_at) With index on (nid, requested_at) and ORDER BY requested_at, we have no sorting, no full scan, and we have full usage of index. : Since the code change a lot, I think it would be better to create a new pull request, or an issue (I don't know yet if I would have time to create a new pull request). |
9129408 to
32421b9
Compare
…R BY clauses to improve query plan performance
32421b9 to
85962db
Compare
85962db to
13d762b
Compare
|
Superseded by #3516 |
Hi,
on large tables, janitor queries could take minutes.
Table like refresh_token can grow before there is some expired tokens.
By adding missing indexes on requested_at columns, and by ordering on this field we can avoid full table scan, and/or in memory sorting.
introduces a new feature.
contributing code guidelines.
vulnerability. If this pull request addresses a security. vulnerability, I
confirm that I got green light (please contact
[email protected]) from the maintainers to push
the changes.
works.