What this PR does:

• Finding the sub ring for a user is a computationally expensive if
• It takes about 46 ms to compute the sub ring for a user under these conditions.
  • 512 tokens per store-gateway
  • 350 store-gateways
  • shard size = 90%
• If there are lots of users in the cluster, the total time can be a lot.
  • For example if there are 20,000 users it'll take about ~15 mins to filter users.
  • 20,000 * 46ms / 1000ms / 60s = ~15 mins
• This is an attempt at optimizing this by using additional cores on the machine.
• Instead of using a single go routine (which runs on a single core), we can use runtime.NumCPU() concurrency to filter users in multiple cores.

Results:
I saw significant improvement in filtering. The time went from 15 mins to 3 mins.

Before:

ts=2023-11-23T02:49:58.737035299Z caller=bucket_stores.go:233 level=info msg="Finished filtering users" users=37115 elapsed=32m37.846198184s

After:

ts=2023-11-23T22:29:07.14464567Z caller=bucket_stores.go:233 level=info msg="Finished filtering users" users=37580 elapsed=3m44.923125472s

Which issue(s) this PR fixes:
Fixes #

Checklist

• Tests updated
• Documentation added
• CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]