Skip to content

[Feature] Implement HitRate metric for RecSys#3530

Merged
vfdev-5 merged 7 commits intopytorch:masterfrom
rwtarpit:feature/hitrate-metric
Feb 20, 2026
Merged

[Feature] Implement HitRate metric for RecSys#3530
vfdev-5 merged 7 commits intopytorch:masterfrom
rwtarpit:feature/hitrate-metric

Conversation

@rwtarpit
Copy link
Copy Markdown
Contributor

@rwtarpit rwtarpit commented Feb 11, 2026

Implements Hitrate metric as proposed in #2631

Description:
Implements hitrate metric for recommendation systems. The hitrate is calculated for the top-k list provided by the user and returns a list of hitrate in sorted order. A "hit" is considered when atleast one of our model's top-k predictions for recommedations is actually correct.
Test cases and docstrings are also implemented.

I followed the code structure of other metrics like loss, KL divergence, etc.

Reference : https://github.com/catalyst-team/catalyst/blob/master/catalyst/metrics/functional/_hitrate.py
Also i used Gemini initially for understanding how recommendation systems and its evaluation/metrics work, and for understanding disrtibuted tests.

Check list:

  • New tests are added (if a new feature is added)
  • New doc strings: description and/or example code are in RST format
  • Documentation is updated (if required)

@github-actions github-actions bot added the module: metrics Metrics module label Feb 11, 2026
@rwtarpit rwtarpit marked this pull request as ready for review February 14, 2026 15:27
Copy link
Copy Markdown
Collaborator

@vfdev-5 vfdev-5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the PR @rwtarpit
There are few things to handle and we should be good to go.

@github-actions github-actions bot added the docs label Feb 19, 2026
@rwtarpit rwtarpit requested a review from vfdev-5 February 20, 2026 17:14
Copy link
Copy Markdown
Collaborator

@vfdev-5 vfdev-5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @rwtarpit !

@vfdev-5 vfdev-5 added this pull request to the merge queue Feb 20, 2026
@rwtarpit
Copy link
Copy Markdown
Contributor Author

rwtarpit commented Feb 20, 2026

next step after hitrate could be recall@top_k which has similar logic but instead of binary hits per user, it calculates fraction/percentage of hits per user.

Merged via the queue into pytorch:master with commit 24535a0 Feb 20, 2026
24 checks passed
@vfdev-5
Copy link
Copy Markdown
Collaborator

vfdev-5 commented Feb 20, 2026

next step after hitrate could be recall@top_k which has similar logic but instead of binary hits per user, it calculates fraction/percentage of hits per user.

We already have Precision/Recall metrics, so if we could add top_k wrapper or arg to the existing metrics, it would be better than introducing new metric class

@rwtarpit
Copy link
Copy Markdown
Contributor Author

yes that sounds better!!
i'll take a look and try to find the best approach

@rwtarpit rwtarpit deleted the feature/hitrate-metric branch February 21, 2026 10:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants