-
-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Fix RTMPose likelihood computation #3078
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR fixes the likelihood computation in SimCCPredictor to make it consistent with the MMPose decoding logic. The main issue was that likelihoods were either exceeding 1 (returning raw logits) or were extremely small due to improper softmax application across the entire SimCC space.
- Updates
SimCCPredictorto usesigmaanddecode_betaparameters for proper likelihood scaling - Changes default behavior to
apply_softmax=Truefor proper probability computation - Adds new
sigmaanddecode_betaparameters to RTMPose configuration files
Reviewed Changes
Copilot reviewed 33 out of 33 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| deeplabcut/pose_estimation_pytorch/models/predictors/sim_cc.py | Core fix - adds sigma/decode_beta scaling and changes default apply_softmax to True |
| Multiple RTMPose config files | Adds sigma and decode_beta parameters to model configurations |
| Multiple other files | Cleanup of superanimal_humanbody specific code and infrastructure improvements |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
|
cc @maximpavliv can you adress copilots suggestions? |
@MMathisLab yes, done! |
Background
The likelihoods (confidences) produced by
SimCCPredictorwere not consistent with the expected behavior of SimCC decoding.apply_softmax=True, likelihoods were extremely small (~1e-3). This is expected mathematically (softmax distributes probability mass across the entire SimCC space), but not useful as a visibility/confidence measure.normalize_outputs=True, most likelihoods were exactly 1, because normalization simply divided by the maximum logit value.This mismatch originated from how the raw SimCC representation vectors (logits over discretized x/y coordinates) were converted into coordinates and visibility scores.
See the original MMPose predictor implementation for reference.
Changes
This PR brings
SimCCPredictorcloser to the original MMPose decoding logic:1. Added parameters
sigma: variance of the Gaussian used to generate SimCC labels during training. This controls the expected sharpness of the label distribution.decode_beta: scaling factor applied together withsigmato logits before softmax. It acts like an inverse temperature in softmax: sharpening peaked distributions (visible keypoints) and flattening diffuse ones (occluded keypoints).Together,
sigma * decode_betaensures that visibility/confidence is decoded consistently with the label distribution used in training.2. Likelihood computation
normalize_outputs=False(which is the default): the raw logits are scaled by (sigma * decode_beta) before callingget_simcc_maximumwithapply_softmax=True. This matches the MMPose decoding step and yields meaningful visibility/confidence scores in[0,1].3. Default behavior
apply_softmaxnow defaults toTrue(to ensure proper probabilities rather than raw logits are returned).normalize_outputsstill defaults toFalse.Outcome
[0,1].normalize_outputsfor compatibility with previous behavior, but it is discouraged for real confidence estimation.