Skip to content

feat: Add HTTP status class label to feature server request metrics - #7138

Merged
ntkathole merged 1 commit into
feast-dev:masterfrom
NopeCharon:feat/request-metrics-status-class
Nov 13, 2026
Merged

ntkathole merged 1 commit into
feast-dev:masterfrom
NopeCharon:feat/request-metrics-status-class

Conversation

@NopeCharon

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

feast_feature_server_request_total is labelled endpoint, status, and status is only ever success or error, depending on whether the handler raised. That has two problems:

  • A client error (unknown feature view or feature service, a denied permission) and a server error (the online store is down) end up in the same error bucket, so you can't alert on server-side failures without also paging on bad client requests.
  • Handlers that return an error response without raising are counted as success. That covers the 404 and 422 from /v1/vector_stores/{id}/search, and the 409 from async /materialize and /materialize-incremental when a materialization is already running.

This PR adds a status_class label (2xx, 4xx, 5xx) based on the HTTP status code the server actually responds with:

  • When the handler raises, the status code is worked out the same way the REST exception handling answers. FeastError subclasses use their own http_status_code(), HTTPException uses its status_code, and anything else is a 500.
  • RequestMetricsContext gets a status_code attribute (default 200), following the existing feature_count / feature_view_count pattern. Handlers set it when they return a non-2xx response. The vector store search and the two materialize conflict paths now do this.
  • The existing status label stays, so dashboards and alerts that use it keep working. It is now derived from the status code too: success below 400, error otherwise. The only behaviour change is that the returned 404/422/409 responses above now count as error, not success.
  • status_class is added only to the counter, not to the latency histogram, so histogram cardinality does not change.

Docs: the metrics table in the Python feature server reference now lists the label, there is a short "Request status labels" section with an example PromQL query, and the performance tuning guide mentions the label.

Example:

sum by (endpoint) (rate(feast_feature_server_request_total{status_class="5xx"}[5m]))
  / sum by (endpoint) (rate(feast_feature_server_request_total[5m]))

Which issue(s) this PR fixes:

Fixes #6641

Checks

  • I've made sure the tests are passing.
  • My commits are signed off (git commit -s)
  • My PR title follows conventional commits format

Testing Strategy

  • Unit tests
  • Integration tests
  • Manual tests
  • Testing is not required for this change

New tests in sdk/python/tests/unit/test_metrics.py:

  • track_request_latency maps exceptions to the right class: a plain error gives 5xx, FeatureViewNotFoundException and FeastPermissionError give 4xx, and HTTPException(404) / HTTPException(503) give 4xx / 5xx.
  • A status_code set on the yielded context is used for both labels (202 gives success/2xx; 404, 409 and 503 give error).
  • Through the FastAPI app: /get-online-features that fails with a not-found error answers 404 and records 4xx, and an unexpected error answers 500 and records 5xx. Async /materialize against a feature view that is already materializing answers 409 and records 4xx.

The existing assertions now include the new label. test_metrics.py passes (141 tests), and 16 of those tests fail without the source change. ruff check, ruff format --check and mypy on feast/metrics.py and feast/feature_server.py are clean.

@NopeCharon
NopeCharon requested a review from a team as a code owner November 12, 2026 09:25
@codecov-commenter

codecov-commenter commented Nov 12, 2026 •

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 86.36364% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 52.38%. Comparing base (001d4a6) to head (f236f76).
⚠️ Report is 4 commits behind head on master.

Files with missing lines Patch % Lines
sdk/python/feast/feature_server.py 50.00% 3 Missing ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #7138      +/-   ##
==========================================
+ Coverage   52.31%   52.38%   +0.07%     
==========================================
  Files         459      459              
  Lines       58080    58154      +74     
  Branches     8556     8572      +16     
==========================================
+ Hits        30382    30464      +82     
+ Misses      25614    25608       -6     
+ Partials     2084     2082       -2     
Flag Coverage Δ
go-feature-server 30.96% <ø> (ø)
python-unit 53.88% <86.36%> (+0.07%) ⬆️
Files with missing lines Coverage Δ
sdk/python/feast/metrics.py 86.45% <100.00%> (+0.56%) ⬆️
sdk/python/feast/feature_server.py 66.88% <50.00%> (+1.25%) ⬆️

... and 3 files with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 839e51e...f236f76. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

feast_feature_server_request_total was labelled with endpoint and a
status that was only ever "success" or "error", and the status came
from whether the handler raised. Client errors and server errors fell
into the same bucket, and handlers that returned an error response
without raising (a 404 or 422 from the vector store search, a 409 when
a materialization is already running) were counted as successes.

The counter now carries a status_class label (2xx, 4xx, 5xx) taken
from the HTTP status code the server responds with:

- When the handler raises, the status code is derived the same way the
  REST exception handling answers: FeastError subclasses use their own
  http_status_code(), HTTPException uses its status_code, and anything
  else is a 500.
- RequestMetricsContext gains a status_code attribute (default 200)
  that handlers set when they return a non-2xx response. The vector
  store search and the async /materialize and
  /materialize-incremental conflict paths now do so.

The existing status label is kept for dashboards and alerts that use
it, and is now derived from the status code as well: "success" below
400 and "error" otherwise.

The metrics reference and the performance tuning guide document the
new label.

Fixes feast-dev#6641

Signed-off-by: NopeCharon <[email protected]>
@ntkathole
ntkathole force-pushed the feat/request-metrics-status-class branch from 485d28f to f236f76 Compare November 13, 2026 12:49
@ntkathole
ntkathole merged commit 029dc95 into feast-dev:master Nov 13, 2026
22 of 27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: Add HTTP status class (2xx/4xx/5xx) segmentation to Feature Server error metrics

4 participants