feat(ai-proxy): add latency and usage in access log and prometheus metrics#12518
Merged
Conversation
dosubot
Bot
added
the
size:XL
Revolyssup
marked this pull request as draft
August 13, 2025 09:19
Revolyssup
marked this pull request as ready for review
August 14, 2025 07:06
Revolyssup
commented
Aug 14, 2025
| "override": { | ||
| "endpoint": "http://localhost:16724" | ||
| } | ||
| "auth": {"header": {"Authorization": "Bearer token"}}, |
Contributor
Author
There was a problem hiding this comment.
This change is to avoid the error coming from luajit that string too long.
kayx23
self-requested a review
August 14, 2025 09:21
kayx23
reviewed
Aug 14, 2025
| | consumer | Name of the Consumer associated with a request. Default to an empty string if no Consumer is associated with the request. | | ||
| | node | IP address of the upstream node. | | ||
| | request_type | traditional_http / ai_chat / ai_stream | | ||
| | llm_model | For non-traditional_http requests, name of the llm_model | |
Contributor
Author
There was a problem hiding this comment.
The _ because this will be shown like this in the prometheus metrics.
| enable_access_log: true # Enable HTTP proxy access logging. | ||
| access_log: logs/access.log # Location of the access log. | ||
| access_log_buffer: 16384 # buffer size of access log. | ||
| # available variables: |
Contributor
There was a problem hiding this comment.
the variables available to be logged should not be documented here i guess, as there are several more variables that can be added to the logs.
|
|
||
| set $request_type 'traditional_http'; | ||
|
|
||
| set $llm_time_to_first_token ''; |
Contributor
There was a problem hiding this comment.
in exporter.lua you have named the metric as llm_latency. It's better to have the name consistent across the codebase and docs.
Contributor
Author
There was a problem hiding this comment.
i think its okay to keep this variable more explicit here. and lkeep it llm_latency in prometheus metric.
This reverts commit 46ae368.
dosubot
Bot
added
size:XXL
kayx23
reviewed
Aug 18, 2025
|
|
||
| :::note | ||
|
|
||
| The usage in this example will become available in APISIX 3.13.0. |
kayx23
reviewed
Aug 18, 2025
| :::note | ||
|
|
||
| The usage will become available in APISIX 3.13.0. | ||
|
|
nic-6443
approved these changes
Aug 19, 2025
5 tasks
membphis
approved these changes
Aug 19, 2025
jizhuozhi
pushed a commit
to jizhuozhi/apisix
that referenced
this pull request
Oct 18, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR adds two things:
request_typeto distinguish between normal requests and ai related requests &llm_modelWhich issue(s) this PR fixes:
Fixes #
Checklist