Skip to content

feat: expose prometheus metrics - #62

Merged
dannykopping merged 12 commits into
mainfrom
metrics
Nov 24, 2025
Merged

dannykopping merged 12 commits into
mainfrom
metrics

Conversation

@dannykopping

@dannykopping dannykopping commented Nov 21, 2025 •

Copy link
Copy Markdown
Collaborator

Closes #25
Required for coder/coder#20865

Metrics are updated abstractly, no provider-specific logic: all updates happen in the recorder/interceptor layers.

Signed-off-by: Danny Kopping <[email protected]>
Signed-off-by: Danny Kopping <[email protected]>
Signed-off-by: Danny Kopping <[email protected]>
Signed-off-by: Danny Kopping <[email protected]>
Signed-off-by: Danny Kopping <[email protected]>
Signed-off-by: Danny Kopping <[email protected]>
Signed-off-by: Danny Kopping <[email protected]>
Signed-off-by: Danny Kopping <[email protected]>
@dannykopping
dannykopping marked this pull request as ready for review November 21, 2025 14:46
@dannykopping
dannykopping requested a review from pawbana November 21, 2025 14:46
@dannykopping dannykopping assigned mtojek and unassigned mtojek Nov 21, 2025
@dannykopping
dannykopping requested a review from mtojek November 21, 2025 14:47
@dannykopping dannykopping self-assigned this Nov 21, 2025
Comment thread api.go Resolved

@mtojek mtojek left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First review round done.

What dragged my attention was initiator_id property, and the risk of metrics increase if there are many initiators, otherwise only minors.

Comment thread metrics.go Resolved
Comment thread metrics.go Resolved
Comment thread metrics.go Resolved
Comment thread metrics.go Resolved
Comment thread metrics.go Resolved
Comment thread interception.go Resolved
Comment thread metrics_integration_test.go Resolved
Comment thread metrics.go
Subsystem: "prompts",
Name: "total",
Help: "The number of prompts issued by users (initiators).",
}, append(baseLabels, "initiator_id")),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How many initiator_ids do we expect? I wanted to confirm - it will not blow up the registry?

EDIT:

actually, why exactly do we need this id? is it to identify tool, user, or something else?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Operators will need to know if one user (or, by extension, agent) is overloading the system.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, but since initiator_id will be uuid, how will operators match and id with specific agent or extension? will bridge support/"deference" it?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

initiator_id == users.id

Comment thread metrics.go Resolved
Comment thread metrics.go Resolved
Signed-off-by: Danny Kopping <[email protected]>
Signed-off-by: Danny Kopping <[email protected]>
@dannykopping

Copy link
Copy Markdown
Collaborator Author

@mtojek @pawbana I removed the initiator labels on the tool-related metrics; this significantly reduces the worst-case cardinality. Please take another look.

@dannykopping
dannykopping requested a review from mtojek November 24, 2025 08:09

@mtojek mtojek left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice job, Danny! Nothing blocking on my side, but please wait for 👍 from @pawbana.

Comment thread bridge.go Resolved
Comment thread intercept_openai_chat_streaming.go Resolved
Comment thread metrics.go
Subsystem: "prompts",
Name: "total",
Help: "The number of prompts issued by users (initiators).",
}, append(baseLabels, "initiator_id")),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, but since initiator_id will be uuid, how will operators match and id with specific agent or extension? will bridge support/"deference" it?

@dannykopping
dannykopping merged commit 9e9491b into main Nov 24, 2025
1 check passed
@dannykopping
dannykopping deleted the metrics branch November 24, 2025 13:33
dannykopping added a commit to coder/coder that referenced this pull request Nov 24, 2025
Upgrades `coder/aibridge` to v0.2.0 which includes
coder/aibridge#62.

Creates a `prometheus.Registerer` with a prefix `coder_aibridged_` and
passes that along to coder/aibridge which actually exposes the metrics.

Also includes a side-effect of a change described in
coder/aibridge#62 (comment).

---------

Signed-off-by: Danny Kopping <[email protected]>
david-fraley pushed a commit to coder/coder that referenced this pull request Nov 25, 2025
Upgrades `coder/aibridge` to v0.2.0 which includes
coder/aibridge#62.

Creates a `prometheus.Registerer` with a prefix `coder_aibridged_` and
passes that along to coder/aibridge which actually exposes the metrics.

Also includes a side-effect of a change described in
coder/aibridge#62 (comment).

---------

Signed-off-by: Danny Kopping <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expose Prometheus metrics

3 participants