-
Notifications
You must be signed in to change notification settings - Fork 14.2k
Closed
Labels
Description
Issues
- Server
/metricsendpoint share the same task event as/health:TASK_TYPE_METRICS. It means metrics are reset on both calls. - the
Process-Start-Time-Unixhttp response header is not set. - metrics
llamacpp:prompt_tokens_secondsandllamacpp:predicted_tokens_secondsare per slots, while the server actually process llamacpp:prompt_tokens_seconds * n_slots
Proposal
- Add a data params in TASK_TYPE_METRICS to reset the metric bucket only in /metrics
- Add
llamacpp:prompt_tokens_seconds_totalandllamacpp:predicted_tokens_seconds_total
blake-hamm