Add per-tool latency and error-rate analytics by wunsir · Pull Request #1521 · HelpCode-ai/anythingmcp

wunsir · 2026-11-05T06:00:54Z

Summary

Adds a Tool performance table to Analytics to help identify slow or failing tools without querying the audit logs manually.

It shows call counts, error rates, p50/p95/p99 latency, and the latest recorded error, grouped by tool and MCP server. The table has its own time-range and connector/server filters, defaulting to the last 24 hours.

The existing Analytics overview and Top tools ranking are unchanged.

The view works the same on Cloud and self-hosted deployments, using the existing audit history without any additional setup.

Changes

Add an organization-scoped, paginated endpoint using the existing invocation data. No database migrations or changes to invocation logging.
Add the performance table with filtering, error details, and loading/retry handling.
Add tests, API documentation, translations (EN/DE/IT), and the generated CLI operation entry. Native-speaker review of the translations is welcome.

Known limitation

The existing audit logger deduplicates repeated failures without including the MCP server in its deduplication key. As a result, matching failures from different servers may be attributed to the first recorded server. Server-level error rates should therefore be treated as estimates rather than exact per-request metrics.

Latency percentiles use the durations actually stored, without duplicating samples for repeated failures. I've left the logging behavior unchanged to keep this PR focused on #984.

Type

Testing

Added/updated unit and integration tests
Tested manually (local development environment)
No tests needed

Local checks:

packages/backend: npm run lint:check, npx --no-install tsc --noEmit -p tsconfig.json, and npm run build passed; 205 audit tests and 8 authentication guard tests passed.
packages/frontend: npm run lint, npx --no-install tsc --noEmit -p tsconfig.json, and npm run build passed; npm run test:e2e -- tests/e2e/analytics-tool-performance.spec.ts passed all 6 targeted Playwright tests.
packages/cli: npm run build and generated operation checks passed. A fresh full npm test run using the installed Git Bash passed: 20 passed, 0 failed, 3 skipped (zsh/fish are not installed).
Repository root: node scripts/generate-cli-operations.mjs --check and node scripts/generate-llms-txt.mjs --check passed.

I also tested the Analytics page manually with the local frontend, backend, and native PostgreSQL. Time-range filters, connector/server filters, pagination, error details, and mobile horizontal scrolling worked as expected. The existing Top tools ranking remained intact.

Remaining test limitations:

The earlier full backend run had 999 passing suites, 2 network-test timeouts, and 6 skipped suites. Rerunning npm test -- src/common/ssrf.util.spec.ts src/connectors/engines/database.engine.ssrf.spec.ts --runInBand --silent reproduced the same two 5-second timeouts, with 36 tests passing. Both test files and their implementations are unchanged from upstream main. Direct DNS lookups for the test hosts (no-such-host.invalid and db.internal) took about 11 seconds to return ENOTFOUND on this machine. The full suite is therefore not marked as passing.
Local SQL integration tests used PGlite. The native PostgreSQL 17 CI run is still pending.

Checklist

My code follows the existing code style
I have tested my changes locally
I have updated documentation
All existing tests pass (npm test)
This PR has a descriptive title

Related Issues

Refs #984

Refs HelpCode-ai#984

github-actions · 2026-11-05T06:01:04Z

👋 Welcome, @wunsir, and thanks for opening your first PR on AnythingMCP!

A few quick pointers:

Make sure CI is green before requesting review (Backend, Frontend, Playwright, CodeQL, Trivy).
If this is a new adapter, the parametrised catalog.spec.ts test will validate it automatically.
Sign off your commits if you can — it's not blocking, just nice to have.

Someone from the core team will look at this within ~48h. If you don't hear back, please ping us in Discussions / Q&A.

⭐ While you wait — if you find AnythingMCP useful, a star helps others discover it.

Copilot

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

github-actions · 2026-11-05T06:01:05Z

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

wunsir · 2026-11-05T07:19:41Z

I have read the CLA Document and I hereby sign the CLA

helpcodeai · 2026-11-05T07:23:36Z

Thanks for picking up #984, @wunsir, and for the detailed write-up, including the test runs that did not pass.

One thing blocks the merge: the CLA. Please post the comment the CLA bot asks for above (I have read the CLA Document and I hereby sign the CLA). The check then goes green.

CI: I approved the workflow runs, so Backend, Frontend, Playwright, CodeQL and Trivy are running now. The integration spec will run against the PostgreSQL service in CI. That covers the native run you could not do locally. The ci.yml change only adds AUDIT_TEST_DATABASE_URL, which is fine.

First read of the code:

The query is scoped to the organization through connectors.organization_id, and servers from another organization are joined out. This is the same pattern as the existing analytics query, and the integration spec checks it with a second organization.
Building the page of groups first and then looking up the latest error per group through LATERAL is a good way to avoid aggregating every error in the window.
The integration spec uses connection-local temp tables, so it cannot touch the CI schema.
The dedup limitation is stated clearly. Keeping the audit logger out of this PR is the right call.
Small addition for the description: please state that the view behaves the same on Cloud and self-hosted. We ask this for every feature (AGENTS.md, "Hard rules and deployment modes").

I'll do the full review once CI is green and the CLA is signed.

helpcodeai

Thanks, @wunsir. I see you had already signed the CLA a few minutes before my comment, sorry for the noise. The CLA check is green, and so is CI. tool-performance.integration.spec.ts ran and passed against the PostgreSQL service in CI, so the native run is covered.

I read the rest of the PR:

Backend: the DTO bounds hours, limit and offset. Filters go through Prisma.sql, so no value is concatenated into the SQL. The endpoint returns the same error text members can already read in the audit log, so it exposes nothing new.
Frontend: a stale response cannot overwrite a newer selection, there is a retry on errors, and lastError is rendered as text. The translations in de and it read naturally.
Tests: the unit, integration and Playwright specs cover pagination, the filters, and isolation between organizations.

Approving. The one thing I'd still like is the Cloud/self-hosted line in the description. It's not a blocker. We'll merge it in the next batch.

wunsir · 2026-11-05T07:53:40Z

Thanks for the review and approval! I've added the Cloud/self-hosted note to the PR description. Appreciate you taking the time to review the changes.

helpcodeai · 2026-11-05T08:50:42Z

Thanks @wunsir, merged. The tool performance view will be on AnythingMCP Cloud with today's deploy, and in the next self-hosted release. I checked the query plan on a large workspace: it uses the tool and date index, so a 90-day window stays fast.

feat(audit): add per-tool latency and error-rate view

1b9976a

Refs HelpCode-ai#984

wunsir requested a review from helpcodeai as a code owner November 5, 2026 06:00

Copilot AI balanced review requested due to automatic review settings November 5, 2026 06:00

Copilot AI reviewed Nov 5, 2026

github-actions Bot added a commit that referenced this pull request Nov 5, 2026

@wunsir has signed the CLA in #1521

5556413

helpcodeai approved these changes Nov 5, 2026

View reviewed changes

Merge branch 'main' into feat/tool-latency-view

94bff9e

helpcodeai enabled auto-merge (squash) November 5, 2026 08:10

Merge branch 'main' into feat/tool-latency-view

5cbb61b

helpcodeai merged commit 5552d54 into HelpCode-ai:main Nov 5, 2026
14 checks passed

github-actions Bot locked and limited conversation to collaborators Nov 5, 2026

Navigation Menu

Repository navigation

Add per-tool latency and error-rate analytics - #1521

Add per-tool latency and error-rate analytics#1521
helpcodeai merged 3 commits into
HelpCode-ai:mainfrom
wunsir:feat/tool-latency-view

wunsir commented Nov 5, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Nov 5, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

github-actions Bot commented Nov 5, 2026 •

edited

Loading

Uh oh!

wunsir commented Nov 5, 2026

Uh oh!

helpcodeai commented Nov 5, 2026

Uh oh!

helpcodeai left a comment

Uh oh!

wunsir commented Nov 5, 2026

Uh oh!

Uh oh!

helpcodeai commented Nov 5, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

wunsir commented Nov 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Known limitation

Type

Testing

Checklist

Related Issues

Uh oh!

github-actions Bot commented Nov 5, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Nov 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wunsir commented Nov 5, 2026

Uh oh!

helpcodeai commented Nov 5, 2026

Uh oh!

helpcodeai left a comment

Choose a reason for hiding this comment

Uh oh!

wunsir commented Nov 5, 2026

Uh oh!

Uh oh!

helpcodeai commented Nov 5, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

wunsir commented Nov 5, 2026 •

edited

Loading

github-actions Bot commented Nov 5, 2026 •

edited

Loading