Add AI Request Logging experiment with observability dashboard#149
Add AI Request Logging experiment with observability dashboard#149Jameswlepage wants to merge 9 commits intoWordPress:developfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds a comprehensive AI Request Logging experiment that provides observability for AI operations through real-time logging, a React-powered admin dashboard, REST API endpoints, and configurable retention settings. The implementation uses the decorator pattern to wrap the SDK's HTTP transporter and supports 14 AI providers out of the box.
Key changes:
- Logging infrastructure with database schema, repository pattern, and cost estimation
- React-based admin dashboard with DataViews for log viewing and filtering
- REST API endpoints for programmatic access to logs and settings
- Automatic cleanup based on retention period and row limits
Reviewed changes
Copilot reviewed 46 out of 47 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| webpack.config.js | Adds webpack entry point for AI request logs admin bundle |
| src/admin/hooks/usePersistedView.ts | Custom hook wrapping @wordpress/views for persisting DataViews state |
| src/admin/components/provider-icons.tsx | Provider icon component mapping for displaying AI provider logos |
| src/admin/components/icons/* | SVG icon components for various AI providers |
| src/admin/ai-request-logs/index.tsx | Main React app component with state management and API integration |
| src/admin/ai-request-logs/components/* | UI components for logs table, summary cards, settings panel, and modal |
| src/admin/ai-request-logs/style.scss | Comprehensive styles for the admin dashboard |
| includes/helpers.php | Helper functions for experiment checking, icon loading, and timeout configuration |
| includes/bootstrap.php | Bootstrap integration for logging initialization |
| includes/Logging/* | Core logging infrastructure (Manager, Schema, Repository, HTTP Transporter) |
| includes/Experiment_Loader.php | Registers the AI Request Logging experiment |
| tests/Integration/Includes/Logging/* | Integration tests for log manager functionality |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
This is great. Building something like this was at the top of my to-do list for January, before I start developing more AI abilities for our staff. |
assets/images/ai-icon.svg
Outdated
There was a problem hiding this comment.
Can we use the colored variant like https://github.com/WordPress/ai/blob/develop/.wordpress-org/icon.svg instead?
|
|
Trying to open PR for review to see if that'll trigger playground generation |
|
The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message. To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook. |
|
In working on a demo video to highlight this feature on make/ai, my first reaction is that the new screen should likely be under |
|
Adding the plugin/theme as a source of the API call would be really important here as well. It's one thing to see that an API use happened, it's another – especially in a common WP environment – to know how or where it was triggered and how to STOP it from continuing (thinking of excessive spend as a problem to solve). |
3731f8f to
f93095e
Compare
|
I force-pushed This was handled as a port onto the current What changed in this refresh:
Verification on this branch:
Manual smoke check:
Remaining limitation here:
If reviewers want, I can follow this with a fresh screenshot/GIF of the updated screen now that the branch is current. |
|
Superseded by the next comment with the correctly formatted CI update. |
|
Pushed This follow-up does two things:
The new run is in progress. The fast checks already restarted cleanly, and |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## develop #149 +/- ##
=============================================
- Coverage 57.85% 49.80% -8.05%
- Complexity 615 1020 +405
=============================================
Files 46 57 +11
Lines 3165 4712 +1547
=============================================
+ Hits 1831 2347 +516
- Misses 1334 2365 +1031
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Pushed follow-up fix at This tightens the request logs screen in two places:
Also rebuilt the request logs assets and added a narrow regression test for the extractor classification path. Local checks:
|
- Replace hand-rolled table, filters, search, and pagination with @wordpress/dataviews for the Request Logs screen - Bundle @wordpress/dataviews and @wordpress/ui (not registered in WP 7.0-RC1) via custom DependencyExtractionWebpackPlugin config - Copy DataViews CSS into build/ and enqueue from PHP with fallback to wp-dataviews style handle when available - Filter unregistered script dependencies in Asset_Loader to prevent wp_enqueue_script notices - Expand AI_Request_Cost_Calculator from 3 providers / 25 models to 8 providers / 50+ models (adds DeepSeek, Mistral, Cohere, Groq, Grok, Ollama; adds o-series reasoning models, Claude 4.5/4.6 IDs) - Add Azure → OpenAI provider alias for cost lookup - Return $0.00 for local providers (Ollama) instead of null - Fix provider tooltip: click-to-toggle instead of hover so links are reachable - Compact operation cell to max 2 rows (code + kind inline) Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
- Fix PHPCS array arrow alignment in cost calculator (phpcbf auto-fix) - Fix PHPStan: cast filemtime() to string for wp_enqueue_style version - Fix ESLint a11y: add role, tabIndex, onKeyDown to provider cell - Fix ESLint prettier: auto-formatted multiline expressions - Fix ESLint import: suppress no-extraneous-dependencies for webpack build-tool requires (dependency-extraction-webpack-plugin, copy-webpack-plugin) Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
jeffpaul
left a comment
There was a problem hiding this comment.
- Please left align the
Log retention daysportion with the rest of the text on the settings page - Let's remove the
Open the...text and link and then update the second description sentence toView detailed logs at Tools > AI Request Logsand link on theAI Request Logstext.
- Not seeing the logo on the upper left of the AI Request Logs page
|
Could someone please provide me a working built version of this? I tried on Playground but it seems that ""https://github.com/WordPress/ai/releases/download/ci-artifacts/pr-149-12cfe8353575986d2fb62b26c0b4533c4c3f7546.zip" is not available. I'm interesting on seeing how it behaves against a similar feature that I was implemented in a another plugin. We're using the CoreAI methods so I believe our requests should be logged there as well and this UI is awesome! |
|
@mateuswetah you can pull a built ZIP via the Build Plugin ZIP action: https://github.com/WordPress/ai/actions/runs/23660730731 |


What?
Adds a comprehensive AI Request Logging experiment that provides observability for all AI operations:
Why?
AI operations are opaque—users have no visibility into what requests are being made, how much they cost, or whether they're succeeding. This experiment provides the observability surface needed for debugging, cost tracking, and usage analysis.
How?
Architecture uses the decorator pattern to wrap the SDK's HTTP transporter:
Logging_Integrationwraps the transporter onwp_loaded/admin_initusing the publicsetHttpTransporter()APILogging_Http_Transporterdecorates requests, capturing timing and delegating toLog_Data_ExtractorLog_Data_Extractorparses request/response payloads with 4 filter hooks for extensibility:ai_request_log_providers- customize provider detectionai_request_log_context- filter context dataai_request_log_tokens- custom token extractionai_request_log_kind- request type detectionAI_Request_Log_Managercoordinates schema, repository, and cost calculationAI_Request_Cost_Calculatorestimates costs based on provider pricing tablesSupports 14 providers out of the box: OpenAI, Anthropic, Google, Fal, Cloudflare, Groq, Grok, HuggingFace, DeepSeek, Ollama, OpenRouter, Azure, Cohere, and Mistral.
Testing Instructions