Adding scripts to track the performance of the agent#158
Conversation
…ken sizes, ttft, and tps for an entire run of an agent.
76ec95d to
83fc05d
Compare
|
@claude review this PR. |
|
Claude finished @eddierichter-amd's task —— View job Reviewing PR
Code Review: Performance Analysis ScriptOverall, this is a well-structured utility script for analyzing llama.cpp server logs. The code is clean, uses good Python practices (dataclasses, enums, type hints), and includes comprehensive documentation. 🟢 Strengths
🟡 Suggestions
🟢 Minor Nits
SummaryThis is solid, production-ready code. The suggestions above are minor improvements. The script will be valuable for profiling GAIA agent performance with Lemonade. Recommendation: Approve with minor fixes for the copyright year and type hint. |
|
@claude I fixed the typing issues. I will address the header issue in another PR. Anything else for this one? |
|
Claude finished @eddierichter-amd's task —— View job Re-reviewing PR Changes
Review: Updated CodeI see the type hint fix on line 166 - you've updated from 🔴 Issue: Missing Import (
|
|
@claude good catch! Fixed that. Anything else? |
|
Claude finished @eddierichter-amd's task —— View job Final Review Check
Review CompleteThe One tiny remaining nit:
Same applies to lines 150, 272, and 275. Otherwise, this PR is ready to merge! |
|
@kovtcharov-amd passed Claude review. let me know what you think. |
…have everythinging in the GAIA cli now
util/token-and-performance-script/perf_analysis.py, a plotting tool that ingests one or more llama.cpp server logs and generates prompt/input/output token count charts along with TTFT and TPS plots.