Repository navigation
Description
Reply with commands, one per line:
/approve all · /approve 1,3-5 · /reject 2 · /edit 3 category=world_terminal tags=benchmark (edit implies approve; tags=- clears tags). Valid category keys: see taxonomy.json.
Auto-skipped 116 out-of-scope; 10 failed (retried next run).
1. Model-Based Agentic Software Engineering
James C. Davis, Kelechi Kalu, Huiyun Peng, et al. · arXiv 2026/08 · paper
proposed: studies · tags: position
Proposes MAGE, a framework and theory for governing coding-agent-driven software engineering by externalizing representations and granting settled obligations proportionate authority, derived from a longitudinal case and six industrial accounts. It argues for durable engineering structure rather than ad hoc prompting or review.
A position/theory paper whose object is how agentic software engineering practice should be governed, not a task-performing agent or benchmark.
2. SPECMINE: A Large-Scale Corpus of Spec-Driven Development Artifacts
Shyam Agarwal, Bogdan Vasilescu · arXiv 2026/08 · paper
proposed: studies · tags: empirical
SPECMINE is a large-scale corpus of spec-driven-development artifacts (spec files, commit history, PRs) mined from GitHub, enabling study of how AI coding agents implement natural-language specifications into code. It links specs to code changes via typed references across hundreds of thousands of repositories.
An empirical corpus study of the emerging spec-driven agentic coding practice rather than a task-performing agent or method.
3. A Few Pages of Markdown: Committed AI Configuration and Lower Quality Cost after Coding-Agent Adoption
Yegor Denisov-Blanch, Shyam Agarwal, Pavel Azaletskiy, et al. · arXiv 2026/08 · paper
proposed: studies · tags: empirical
Introduces RAMP, a maturity model for version-controlled AI configuration artifacts, and studies 441 repositories to show coding-agent adoption is cumulative and correlates with lower quality-cost increases when configuration is committed. It is a behavioral/empirical study of agent adoption practices, not a task-performing agent.
Object of study is developer/team practice around coding-agent adoption, matching the studies/empirical definition.
4. Can your AI agent be cheaper? Investigating the effects of task specifications on token spend in agentic coding tasks
Jakub Smékal · arXiv 2026/08 · paper
proposed: studies · tags: empirical
An empirical study examining how the specificity of task descriptions affects token spend in agentic coding workflows using the Kimi K3 model across 2,700 runs. It also proposes a predictor to estimate token cost distributions from a single probe run.
Object of study is the behavior/cost of coding agents themselves, making it an empirical study rather than a task-performing method.
5. TraceML: An Empirical Analysis of Human-Agent Planning in Machine Learning Development
Jiarui Yan, Weiwei Sun, Sijie Li, et al. · arXiv 2026/08 · paper
proposed: studies · tags: empirical
TraceML is a paired human-agent trajectory dataset and empirical analysis of machine-learning development on Kaggle, comparing how human experts versus agent scaffolds (Codex, MLEvolve) plan, pivot, and iterate over long-horizon ML pipelines. It finds agents collapse into narrow behavioral loops compared to human practice.
An empirical/behavioral study whose object is coding-agent practice and its gap from humans, not a proposed task-performing agent, matching studies.
6. Metis: Typed Runtime Mediation for Tool-Using Software Agents
Jun Yu · arXiv 2026/08 · paper
proposed: software_infrastructure · tags: none
Metis is a typed runtime that mediates tool-using software agents' calls before they reach external effects like repositories, processes, and networks, enforcing permission and interference controls. Evaluations show reduced latency and successful blocking of unauthorized effects across model and provider conditions.
A runtime layer enabling safe agent-driven code/repository operations is enabling infrastructure work in service of code production.
7. Evaluating and Preventing Security Smells in AI-Generated Ansible Code
Pandu Ranga Reddy Konala, Vimal Kumar, David Bainbridge, et al. · arXiv 2026/08 · paper
proposed: software_security · tags: benchmark
Evaluates 16 AI models generating Ansible Infrastructure-as-Code roles for security smells against CIS benchmarks, then introduces a CO-STAR prompting approach that prevents smells during synthesis, achieving up to 100% compliance.
Secure code generation and its improvement is explicitly listed under software_security's scope.
8. QLCoder: A Query Synthesizer For Static Analysis of Security Vulnerabilities
Claire Wang, Ziyang Li, Saikat Dutta, et al. · arXiv 2026/08 · paper
proposed: software_security · tags: none
QLCoder is an agentic framework that synthesizes CodeQL static-analysis queries from CVE metadata using an LLM with execution feedback, LSP syntax guidance, and RAG retrieval, detecting 53.4% of evaluated CVEs.
The task is vulnerability detection/query synthesis for security auditing, matching software_security's scope of finding and localizing vulnerabilities.
9. MACGen: Toward Functionally Correct and Secure Code Generation via Multi-Agent Collaboration
Miseon Yu, Jaehoon Choi, Younghan Lee, et al. · arXiv 2026/08 · paper
proposed: software_security · tags: none
MACGen is a multi-agent framework (planner, security advisor, coder, reviewer) that jointly optimizes functional correctness and security in code generation by passing structured artifacts instead of shared dialogue. It improves security-and-functionality metrics on CWEval and BaxBench benchmarks.
The served task is secure code generation, matching the software_security leaf's inclusion of secure code generation via multi-agent methods.
10. AI Slop and Hallucinations in Vulnerability Assessment: A Survey on Reasoning Failures and Trustworthy Mitigation
Junchen Ding, Jialiang Dong, Yichen Zhu, et al. · arXiv 2026/08 · paper
proposed: software_security · tags: survey
Surveys reasoning failures and hallucinations ('AI slop') in LLM-based vulnerability assessment, proposing a taxonomy, a Deductive Coverage Score proxy, and evaluation instruments (CVE-Bench, Slop-Score) toward trustworthy triage.
A survey focused on one activity, security vulnerability assessment, so it stays at that leaf per the single-activity survey boundary.
11. From Verdict to Diagnosis: Attributable Security Review of Pull Requests
Zhuo Chen, Boyang Wang, Xiyue Zhang, et al. · arXiv 2026/08 · paper
proposed: software_security · tags: benchmark
Introduces MalPR-Bench, a benchmark exposing the Verdict-Diagnosis gap in automated PR security review, and PRGuard, an agent that constructs and validates candidate vulnerabilities against repository evidence, outperforming CodeRabbit on attributable diagnosis.
The served task is vulnerability identification and validation in pull requests, a security-assurance activity on code.
12. DBcover: A White-box SQL Test Generation Framework for Coverage Improvement
Yankai Rong, Shuang Liu, Jinhao Dong, et al. · arXiv 2026/08 · paper
proposed: software_testing · tags: none
DBcover is an LLM-driven white-box SQL test generation framework that uses dynamic analysis and a knowledge graph of SQL-to-path context to guide generation of SQL tests targeting uncovered code regions in RDBMSs. It achieves high coverage on PostgreSQL, MySQL, and a closed-source enterprise database.
Proactive coverage-guided test generation for correctness assurance fits software_testing (agentic fuzzing/test generation).
13. XREPOTEST: Benchmarking Multilingual Repository-Level Unit Test Generation for Large Language Models
Dung Le Quang, Dong Cao Van, Nam Le Hai, et al. · arXiv 2026/08 · paper
proposed: software_testing · tags: benchmark
Introduces XREPOTEST, a multilingual repository-level benchmark for unit test generation across five underexplored languages (Rust, Go, Julia, PHP, Ruby), evaluating LLMs under realistic repo constraints with a new Invocation Rate metric.
Proactive unit test generation for correctness assurance matches the software_testing leaf, with benchmark release as the primary contribution.
14. DataKernelBench: Can LLMs Optimize Database Queries on GPUs?
Gokul Karthik Kumar, Yotam Perlitz, Corey Lammie, et al. · arXiv 2026/08 · paper
proposed: systems · tags: benchmark
DataKernelBench translates SQL queries into validated TorchPlan programs and benchmarks LLMs that generate and repair optimized CUDA/Triton kernels for database-style GPU workloads. The best full-query CUDA configurations achieve over 2x speedup versus torch.compile through execution-guided repair.
The deliverable is systems-level GPU kernel code produced and repaired via execution-guided LLM agents, matching the systems leaf's kernel-generation example.
15. Heimdall: Formally Verified Automated Migration of Legacy eBPF Programs to Rust
Vishnu Asutosh Dasu, Monika Santra, Md Rafi Ur Rashid, et al. · arXiv 2026/08 · paper
proposed: systems · tags: none
Heimdall is an LLM-driven pipeline that automatically migrates legacy libbpf C eBPF kernel programs to Aya Rust, iteratively repairing compiler and kernel-verifier failures. It uses static analysis and Z3-based symbolic execution to formally prove behavioral equivalence, fixing nine real-world kernel bugs in the process.
An agent that writes and iteratively repairs kernel-level eBPF code; the code domain (kernel extensions) is systems-specific rather than domain-agnostic software.
16. Beyond Scaling: Self-Evolving LLM Agents for Hardware Kernel Optimization via an Experience-Driven Workflow and Experience Graph Memory
Siyuan Chen, Runlin Hou, Shenxiu Wu, et al. · arXiv 2026/08 · paper
proposed: systems · tags: none
KOPE is an experience-driven LLM agent framework for hardware kernel optimization that stores optimization trajectories with correctness/performance feedback in an Experience Graph Memory and retrieves relevant experience under a token budget. It substantially improves pass rate, speedup, and token efficiency over prior kernel-optimization baselines.
Agentic kernel optimization is systems-level code production/optimization, matching the KernelBench-style routing to systems.
17. FrontierChallenge: Evaluating Scientific Workflow Completion
Liangcai Su, Zhaopeng Feng, Zhuo Chen, et al. · arXiv 2026/08 · paper
proposed: world_research · tags: benchmark
FrontierChallenge is a cross-domain benchmark of 300 (97 released) end-to-end scientific workflows across chemistry, materials science, and life science, evaluating agents that execute code to produce complete research deliverables. Frontier models with agent scaffolds achieve only ~20% full completion, revealing a gap between partial progress and true task completion.
Agents execute code to run scientific workflows and produce research findings, matching world_research's discovery-via-interaction definition.
18. Algorithmic algorithm development with LLMs: A Case Study on LLM-Usage for Contraction Order Optimization in Tensor Networks
Fabian Hoppe, Melven Röhrig-Zöllner, Philipp Knechtges · arXiv 2026/08 · paper
proposed: world_research · tags: empirical
This case study uses the OpenEvolve evolutionary coding agent, guided by an LLM and a verifier, to discover improved contraction-order algorithms for tensor networks. It reflects on LLM/design choices and highlights both the promise and continuing need for human validation in agent-driven algorithm development.
An LLM-driven evolutionary coding agent pursues algorithmic discovery for a scientific computing problem, matching world_research's AlphaEvolve-style scientific-discovery agents.
19. FLARE: Verifying MILP Reformulations with LLM-Based Theorem Proving
Henry Robbins, Connor Lawless, Madeleine Udell, et al. · arXiv 2026/08 · paper
proposed: world_research · tags: none
FLARE uses an LLM-based agent with the Lean proof assistant to formally verify that proposed MILP reformulations preserve the original optimization problem, producing machine-checkable certificates, evaluated on a new FormulationBench dataset.
An LLM/Lean theorem-proving agent whose purpose is mathematical verification of optimization formulations, matching the pure-math theorem-proving boundary for world_research.
20. BixBench3: Benchmarking AI agents on research-study-scale computational biology tasks
Zane Koch, Asmamaw T. Wassie, Javier Valdes-Aleman, et al. · arXiv 2026/08 · paper
proposed: world_research · tags: benchmark
BixBench3 benchmarks AI agents on end-to-end computational biology research tasks, requiring agents to write and execute analysis code on raw data to reproduce published scientific results. Frontier models are scored on 20 tasks producing 138 artifacts, revealing large gaps in handling large data and multi-step analyses.
Agents use code as an instrument to conduct scientific/data analysis, matching world_research's data-science and discovery scope.
21. MathAdv: What Theorem Provers Know, Reason, Formalize, and Generalize
Jiaxin Yuan, Connor Martinez Lockhart, Xiaoyu Liu, et al. · arXiv 2026/08 · paper
proposed: world_research · tags: benchmark
MathAdv is a diagnostic benchmark spanning 13 mathematical domains that evaluates Lean 4 theorem provers alongside auxiliary tasks probing knowledge, informal reasoning, and robustness to reformulation. It reveals formalization bottlenecks and robustness gaps in current provers.
Pure mathematical theorem proving via Lean code maps to the world_research theorem-proving example, with benchmark as the released artifact.
22. Praxist: From Experimental Artifacts to Solution Lineages
Jin Li, Ahmed Murtadha, Zhiyu Wang, et al. · arXiv 2026/08 · paper
proposed: world_research · tags: none
Praxist is a lineage-centered generational system that converts R&D agents' experimental artifacts and evaluator outcomes into a typed evidence graph, letting later attempts inherit validated findings. Evaluated on MLE-bench and open-ended engineering case studies (SLAM, tokamak control, rocket landing), it outperforms a Claude Code baseline at a fraction of the cost.
An autonomous agent that iteratively runs and improves ML/engineering experiments; code is the instrument for research outcomes, matching world_research (MLE-bench style).
23. Planetary Prediction Engine: Autonomous Geospatial Prediction via Intelligent Data Selection and Foundation Model Embeddings
Evelyn Ma, Rama Kumar Pasumarthi, Kishwar Shafin, et al. · arXiv 2026/08 · paper
proposed: world_research · tags: none
Planetary Prediction Engine (PPE) is an autonomous AI system that executes end-to-end geospatial predictive modeling from natural-language queries, autonomously retrieving and fusing multimodal Earth-observation data and searching model architectures. It outperforms manually tuned expert baselines across health, disaster-risk, food-security, and epidemiological forecasting tasks.
An autonomous data-science/ML-engineering agent whose deliverable is predictive insight, not code artifacts, matching world_research.
24. Agentic Autoresearch for Cell-Edge Power Control: Radically Redefining the Researcher's Role
Ahmad Khan, Akram Bin Sediq, Sara Azadegi Naeini, et al. · arXiv 2026/08 · paper
proposed: world_research · tags: empirical
An autonomous AI coding agent is granted full control over architecture, loss, and training design to solve a wireless power-control ML problem, iteratively editing and testing training scripts under safeguards. It discovers an inference-efficient solution matching a converged optimization reference.
An ML-engineering agent that edits code and runs experiments to produce a trained model serves world_research (MLE-bench-style interaction), not a code artifact.
25. EvoCurr: Self-evolving Curriculum with Behavior Code Generation for Complex Decision-making
Yang Cheng, Weiyu Ma, Zilai Wang, et al. · arXiv 2026/08 · paper
proposed: world_game · tags: none
EvoCurr co-evolves adaptive task curricula and executable code-as-policy solutions via a Designer-Solver loop, achieving strong win rates on StarCraft II super-late-game micromanagement plus other MARL and code benchmarks.
Code is generated as the policy/action language to act in game worlds, with StarCraft II as the headline stress test, so it routes to world_game.
machine payload (do not edit)
[{"paper": {"id": "2608.25174", "title": "Model-Based Agentic Software Engineering", "authors": ["James C. Davis", "Kelechi Kalu", "Huiyun Peng", "Parth V. Patil"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-25", "links": {"paper": "https://arxiv.org/abs/2608.25174", "github": "", "website": ""}}, "category": "studies", "tags": ["position"], "summary": "Proposes MAGE, a framework and theory for governing coding-agent-driven software engineering by externalizing representations and granting settled obligations proportionate authority, derived from a longitudinal case and six industrial accounts. It argues for durable engineering structure rather than ad hoc prompting or review.", "reason": "A position/theory paper whose object is how agentic software engineering practice should be governed, not a task-performing agent or benchmark.", "source": "crawl"}, {"paper": {"id": "2608.25202", "title": "SPECMINE: A Large-Scale Corpus of Spec-Driven Development Artifacts", "authors": ["Shyam Agarwal", "Bogdan Vasilescu"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-25", "links": {"paper": "https://arxiv.org/abs/2608.25202", "github": "", "website": ""}}, "category": "studies", "tags": ["empirical"], "summary": "SPECMINE is a large-scale corpus of spec-driven-development artifacts (spec files, commit history, PRs) mined from GitHub, enabling study of how AI coding agents implement natural-language specifications into code. It links specs to code changes via typed references across hundreds of thousands of repositories.", "reason": "An empirical corpus study of the emerging spec-driven agentic coding practice rather than a task-performing agent or method.", "source": "crawl"}, {"paper": {"id": "2608.25241", "title": "A Few Pages of Markdown: Committed AI Configuration and Lower Quality Cost after Coding-Agent Adoption", "authors": ["Yegor Denisov-Blanch", "Shyam Agarwal", "Pavel Azaletskiy", "Hao He", "Rylan Schaeffer", "Brando Miranda", "Bogdan Vasilescu", "Sanmi Koyejo"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-26", "links": {"paper": "https://arxiv.org/abs/2608.25241", "github": "", "website": ""}}, "category": "studies", "tags": ["empirical"], "summary": "Introduces RAMP, a maturity model for version-controlled AI configuration artifacts, and studies 441 repositories to show coding-agent adoption is cumulative and correlates with lower quality-cost increases when configuration is committed. It is a behavioral/empirical study of agent adoption practices, not a task-performing agent.", "reason": "Object of study is developer/team practice around coding-agent adoption, matching the studies/empirical definition.", "source": "crawl"}, {"paper": {"id": "2608.25399", "title": "Can your AI agent be cheaper? Investigating the effects of task specifications on token spend in agentic coding tasks", "authors": ["Jakub Smékal"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-26", "links": {"paper": "https://arxiv.org/abs/2608.25399", "github": "", "website": ""}}, "category": "studies", "tags": ["empirical"], "summary": "An empirical study examining how the specificity of task descriptions affects token spend in agentic coding workflows using the Kimi K3 model across 2,700 runs. It also proposes a predictor to estimate token cost distributions from a single probe run.", "reason": "Object of study is the behavior/cost of coding agents themselves, making it an empirical study rather than a task-performing method.", "source": "crawl"}, {"paper": {"id": "2608.26086", "title": "TraceML: An Empirical Analysis of Human-Agent Planning in Machine Learning Development", "authors": ["Jiarui Yan", "Weiwei Sun", "Sijie Li", "Wenhan Li", "Yiming Yang"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-26", "links": {"paper": "https://arxiv.org/abs/2608.26086", "github": "", "website": ""}}, "category": "studies", "tags": ["empirical"], "summary": "TraceML is a paired human-agent trajectory dataset and empirical analysis of machine-learning development on Kaggle, comparing how human experts versus agent scaffolds (Codex, MLEvolve) plan, pivot, and iterate over long-horizon ML pipelines. It finds agents collapse into narrow behavioral loops compared to human practice.", "reason": "An empirical/behavioral study whose object is coding-agent practice and its gap from humans, not a proposed task-performing agent, matching studies.", "source": "crawl"}, {"paper": {"id": "2608.25322", "title": "Metis: Typed Runtime Mediation for Tool-Using Software Agents", "authors": ["Jun Yu"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-26", "links": {"paper": "https://arxiv.org/abs/2608.25322", "github": "", "website": ""}}, "category": "software_infrastructure", "tags": [], "summary": "Metis is a typed runtime that mediates tool-using software agents' calls before they reach external effects like repositories, processes, and networks, enforcing permission and interference controls. Evaluations show reduced latency and successful blocking of unauthorized effects across model and provider conditions.", "reason": "A runtime layer enabling safe agent-driven code/repository operations is enabling infrastructure work in service of code production.", "source": "crawl"}, {"paper": {"id": "2608.24962", "title": "Evaluating and Preventing Security Smells in AI-Generated Ansible Code", "authors": ["Pandu Ranga Reddy Konala", "Vimal Kumar", "David Bainbridge", "Junaid Haseeb"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-25", "links": {"paper": "https://arxiv.org/abs/2608.24962", "github": "", "website": ""}}, "category": "software_security", "tags": ["benchmark"], "summary": "Evaluates 16 AI models generating Ansible Infrastructure-as-Code roles for security smells against CIS benchmarks, then introduces a CO-STAR prompting approach that prevents smells during synthesis, achieving up to 100% compliance.", "reason": "Secure code generation and its improvement is explicitly listed under software_security's scope.", "source": "crawl"}, {"paper": {"id": "2511.08462", "title": "QLCoder: A Query Synthesizer For Static Analysis of Security Vulnerabilities", "authors": ["Claire Wang", "Ziyang Li", "Saikat Dutta", "Mayur Naik"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-26", "links": {"paper": "https://arxiv.org/abs/2511.08462", "github": "", "website": ""}}, "category": "software_security", "tags": [], "summary": "QLCoder is an agentic framework that synthesizes CodeQL static-analysis queries from CVE metadata using an LLM with execution feedback, LSP syntax guidance, and RAG retrieval, detecting 53.4% of evaluated CVEs.", "reason": "The task is vulnerability detection/query synthesis for security auditing, matching software_security's scope of finding and localizing vulnerabilities.", "source": "crawl"}, {"paper": {"id": "2608.25457", "title": "MACGen: Toward Functionally Correct and Secure Code Generation via Multi-Agent Collaboration", "authors": ["Miseon Yu", "Jaehoon Choi", "Younghan Lee", "Yunheung Paek"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-26", "links": {"paper": "https://arxiv.org/abs/2608.25457", "github": "", "website": ""}}, "category": "software_security", "tags": [], "summary": "MACGen is a multi-agent framework (planner, security advisor, coder, reviewer) that jointly optimizes functional correctness and security in code generation by passing structured artifacts instead of shared dialogue. It improves security-and-functionality metrics on CWEval and BaxBench benchmarks.", "reason": "The served task is secure code generation, matching the software_security leaf's inclusion of secure code generation via multi-agent methods.", "source": "crawl"}, {"paper": {"id": "2608.25667", "title": "AI Slop and Hallucinations in Vulnerability Assessment: A Survey on Reasoning Failures and Trustworthy Mitigation", "authors": ["Junchen Ding", "Jialiang Dong", "Yichen Zhu", "Yi Liu", "Gelei Deng", "Willy Susilo", "Siqi Ma", "Yuekang Li"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-26", "links": {"paper": "https://arxiv.org/abs/2608.25667", "github": "", "website": ""}}, "category": "software_security", "tags": ["survey"], "summary": "Surveys reasoning failures and hallucinations ('AI slop') in LLM-based vulnerability assessment, proposing a taxonomy, a Deductive Coverage Score proxy, and evaluation instruments (CVE-Bench, Slop-Score) toward trustworthy triage.", "reason": "A survey focused on one activity, security vulnerability assessment, so it stays at that leaf per the single-activity survey boundary.", "source": "crawl"}, {"paper": {"id": "2608.25730", "title": "From Verdict to Diagnosis: Attributable Security Review of Pull Requests", "authors": ["Zhuo Chen", "Boyang Wang", "Xiyue Zhang", "Xiaoyun Xu", "Ahmad-Reza Sadeghi", "Stjepan Picek", "Lichao Wu"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-26", "links": {"paper": "https://arxiv.org/abs/2608.25730", "github": "", "website": ""}}, "category": "software_security", "tags": ["benchmark"], "summary": "Introduces MalPR-Bench, a benchmark exposing the Verdict-Diagnosis gap in automated PR security review, and PRGuard, an agent that constructs and validates candidate vulnerabilities against repository evidence, outperforming CodeRabbit on attributable diagnosis.", "reason": "The served task is vulnerability identification and validation in pull requests, a security-assurance activity on code.", "source": "crawl"}, {"paper": {"id": "2608.25573", "title": "DBcover: A White-box SQL Test Generation Framework for Coverage Improvement", "authors": ["Yankai Rong", "Shuang Liu", "Jinhao Dong", "Qiang Yin", "Wei Lu", "Jianhua Wang", "Xiaoyong Du"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-26", "links": {"paper": "https://arxiv.org/abs/2608.25573", "github": "", "website": ""}}, "category": "software_testing", "tags": [], "summary": "DBcover is an LLM-driven white-box SQL test generation framework that uses dynamic analysis and a knowledge graph of SQL-to-path context to guide generation of SQL tests targeting uncovered code regions in RDBMSs. It achieves high coverage on PostgreSQL, MySQL, and a closed-source enterprise database.", "reason": "Proactive coverage-guided test generation for correctness assurance fits software_testing (agentic fuzzing/test generation).", "source": "crawl"}, {"paper": {"id": "2608.25939", "title": "XREPOTEST: Benchmarking Multilingual Repository-Level Unit Test Generation for Large Language Models", "authors": ["Dung Le Quang", "Dong Cao Van", "Nam Le Hai", "Linh Ngo Van", "Anh M. T. Bui", "Phuong T. Nguyen"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-26", "links": {"paper": "https://arxiv.org/abs/2608.25939", "github": "", "website": ""}}, "category": "software_testing", "tags": ["benchmark"], "summary": "Introduces XREPOTEST, a multilingual repository-level benchmark for unit test generation across five underexplored languages (Rust, Go, Julia, PHP, Ruby), evaluating LLMs under realistic repo constraints with a new Invocation Rate metric.", "reason": "Proactive unit test generation for correctness assurance matches the software_testing leaf, with benchmark release as the primary contribution.", "source": "crawl"}, {"paper": {"id": "2608.25061", "title": "DataKernelBench: Can LLMs Optimize Database Queries on GPUs?", "authors": ["Gokul Karthik Kumar", "Yotam Perlitz", "Corey Lammie", "Andrea Giovannini", "Katja Hose"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-25", "links": {"paper": "https://arxiv.org/abs/2608.25061", "github": "", "website": ""}}, "category": "systems", "tags": ["benchmark"], "summary": "DataKernelBench translates SQL queries into validated TorchPlan programs and benchmarks LLMs that generate and repair optimized CUDA/Triton kernels for database-style GPU workloads. The best full-query CUDA configurations achieve over 2x speedup versus torch.compile through execution-guided repair.", "reason": "The deliverable is systems-level GPU kernel code produced and repaired via execution-guided LLM agents, matching the systems leaf's kernel-generation example.", "source": "crawl"}, {"paper": {"id": "2605.25411", "title": "Heimdall: Formally Verified Automated Migration of Legacy eBPF Programs to Rust", "authors": ["Vishnu Asutosh Dasu", "Monika Santra", "Md Rafi Ur Rashid", "Ashish Kumar", "Saeid Tizpaz-Niari", "Gang Tan"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-26", "links": {"paper": "https://arxiv.org/abs/2605.25411", "github": "", "website": ""}}, "category": "systems", "tags": [], "summary": "Heimdall is an LLM-driven pipeline that automatically migrates legacy libbpf C eBPF kernel programs to Aya Rust, iteratively repairing compiler and kernel-verifier failures. It uses static analysis and Z3-based symbolic execution to formally prove behavioral equivalence, fixing nine real-world kernel bugs in the process.", "reason": "An agent that writes and iteratively repairs kernel-level eBPF code; the code domain (kernel extensions) is systems-specific rather than domain-agnostic software.", "source": "crawl"}, {"paper": {"id": "2608.25570", "title": "Beyond Scaling: Self-Evolving LLM Agents for Hardware Kernel Optimization via an Experience-Driven Workflow and Experience Graph Memory", "authors": ["Siyuan Chen", "Runlin Hou", "Shenxiu Wu", "Yansong Sun", "Junming Cao", "Yiyu Zhang", "Shudi Shao", "Junhao Qiu", "Zhichao Lu", "Qingfu Zhang"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-26", "links": {"paper": "https://arxiv.org/abs/2608.25570", "github": "", "website": ""}}, "category": "systems", "tags": [], "summary": "KOPE is an experience-driven LLM agent framework for hardware kernel optimization that stores optimization trajectories with correctness/performance feedback in an Experience Graph Memory and retrieves relevant experience under a token budget. It substantially improves pass rate, speedup, and token efficiency over prior kernel-optimization baselines.", "reason": "Agentic kernel optimization is systems-level code production/optimization, matching the KernelBench-style routing to systems.", "source": "crawl"}, {"paper": {"id": "2608.24979", "title": "FrontierChallenge: Evaluating Scientific Workflow Completion", "authors": ["Liangcai Su", "Zhaopeng Feng", "Zhuo Chen", "Zhen Zhang", "Xiang Lin", "Ruilin Li", "Handuo Zhang", "Ning Wang", "Kailong Wen", "Yueqi Guo", "Feng Xing", "Yiling Guo", "Chenxiong Qian", "Simon Shaolei Du", "Lidong Bing", "Xinyu Wang"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-25", "links": {"paper": "https://arxiv.org/abs/2608.24979", "github": "", "website": ""}}, "category": "world_research", "tags": ["benchmark"], "summary": "FrontierChallenge is a cross-domain benchmark of 300 (97 released) end-to-end scientific workflows across chemistry, materials science, and life science, evaluating agents that execute code to produce complete research deliverables. Frontier models with agent scaffolds achieve only ~20% full completion, revealing a gap between partial progress and true task completion.", "reason": "Agents execute code to run scientific workflows and produce research findings, matching world_research's discovery-via-interaction definition.", "source": "crawl"}, {"paper": {"id": "2606.01975", "title": "Algorithmic algorithm development with LLMs: A Case Study on LLM-Usage for Contraction Order Optimization in Tensor Networks", "authors": ["Fabian Hoppe", "Melven Röhrig-Zöllner", "Philipp Knechtges"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-26", "links": {"paper": "https://arxiv.org/abs/2606.01975", "github": "", "website": ""}}, "category": "world_research", "tags": ["empirical"], "summary": "This case study uses the OpenEvolve evolutionary coding agent, guided by an LLM and a verifier, to discover improved contraction-order algorithms for tensor networks. It reflects on LLM/design choices and highlights both the promise and continuing need for human validation in agent-driven algorithm development.", "reason": "An LLM-driven evolutionary coding agent pursues algorithmic discovery for a scientific computing problem, matching world_research's AlphaEvolve-style scientific-discovery agents.", "source": "crawl"}, {"paper": {"id": "2608.25220", "title": "FLARE: Verifying MILP Reformulations with LLM-Based Theorem Proving", "authors": ["Henry Robbins", "Connor Lawless", "Madeleine Udell", "Ellen Vitercik"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-25", "links": {"paper": "https://arxiv.org/abs/2608.25220", "github": "", "website": ""}}, "category": "world_research", "tags": [], "summary": "FLARE uses an LLM-based agent with the Lean proof assistant to formally verify that proposed MILP reformulations preserve the original optimization problem, producing machine-checkable certificates, evaluated on a new FormulationBench dataset.", "reason": "An LLM/Lean theorem-proving agent whose purpose is mathematical verification of optimization formulations, matching the pure-math theorem-proving boundary for world_research.", "source": "crawl"}, {"paper": {"id": "2608.25286", "title": "BixBench3: Benchmarking AI agents on research-study-scale computational biology tasks", "authors": ["Zane Koch", "Asmamaw T. Wassie", "Javier Valdes-Aleman", "Jason Lee", "Michaela M. Hinks", "Samuel G. Rodriques", "Andrew D. White", "Jon M. Laurent"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-26", "links": {"paper": "https://arxiv.org/abs/2608.25286", "github": "", "website": ""}}, "category": "world_research", "tags": ["benchmark"], "summary": "BixBench3 benchmarks AI agents on end-to-end computational biology research tasks, requiring agents to write and execute analysis code on raw data to reproduce published scientific results. Frontier models are scored on 20 tasks producing 138 artifacts, revealing large gaps in handling large data and multi-step analyses.", "reason": "Agents use code as an instrument to conduct scientific/data analysis, matching world_research's data-science and discovery scope.", "source": "crawl"}, {"paper": {"id": "2608.25449", "title": "MathAdv: What Theorem Provers Know, Reason, Formalize, and Generalize", "authors": ["Jiaxin Yuan", "Connor Martinez Lockhart", "Xiaoyu Liu", "Jiaqi Wang", "Chenghao Deng", "Xiayimei Han", "Vlasios Mastrantonis", "Dmitrii Gudin", "Shaopeng Zhu", "Abdirisak Abdullahi Mohamed", "Bilal Hamdi Aytekin", "Jiewen Lang", "Zezheng Song", "Furong Huang"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-26", "links": {"paper": "https://arxiv.org/abs/2608.25449", "github": "", "website": ""}}, "category": "world_research", "tags": ["benchmark"], "summary": "MathAdv is a diagnostic benchmark spanning 13 mathematical domains that evaluates Lean 4 theorem provers alongside auxiliary tasks probing knowledge, informal reasoning, and robustness to reformulation. It reveals formalization bottlenecks and robustness gaps in current provers.", "reason": "Pure mathematical theorem proving via Lean code maps to the world_research theorem-proving example, with benchmark as the released artifact.", "source": "crawl"}, {"paper": {"id": "2608.25955", "title": "Praxist: From Experimental Artifacts to Solution Lineages", "authors": ["Jin Li", "Ahmed Murtadha", "Zhiyu Wang", "Qiwen Chen", "William Chen", "Yifei Wu", "Guan Wang", "Andy L. Siy", "Jiayi Yang", "Mengsha Huang", "Wenhao Li", "Yixuan Liu", "Shuailin Pan", "Mingli Yuan", "Sen Song", "Yuhao Sun"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-26", "links": {"paper": "https://arxiv.org/abs/2608.25955", "github": "", "website": ""}}, "category": "world_research", "tags": [], "summary": "Praxist is a lineage-centered generational system that converts R&D agents' experimental artifacts and evaluator outcomes into a typed evidence graph, letting later attempts inherit validated findings. Evaluated on MLE-bench and open-ended engineering case studies (SLAM, tokamak control, rocket landing), it outperforms a Claude Code baseline at a fraction of the cost.", "reason": "An autonomous agent that iteratively runs and improves ML/engineering experiments; code is the instrument for research outcomes, matching world_research (MLE-bench style).", "source": "crawl"}, {"paper": {"id": "2608.26088", "title": "Planetary Prediction Engine: Autonomous Geospatial Prediction via Intelligent Data Selection and Foundation Model Embeddings", "authors": ["Evelyn Ma", "Rama Kumar Pasumarthi", "Kishwar Shafin", "Mandar Sharma", "Mimi Sun", "Hamed Sadeghi", "Dav M. Ebengo", "Mbulayi Onesime", "Rouslan Solomakhin", "John Wamburu", "William Ogallo", "Aisha Walcott-Bryant", "Sanxing Chen", "Arbaaz Muslim", "Yael Mayer", "Ronald Ho", "Roy Lee", "Ruth Alcantara", "Abdoulaye Diack", "Monica Bharel", "Lambert Rosique", "Jeremy Amez-Droz", "Christopher Haire", "James Manyika", "Yossi Matias", "Niv Efron", "Gautam Prasad", "Shravya Shetty"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-26", "links": {"paper": "https://arxiv.org/abs/2608.26088", "github": "", "website": ""}}, "category": "world_research", "tags": [], "summary": "Planetary Prediction Engine (PPE) is an autonomous AI system that executes end-to-end geospatial predictive modeling from natural-language queries, autonomously retrieving and fusing multimodal Earth-observation data and searching model architectures. It outperforms manually tuned expert baselines across health, disaster-risk, food-security, and epidemiological forecasting tasks.", "reason": "An autonomous data-science/ML-engineering agent whose deliverable is predictive insight, not code artifacts, matching world_research.", "source": "crawl"}, {"paper": {"id": "2608.26093", "title": "Agentic Autoresearch for Cell-Edge Power Control: Radically Redefining the Researcher's Role", "authors": ["Ahmad Khan", "Akram Bin Sediq", "Sara Azadegi Naeini", "Raviraj S. Adve"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-26", "links": {"paper": "https://arxiv.org/abs/2608.26093", "github": "", "website": ""}}, "category": "world_research", "tags": ["empirical"], "summary": "An autonomous AI coding agent is granted full control over architecture, loss, and training design to solve a wireless power-control ML problem, iteratively editing and testing training scripts under safeguards. It discovers an inference-efficient solution matching a converged optimization reference.", "reason": "An ML-engineering agent that edits code and runs experiments to produce a trained model serves world_research (MLE-bench-style interaction), not a code artifact.", "source": "crawl"}, {"paper": {"id": "2508.09586", "title": "EvoCurr: Self-evolving Curriculum with Behavior Code Generation for Complex Decision-making", "authors": ["Yang Cheng", "Weiyu Ma", "Zilai Wang", "Wenhui Zhu", "Tangjie Lv", "Yujing Hu", "Mohamed Elhoseiny", "Yue Deng", "Jian Zhao"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-26", "links": {"paper": "https://arxiv.org/abs/2508.09586", "github": "", "website": ""}}, "category": "world_game", "tags": [], "summary": "EvoCurr co-evolves adaptive task curricula and executable code-as-policy solutions via a Designer-Solver loop, achieving strong win rates on StarCraft II super-late-game micromanagement plus other MARL and code benchmarks.", "reason": "Code is generated as the policy/action language to act in game worlds, with StarCraft II as the headline stress test, so it routes to world_game.", "source": "crawl"}]
Activity