Skip to content

Paper review 2026-08-05: 25 candidates #354

Open

Description

@github-actions

Reply with commands, one per line:
/approve all · /approve 1,3-5 · /reject 2 · /edit 3 category=world_terminal tags=benchmark (edit implies approve; tags=- clears tags). Valid category keys: see taxonomy.json.

Auto-skipped 149 out-of-scope; 0 failed (retried next run).

1. A Study of Library Usage in Agent-Authored Pull Requests

Lukas Twist, Jie M. Zhang · arXiv 2026/01 · paper
proposed: studies · tags: empirical

An empirical study analyzing 26,760 agent-authored pull requests to characterize how coding agents import libraries, introduce dependencies, and follow versioning practices compared to non-agentic LLM usage.
An empirical/behavioral study whose object is coding agents' practices, not a proposed agent or task, matching the studies leaf.

2. What Breaks When LLMs Code? Characterizing Operational Safety Failures of Agentic Code Assistants

Alif Al Hasan, Sumon Biswas · arXiv 2026/08 · paper
proposed: studies · tags: empirical

An incident-driven empirical study mining 68,816 papers and 16,586 GitHub issues to build a taxonomy of 33 operational safety failure types in agentic coding assistants, finding severe risks concentrated in bug fixing and environment setup. It argues guardrails must address environmental constraints and failure transparency, not just adversarial prompts.
The paper's object of study is coding agents' operational failures, an empirical/behavioral study rather than a proposed task-performing method.

3. Self-Evolving Coding Agents

Hao Zhou, Haichuan Hu, Ye Shang, et al. · arXiv 2026/08 · paper
proposed: studies · tags: survey

A systematic survey of self-evolving coding agents that update their frameworks, memory, skills, tools, or models from prior coding interactions, proposing an object-centered taxonomy of what/when/why evolution occurs. It synthesizes challenges like feedback reliability, benchmark overfitting, and safety in this emerging area.
A field-wide survey whose object of study is coding agents themselves, not a proposed task-solving method.

4. When AI Joins the Team! A Model of How AI Adoption Relates To Social Patterns in Software Engineering Teams

Giusy Annunziata, Rudrajit Choudhuri, Anita Sarma, et al. · arXiv 2026/08 · paper
proposed: studies · tags: empirical

An empirical, TMS-grounded survey study of 152 software professionals examining how AI-assisted development tool adoption relates to community smells and social coordination patterns in software teams. It finds AI's relationship to team dynamics differs by whether work involves specialization or coordination.
An empirical/behavioral study of human-AI interaction in software engineering teams, whose object is AI adoption practice rather than a proposed agent performing a task.

5. From Social Coding to Agentic Coding: Productivity and Relational Reconfiguration in Open-Source Communities

Mengying Zhou, Yongjie Yin, Yang Chen · arXiv 2026/08 · paper
proposed: studies · tags: empirical

Using an LLM-based multi-agent simulation seeded with real GitHub data, this study compares open-source community dynamics with and without coding agents, finding productivity gains but reduced public human interaction and knowledge coverage. It reveals a tension between technical productivity and community knowledge sharing.
An empirical/behavioral study of how coding agents reshape developer practice and community relations, matching the studies leaf.

6. ConFL: Explainable Concurrent Fault Localization via Hierarchy-Guided LLM Reasoning

Shuai Shao, Dingbang Wang, Yiming Zeng, et al. · arXiv 2026/08 · paper
proposed: software_debugging · tags: none

ConFL is an explainable concurrent fault localization framework building a Concurrent Knowledge Base and using LLM-guided hierarchical reasoning to narrow the search space for cross-thread bugs, outperforming IR- and LLM-based baselines on real-world Java bugs.
An LLM-guided reasoning system performing fault localization, a core software_debugging lifecycle activity.

7. Fail-Fast, Restart-Smart: Early Failure Prediction and Restart for SWE Agentic Tasks

Chenyu Wang, Yunbo Lyu, Junda He, et al. · arXiv 2026/08 · paper
proposed: software_debugging · tags: none

FailFast-RestartSmart is a two-stage controller for SWE agentic issue-resolution tasks that predicts trajectory failure early using a lightweight monitor and restarts with a fresh rollout, saving execution tokens. It improves resolution rate on SWE-bench Verified while reducing wasted compute on failing trajectories.
Serves SWE-bench issue-resolution agents by optimizing their execution, so it belongs to software_debugging.

8. From Bug Reports to Browser-Executable Procedures: An LLM-Driven Agent for Web GUI Bug Reproduction

Cunming Zhang, Yu Pei, Michail Papadakis · arXiv 2026/08 · paper
proposed: software_debugging · tags: none

ReBug is a context-aware agent that reconstructs missing prerequisites from web GUI bug reports and drives a real browser to reproduce and validate the reported bug. Evaluated on 667 real-world bug reports across four open-source web apps, it outperforms baselines on reproduction success rate.
Task is bug reproduction from reports, an issue-resolution lifecycle activity serving software maintenance.

9. Route-Align-Verify for Functional Correctness in Code Generation

Erxue Zhou, Jingxiang Meng, Aofan Liu · arXiv 2026/08 · paper
proposed: software_code_generation · tags: none

RAV is a modular code-generation pipeline combining task-aware prompt routing, aligned LoRA adaptation, and execution-based verification over multiple candidates to improve functional correctness. Evaluated on MBPP, it improves accuracy by 6-10 points over the base model.
The task's purpose is producing functionally correct code, placing it in software_code_generation.

10. CodeAssay: A Multi-Metric Benchmark with Audited Ground Truth for LLM Code Generation

Shahbaz Siddeeq, Muhammad Waseem, Umar Subhan Malhi, et al. · arXiv 2026/08 · paper
proposed: software_code_generation · tags: benchmark

CodeAssay is a taxonomy-first benchmark of 185 Python tasks with audited ground truth, hidden/mutation-validated tests, and code-property measures for evaluating LLM code generation. Auditing revealed substantial correctness-label errors, showing that reliable evaluation needs validated ground truth and multiple measures beyond pass/fail.
A benchmark resource whose downstream task is evaluating LLM code generation quality, so it routes to software_code_generation.

11. GenOS: Compositional Certificates for Semantic Robustness in AI Code Generation

Corrado Priami · arXiv 2026/08 · paper
proposed: software_code_generation · tags: none

GenOS introduces a probabilistic operational semantics with compositional equivalence certificates for safely replacing components (prompts, contracts, generators) within AI coding-agent pipelines. It proves robustness bounds and demonstrates the theory on an executable insertion-sort audit across paraphrased prompts and programs.
The framework specifically targets certifying robustness of AI code-generation agentic workflows, not a generic technique with an incidental code case study.

12. BulkPR-Bench: Benchmarking Queue-Level Governance of Interacting Pull Requests

Zetong Xiong, Qiao Zhao, Jun Zhang, et al. · arXiv 2026/08 · paper
proposed: software_infrastructure · tags: benchmark

BulkPR-Bench is an executable benchmark where agents must jointly decide which of many interacting pull requests to merge and in what order to maximize safe delivery across 18 real repositories. Results show current models struggle with whole-queue governance despite decent per-relation performance.
Task is merge/version-control governance across interacting PRs, matching the software_infrastructure boundary on branching/merging operations.

13. Evaluating LLM Trade-offs for Enterprise Automation: Lessons from Workflow Generation in a Production Enterprise Platform

Xavier Wrenn, Radoslav Raykov, Aleksandar Angelov, et al. · arXiv 2026/08 · paper
proposed: software_infrastructure · tags: empirical

An empirical study comparing six LLMs and two generation pipelines for producing IT automation workflows in a production enterprise compliance platform. It reports structural-success, cost, and latency trade-offs, showing a piecewise generation pipeline lets smaller models reach production viability.
Task is generating enterprise automation/remediation workflows, an enabling CI/CD-style automation artifact, fitting software_infrastructure.

14. AgentModernize: Preserving Business Logic in Legacy Modernization with Multi-Agent LLMs and Behavioral Specification Graphs

Sheikh Nazib Ahmed, Marnim Galib · arXiv 2026/08 · paper
proposed: software_maintenance · tags: benchmark

AgentModernize is a multi-agent LLM framework that treats legacy code modernization (e.g., COBOL to Java) as behavioral preservation, extracting business logic into a Behavioral Specification Graph before code generation. Evaluated on a new 8-scenario benchmark, it shows feedback-driven pipelines outperform single-prompt baselines on preserving business logic.
Task is behavior-preserving code migration/translation, matching software_maintenance.

15. Lean Refactor: Multi-Objective Controllable Proof Optimization via Agentic Strategy Search

Jialin Lu, Soonho Kong, Rodrigo Stehling, et al. · arXiv 2026/08 · paper
proposed: software_maintenance · tags: none

Lean Refactor is a retrieval-augmented agentic framework that refactors Lean proofs for length, compilation cost, and version compatibility while preserving correctness. It achieves large token and compilation-time reductions and improves cross-version proof portability.
The task refactors existing proof code to reduce length/cost while preserving behavior, matching software_maintenance's refactoring/optimization scope rather than new theorem proving.

16. EffiHolmes: Differential Profiling-Guided Repository Level Time Inefficiency Fix Localization

Haowen Yang, Yun Peng, Zishuo Ding · arXiv 2026/08 · paper
proposed: software_maintenance · tags: benchmark

EffiHolmes is an LLM-based framework that uses differential profiling and domain-guided reasoning to localize repository-level time-inefficiency fix locations, introducing RepoEffi-Bench for evaluation. It outperforms retrieval-, agent-, and profiling-based baselines at pinpointing performance bug fix sites.
Targets performance optimization/inefficiency (behavior-preserving) rather than functional bugs, matching software_maintenance's performance-optimization scope.

17. Can Large Language Models Recover Semantic Optimization Opportunities That Compilers Miss?

Hailong Jiang, Feng Yu, Emran Hossain, et al. · arXiv 2026/08 · paper
proposed: software_maintenance · tags: benchmark

SeGaBench is an executable benchmark testing whether LLMs can recover semantic optimization opportunities missed by compilers and realize them as validated, contract-preserving performance transformations in C/C++. The strongest model achieves 94.8% correctness and speedups in most cases.
Behavior-preserving performance optimization of existing code matches software_maintenance.

18. AgenticSCR: An Autonomous Agentic Secure Code Review for Immature Vulnerabilities Detection

Wachiraphan Charoenwet, Kla Tantithamthavorn, Patanamon Thongtanunam, et al. · arXiv 2026/08 · paper
proposed: software_security · tags: none

AgenticSCR is an autonomous agentic secure code reviewer that uses security-focused semantic memory and tool interaction to detect vulnerabilities before pre-integration, outperforming static LLM and SAST baselines. It was validated in a shadow deployment where over half its comments were confirmed useful by security engineers.
An agent that reviews code for vulnerabilities is a security-assurance task on code, matching software_security.

19. Security-First Evaluation of Text-to-Terraform: Benchmarking LLMs and SLMs for Secure IaC Generation

Francis Luis Santos Vargas, Rodrigo Brandão Mansilha, Diego Kreutz · arXiv 2026/08 · paper
proposed: software_security · tags: benchmark empirical

Benchmarks seven LLMs/SLMs on generating AWS Terraform infrastructure-as-code across 17 scenarios, scanning outputs with Checkov and Trivy to assess security compliance versus syntactic validity. Finds syntactic correctness and security compliance are largely orthogonal, arguing scanning must complement generation.
Evaluates and benchmarks secure code generation, matching software_security's 'secure code generation and completion' scope.

20. PRWeaver: Evaluating LLM-Based Code Auditors against Long-Horizon Malicious Pull Requests

Yuekun Wang, Mingfei Cheng, Xiaofei Xie · arXiv 2026/08 · paper
proposed: software_security · tags: benchmark

PRWeaver is a benchmark of 208 execution-validated malicious pull-request attacks across four review renderings, used to evaluate how well LLM-based code auditors detect adversarial changes hidden across repository evolution. It finds detection drops sharply when malicious and benign changes are interleaved or fused.
Evaluates agents that audit repositories for malicious/backdoor code changes, matching software_security's repository-auditing scope.

21. CLEAR: Causal Context-Based Agentic Reasoning for Vulnerability Detection

Sungju Yun, Sijune Hwang, Yeonjoon Lee, et al. · arXiv 2026/08 · paper
proposed: software_security · tags: none

CLEAR is a multi-agent vulnerability detection framework that builds a Vulnerability Causal Knowledge Graph capturing causal chains among entrypoints, root causes, and fixes. Four specialized agents (Collector, Claim, Critic, Judge) collaboratively verify vulnerability hypotheses, improving detection accuracy on C/C++ and Java benchmarks.
A multi-agent system performing vulnerability detection on source code serves the security-assurance activity on code.

22. SkillSentry: Adaptive Honey Worlds for Dynamic Safety Testing of Agent Skills

Nizhang Li, Zonghao Ying, Xiangfan Wu, et al. · arXiv 2026/08 · paper
proposed: software_security · tags: none

SkillSentry is a dynamic safety-testing framework that constructs adaptive 'honey world' environments to elicit and detect conditionally malicious behavior in executable LLM agent skills, grounding findings in source code and execution traces. It substantially outperforms static/one-shot scanners at detecting evasive malicious skill behavior.
Dynamic analysis that localizes malicious behavior in skill code via execution traces is a security-assurance activity on code, matching software_security.

23. We Must Have Missed This Comment: Detecting and Repairing Stale Function References in Linux Kernel Comments

Kexin Sun, Yunbo Lyu, Xutong Ma, et al. · arXiv 2026/08 · paper
proposed: software_comprehension · tags: none

ReCite is a three-stage LLM-based approach that detects stale function references in Linux kernel comments, traces their evolution via Git history, and generates repair suggestions to keep documentation accurate. Applied to Linux v6.18-rc1, it found 869 stale references, with most sampled repairs judged useful and dozens of patches accepted upstream.
Task is documentation freshness/repair to aid code comprehension, matching software_comprehension's documentation-generation scope.

24. Can LLMs Test Terminal User Interfaces?

Chao Peng, Ruida Hu, Ajitha Rajan, et al. · arXiv 2026/08 · paper
proposed: software_testing · tags: benchmark

Surveys 197 real-world Terminal User Interface apps, finding most lack interactive test coverage, then builds a headless benchmark and LLM-driven exploration framework (tuibot) to test TUI apps for crashes and coverage across four frameworks. Frontier LLMs are compared against random exploration, showing LLM guidance is more input-efficient though automated TUI testing remains unsolved.
Proactive, undirected crash/coverage exploration of software is agentic fuzzing/testing, per software_testing's scope.

25. TheBotCompany: Self-Organizing Multi-agent Systems for Continuous Software Development

Wenhan Lyu, Yue Xiao, Yixuan Zhang, et al. · arXiv 2026/08 · paper
proposed: software_development · tags: none

TheBotCompany is an orchestration framework for continuous, milestone-driven multi-agent software development, featuring self-organizing agent teams and a verification phase to catch defects. It is evaluated on real-world software projects over multiple days of continuous development.
Agents evolve real software projects via ongoing milestone/feature work rather than one-shot generation, matching software_development.

machine payload (do not edit)
[{"paper": {"id": "2512.11589", "title": "A Study of Library Usage in Agent-Authored Pull Requests", "authors": ["Lukas Twist", "Jie M. Zhang"], "venue": "arXiv 2026/01", "category": "", "published": "2026-01-27", "links": {"paper": "https://arxiv.org/abs/2512.11589", "github": "", "website": ""}}, "category": "studies", "tags": ["empirical"], "summary": "An empirical study analyzing 26,760 agent-authored pull requests to characterize how coding agents import libraries, introduce dependencies, and follow versioning practices compared to non-agentic LLM usage.", "reason": "An empirical/behavioral study whose object is coding agents' practices, not a proposed agent or task, matching the studies leaf.", "source": "crawl"}, {"paper": {"id": "2605.30777", "title": "What Breaks When LLMs Code? Characterizing Operational Safety Failures of Agentic Code Assistants", "authors": ["Alif Al Hasan", "Sumon Biswas"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-04", "links": {"paper": "https://arxiv.org/abs/2605.30777", "github": "", "website": ""}}, "category": "studies", "tags": ["empirical"], "summary": "An incident-driven empirical study mining 68,816 papers and 16,586 GitHub issues to build a taxonomy of 33 operational safety failure types in agentic coding assistants, finding severe risks concentrated in bug fixing and environment setup. It argues guardrails must address environmental constraints and failure transparency, not just adversarial prompts.", "reason": "The paper's object of study is coding agents' operational failures, an empirical/behavioral study rather than a proposed task-performing method.", "source": "crawl"}, {"paper": {"id": "2608.03392", "title": "Self-Evolving Coding Agents", "authors": ["Hao Zhou", "Haichuan Hu", "Ye Shang", "Quanjun Zhang"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-04", "links": {"paper": "https://arxiv.org/abs/2608.03392", "github": "", "website": ""}}, "category": "studies", "tags": ["survey"], "summary": "A systematic survey of self-evolving coding agents that update their frameworks, memory, skills, tools, or models from prior coding interactions, proposing an object-centered taxonomy of what/when/why evolution occurs. It synthesizes challenges like feedback reliability, benchmark overfitting, and safety in this emerging area.", "reason": "A field-wide survey whose object of study is coding agents themselves, not a proposed task-solving method.", "source": "crawl"}, {"paper": {"id": "2608.03462", "title": "When AI Joins the Team! A Model of How AI Adoption Relates To Social Patterns in Software Engineering Teams", "authors": ["Giusy Annunziata", "Rudrajit Choudhuri", "Anita Sarma", "Gemma Catolino", "Filomena Ferrucci"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-04", "links": {"paper": "https://arxiv.org/abs/2608.03462", "github": "", "website": ""}}, "category": "studies", "tags": ["empirical"], "summary": "An empirical, TMS-grounded survey study of 152 software professionals examining how AI-assisted development tool adoption relates to community smells and social coordination patterns in software teams. It finds AI's relationship to team dynamics differs by whether work involves specialization or coordination.", "reason": "An empirical/behavioral study of human-AI interaction in software engineering teams, whose object is AI adoption practice rather than a proposed agent performing a task.", "source": "crawl"}, {"paper": {"id": "2608.03585", "title": "From Social Coding to Agentic Coding: Productivity and Relational Reconfiguration in Open-Source Communities", "authors": ["Mengying Zhou", "Yongjie Yin", "Yang Chen"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-04", "links": {"paper": "https://arxiv.org/abs/2608.03585", "github": "", "website": ""}}, "category": "studies", "tags": ["empirical"], "summary": "Using an LLM-based multi-agent simulation seeded with real GitHub data, this study compares open-source community dynamics with and without coding agents, finding productivity gains but reduced public human interaction and knowledge coverage. It reveals a tension between technical productivity and community knowledge sharing.", "reason": "An empirical/behavioral study of how coding agents reshape developer practice and community relations, matching the studies leaf.", "source": "crawl"}, {"paper": {"id": "2608.02974", "title": "ConFL: Explainable Concurrent Fault Localization via Hierarchy-Guided LLM Reasoning", "authors": ["Shuai Shao", "Dingbang Wang", "Yiming Zeng", "Tingting Yu"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-04", "links": {"paper": "https://arxiv.org/abs/2608.02974", "github": "", "website": ""}}, "category": "software_debugging", "tags": [], "summary": "ConFL is an explainable concurrent fault localization framework building a Concurrent Knowledge Base and using LLM-guided hierarchical reasoning to narrow the search space for cross-thread bugs, outperforming IR- and LLM-based baselines on real-world Java bugs.", "reason": "An LLM-guided reasoning system performing fault localization, a core software_debugging lifecycle activity.", "source": "crawl"}, {"paper": {"id": "2608.03222", "title": "Fail-Fast, Restart-Smart: Early Failure Prediction and Restart for SWE Agentic Tasks", "authors": ["Chenyu Wang", "Yunbo Lyu", "Junda He", "Zhou Yang", "Chenxing Zhong", "Yaniv Harel", "David Lo"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-04", "links": {"paper": "https://arxiv.org/abs/2608.03222", "github": "", "website": ""}}, "category": "software_debugging", "tags": [], "summary": "FailFast-RestartSmart is a two-stage controller for SWE agentic issue-resolution tasks that predicts trajectory failure early using a lightweight monitor and restarts with a fresh rollout, saving execution tokens. It improves resolution rate on SWE-bench Verified while reducing wasted compute on failing trajectories.", "reason": "Serves SWE-bench issue-resolution agents by optimizing their execution, so it belongs to software_debugging.", "source": "crawl"}, {"paper": {"id": "2608.03598", "title": "From Bug Reports to Browser-Executable Procedures: An LLM-Driven Agent for Web GUI Bug Reproduction", "authors": ["Cunming Zhang", "Yu Pei", "Michail Papadakis"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-04", "links": {"paper": "https://arxiv.org/abs/2608.03598", "github": "", "website": ""}}, "category": "software_debugging", "tags": [], "summary": "ReBug is a context-aware agent that reconstructs missing prerequisites from web GUI bug reports and drives a real browser to reproduce and validate the reported bug. Evaluated on 667 real-world bug reports across four open-source web apps, it outperforms baselines on reproduction success rate.", "reason": "Task is bug reproduction from reports, an issue-resolution lifecycle activity serving software maintenance.", "source": "crawl"}, {"paper": {"id": "2608.03341", "title": "Route-Align-Verify for Functional Correctness in Code Generation", "authors": ["Erxue Zhou", "Jingxiang Meng", "Aofan Liu"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-04", "links": {"paper": "https://arxiv.org/abs/2608.03341", "github": "", "website": ""}}, "category": "software_code_generation", "tags": [], "summary": "RAV is a modular code-generation pipeline combining task-aware prompt routing, aligned LoRA adaptation, and execution-based verification over multiple candidates to improve functional correctness. Evaluated on MBPP, it improves accuracy by 6-10 points over the base model.", "reason": "The task's purpose is producing functionally correct code, placing it in software_code_generation.", "source": "crawl"}, {"paper": {"id": "2608.03535", "title": "CodeAssay: A Multi-Metric Benchmark with Audited Ground Truth for LLM Code Generation", "authors": ["Shahbaz Siddeeq", "Muhammad Waseem", "Umar Subhan Malhi", "Pekka Abrahamsson"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-04", "links": {"paper": "https://arxiv.org/abs/2608.03535", "github": "", "website": ""}}, "category": "software_code_generation", "tags": ["benchmark"], "summary": "CodeAssay is a taxonomy-first benchmark of 185 Python tasks with audited ground truth, hidden/mutation-validated tests, and code-property measures for evaluating LLM code generation. Auditing revealed substantial correctness-label errors, showing that reliable evaluation needs validated ground truth and multiple measures beyond pass/fail.", "reason": "A benchmark resource whose downstream task is evaluating LLM code generation quality, so it routes to software_code_generation.", "source": "crawl"}, {"paper": {"id": "2608.03588", "title": "GenOS: Compositional Certificates for Semantic Robustness in AI Code Generation", "authors": ["Corrado Priami"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-04", "links": {"paper": "https://arxiv.org/abs/2608.03588", "github": "", "website": ""}}, "category": "software_code_generation", "tags": [], "summary": "GenOS introduces a probabilistic operational semantics with compositional equivalence certificates for safely replacing components (prompts, contracts, generators) within AI coding-agent pipelines. It proves robustness bounds and demonstrates the theory on an executable insertion-sort audit across paraphrased prompts and programs.", "reason": "The framework specifically targets certifying robustness of AI code-generation agentic workflows, not a generic technique with an incidental code case study.", "source": "crawl"}, {"paper": {"id": "2608.02685", "title": "BulkPR-Bench: Benchmarking Queue-Level Governance of Interacting Pull Requests", "authors": ["Zetong Xiong", "Qiao Zhao", "Jun Zhang", "Xueying Lyu", "Zhi Li", "Yixiang Tu", "Xiaowen Yang", "Yunjie Zhang", "Yufeng Wang", "Zhe Zhang", "Kaize Yu", "Hanwen Du", "Zhongkai Sun", "Zhuoxin Liu", "Zekun Lin", "Jianwen Yang", "Ruining Chen", "Ying Zhang", "Tingxuan Pan", "Ke Chen", "Shubin Han", "Chuanhao Sun", "Yehua Yang"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-03", "links": {"paper": "https://arxiv.org/abs/2608.02685", "github": "", "website": ""}}, "category": "software_infrastructure", "tags": ["benchmark"], "summary": "BulkPR-Bench is an executable benchmark where agents must jointly decide which of many interacting pull requests to merge and in what order to maximize safe delivery across 18 real repositories. Results show current models struggle with whole-queue governance despite decent per-relation performance.", "reason": "Task is merge/version-control governance across interacting PRs, matching the software_infrastructure boundary on branching/merging operations.", "source": "crawl"}, {"paper": {"id": "2608.03311", "title": "Evaluating LLM Trade-offs for Enterprise Automation: Lessons from Workflow Generation in a Production Enterprise Platform", "authors": ["Xavier Wrenn", "Radoslav Raykov", "Aleksandar Angelov", "Hirokuni Kitahara", "Yuji Watanabe", "Anca Sailer"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-04", "links": {"paper": "https://arxiv.org/abs/2608.03311", "github": "", "website": ""}}, "category": "software_infrastructure", "tags": ["empirical"], "summary": "An empirical study comparing six LLMs and two generation pipelines for producing IT automation workflows in a production enterprise compliance platform. It reports structural-success, cost, and latency trade-offs, showing a piecewise generation pipeline lets smaller models reach production viability.", "reason": "Task is generating enterprise automation/remediation workflows, an enabling CI/CD-style automation artifact, fitting software_infrastructure.", "source": "crawl"}, {"paper": {"id": "2605.17535", "title": "AgentModernize: Preserving Business Logic in Legacy Modernization with Multi-Agent LLMs and Behavioral Specification Graphs", "authors": ["Sheikh Nazib Ahmed", "Marnim Galib"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-04", "links": {"paper": "https://arxiv.org/abs/2605.17535", "github": "", "website": ""}}, "category": "software_maintenance", "tags": ["benchmark"], "summary": "AgentModernize is a multi-agent LLM framework that treats legacy code modernization (e.g., COBOL to Java) as behavioral preservation, extracting business logic into a Behavioral Specification Graph before code generation. Evaluated on a new 8-scenario benchmark, it shows feedback-driven pipelines outperform single-prompt baselines on preserving business logic.", "reason": "Task is behavior-preserving code migration/translation, matching software_maintenance.", "source": "crawl"}, {"paper": {"id": "2605.20244", "title": "Lean Refactor: Multi-Objective Controllable Proof Optimization via Agentic Strategy Search", "authors": ["Jialin Lu", "Soonho Kong", "Rodrigo Stehling", "Kaiyu Yang", "Zhangyang Wang", "Weiran Sun", "Wuyang Chen"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-04", "links": {"paper": "https://arxiv.org/abs/2605.20244", "github": "", "website": ""}}, "category": "software_maintenance", "tags": [], "summary": "Lean Refactor is a retrieval-augmented agentic framework that refactors Lean proofs for length, compilation cost, and version compatibility while preserving correctness. It achieves large token and compilation-time reductions and improves cross-version proof portability.", "reason": "The task refactors existing proof code to reduce length/cost while preserving behavior, matching software_maintenance's refactoring/optimization scope rather than new theorem proving.", "source": "crawl"}, {"paper": {"id": "2608.03558", "title": "EffiHolmes: Differential Profiling-Guided Repository Level Time Inefficiency Fix Localization", "authors": ["Haowen Yang", "Yun Peng", "Zishuo Ding"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-04", "links": {"paper": "https://arxiv.org/abs/2608.03558", "github": "", "website": ""}}, "category": "software_maintenance", "tags": ["benchmark"], "summary": "EffiHolmes is an LLM-based framework that uses differential profiling and domain-guided reasoning to localize repository-level time-inefficiency fix locations, introducing RepoEffi-Bench for evaluation. It outperforms retrieval-, agent-, and profiling-based baselines at pinpointing performance bug fix sites.", "reason": "Targets performance optimization/inefficiency (behavior-preserving) rather than functional bugs, matching software_maintenance's performance-optimization scope.", "source": "crawl"}, {"paper": {"id": "2608.03983", "title": "Can Large Language Models Recover Semantic Optimization Opportunities That Compilers Miss?", "authors": ["Hailong Jiang", "Feng Yu", "Emran Hossain", "Jianfeng Zhu", "Mengfei Ren", "Qiang Guan", "Chunwei Xia"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-04", "links": {"paper": "https://arxiv.org/abs/2608.03983", "github": "", "website": ""}}, "category": "software_maintenance", "tags": ["benchmark"], "summary": "SeGaBench is an executable benchmark testing whether LLMs can recover semantic optimization opportunities missed by compilers and realize them as validated, contract-preserving performance transformations in C/C++. The strongest model achieves 94.8% correctness and speedups in most cases.", "reason": "Behavior-preserving performance optimization of existing code matches software_maintenance.", "source": "crawl"}, {"paper": {"id": "2601.19138", "title": "AgenticSCR: An Autonomous Agentic Secure Code Review for Immature Vulnerabilities Detection", "authors": ["Wachiraphan Charoenwet", "Kla Tantithamthavorn", "Patanamon Thongtanunam", "Hong Yi Lin", "Minwoo Jeong", "Ming Wu"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-03", "links": {"paper": "https://arxiv.org/abs/2601.19138", "github": "", "website": ""}}, "category": "software_security", "tags": [], "summary": "AgenticSCR is an autonomous agentic secure code reviewer that uses security-focused semantic memory and tool interaction to detect vulnerabilities before pre-integration, outperforming static LLM and SAST baselines. It was validated in a shadow deployment where over half its comments were confirmed useful by security engineers.", "reason": "An agent that reviews code for vulnerabilities is a security-assurance task on code, matching software_security.", "source": "crawl"}, {"paper": {"id": "2608.02672", "title": "Security-First Evaluation of Text-to-Terraform: Benchmarking LLMs and SLMs for Secure IaC Generation", "authors": ["Francis Luis Santos Vargas", "Rodrigo Brandão Mansilha", "Diego Kreutz"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-02", "links": {"paper": "https://arxiv.org/abs/2608.02672", "github": "", "website": ""}}, "category": "software_security", "tags": ["benchmark", "empirical"], "summary": "Benchmarks seven LLMs/SLMs on generating AWS Terraform infrastructure-as-code across 17 scenarios, scanning outputs with Checkov and Trivy to assess security compliance versus syntactic validity. Finds syntactic correctness and security compliance are largely orthogonal, arguing scanning must complement generation.", "reason": "Evaluates and benchmarks secure code generation, matching software_security's 'secure code generation and completion' scope.", "source": "crawl"}, {"paper": {"id": "2608.02693", "title": "PRWeaver: Evaluating LLM-Based Code Auditors against Long-Horizon Malicious Pull Requests", "authors": ["Yuekun Wang", "Mingfei Cheng", "Xiaofei Xie"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-03", "links": {"paper": "https://arxiv.org/abs/2608.02693", "github": "", "website": ""}}, "category": "software_security", "tags": ["benchmark"], "summary": "PRWeaver is a benchmark of 208 execution-validated malicious pull-request attacks across four review renderings, used to evaluate how well LLM-based code auditors detect adversarial changes hidden across repository evolution. It finds detection drops sharply when malicious and benign changes are interleaved or fused.", "reason": "Evaluates agents that audit repositories for malicious/backdoor code changes, matching software_security's repository-auditing scope.", "source": "crawl"}, {"paper": {"id": "2608.03134", "title": "CLEAR: Causal Context-Based Agentic Reasoning for Vulnerability Detection", "authors": ["Sungju Yun", "Sijune Hwang", "Yeonjoon Lee", "Kyungtae Kang", "Sungbin Park"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-04", "links": {"paper": "https://arxiv.org/abs/2608.03134", "github": "", "website": ""}}, "category": "software_security", "tags": [], "summary": "CLEAR is a multi-agent vulnerability detection framework that builds a Vulnerability Causal Knowledge Graph capturing causal chains among entrypoints, root causes, and fixes. Four specialized agents (Collector, Claim, Critic, Judge) collaboratively verify vulnerability hypotheses, improving detection accuracy on C/C++ and Java benchmarks.", "reason": "A multi-agent system performing vulnerability detection on source code serves the security-assurance activity on code.", "source": "crawl"}, {"paper": {"id": "2608.03485", "title": "SkillSentry: Adaptive Honey Worlds for Dynamic Safety Testing of Agent Skills", "authors": ["Nizhang Li", "Zonghao Ying", "Xiangfan Wu", "Zonglei Jing", "Xixun Lin", "Hao Zhang", "Wenxin Zhang", "Jiaye Lin", "Quanchen Zou", "Xiangzheng Zhang"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-04", "links": {"paper": "https://arxiv.org/abs/2608.03485", "github": "", "website": ""}}, "category": "software_security", "tags": [], "summary": "SkillSentry is a dynamic safety-testing framework that constructs adaptive 'honey world' environments to elicit and detect conditionally malicious behavior in executable LLM agent skills, grounding findings in source code and execution traces. It substantially outperforms static/one-shot scanners at detecting evasive malicious skill behavior.", "reason": "Dynamic analysis that localizes malicious behavior in skill code via execution traces is a security-assurance activity on code, matching software_security.", "source": "crawl"}, {"paper": {"id": "2608.03734", "title": "We Must Have Missed This Comment: Detecting and Repairing Stale Function References in Linux Kernel Comments", "authors": ["Kexin Sun", "Yunbo Lyu", "Xutong Ma", "Hongyu Kuang", "Ratnadira Widyasari", "He Zhang", "Xiaoxing Ma", "Julia Lawall", "David Lo"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-04", "links": {"paper": "https://arxiv.org/abs/2608.03734", "github": "", "website": ""}}, "category": "software_comprehension", "tags": [], "summary": "ReCite is a three-stage LLM-based approach that detects stale function references in Linux kernel comments, traces their evolution via Git history, and generates repair suggestions to keep documentation accurate. Applied to Linux v6.18-rc1, it found 869 stale references, with most sampled repairs judged useful and dozens of patches accepted upstream.", "reason": "Task is documentation freshness/repair to aid code comprehension, matching software_comprehension's documentation-generation scope.", "source": "crawl"}, {"paper": {"id": "2608.03743", "title": "Can LLMs Test Terminal User Interfaces?", "authors": ["Chao Peng", "Ruida Hu", "Ajitha Rajan", "Tegawendé F Bissyandé", "Jacques Klein", "Cuiyun Gao"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-04", "links": {"paper": "https://arxiv.org/abs/2608.03743", "github": "", "website": ""}}, "category": "software_testing", "tags": ["benchmark"], "summary": "Surveys 197 real-world Terminal User Interface apps, finding most lack interactive test coverage, then builds a headless benchmark and LLM-driven exploration framework (tuibot) to test TUI apps for crashes and coverage across four frameworks. Frontier LLMs are compared against random exploration, showing LLM guidance is more input-efficient though automated TUI testing remains unsolved.", "reason": "Proactive, undirected crash/coverage exploration of software is agentic fuzzing/testing, per software_testing's scope.", "source": "crawl"}, {"paper": {"id": "2603.25928", "title": "TheBotCompany: Self-Organizing Multi-agent Systems for Continuous Software Development", "authors": ["Wenhan Lyu", "Yue Xiao", "Yixuan Zhang", "Yifan Sun"], "venue": "arXiv 2026/08", "category": "", "published": "2026-08-04", "links": {"paper": "https://arxiv.org/abs/2603.25928", "github": "", "website": ""}}, "category": "software_development", "tags": [], "summary": "TheBotCompany is an orchestration framework for continuous, milestone-driven multi-agent software development, featuring self-organizing agent teams and a verification phase to catch defects. It is evaluated on real-world software projects over multiple days of continuous development.", "reason": "Agents evolve real software projects via ongoing milestone/feature work rather than one-shot generation, matching software_development.", "source": "crawl"}]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    paper-reviewPapers awaiting the owner's review

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions