BuildBench: Benchmarking LLM Agents on Compiling Real-World Open-Source Software
Zehua Zhang, Ati Priya Bajaj, Divij Handa, Siyu Liu, Arvind S Raj, Hongkai Chen, Hulin Wang, Yibo Liu, Zion Leonahenahe Basque, Souradip Nath, Vishal Juneja, Nikhil Chapre, Tiffany Bao, Yan Shoshitaishvili, Adam Doupé, Chitta Baral, Ruoyu Wang
April, 2026Abstract
Automatically compiling open-source software is a prerequisite for many downstream security and software-engineering tasks. BuildBench is a benchmark designed to reflect practical compilation difficulty, featuring diverse, low-resource, and heterogeneous open-source projects, together with OSS-Build-Agent, an LLM agent that combines build-instruction retrieval, dependency inference, and context-aware code modification.
Publication
Transactions on Machine Learning Research (TMLR)

Ph.D. Candidate · LLM Agents for Vulnerability Discovery & Repair
Ph.D. candidate at ASU SEFCOM building LLM agents that find and fix real-world software vulnerabilities.