Presentation

· Contributors · Organizations · Search Program · My Schedule

Large-Scale Randomized Program Generation with Large Language Models

SessionResearch, ACM SRC, and Doctoral Showcase Posters

DescriptionLarge, diverse datasets of executable programs are required for training and running machine learning models to find insights in program performance. While many open-source code repositories exist freely on popular software development websites such as GitHub, the safety and executability of such programs cannot be guaranteed. To bridge this gap, this study proposes LLMRPG (Large Language Model-based Randomized Program Generator), a program generator that harnesses open-source large language models (LLMs) fine-tuned for code generation to generate error-free, executable, and human-like programs on demand. The performance of LLMRPG was evaluated across popular open-source LLMs using heuristics such as the semantic similarity between programs, and the proportion of compilable and executable programs generated by LLMRPG. Analysis on the programs generated by LLMRPG demonstrates that these programs have satisfactory compilability and executability, as well as high diversity.

Authors

Tyler Lam

University of California, Berkeley

Lingda Li

Brookhaven National Laboratory