The 14th International Workshop on Genetic Improvement @ICSE 2025

Ottawa and Gatineau, Canada (Michel Gagnon, CC BY-SA 3.0, via Wikimedia Commons)

Navigation: Attending, Important Dates, Keynote, CFP, Workshops Chairs, Program Committee

Attending

The 14th instalment of the GI workshop is will take place in Ottawa, collocated with the 47th International Conference on Software Engineering, ICSE 2025, which will be held in Ottawa’s Shaw Centre.

The Shaw Centre is part of a large building complex that also includes the Rideau Centre (a shopping mall), the Westin Hotel, and the underground Rideau light rail station. It overlooks the Rideau Canal, a UNESCO World Heritage site, and is a short walk from the Byward market, Parliament, the National Arts Centre (performing arts), the National Gallery of Canada, and the University of Ottawa. It is also a short walk from most of the other hotels that have been booked for the ICSE conference.

Click here for information about how to get to the Shaw Centre, and click here for information about how to get to Ottawa, as well as visa and travel authorization information.

Click here for information about registration.

The workshop is expected to be held in-person. In case of a virtual or hybrid event, virtual presentations may be possible.

Important Dates

Submission Deadline: 11 November 2024 (Mon)
Extended Submission Deadline: 14 November (Thurs)
Notification: 8 December 2024 (Sun)
Camera-ready: 5 February 2025 (Wed)
Workshop: 27 April 2025 (Sun)

Keep up to date with the latest event news via our Twitter: https://twitter.com/gi_of_software.

Keynote

We are happy to announce that Dr. Shin Hwei Tan, head of the Automated Program RepaIr and MaintEnance Lab (A-PRIME), Concordia University Montreal, Canada, will give the keynote speech at GI@ICSE 2025.

Put on Your Tester Hat: Improving programs for Automated Program Generation
Given either a specification written in natural language or an input program, automated program generation techniques produce a program according to the given specification or by modifying the input program. Automated program generation is a powerful technique that can be used for finding bugs in software systems that take programs as input or fixing bugs in the input programs. However, most existing techniques focus on automated program generation for automated program repair or program synthesis. This talk introduces a different perspective of automated program generation where we will discuss our latest results on automated program generation for testing static program analyzers by designing different types of program transformations. We will also explore and rethink about the automated program generation problem from the tester perspective. The new perspective could have huge potential for the design of new genetic improvement techniques to improve programs for automated program generation.

Shin Hwei Tan is an Associate Professor (Gina Cody Research Chair) in Concordia University. Before moving to Concordia University, she was an Assistant Professor in Southern University of Science and Technology in Shenzhen, China. She obtained her PhD degree from National University of Singapore and her B.S (Hons) and MSc degree from University of Illinois at Urbana-Champaign. Her main research interests are in automated program repair, software testing and open-source development. She is an Associate Editor for TOSEM and the Guest Editors-in-Chief for the New Frontier in Software Engineering track in TOSEM. She has also served as PCs for top-tier software engineering conferences, where she won 3 best reviewers award (FSE 2020, ASE 2020, ICSE 2022 NIER-track). She is also the general chair of FSE26 which will be held in Concordia University.

Accepted Papers

LLM-Assisted Crossover in Genetic Improvement of Software
by Dimitrios Stamatios Bouras, Justyna Petke, and Sergey Mechtaev
PDF Abstract

This study explores the use of Large Language Models to improve the crossover process in genetic programming. Traditional crossover techniques typically combine parent variants by selecting modifications uniformly or even randomly, without consideration of contextual relevance, often resulting in inefficient searches and suboptimal solutions due to incompatible or redundant modifications. In contrast, our LLM-assisted crossover leverages contextual understanding to select and combine edits from parent solutions that are more likely to work well together, with the goal of producing higher quality variants and accelerating optimization. We implemented this approach within MAGPIE, a unified genetic improvement framework. We evaluated against five traditional crossover methods across seven benchmarks, measuring performance on four key metrics: average ranking, best variant execution time, efficiency in reaching performance milestones, and viable variant count. Results show that LLM-assisted crossover achieved an average ranking of 2.27 (on a scale where 1 is best and 6 is worst), making it the top-performing method across benchmarks based on the quality of the optimal variants produced. The LLM-based approach also improved the fitness (execution time) by an average of 8.5% over the best variant produced by the traditional methods. In terms of efficiency, the LLM-assisted crossover required on average 25.6% fewer variants to reach 25%, 50%, 75%, and 100% of the final performance improvement, compared to the traditional methods. Additionally, the LLM-assisted crossover produced 4.8% more viable variants across scenarios, including both source code modification and parameter tuning cases.These findings suggest that LLMs can significantly enhance genetic programming by guiding the crossover process toward more effective and viable solutions, providing motivation for further research in LLM-assisted evolutionary algorithms.

Large Language Model based Code Completion is an Effective Genetic Improvement Mutation
by Jingyuan Wang, Carol Hanna, and Justyna Petke
PDF Abstract

In this work, we introduce a novel large language model (LLM)-based masking mutation operator for Genetic Improvement (GI), which leverages code completion capabilities of large language models to replace masked code segments with contextually relevant modifications. Our approach was tested on five open-source Java projects, where we compared its effective- ness against both traditional GI mutations and an existing LLM- based replacement mutation operator using random sampling and local search algorithms. Results show that the masking mutation operator creates a search space with more compiling and test-passing patches, reducing model response time by up to 60.7% compared to the replacement mutation. Additionally, it outperforms the replacement mutation in achieving the highest runtime improvement on four out of five projects and discovers more runtime-improving patches across all projects. However, combining the masking mutation with traditional GI mutations yielded inconsistent results, suggesting further investigation is needed. This study highlights the promise of LLM-based code completion to boost the efficiency and effectiveness of GI for automated software optimisation.

Enhancing Software Runtime with Reinforcement Learning-Driven Mutation Operator Selection in Genetic Improvement
by Damien Bose, Carol Hanna, and Justyna Petke
PDF Abstract

Genetic Improvement employs heuristic search algo- rithms to explore the search space of program variants by mod- ifying code using mutation operators. This research focuses on operators that delete, insert and replace source code statements. Traditionally, in GI, an operator is chosen uniformly at random at each search iteration. This work leverages Reinforcement Learning to intelligently guide the selection of these operators specifically to improve program runtime. We propose to integrate RL into the operator selection pro- cess. Four Multi-Armed bandit RL algorithms (Epsilon Greedy, UCB, Probability Matching, and Policy Gradient) were inte- grated within a GI framework, and their efficacy and efficiency were benchmarked against the traditional GI operator selection approach. These RL-guided operator selection strategies have demonstrated empirical superiority over the traditional GI meth- ods of randomly selecting a search operator, with UCB emerging as the top-performing RL algorithm. On average, the UCB- guided Hill Climbing search algorithm produced variants that compiled and passed all tests 44% of the time, while only 22% of the variants produced by the traditional uniform random selection strategies compiled and passed all tests. Index Terms—Reinforcement learning, genetic improvement.

Empirical Comparison of Runtime Improvement Approaches: Genetic Improvement, Parameter Tuning, and Their Combination
by Thanatad Songpetchmongkol, Aymeric Blot, and Justyna Petke
PDF Abstract

Software can be optimised in various ways, e.g., by changing the code directly, modifying compiler or software’s paramters. To automate these tasks, algorithm configuration and genetic improvement have been proposed where one modifies parameters and the other source code. Several tools have been introduced to facilitate such changes automatically. However, these tools only work at a single code level, either optimising a parameter or modifying source code. In 2022, Blot and Petke introduced MAGPIE, which is a framework that is capable of simultaneously searching for improvement at different granu- larity levels. From our literature review, we found that the best search strategies in genetic improvement and algorithm configuration, that generalise to both domains, are based on local search and genetic algorithms, respectively. We thus compared the two approaches for runtime improvement of the MiniSAT solver. We also explored the two search strategies on the joint search space of parameter and source code edits. We found that genetic improvement with first improvement local search led to the best results by improving MiniSAT’s runtime by 18.05%.

The gem5 C++ glibc Heap Fitness Landscape
by William B. Langdon and Bobby R. Bruce
PDF Abstract

Using the language independent genetic improvement tool MAGPIE (Machine Automated General Performance Improvement via Evolution of software) and logarithmic sampling, we measure the parameter fitness landscape when optimising the GNU glibc heap management of a million line C++ application, gem5. The malloc_info landscape is far smoother than is commonly assumed and savings of 11 percent with no loss of runtime speed are readily obtained by both Magpie and CMA-ES.

A Three-Stage Genetic Algorithm for Compiler Flag and Library Version Selection to Minimize Execution Time
by Chi Ho Chan and Spyro Nita
PDF Abstract

Existing research in compiler autotuning mainly focuses on selecting optimization flags without configurable values. However, the potential of selecting optimization flags with configurable values, alongside using directory and link flags for library version selection to improve performance, remains largely unexplored. We propose a three-stage Genetic Algorithm (GA) that incrementally selects optimization flags without configurable values, then optimization flags with configurable values, and finally library versions, to minimize software execution time. We also discuss the implementation challenges of the proposed algorithm and outline potential future work.

Call For Submissions - Submit Here

We invite submissions that discuss recent developments in all areas of research on, and applications of, Genetic Improvement. The International Workshop on Genetic Improvement is the premier workshop in the field and provides an opportunity for researchers interested in automated program repair and software optimisation to disseminate their work, exchange ideas, and discover new research directions.

Topics of interest include both the theory and practice of Genetic Improvement. Applications of GI include, but are not limited to, using GI to:

Improve runtime efficiency
Decrease memory consumption
Decrease energy consumption
Transplant new functionality
Specialise software
Generate multiple versions of software
Improve low level or binary code
Use of Large Language Models with GI
Use of AI with GI
Repair bugs
GI techniques in industrial settings

The workshop emphasises interaction and discussion between participants.
Authors are encouraged to submit early and in-progress work.

We invite two types of submissions:

Research papers (eight page limit, including references)
Position papers (two page limit, including references)

Papers should be submitted electronically here and must conform to the IEEE conference proceedings template as per the ICSE submission process (template)

Accepted papers must be presented at GI 2025 and will appear in the ICSE workshops volume. The official publication date of the workshop proceedings is the date the proceedings are made available by IEEE. This date may be up to two weeks prior to the first day of ICSE 2025. The official publication date affects the deadline for any patent filings related to published work.

The best paper and best presentation will be awarded during the workshop.

Following the workshop selected papers will be invited to submit to a special issue of the Automated Software Engineering journal on “Genetic Improvement: Pushing the Boundaries of Automated Software Development”

Workshop Chairs

Aymeric Blot is a Senior Lecturer at the University of Rennes and a member of the IRISA research centre in the joint Inria/IRISA DiverSE team. After receiving a doctorate from the University of Lille, focused on automated algorithm design for multi-objective combinatorial optimisation, they moved to University College London to work on software specialization using genetic improvement. Currently working on developing and maintaining the Magpie automated software improvement framework.

Vesna Nowack received her PhD in Computer Architecture at UPC, Spain (2016). Recently, she worked on APR in academia (Queen Mary University of London, Lancaster University) and industry (Bloomberg). She is currently a Research Associate at Imperial College London with a focus on human-in-the-loop ML systems.

Oliver Krauss received his doctorate in 2022 in Pattern Mining and Genetic Improvement in Compilers and Interpeters. His research focuses on mining patterns in software, as well as data, to improve runtime performance and energy consumption. He maintains several open source frameworks, such as Amaru.

Penn Faulkner Rainford received their PhD in 2019 from the University of York. They work on the interface of Biochemistry and Computing (University of York). Their GI research centres on applications of principles of natural evolution to genetic improvement. Currently focused on phylogenetic analysis for systems designed for real-time evolutionary adaptation (Lancaster University).

See on Researchr.