The 13th International Workshop on Genetic Improvement @ICSE 2024

Lisbon, Portugal (Deensel, CC BY 2.0, via Wikimedia Commons)

Navigation: Registration, Important Dates, Keynote, Tutorial, Accepted Papers, CFP

Event

The 13th instalment of the GI workshop took place in Lisbon, collocated with the International Conference on Software Engineering, ICSE 2024.

GI 2023 was a hybrid workshop that ran in-person and virtually.

Authors of selected accepted papers have been invited to submit an extended version of their papers to an ASE special issue on Genetic Improvement.

best paper award: Deep Mutations have Little Impact — Bill Langdon and David Clark
best position paper award: Ecosystem Curation in Genetic Improvement for Emergent Software Systems — Zsolt Nemeth, Penn Faulkner Rainford and Barry Porter
best presentation award: Genetic Improvement for DNN Security — Hunter Baxter, Yu Huang, and Kevin Leach

Registration

Registration details are available on the ICSE website: https://conf.researchr.org/attending/icse-2024/registration

Early Bird deadline: 12th February 2024

At least one author of an accepted paper needs to register for the event.

Important Dates

Submission Deadline: ~~9 November~~ 7 December 2023 (Thu)
Notification: ~~21 December~~ 11 January 2023 (Thu)
Camera-ready: 25 January 2024 (Thu)
Workshop: 16 April 2024 (Tue)

Keep up to date with the latest event news via our Twitter: https://twitter.com/gi_of_software.

Keynote

We are happy to announce that Prof. Shin Yoo, head of Computational Intelligence for Software Engineering, KAIST, Korea, will give the keynote speech at GI@ICSE 2024.

Executing One’s Way out of the Chinese Room
One very attractive property of Large Language Models (LLMs) is their emergent in-context learning capability, which enables us to simply describe our requirements in natural languages and get the corresponding souce code generated in programming languages. While LLMs as a generative model are known to hallucinate, i.e., generate factually incorrect contents, the fact that code can be executed can be used to fight this phenomenon. We briefly look at existing techniques designed to improve the correctness of code generated by LLMs, and will try to imagine the future of Genetic Improvement that is supported, enhanced, and driven by LLMs.

Prof. Shin Yoo gained his PhD under Mark Harman in the CREST group at King’s College, London and was a lecture in UCL’s Centre for Research on Evolution, Search and Testing before returning to his native Korea to join in the Korea Advanced Institute of Science and Technology (KAIST) in 2015. Shin has been active in software engineering research, particularly efficient mutation testing, code slicing, and fault localisation, for more than fifteen years. He is an expert in mining fixes and GI and recently has shown large language models (LLMs) may be used in advanced software testing. Prof. Yoo is an associate editor of ACM TOSEM and editorial board member of Empirical Software Engineering and Genetic Programming and Evolvable Machines. He has served on many conferences, including co-chair of ICST 2018, GI 2020, SSBSE 2022 and this year is the ICSE 2024 area-chair for testing and analysis.

Tutorial

In addition to the main keynote, we have the pleasure to announce a tutorial led by Dr. Aymeric Blot, senior lecturer at the Université of Rennes, France, and maintainer of the Magpie project.

Automated Software Performance Improvement with Magpie
In this tutorial, we present Magpie, a powerful tool for both Genetic Improvement researchers and practitioners. Magpie stands at the forefront of software evolution, providing a streamlined approach to model, evolve, and automatically improve software systems. Addressing both functional and non-functional concerns, Magpie offers a user-friendly no-code interface that seamlessly integrates various search processes as well as enabling easy Python code injection for advanced users to further tailor and specialise the improvement process to meet their specific needs. We will provide a concise overview of Magpie’s internals before exploring diverse real-world scenarios.

Accepted Papers

Authors of accepted papers are invited to submit an extended version of their papers to ASE’s Special Issue on Genetic Improvement.

Automated Software Performance Improvement with Magpie
by Aymeric Blot
DOI PDF SLIDES VIDEO VIDEO URL URL Abstract

we present Magpie, a powerful tool for both Genetic Improvement researchers and practitioners. Magpie stands at the forefront of software evolution, providing a streamlined approach to model, evolve, and automatically improve software systems. Addressing both functional and non-functional concerns, Magpie offers a user-friendly no-code interface that seamlessly integrates various search processes, as well as enabling easy Python code injection for advanced users to further tailor and specialise the improvement process to meet their specific needs. We will provide a concise overview of Magpie's internals before exploring diverse real-world scenarios. Aymeric Blot is a Senior Lecturer at the Universite of Rennes, France.

Deep Mutations have Little Impact
by William B. Langdon and David Clark
DOI PDF PDF SLIDES VIDEO VIDEO VIDEO Abstract

Using the language independent genetic improvement tool MAGPIE (Machine Automated General Performance Improvement via Evolution of software), we measure the impact of GI on a non-deterministic deeply nested PARSEC VIPS parallel computing multi-threaded image processing benchmark written in C. More than 53% of mutants compile and generate identical output. Overall we find approximately 10% Failed Disruption Propagation (FDP). Excluding software detected internal errors and asserts, almost all changes deeper than 30 nested functions which are Executed and Infect data or change control are not Propagated. That is, almost all deep PIE code changes have no external impact. Suggesting (where it relies on testing) automatic software engineering on deeply nested code will be hard.

Ecosystem Curation in Genetic Improvement for Emergent Software Systems
by Zsolt Nemeth, Penn Faulkner Rainford, and Barry Porter
DOI PDF SLIDES VIDEO VIDEO Abstract

Emergent software systems are composed, and continuously recomposed at runtime, from a large pool of small potential building blocks with the aim of responding to changes in the deployment environment [REX]. The approach assumes that each building block has a set of available variations in the selection pool, such that the most appropriate collection of variants can be composed according to the current conditions, with some objective function in mind. Populating such a pool of implementation variation, however, is not a trivial task, and existing work has examined the use of Genetic Improvement (GI) to drive this process [rainford:2022:GECCO]

Executing One's Way out of the Chinese Room
by Shin Yoo
DOI PDF SLIDES VIDEO Abstract

One very attractive property of Large Language Models (LLMs) is their emergent in-context learning capability, which enables us to simply describe our requirements in natural languages and get the corresponding source code generated in programming languages. While LLMs as a generative model are known to hallucinate, i.e., generate factually incorrect contents, the fact that code can be executed can be used to fight this phenomenon. We briefly look at existing techniques designed to improve the correctness of code generated by LLMs, and will try to imagine the future of Genetic Improvement that is supported, enhanced, and driven by LLMs.

Genetic Improvement for DNN Security
by Hunter Baxter, Yu Huang, and Kevin Leach
DOI PDF SLIDES VIDEO VIDEO VIDEO Abstract

Genetic improvement (GI) in Deep Neural Networks (DNNs) has traditionally enhanced neural architecture and trained DNN parameters. Recently, GI has supported large language models by optimising DNN operator scheduling on accelerator clusters. However, with the rise of adversarial AI, particularly model extraction attacks, there is an unexplored potential for GI in fortifying Machine Learning as a Service (MLaaS) models. We suggest a novel application of GI, not to improve model performance, but to diversify operator parallelism for the purpose of a moving target defence against model extraction attacks. We discuss an application of GI to create a DNN model defense strategy that uses probabilistic isolation, offering unique benefits not present in current DNN defense systems.

Grammar evolution and symbolic regression for astrometric centering of Hubble Space Telescope images
by Ricardo Sarmiento, Marina de la Cruz, Alfonso Ortega, Terrence M. Girard, Dana I. Casetti-Dinescu, Alejandro Cervantes, and Roberto Baena-Galle
DOI PDF SLIDES VIDEO VIDEO URL Abstract

Symbolic regression, in general, and genetic models, in particular, are promising approaches to mathematical modeling in astrometry where it is not always clear which is the fittest analytic expression depending on the problem under consideration. Several attempts and increasing research efforts are being made in this direction mainly from the Genetic Programming (GP) viewpoint. Our proposal is, as far as we know, the first one to apply Grammatical Evolution (GE) in this domain. GE (and further GE extensions) aim to outperform GP limitations by incorporating formal language stools to guarantee the correctness (both syntactic and semantic) of the generated expressions. The current contribution is a first proof to check the viability of GE on astrometric real datasets. Its success in finding adequate parameters for predefined families of functions in star centering (Gaussian and Moffat PSFs) with simple and naive GE experiments supports our hypothesis on taking advantage of the expressive power of GE to tackle astrometry scenarios of interest and hence greatly improve current astrometric software thanks to specific genetic approaches.

Human Guidance Approaches for the Genetic Improvement of Software
by Benjamin J. Craine, Penn Faulkner Rainford, and Barry Porter
DOI PDF SLIDES VIDEO VIDEO Abstract

Existing research on Genetic Improvement (GI) of source code to improve performance [rainford:2022:GECCO] has examined the mixed application of code synthesis and traditional GI mutation/crossover to gain higher-performing individuals that are tailored to particular deployment contexts, for examples such as hash tables or scheduling algorithms. While demonstrating successful improvements, this research presents a host of challenges [Rainford:2021:GI], from search space size to fitness landscape shape, which raise questions on whether GI alone is able to present a complete solution. In this position paper we propose to augment GI processes with Human Guidance (HG) to offer a co-pilot paradigm which may overcome these challenges.

On Reducing Network Usage with Genetic Improvement
by James Callan, William B. Langdon, and Justyna Petke
DOI PDF SLIDES URL URL Abstract

Mobile applications can be very network-intensive. Mobile phone users are often on limited data plans, while network infrastructure has limited capacity. There is little work on optimising network usage of mobile applications. The most popular approach has been prefetching and caching assets. However, past work has shown that developers can improve the network usage of Android applications by making changes to Java source code. We built upon this insight and investigated the effectiveness of automated, heuristic application of software patches, a technique known as Genetic Improvement (GI), to improve network usage. Genetic improvement has already shown effective at reducing the execution time and memory usage of Android applications. We thus adapt our existing GIdroid framework with a new mutation operator and develop a new profiler to identify network-intensive methods to target. Unfortunately, our approach is unable to find improvements. We conjecture this is due to the fact source code changes affecting network might be rare in the large patch search space. We thus advocate use of more intelligent search strategies in future work.

Schedule

This schedule appears in Lisbon’s time zone (GMT+1, Greenwich Mean Time, summer); compare to your timezone here.
Presentations for full papers are 20 minutes long, followed by 10 minutes for questions.
Presentations for short papers consist of a 10 minute talk, followed by 5 minutes for questions.

The workshop will take place in Vianna da Motta.
See also the schedule on the ICSE website for easy bookmarking.

Tuesday, April 16, 09:00–10:30 (90 mins; chair: Bill Langdon)
- 09:00: Welcome and Introductions (15 minutes)
- 09:15: Keynote speech — Executing One’s Way out of the Chinese Room — Shin Yoo (5+55+15 minutes Q&A)
- 10:30: Break & Morning tea (30 minutes)
Tuesday, April 16, 11:00–12:30 (90 minutes; chair: Gabin An, Justyna Petke)
- 11:00: Deep Mutations have Little Impact — Bill Langdon and David Clark (20+10 minutes)
- 11:30: Grammar evolution and symbolic regression for astrometric centering of Hubble Space Telescope images — Ricardo Sarmiento, Marina de la Cruz, Alfonso Ortega, Roberto Baena-Galle, Terrence Girard, Dana Casetti-Dinescu and Alejandro Cervantes (20+10 minutes)
- 12:00: Genetic Improvement for DNN Security — Hunter Baxter, Yu Huang, and Kevin Leach (10+5 minutes)
- 12:15: Discussion (15 minutes)
- 12:30: Lunch & Social event (90 minutes)
Tuesday, April 16, 14:00–15:30 (90 minutes; chair: Sungmin Kang, Oliver Krauss)
- 14:00: Invited Tutorial — Automated Software Performance Improvement with Magpie — Aymeric Blot (5+55+15 minutes Q&A)
- 15:15: Ecosystem Curation in Genetic Improvement for Emergent Software Systems — Zsolt Nemeth, Penn Faulkner Rainford and Barry Porter (10+5 minutes)
- 15:30: Break & Afternoon tea (30 minutes)
Tuesday, April 16, 16:00–17:30 (90 minutes; chair: Aymeric Blot, Vesna Nowack)
- 16:00: Human Guidance Approaches for the Genetic Improvement of Software — Benjamin Craine, Penn Rainford and Barry Porter (10+5 minutes)
- 16:15: On Reducing Network Usage with Genetic Improvement — James Callan, Bill Langdon and Justyna Petke (20+10 minutes)
- 16:45: Discussion, Awards, and Closing (45 minutes)

Call For Submissions [pdf]

We invite submissions that discuss recent developments in all areas of research on, and applications of, Genetic Improvement. The International Workshop on Genetic Improvement is the premier workshop in the field and provides an opportunity for researchers interested in automated program repair and software optimisation to disseminate their work, exchange ideas, and discover new research directions. Topics of interest include both the theory and practice of Genetic Improvement. Applications of GI include, but are not limited to:

Improve runtime efficiency
Decrease memory consumption
Decrease energy consumption
Transplant new functionality
Specialise software
Generate multiple versions of software
Improve low level or binary code
Use of AI/large language models with GI
Repair bugs
GI techniques in industrial settings

We invite submissions of two paper types:

Research papers (eight page limit, including references)
Position papers (two page limit, including references)

The best paper and best presentation will be awarded during the workshop.

Detailed formatting instructions for authors are listed here.
We encourage authors to submit early and in-progress work. The workshop emphasises interaction and discussion.

Papers must be submitted through the paper submission website: https://gi-at-icse2024-workshop.hotcrp.com/
These papers will be reviewed in a double-blind manner.
All accepted papers must be presented at the workshop.

Special Issue on GI in the ASE journal

Authors of selected accepted papers are invited to submit an extended version of their papers to ASE Journal’s second Special Issue on Genetic Improvement.

Workshop Chairs

Gabin An is a doctoral candidate at KAIST in South Korea, advised by Prof. Shin Yoo. Gabin’s work focuses on faster and more accurate bug assignments and, in close collaboration with industry, improving test suites. She is also behind PyGGI, a Python library for fast prototyping of GI techniques for multiple programming languages. Gabin served as the web chair for GI@ICSE 2020, the GI workshop PC 2021-2022, and was a GI workshop co-chair in 2023.

Aymeric Blot is a Senior Lecturer at the Université of Rennes, France. He received in 2018 a doctorate from the University of Lille following work on automated algorithm design for multi-objective combinatorial optimisation, before moving to University College London to work on software specialisation using GI. His research focuses on strengthening GI techniques using knowledge from automated machine learning, algorithm configuration, and evolutionary computation.

Vesna Nowack received her PhD in Computer Architecture at UPC, Spain (2016). Recently, she worked on APR in academia (Queen Mary University of London, Lancaster University) and industry (Bloomberg). She is currently a Research Associate at Imperial College London with a focus on human-in-the-loop ML systems.

Oliver Krauss received his doctorate in 2022 in Pattern Mining and Genetic Improvement in Compilers and Interpeters. His research focuses on mining patterns in software, as well as data, to improve runtime performance and energy consumption. He maintains several open source frameworks, such as Amaru).

Justyna Petke leads research in genetic improvement at University College London. Her work on GI was awarded a Silver (2014) and a Gold ’Humie’ (2016) and an ACM SIGSOFT Distinguished Paper Award at ISSTA 2015. She was the PC co-Chair for the International Symposium on Search-Based Software Engineering in 2017, has organised 8 GI workshops and serves on the editorial boards of 5 journals.