The 11th International Workshop on Genetic Improvement @GECCO 2022

Navigation: Keynote, Accepted Papers, Schedule, CFP, Workshops Chairs, PC

Event

The 11th instalment of the GI workshop took place in Boston on 9 July 2022, collocated with the Genetic and Evolutionary Computation Conference, GECCO 2022.

best paper award: Evaluation of Genetic Improvement Tools for Improvement of Non-functional Properties of Software — Shengjie Zuo, Aymeric Blot, and Justyna Petke
best presentation award: Genetic Improvement in the Shackleton Framework for Optimizing LLVM Pass Sequences — Shuyue Stella Li, Hannah Peeler, Andrew N. Sloss, Kenneth N. Reid, and Wolfgang Banzhaf

Group photo:

Presentation Recordings

A YouTube Playlist has been created containing recordings of all the 11th GI Workshop presentations:

Important Dates

Submission Deadline: 11 April 2022 (Mon)
Notification: 25 April 2022 (Mon)
Camera-ready: 2 May 2022 (Mon)
Workshop: 9 July 2022 (Sat)

Keep up to date with the latest event news via our Twitter: https://twitter.com/gi_of_software.

Keynote

We are happy to announce that Westley Weimer (University of Michigan, USA) will give the keynote speech in GI@GECCO 2022: From Deep Learning to Human Judgments: Lessons for Genetic Improvement.
Many new techniques and evaluation approaches are being considered in subfields like automated program repair or manual program improvement, but they have not yet translated to popular success in the subfield of Genetic Improvement. For example, automated program repair researchers are increasingly considering deep neural network or neural machine translation approaches to fix defects – approaches which may seem, at first glance, to be incompatible with genetic methods. Similarly, there has been an increasing use of advanced measurements, such as eye tracking or medical imaging, or experimental designs, such as the use of deception, in the evaluation of bug-fixing patches – but those evaluations are not as common for genetic improvement. This keynote address summarizes some recent techniques, placing them in a GI context and arguing that many of these techniques are compatible with a continuous notion of genetic improvement (as opposed to a discrete notion of program repair). This keynote also summarizes recent evaluations and their use in the field, as well as lessons that can be learned from industrial deployments (e.g., from Facebook to Fujitsu to Bloomberg) about human trust and acceptability. It concludes with a call to arms about the potential impact of Genetic Improvement.

Prof. Westley Weimer received his PhD from the University of California at Berkeley. His research interests include reducing the costs associated with software development at scale (particularly through automated program repair) as well as program analysis, formal verification, and human linguistic and visual interaction with software. He is a senior member of the Association for Computing Machinery and his work has led to over fifteen thousand citations and several awards, including three ‘Humies’ and ICSE 2019 Most Influential paper for his work on using Genetic Improvement for bug fixing.

An archive of the keynote slides are available here.

Accepted Papers

Amaru - A Framework for combining Genetic Improvement with Pattern Mining
by Oliver Krauss
DOI PDF SLIDES VIDEO URL Abstract

We present Amaru, a framework for Genetic Improvement using Abstract Syntax Trees directly at the interpreter and compiler level. Amaru also enables the mining of frequent, discriminative patterns from Genetic Improvement populations. These patterns in turn can be used to improve the crossover and mutation operators to increase population diversity, reduce the number of individuals failing at run-time and increasing the amount of successful individuals.

Automatically Exploring Computer System Design Spaces
by Bobby R. Bruce
DOI PDF SLIDES Abstract

While much research has focused on using search to optimize software, it is possible to use these same approaches to optimise other parts of the computer system. With decreased costs in silicon customisation, and the return of centralized systems carrying out specialized tasks, the time is right to begin thinking about tools to optimise systems for the needs of specific software targets. In the approach outlined, the standard Genetic Improvement process is flipped with source code considered static and the remainder of the computer system altered to the needs of the software. The project proposed is preliminary research into incorporating grammar-based GP with an advanced computer architecture simulator to automatically design computer systems. I argue this approach has the potential to significantly improve the design of computer systems while reducing manual effort.

Dissecting Copy/Delete/Replace/Swap mutations: Insights from a GIN Case Study
by Sherlock A. Licorish and Markus Wagner
DOI PDF SLIDES VIDEO Abstract

Research studies are increasingly critical of publicly available code due to evidence of faults. This has led researchers to explore ways to improve such code, with static analysis and genetic code improvement previously singled out. Previous work has evaluated the feasibility of these techniques, using PMD and GIN for enhancing Stack Overflow code snippets. Results reported in this regard pointed to the potential of these techniques, especially in terms of GIN removal of PMD performance faults from 58 programs. We use a contextual lens to explore these mutations in this study, to evaluate the promise of these techniques. The outcomes show that while the programs were syntactically correct after GIN mutations, many of GIN mutations changed the semantics of the code, rendering its purpose questionable. However, certain code mutations tend to retain code semantics more than others. In addition, GINs mutations at times affected PMDs parsing ability, potentially increasing false negatives. Overall, while these approaches may prove useful, full utility may not be claimed at this time. For enhancing the outcomes of these approaches, we outline multiple strategies.

Evaluation of Genetic Improvement Tools for Improvement of Non-functional Properties of Software
by Shengjie Zuo, Aymeric Blot, and Justyna Petke
DOI PDF URL Abstract

Genetic improvement (GI) improves both functional properties of software, such as bug repair, and non-functional properties, such as execution time, energy consumption, or source code size. There are studies summarising and comparing GI tools for improving functional properties of software; however there is no such study for improvement of its non-functional properties using GI. Therefore, this research aims to survey and report on the existing GI tools for improvement of non-functional properties of software. We conducted a literature review of available GI tools, and ran multiple experiments on the found open-source tools to examine their usability. We applied a cross-testing strategy to check whether the available tools can work on different programs. Overall, we found 63 GI papers that use a GI tool to improve non-functional properties of software, within which 31 are accompanied with open-source code. We were able to successfully run nine GI tools, and found that ultimately only two, Gin and PyGGI, can be readily applied to new general software.

From Deep Learning to Human Judgments: Lessons for Genetic Improvement
by Westley Weimer
SLIDES VIDEO

Genetic Improvement in the Shackleton Framework for Optimizing LLVM Pass Sequences
by Shuyue Stella Li, Hannah Peeler, Andrew N. Sloss, Kenneth N. Reid, and Wolfgang Banzhaf
DOI PDF PDF SLIDES VIDEO URL URL Abstract

Genetic Improvement is a search technique that aims to improve a given acceptable solution to a problem. we present the novel use of genetic improvement to find problem-specific optimized LLVM Pass sequences. We develop a Pass-level edit representation in the linear genetic programming framework, Shackleton, to evolve the modifications to be applied to the default optimization Pass sequences. Our GI-evolved solution has a mean of 3.7percent runtime improvement compared to the default LLVM optimisation level -O3 which targets runtime. The proposed GI method provides an automatic way to find a problem-specific optimization sequence that improves upon a general solution without any expert domain knowledge. we discuss the advantages and limitations of the GI feature in the Shackleton Framework and present our results.

Genetic Improvement of Shoreline Evolution Forecasting Models
by Mahmoud Al Najar, Rafael Almar, Erwin W. J. Bergsma, Jean-Marc Delvit, and Dennis G. Wilson
DOI PDF SLIDES SLIDES VIDEO Abstract

Coastal development and climate change are changing the geography of our coasts, while more and more people are moving towards the coasts. Recent advances in artificial intelligence allow for automatic analysis of observational data. Cartesian Genetic Programming (CGP) is a form of Genetic Programming (GP) that has been successfully used in a large variety of tasks including data-driven symbolic regression. We formulate the problem of shoreline evolution forecasting as a Genetic Improvement (GI) problem using CGP to encode and improve upon ShoreFor, an equilibrium shoreline prediction model, to study the effectiveness of CGP in GI in forecasting tasks. This work presents an empirical study of the sensitivity of CGP to a number of evolutionary configurations and constraints and compares the performances of the evolved models to the base ShoreFor model.

Leveraging Fuzzy System to Reduce Uncertainty of Decision Making in Software Engineering Automation
by Yueke Zhang and Yu Huang
DOI PDF SLIDES VIDEO Abstract

Decision making in automated GI-based software engineering tasks can significantly affect the performance of the system. However, modern SE usually presents high uncertainty in such decision making process due to the existence of multiple solutions that reply on different heuristics. We propose to apply the theory of Fuzzy System to obtain a final decision with lower uncertainty and higher accuracy. We also demonstrate a motivating example and discuss the challenges and opportunities for applying fuzzy system to SE tasks.

Opportunities for Genetic Improvement of Cryptographic Code
by Chitchanok Chuengsatiansup, Markus Wagner, and Yuval Yarom
DOI PDF SLIDES VIDEO Abstract

Cryptography is one of the main tools underlying the security of our connected world. Cryptographic code must meet not only high security requirements, but also exhibit excellent non-functional properties, such as high performance and unique security requirements. As both automatic code generation and genetic improvement of such code are under explored, we motivate here what makes cryptographic code a prime target for future research.

Py2Cy: A Genetic Improvement Tool To Speed Up Python
by James Zhong, Max Hort, and Federica Sarro
DOI PDF SLIDES VIDEO URL Abstract

Due to its ease of use and wide range of custom libraries, Python has quickly gained popularity and is used by a wide range of developers all over the world. While Python allows for fast writing of source code, the resulting programs are slow to execute when compared to programs written in other programming languages like C. One of the reasons for its slow execution time is the dynamic typing of variables. Cython is an extension to Python, which can achieve execution speed-ups by compiler optimization. One possibility for improvements is the use of static typing, which can be added to Python scripts by developers. To alleviate the need for manual effort,we create Py2Cy, a Genetic Improvement tool for automatically converting Python scripts to statically typed Cython scripts. To show the feasibility of improving runtime with Py2Cy, we optimise a Python script for generating Fibonacci numbers. The results show that Py2Cy is able to speed up the execution time by up to a factor of 18.

The case for Grammatical Evolution in test generation
by Aidan Murphy, Thomas Laurent, and Anthony Ventresque
DOI PDF SLIDES VIDEO Abstract

Generating tests for software is an important, but difficult, task. Search-based test generation is promising, as it reduces the time required from human experts, but suffers from many problems and limitations. Namely, the inability to fully incorporate a testers domain knowledge into the search, its difficulty in creating very complex objects, and the problems associated with variable length tests. This paper illustrates how Grammatical Evolution could address and provide a possible solution to each of these concerns.

Towards evolution-based autonomy in large-scale systems
by Damien Anderson, Paul Harvey, Yusaku Kaneta, Petros Papadopoulos, Philip Rodgers, and Marc Roper
DOI PDF SLIDES VIDEO Abstract

To achieve truly autonomous behaviour systems need to adapt to not only previously unseen circumstances but also previously unimagined ones. A hierarchical evolutionary system is proposed which is capable of displaying the emergent behaviour necessary to achieve this goal. we report the practical challenges encountered in implementing this proposed approach in a large-scale open-source system.

Schedule

This schedule appears in Boston’s time zone (UTC/GMT-4, Eastern Daylight Time); compare to your timezone here.
Presentations for full papers are 20 minutes long, followed by 5 minutes for questions.
Presentations for short papers consist of a 10 minute talk, followed by 5 minutes for questions.

Saturday, July 09, 10:50–12:40 (110 mins; chair: Bobby R. Bruce, Justyna Petke)
- 10:50: Welcome — Bobby R. Bruce (5 mins)
- 10:55: Keynote speech — From Deep Learning to Human Judgments: Lessons for Genetic Improvement — Westley Weimer (5+45+15 mins Q&A)
- 12:00: Amaru: A Framework for Combining Genetic Improvement with Pattern Mining — Oliver Krauss (20+5 mins)
- 12:25: Genetic Improvement in the Shackleton Framework for Optimizing LLVM Pass Sequences — Shuyue Stella Li, Hannah Peeler, Andrew N. Sloss, Kenneth N. Reid, and Wolfgang Banzhaf (10+5 mins)
Saturday, July 09, 13:40–15:30 (110 mins; chair: Bobby R. Bruce, Vesna Nowack)
- 13:40: Evaluation of Genetic Improvement Tools for Improvement of Non-functional Properties of Software — Shengjie Zuo, Aymeric Blot, and Justyna Petke (20+5 mins)
- 14:05: Opportunities for Genetic Improvement of Cryptographic Code — Chitchanok Chuengsatiansup, Markus Wagner, and Yuval Yarom (10+5 mins)
- 14:20: Towards evolution-based autonomy in large-scale systems — Damien Anderson, Paul Harvey, Yusaku Kaneta, Petros Papadopoulos, Philip Rodgers, and Marc Roper (10+5 mins)
- 14:35: Genetic Improvement of Shoreline Evolution Forecasting Models — Mahmoud Al Najar, Rafael Almar, Erwin W. J. Bergsma, Jean-Marc Delvit, and Dennis G. Wilson (20+5 mins)
- 15:00: The case for Grammatical Evolution in test generation — Aidan Murphy, Thomas Laurent, and Anthony Ventresque (10+5 mins)
- 15:15: Leveraging Fuzzy System to Reduce Uncertainty of Decision Making in Software Engineering Automation — Yueke Zhang and Yu Huang (10+5 mins)
Saturday, July 09, 16:00–17:50 (110 mins; chair: Aymeric Blot, Vesna Nowack)
- 16:00: Dissecting Copy/Delete/Replace/Swap mutations: Insights from a GIN Case Study — Sherlock A. Licorish and Markus Wagner (20+5 mins)
- 16:25: Py2Cy: A Genetic Improvement Tool To Speed Up Python — James Zhong, Max Hort, and Federica Sarro (20+5 mins)
- 16:50: Automatically Exploring Computer System Design Spaces — Bobby R. Bruce (10+5 mins)
- 17:05: Industry+Student+Faculty Open Discussion — Aymeric Blot (40 mins)
- 17:45: Prizes & Closing — Vesna Nowack (5 mins)

Call For Submissions [pdf]

We invite submissions that discuss recent developments in all areas of research on, and applications of, Genetic Improvement. The International Workshop on Genetic Improvement is the premier workshop in the field and provides an opportunity for researchers interested in automated program repair and software optimisation to disseminate their work, exchange ideas, and discover new research directions. Topics of interest include both the theory and practice of Genetic Improvement. Applications of GI include, but are not limited to:

Improve efficiency
Decrease memory consumption
Decrease energy consumption
Transplant new functionality
Specialise software
Translate between programming languages
Generate multiple versions of software
Repair bugs

We invite submissions of two paper types:

Research papers (eight page limit, excluding references)
Position papers (two page limit, including references)

The best paper and best presentation will be awarded during the workshop.

Detailed formatting instructions for authors are listed here.
We encourage authors to submit early and in-progress work. The workshop emphasises interaction and discussion.

Papers must be submitted through the paper submission website: https://ssl.linklings.net/conferences/gecco/
These papers will be reviewed in a double-blind manner.
All accepted papers must be presented at GI-2022 and will appear in the GECCO companion volume.

As a published ACM author, you and your co-authors are subject to all ACM Publications Policies (https://www.acm.org/publications/policies/toc), including ACM’s new Publications Policy on Research Involving Human Participants and Subjects (https://www.acm.org/publications/policies/research-involving-human-participants-and-subjects).

Funding Opportunity For Students

We will support up to 5 students by offering to partially reimburse (up to $250 each) registration and travel costs for GECCO and all associated workshops for students whose work is accepted to the GI workshop. Priority will be given based on the student’s need and submission quality.

Students applying for a scholarship should submit a first-author regular paper to the workshop (up to 8 pages long) and plan to present their work in Boston in person. Moreover, their supervisor should send a one-paragraph note of recommendation to [email protected] listing:

the student’s area of work.
the supervisor’s support of the student’s application.

Workshop Chairs

Bobby Bruce is a Project Scientist at UC Davis where he primarily works on the gem5 computer architecture simulator. Prior to UC Davis, Bobby carried out research into the automatic optimization of Java bytecode at UCLA. His research interests are centred around Search-based Software Engineering, and its application to improving software performance.

Vesna Nowack. Since she gained her PhD in Software Engineering in 2016 from the Universitat Politecnica de Catalunya in Barcelona, she has conducted research in supercomputing (Spain) and taught robotics in Germany. Her recent research has been on APR in the UK, including 12 months with Bloomberg (London) published this summer as “On the Introduction of Automatic Program Repair in Bloomberg” by IEEE Software. She is now a Senior Research Assistant at Lancaster University where she continues her work on using GI to automatically fix bugs.

Aymeric Blot is a Research Associate conducting research in genetic improvement at the CREST and SOLAR groups in University College London. He received in 2018 a doctorate from the University of Lille following work on automated algorithm design for multi-objective combinatorial optimisation. His research focuses on strengthening GI techniques using knowledge from automated machine learning, algorithm configuration, and evolutionary computation. He maintains and evolves the community website on genetic improvement.

Emily Winter is a Senior Research Associate at Lancaster University, specialising in the human and socio-technical aspects of software engineering. She works on the Fixie Project: Exploiting Defect Prediction for Automatic Software Repair, investigating developer needs and preferences for how they interact with an Automatic Software Repair tool. As part of her research, she is seconded as a contractor to Bloomberg LP.

Bill Langdon has been working on GP since 1993. His PhD was the first book to be published in John Koza and Dave Goldberg's book series. He has previously run the GP track for GECCO 2001 and 2014 and was programme chair for GECCO 2002 having previously chaired EuroGP for 3 years. More recently he edited FOGA 2011 and run the computational intelligence on GPUs (CIGPU 2008—2012) and EvoPAR workshops. His books include A Field Guide to Genetic Programming, Foundations of Genetic Programming and Advances in Genetic Programming III. He also maintains the genetic programming bibliography. His current research uses GP to genetically improve existing software, CUDA, search based software engineering and Bioinformatics. He co-organised GI 2015, GI 2017, GI 2018, GI 2019, GI 2021 and the 2018 Genetic Improvement of Software Seminar at Dagstuhl.

Justyna Petke is a Principal Research Fellow and Proleptic Senior Lecturer (Associate Prof.), conducting research in genetic improvement. She has a doctorate in Computer Science from University of Oxford and is now at the Centre for Research on Evolution, Search and Testing (CREST) in University College London. She has published on applications of genetic improvement. Her work on the subject was awarded a Silver and a Gold ’Humie’ at GECCO 2014 and GECCO 2016 as well as an ACM SIGSOFT Distinguished Paper Award at ISSTA 2015. She was the PC co-Chair for the International Symposium on Search-Based Software Engineering in 2017. She currently serves on the editorial board of the Genetic Programming and Evolvable Machines journal.

PC