Denoising autoencoder genetic programming (DAE-GP) is a novel neural network-based estimation of ... more Denoising autoencoder genetic programming (DAE-GP) is a novel neural network-based estimation of distribution genetic programming approach that uses denoising autoencoder long short-term memory networks as a probabilistic model to replace the standard mutation and recombination operators of genetic programming. At each generation, the idea is to capture promising properties of the parent population in a probabilistic model and to use corruption to transfer variations of these properties to the offspring. This work studies the influence of corruption and sampling steps on search. Corruption partially mutates candidate solutions that are used as input to the model, whereas the number of sampling steps defines how often we re-use the output during model sampling as input to the model. We study the generalization of the royal tree problem, the Airfoil problem, and the Pagie-1 problem, and find that both corruption strength and the number of sampling steps influence exploration and explo...
We introduce GPLS (Genetic Programming for Linear Systems) as a GP system that finds mathematical... more We introduce GPLS (Genetic Programming for Linear Systems) as a GP system that finds mathematical expressions defining an iteration matrix. Stationary iterative methods use this iteration matrix to solve a system of linear equations numerically. GPLS aims at finding iteration matrices with a low spectral radius and a high sparsity, since these properties ensure a fast error reduction of the numerical solution method and enable the efficient implementation of the methods on parallel computer architectures. We study GPLS for various types of system matrices and find that it easily outperforms classical approaches like the Gauss–Seidel and Jacobi methods. GPLS not only finds iteration matrices for linear systems with a much lower spectral radius, but also iteration matrices for problems where classical approaches fail. Additionally, solutions found by GPLS for small problem instances show also good performance for larger instances of the same problem.
Proceedings of the Genetic and Evolutionary Computation Conference
Recent work discovered that tness landscapes induced by Iterated Local Search (ILS) may consist o... more Recent work discovered that tness landscapes induced by Iterated Local Search (ILS) may consist of multiple clusters, denoted as funnels or communities of local optima. Such studies exist only for perturbation operators (kicks) with low strength. We examine how di erent strengths of the ILS perturbation operator a ect the number and size of clusters. We present an empirical study based on local optima networks from NK tness landscapes. Our results show that a properly selected perturbation strength can help overcome the e ect of ILS ge ing trapped in clusters of local optima. is has implications for designing e ective ILS approaches in practice, where traditionally only small perturbations or complete restarts are applied, with the middle ground of intermediate perturbation strengths largely unexplored.
GitHub Copilot, an extension for the Visual Studio Code development environment powered by the la... more GitHub Copilot, an extension for the Visual Studio Code development environment powered by the large-scale language model Codex, makes automatic program synthesis available for software developers. This model has been extensively studied in the field of deep learning, however, a comparison to genetic programming, which is also known for its performance in automatic program synthesis, has not yet been carried out. In this paper, we evaluate GitHub Copilot on standard program synthesis benchmark problems and compare the achieved results with those from the genetic programming literature. In addition, we discuss the performance of both approaches. We find that the performance of the two approaches on the benchmark problems is quite similar, however, in comparison to GitHub Copilot, the program synthesis approaches based on genetic programming are not yet mature enough to support programmers in practical software development. Genetic programming usually needs a huge amount of expensive hand-labeled training cases and takes too much time to generate solutions. Furthermore, source code generated by genetic programming approaches is often bloated and difficult to understand. For future work on program synthesis with genetic programming, we suggest researchers to focus on improving the execution time, readability, and usability. CCS CONCEPTS • Software and its engineering → Search-based software engineering; Genetic programming; • Computing methodologies → Neural networks.
Proceedings of the Genetic and Evolutionary Computation Conference Companion
Program synthesis techniques offer the potential to allow nonprogrammers to create computer progr... more Program synthesis techniques offer the potential to allow nonprogrammers to create computer programs. Approaches based on evolutionary algorithms are known for their performance in program synthesis. Since 2015, the general program synthesis benchmark suite offers a common pool of benchmark problems, which makes different approaches comparable. To analyze what has been achieved so far, we identified in a recent literature review the main evolutionary program synthesis approaches, determined the difficulty of the common benchmark problems, and discussed current challenges. In this short paper, we present the difficulty ranking of the benchmark problems and summarize the current challenges in program synthesis with evolutionary algorithms. CCS CONCEPTS • Theory of computation → Evolutionary algorithms; • Software and its engineering → Search-based software engineering.
Proceedings of the Genetic and Evolutionary Computation Conference
GitHub Copilot, an extension for the Visual Studio Code development environment powered by the la... more GitHub Copilot, an extension for the Visual Studio Code development environment powered by the large-scale language model Codex, makes automatic program synthesis available for software developers. This model has been extensively studied in the field of deep learning, however, a comparison to genetic programming, which is also known for its performance in automatic program synthesis, has not yet been carried out. In this paper, we evaluate GitHub Copilot on standard program synthesis benchmark problems and compare the achieved results with those from the genetic programming literature. In addition, we discuss the performance of both approaches. We find that the performance of the two approaches on the benchmark problems is quite similar, however, in comparison to GitHub Copilot, the program synthesis approaches based on genetic programming are not yet mature enough to support programmers in practical software development. Genetic programming usually needs a huge amount of expensive hand-labeled training cases and takes too much time to generate solutions. Furthermore, source code generated by genetic programming approaches is often bloated and difficult to understand. For future work on program synthesis with genetic programming, we suggest researchers to focus on improving the execution time, readability, and usability. CCS CONCEPTS • Software and its engineering → Search-based software engineering; Genetic programming; • Computing methodologies → Neural networks.
Over-parameterized models can perfectly learn various types of data distributions, however, gener... more Over-parameterized models can perfectly learn various types of data distributions, however, generalization error is usually lower for real data in comparison to artificial data. This suggests that the properties of data distributions have an impact on generalization capability. This work focuses on the search space defined by the input data and assumes that the correlation between labels of neighboring input values influences generalization. If correlation is low, the randomness of the input data space is high leading to high generalization error. We suggest to measure the randomness of an input data space using Maurer's universal. Results for synthetic classification tasks and common image classification benchmarks (MNIST, CIFAR10, and Microsoft's cats vs. dogs data set) find a high correlation between the randomness of input data spaces and the generalization error of deep neural networks for binary classification problems.
Digital Transformation – Meeting the challenges, 2018
As part of an advancing digitalization, many enterprises feel the need to explore the possibiliti... more As part of an advancing digitalization, many enterprises feel the need to explore the possibilities big data may provide for their business. However, only a few companies use big data applications productively, despite its high expected potential. How companies examine the possibilities of big data, is therefore a highly interesting and relevant question. Based on a multiple case study we identify three different approaches: Companies either initially focus entirely on business aspects, or on a systematic build-up of a big data technology and data platform. Innovation adoption research is used as a theoretical basis.
Proceedings of the Genetic and Evolutionary Computation Conference Companion, 2021
Recent studies show that program synthesis with GE produces code that has different structure com... more Recent studies show that program synthesis with GE produces code that has different structure compared to human-generated code, e.g., loops and conditions are hardly used. In this article, we extract knowledge from human-generated code to guide evolutionary search. We use a large code-corpus that was mined from the open software repository service GitHub and measure software metrics and properties describing the code-base. We use this knowledge to guide the search by incorporating a new selection scheme. Our new selection scheme favors programs that are structurally similar to the programs in the GitHub code-base. We find noticeable evidence that software metrics can help in guiding evolutionary search. CCS CONCEPTS • Software and its engineering → Search-based software engineering; Genetic programming.
Proceedings of the Genetic and Evolutionary Computation Conference Companion, 2021
The initial population in evolutionary algorithms (EAs) should form a representative sample of al... more The initial population in evolutionary algorithms (EAs) should form a representative sample of all possible solutions (the search space). While large populations accurately approximate the distribution of possible solutions, small populations tend to incorporate a sampling error. A low sampling error at initialization is necessary (but not sufficient) for a reliable search since a low sampling error reduces the overall random variations in a random sample. For this reason, we have recently presented a model to determine a minimum initial population size so that the sampling error is lower than a threshold, given a confidence level. Our model allows practitioners of, for example, genetic programming (GP) and other EA variants to estimate a reasonable initial population size. CCS CONCEPTS • Computing methodologies → Model development and analysis; • Mathematics of computing → Stochastic processes; Mathematical analysis; Evolutionary algorithms; • Theory of computation → Design and analysis of algorithms.
Recent literature suggests that local optima in fitness landscapes are clustered, which offers an... more Recent literature suggests that local optima in fitness landscapes are clustered, which offers an explanation of why perturbationbased metaheuristics often fail to find the global optimum: they become trapped in a sub-optimal cluster. We introduce a method to extract and visualize the global organization of these clusters in form of a barrier tree. Barrier trees have been used to visualize the barriers between local optima basins in fitness landscapes. Our method computes a more coarsely grained tree to reveal the barriers between clusters of local optima. The core element is a new variant of the flooding algorithm, applicable to local optima networks. We use local optima networks as a compressed representation of fitness landscapes. To identify the clusters, we apply a community detection algorithm. A sample of 200 NK fitness landscapes suggests that the depth of their coarse-grained barrier tree is related to their search difficulty for perturbation-based metaheuristics.
Recently, advances have been made in continuous, normal- distribution-based Estimation-of-Distrib... more Recently, advances have been made in continuous, normal- distribution-based Estimation-of-Distribution Algorithms (EDAs) by scaling the variance up from the maximum-like- lihood estimate. When done properly, such scaling has been shown to prevent premature convergence on slope-like re- gions of the search space. In this paper we specifically fo- cus on one way of scaling that was previously introduced as Adaptive
Genetic and Evolutionary Computation Conference, 2002
When using representations for genetic algorithms (GAs) every optimization problem can be separat... more When using representations for genetic algorithms (GAs) every optimization problem can be separated into a genotype-phenotype and a phenotype-tness mapping. The genotype- phenotype mapping is the used representation and the phenotype-tness mapping is the problem that should be solved. This paper investigates how the use of dierent binary representations of integers inuences the performance of selectorecombinative GAs using only crossover
This paper investigates how the performance of evolutionary search is inuenced by the locality an... more This paper investigates how the performance of evolutionary search is inuenced by the locality and distance distortion of the used representation. The locality of a representation describes how well neighboring phenotypes correspond to neighboring genotypes. The distance distortion measures whether the distances between individuals are preserved when mapping the phenotypes onto the genotypes. When using mutation as the main search operator, perfect locality ensures that the dicult y of the problem is not changed by the representation. In analogy, representations that preserve the distances between individuals guarantee that the dicult y of the problem remains unchanged for crossover-based search. Applying these concepts to binary representations of integers reveals that in contrast to the binary encoding the Gray encoding has perfect locality, and therefore guarantees that for mutation-based search, easy problems remain easy. However, because both Gray and binary encoding change the...
The automatic generation of computer programs is one of the main applications with practical rele... more The automatic generation of computer programs is one of the main applications with practical relevance in the field of evolutionary computation. With program synthesis techniques not only software developers could be supported in their everyday work but even users without any programming knowledge could be empowered to automate repetitive tasks and implement their own new functionality. In recent years, many novel program synthesis approaches based on evolutionary algorithms have been proposed and evaluated on common benchmark problems. Therefore, we identify in this work the relevant evolutionary program synthesis approaches and provide an in-depth analysis of their performance. The most influential approaches we identify are stack-based, grammar-guided, as well as linear genetic programming. Further, we find that these approaches perform well on benchmark problems if there is a simple mapping from the given input to the correct output. On problems where this mapping is complex, e.g., if the problem consists of several subproblems or requires iteration/recursion for a correct solution, results tend to be worse. Consequently, for future work, we encourage researchers not only to use a program's output for assessing the quality of a solution but also the way towards a solution (e.g., correctly solved sub-problems).
Proceedings of the Genetic and Evolutionary Computation Conference Companion
Ephemeral random constants are commonly used for symbolic regression with genetic programming. Ho... more Ephemeral random constants are commonly used for symbolic regression with genetic programming. However, due to their random nature, it is difficult for genetic programming to find proper constants. This often leads to large and complex non-interpretable solutions. Therefore, in this short paper, we analyze a method that replaces ephemeral random constants with constant tokens whose values are optimized during evolution with the Sequential Least Squares Programming method. The results achieved on the studied regression problem indicate that replacing ephemeral random constants by optimized constant tokens leads to solutions with a significantly lower error which are also smaller than the solutions generated by a standard GP approach using ephemeral random constants. CCS CONCEPTS • Software and its engineering → Genetic programming; • Computing methodologies → Supervised learning by regression.
Proceedings of the Genetic and Evolutionary Computation Conference Companion
Denoising Autoencoder Genetic Programming (DAE-GP) is a novel neural-network based estimation of ... more Denoising Autoencoder Genetic Programming (DAE-GP) is a novel neural-network based estimation of distribution genetic programming algorithm that uses denoising autoencoder long short-term memory networks as probabilistic model to replace the standard recombination and mutation operators of genetic programming (GP). Recent work demonstrated that the DAE-GP outperforms standard GP. However, results are limited to the generalization of the royal tree problem. In this work, we apply the DAE-GP to real-world symbolic regression. On the Airfoil dataset and given a fixed number of fitness evaluations, we find that the DAE-GP generates significantly better and smaller (number of nodes) best candidate solutions than standard GP. The results highlight that the DAE-GP may be a good alternative for generating good and interpretable solutions for real-world symbolic regression. CCS CONCEPTS • Computing methodologies → Genetic programming; Supervised learning by regression; • Mathematics of computing → Probabilistic algorithms.
Estimation of Distribution Algorithms (EDAs) are metaheuristics where learning a model and sampli... more Estimation of Distribution Algorithms (EDAs) are metaheuristics where learning a model and sampling new solutions replaces the variation operators recombination and mutation used in standard Genetic Algorithms. The choice of these models as well as the corresponding training processes are subject to the bias/variance tradeoff, also known as underand overfitting: simple models cannot capture complex interactions between problem variables, whereas complex models are susceptible to modeling random noise. This paper suggests using Denoising Autoencoders (DAEs) as generative models within EDAs (DAE-EDA). The resulting DAE-EDA is able to model complex probability distributions. Furthermore, overfitting is less harmful, since DAEs overfit by learning the identity function. This overfitting behavior introduces unbiased random noise into the samples, which is no major problem for the EDA but just leads to higher population diversity. As a result, DAE-EDA runs for more generations before conv...
Denoising autoencoder genetic programming (DAE-GP) is a novel neural network-based estimation of ... more Denoising autoencoder genetic programming (DAE-GP) is a novel neural network-based estimation of distribution genetic programming approach that uses denoising autoencoder long short-term memory networks as a probabilistic model to replace the standard mutation and recombination operators of genetic programming. At each generation, the idea is to capture promising properties of the parent population in a probabilistic model and to use corruption to transfer variations of these properties to the offspring. This work studies the influence of corruption and sampling steps on search. Corruption partially mutates candidate solutions that are used as input to the model, whereas the number of sampling steps defines how often we re-use the output during model sampling as input to the model. We study the generalization of the royal tree problem, the Airfoil problem, and the Pagie-1 problem, and find that both corruption strength and the number of sampling steps influence exploration and explo...
We introduce GPLS (Genetic Programming for Linear Systems) as a GP system that finds mathematical... more We introduce GPLS (Genetic Programming for Linear Systems) as a GP system that finds mathematical expressions defining an iteration matrix. Stationary iterative methods use this iteration matrix to solve a system of linear equations numerically. GPLS aims at finding iteration matrices with a low spectral radius and a high sparsity, since these properties ensure a fast error reduction of the numerical solution method and enable the efficient implementation of the methods on parallel computer architectures. We study GPLS for various types of system matrices and find that it easily outperforms classical approaches like the Gauss–Seidel and Jacobi methods. GPLS not only finds iteration matrices for linear systems with a much lower spectral radius, but also iteration matrices for problems where classical approaches fail. Additionally, solutions found by GPLS for small problem instances show also good performance for larger instances of the same problem.
Proceedings of the Genetic and Evolutionary Computation Conference
Recent work discovered that tness landscapes induced by Iterated Local Search (ILS) may consist o... more Recent work discovered that tness landscapes induced by Iterated Local Search (ILS) may consist of multiple clusters, denoted as funnels or communities of local optima. Such studies exist only for perturbation operators (kicks) with low strength. We examine how di erent strengths of the ILS perturbation operator a ect the number and size of clusters. We present an empirical study based on local optima networks from NK tness landscapes. Our results show that a properly selected perturbation strength can help overcome the e ect of ILS ge ing trapped in clusters of local optima. is has implications for designing e ective ILS approaches in practice, where traditionally only small perturbations or complete restarts are applied, with the middle ground of intermediate perturbation strengths largely unexplored.
GitHub Copilot, an extension for the Visual Studio Code development environment powered by the la... more GitHub Copilot, an extension for the Visual Studio Code development environment powered by the large-scale language model Codex, makes automatic program synthesis available for software developers. This model has been extensively studied in the field of deep learning, however, a comparison to genetic programming, which is also known for its performance in automatic program synthesis, has not yet been carried out. In this paper, we evaluate GitHub Copilot on standard program synthesis benchmark problems and compare the achieved results with those from the genetic programming literature. In addition, we discuss the performance of both approaches. We find that the performance of the two approaches on the benchmark problems is quite similar, however, in comparison to GitHub Copilot, the program synthesis approaches based on genetic programming are not yet mature enough to support programmers in practical software development. Genetic programming usually needs a huge amount of expensive hand-labeled training cases and takes too much time to generate solutions. Furthermore, source code generated by genetic programming approaches is often bloated and difficult to understand. For future work on program synthesis with genetic programming, we suggest researchers to focus on improving the execution time, readability, and usability. CCS CONCEPTS • Software and its engineering → Search-based software engineering; Genetic programming; • Computing methodologies → Neural networks.
Proceedings of the Genetic and Evolutionary Computation Conference Companion
Program synthesis techniques offer the potential to allow nonprogrammers to create computer progr... more Program synthesis techniques offer the potential to allow nonprogrammers to create computer programs. Approaches based on evolutionary algorithms are known for their performance in program synthesis. Since 2015, the general program synthesis benchmark suite offers a common pool of benchmark problems, which makes different approaches comparable. To analyze what has been achieved so far, we identified in a recent literature review the main evolutionary program synthesis approaches, determined the difficulty of the common benchmark problems, and discussed current challenges. In this short paper, we present the difficulty ranking of the benchmark problems and summarize the current challenges in program synthesis with evolutionary algorithms. CCS CONCEPTS • Theory of computation → Evolutionary algorithms; • Software and its engineering → Search-based software engineering.
Proceedings of the Genetic and Evolutionary Computation Conference
GitHub Copilot, an extension for the Visual Studio Code development environment powered by the la... more GitHub Copilot, an extension for the Visual Studio Code development environment powered by the large-scale language model Codex, makes automatic program synthesis available for software developers. This model has been extensively studied in the field of deep learning, however, a comparison to genetic programming, which is also known for its performance in automatic program synthesis, has not yet been carried out. In this paper, we evaluate GitHub Copilot on standard program synthesis benchmark problems and compare the achieved results with those from the genetic programming literature. In addition, we discuss the performance of both approaches. We find that the performance of the two approaches on the benchmark problems is quite similar, however, in comparison to GitHub Copilot, the program synthesis approaches based on genetic programming are not yet mature enough to support programmers in practical software development. Genetic programming usually needs a huge amount of expensive hand-labeled training cases and takes too much time to generate solutions. Furthermore, source code generated by genetic programming approaches is often bloated and difficult to understand. For future work on program synthesis with genetic programming, we suggest researchers to focus on improving the execution time, readability, and usability. CCS CONCEPTS • Software and its engineering → Search-based software engineering; Genetic programming; • Computing methodologies → Neural networks.
Over-parameterized models can perfectly learn various types of data distributions, however, gener... more Over-parameterized models can perfectly learn various types of data distributions, however, generalization error is usually lower for real data in comparison to artificial data. This suggests that the properties of data distributions have an impact on generalization capability. This work focuses on the search space defined by the input data and assumes that the correlation between labels of neighboring input values influences generalization. If correlation is low, the randomness of the input data space is high leading to high generalization error. We suggest to measure the randomness of an input data space using Maurer's universal. Results for synthetic classification tasks and common image classification benchmarks (MNIST, CIFAR10, and Microsoft's cats vs. dogs data set) find a high correlation between the randomness of input data spaces and the generalization error of deep neural networks for binary classification problems.
Digital Transformation – Meeting the challenges, 2018
As part of an advancing digitalization, many enterprises feel the need to explore the possibiliti... more As part of an advancing digitalization, many enterprises feel the need to explore the possibilities big data may provide for their business. However, only a few companies use big data applications productively, despite its high expected potential. How companies examine the possibilities of big data, is therefore a highly interesting and relevant question. Based on a multiple case study we identify three different approaches: Companies either initially focus entirely on business aspects, or on a systematic build-up of a big data technology and data platform. Innovation adoption research is used as a theoretical basis.
Proceedings of the Genetic and Evolutionary Computation Conference Companion, 2021
Recent studies show that program synthesis with GE produces code that has different structure com... more Recent studies show that program synthesis with GE produces code that has different structure compared to human-generated code, e.g., loops and conditions are hardly used. In this article, we extract knowledge from human-generated code to guide evolutionary search. We use a large code-corpus that was mined from the open software repository service GitHub and measure software metrics and properties describing the code-base. We use this knowledge to guide the search by incorporating a new selection scheme. Our new selection scheme favors programs that are structurally similar to the programs in the GitHub code-base. We find noticeable evidence that software metrics can help in guiding evolutionary search. CCS CONCEPTS • Software and its engineering → Search-based software engineering; Genetic programming.
Proceedings of the Genetic and Evolutionary Computation Conference Companion, 2021
The initial population in evolutionary algorithms (EAs) should form a representative sample of al... more The initial population in evolutionary algorithms (EAs) should form a representative sample of all possible solutions (the search space). While large populations accurately approximate the distribution of possible solutions, small populations tend to incorporate a sampling error. A low sampling error at initialization is necessary (but not sufficient) for a reliable search since a low sampling error reduces the overall random variations in a random sample. For this reason, we have recently presented a model to determine a minimum initial population size so that the sampling error is lower than a threshold, given a confidence level. Our model allows practitioners of, for example, genetic programming (GP) and other EA variants to estimate a reasonable initial population size. CCS CONCEPTS • Computing methodologies → Model development and analysis; • Mathematics of computing → Stochastic processes; Mathematical analysis; Evolutionary algorithms; • Theory of computation → Design and analysis of algorithms.
Recent literature suggests that local optima in fitness landscapes are clustered, which offers an... more Recent literature suggests that local optima in fitness landscapes are clustered, which offers an explanation of why perturbationbased metaheuristics often fail to find the global optimum: they become trapped in a sub-optimal cluster. We introduce a method to extract and visualize the global organization of these clusters in form of a barrier tree. Barrier trees have been used to visualize the barriers between local optima basins in fitness landscapes. Our method computes a more coarsely grained tree to reveal the barriers between clusters of local optima. The core element is a new variant of the flooding algorithm, applicable to local optima networks. We use local optima networks as a compressed representation of fitness landscapes. To identify the clusters, we apply a community detection algorithm. A sample of 200 NK fitness landscapes suggests that the depth of their coarse-grained barrier tree is related to their search difficulty for perturbation-based metaheuristics.
Recently, advances have been made in continuous, normal- distribution-based Estimation-of-Distrib... more Recently, advances have been made in continuous, normal- distribution-based Estimation-of-Distribution Algorithms (EDAs) by scaling the variance up from the maximum-like- lihood estimate. When done properly, such scaling has been shown to prevent premature convergence on slope-like re- gions of the search space. In this paper we specifically fo- cus on one way of scaling that was previously introduced as Adaptive
Genetic and Evolutionary Computation Conference, 2002
When using representations for genetic algorithms (GAs) every optimization problem can be separat... more When using representations for genetic algorithms (GAs) every optimization problem can be separated into a genotype-phenotype and a phenotype-tness mapping. The genotype- phenotype mapping is the used representation and the phenotype-tness mapping is the problem that should be solved. This paper investigates how the use of dierent binary representations of integers inuences the performance of selectorecombinative GAs using only crossover
This paper investigates how the performance of evolutionary search is inuenced by the locality an... more This paper investigates how the performance of evolutionary search is inuenced by the locality and distance distortion of the used representation. The locality of a representation describes how well neighboring phenotypes correspond to neighboring genotypes. The distance distortion measures whether the distances between individuals are preserved when mapping the phenotypes onto the genotypes. When using mutation as the main search operator, perfect locality ensures that the dicult y of the problem is not changed by the representation. In analogy, representations that preserve the distances between individuals guarantee that the dicult y of the problem remains unchanged for crossover-based search. Applying these concepts to binary representations of integers reveals that in contrast to the binary encoding the Gray encoding has perfect locality, and therefore guarantees that for mutation-based search, easy problems remain easy. However, because both Gray and binary encoding change the...
The automatic generation of computer programs is one of the main applications with practical rele... more The automatic generation of computer programs is one of the main applications with practical relevance in the field of evolutionary computation. With program synthesis techniques not only software developers could be supported in their everyday work but even users without any programming knowledge could be empowered to automate repetitive tasks and implement their own new functionality. In recent years, many novel program synthesis approaches based on evolutionary algorithms have been proposed and evaluated on common benchmark problems. Therefore, we identify in this work the relevant evolutionary program synthesis approaches and provide an in-depth analysis of their performance. The most influential approaches we identify are stack-based, grammar-guided, as well as linear genetic programming. Further, we find that these approaches perform well on benchmark problems if there is a simple mapping from the given input to the correct output. On problems where this mapping is complex, e.g., if the problem consists of several subproblems or requires iteration/recursion for a correct solution, results tend to be worse. Consequently, for future work, we encourage researchers not only to use a program's output for assessing the quality of a solution but also the way towards a solution (e.g., correctly solved sub-problems).
Proceedings of the Genetic and Evolutionary Computation Conference Companion
Ephemeral random constants are commonly used for symbolic regression with genetic programming. Ho... more Ephemeral random constants are commonly used for symbolic regression with genetic programming. However, due to their random nature, it is difficult for genetic programming to find proper constants. This often leads to large and complex non-interpretable solutions. Therefore, in this short paper, we analyze a method that replaces ephemeral random constants with constant tokens whose values are optimized during evolution with the Sequential Least Squares Programming method. The results achieved on the studied regression problem indicate that replacing ephemeral random constants by optimized constant tokens leads to solutions with a significantly lower error which are also smaller than the solutions generated by a standard GP approach using ephemeral random constants. CCS CONCEPTS • Software and its engineering → Genetic programming; • Computing methodologies → Supervised learning by regression.
Proceedings of the Genetic and Evolutionary Computation Conference Companion
Denoising Autoencoder Genetic Programming (DAE-GP) is a novel neural-network based estimation of ... more Denoising Autoencoder Genetic Programming (DAE-GP) is a novel neural-network based estimation of distribution genetic programming algorithm that uses denoising autoencoder long short-term memory networks as probabilistic model to replace the standard recombination and mutation operators of genetic programming (GP). Recent work demonstrated that the DAE-GP outperforms standard GP. However, results are limited to the generalization of the royal tree problem. In this work, we apply the DAE-GP to real-world symbolic regression. On the Airfoil dataset and given a fixed number of fitness evaluations, we find that the DAE-GP generates significantly better and smaller (number of nodes) best candidate solutions than standard GP. The results highlight that the DAE-GP may be a good alternative for generating good and interpretable solutions for real-world symbolic regression. CCS CONCEPTS • Computing methodologies → Genetic programming; Supervised learning by regression; • Mathematics of computing → Probabilistic algorithms.
Estimation of Distribution Algorithms (EDAs) are metaheuristics where learning a model and sampli... more Estimation of Distribution Algorithms (EDAs) are metaheuristics where learning a model and sampling new solutions replaces the variation operators recombination and mutation used in standard Genetic Algorithms. The choice of these models as well as the corresponding training processes are subject to the bias/variance tradeoff, also known as underand overfitting: simple models cannot capture complex interactions between problem variables, whereas complex models are susceptible to modeling random noise. This paper suggests using Denoising Autoencoders (DAEs) as generative models within EDAs (DAE-EDA). The resulting DAE-EDA is able to model complex probability distributions. Furthermore, overfitting is less harmful, since DAEs overfit by learning the identity function. This overfitting behavior introduces unbiased random noise into the samples, which is no major problem for the EDA but just leads to higher population diversity. As a result, DAE-EDA runs for more generations before conv...
Uploads
Papers by Franz Rothlauf