Papers by Julian Togelius
IEEE Transactions on Computational Intelligence and Ai in Games, 2011
The focus of this survey is on research in applying evolutionary and other metaheuristic search a... more The focus of this survey is on research in applying evolutionary and other metaheuristic search algorithms to automatically generating content for games, both digital and nondigital (such as board games). The term search-based procedural content generation is proposed as the name for this emerging field, which at present is growing quickly. A taxonomy for procedural content generation is devised, centering on what kind of content is generated, how the content is represented and how the quality/fitness of the content is evaluated; search-based procedural content generation in particular is situated within this taxonomy. This article also contains a survey of all published papers known to the authors in which game content is generated through search or optimisation, and ends with an overview of important open research problems.
We try to clarify the concept of procedural content generation (PCG) through contrasting it to ot... more We try to clarify the concept of procedural content generation (PCG) through contrasting it to other forms of content generation in games with which it could easily be mistaken, and through discussing some properties of PCG which are sometimes thought of as necessary but are actually not. After drawing up some clear demarcations for what is and what is not PCG, we present two versions of a content generation system for Infinite Mario Bros which is intentionally designed to question these same demarcations. We argue that, according to our own definition, one version of the system is an example of PCG while the other is not, even though they are mostly identical. We hope that this paper answers some questions but raises others, and inspires researchers and developers to thread some less common ground in developing content generation techniques.
IEEE Transactions on Computational Intelligence and Ai in Games, 2011
The Level Generation Competition, part of the IEEE Computational Intelligence Society (CIS)-spons... more The Level Generation Competition, part of the IEEE Computational Intelligence Society (CIS)-sponsored 2010 Mario AI Championship, was to our knowledge the world's first procedural content generation competition. Competitors participated by submitting level generators - software that generates new levels for a version of Super Mario Bros tailored to individual players' playing style. This paper presents the rules of the competition, the software used, the scoring procedure, the submitted level generators, and the results of the competition. We also discuss what can be learned from this competition, both about organizing procedural content generation competitions and about automatically generating levels for platform games. The paper is coauthored by the organizers of the competition (the first three authors) and the competitors.
Model complexity is key concern to any artificial learning system due its critical impact on gene... more Model complexity is key concern to any artificial learning system due its critical impact on generalization. However, EC research has only focused phenotype structural complexity for static problems. For sequential decision tasks, phenotypes that are very similar in structure, can produce radically different behaviors, and the trade-off between fitness and complexity in this context is not clear. In this paper, behavioral complexity is measured explicitly using compression, and used as a separate objective to be optimized (not as an additional regularization term in a scalar fitness), in order to study this trade-off directly.
The simulated car racing competition of CIG-2009 is the final event of the 2009 Simulated Car Rac... more The simulated car racing competition of CIG-2009 is the final event of the 2009 Simulated Car Racing Championship, an event joining the three competitions held at CEC-2009, GECCO-2009, and CIG-2009.
We address the problem of automatically designing maps for first-person shooter (FPS) games. An e... more We address the problem of automatically designing maps for first-person shooter (FPS) games. An efficient solution to this procedural content generation (PCG) problem could allow the design of FPS games of lower development cost with near-infinite replay value and capability to adapt to the skills and preferences of individual players. We propose a search-based solution, where maps are evolved to optimize a fitness function that is based on the players' average fighting time. For that purpose, four different map representations are tested and compared. Results obtained showcase the clear advantage of some representations in generating interesting FPS maps and demonstrate the promise of the approach followed for automatic level design in that game genre.
The Strategy Game Description Game Language (SGDL) is intended to become a complete description o... more The Strategy Game Description Game Language (SGDL) is intended to become a complete description of all aspects of strategy games, including rules, parameters, scenarios, maps, and unit types. One of the main envisioned uses of SGDL, in combination with an evolutionary algorithm and appropriate fitness functions, is to allow the generation of complete new strategy games or variations of old ones. This paper presents a first version of SGDL, capable of describing unit types and their properties, together with plans for how it will be extended to other sub-domains of strategy games. As a proof of the viability of the idea and implementation, an experiment is presented where unit types are evolved so as to generate complementary properties. A fitness function based on Monte Carlo simulation of gameplay is devised to test complementarity.
Over the last years, researchers have added neutrality in the evolutionary search in the hope tha... more Over the last years, researchers have added neutrality in the evolutionary search in the hope that it can aid evolution. In this paper, we study the presence of neutrality that is already and to do so, we analised the fitness landscape of the Sudoku problem. How and why neutrality affects evolutionary search is a reasonably well-studied but still not clearly understood topic. Here, we use neutral walks, neutrality trajectories and fitness distance correlation to attempt to throw new light on this topic.
The aim of this paper is to introduce the use of Tower Defence (TD) games in Computational Intell... more The aim of this paper is to introduce the use of Tower Defence (TD) games in Computational Intelligence (CI) research. We show how TD games can provide an important test-bed for the often under-represented casual games research area. Additionally, the use of CI in the TD games has the potential to create a more interesting, interactive and ongoing game experience for casual gamers. We present a definition of the current state and development of TD games, and include a classification of TD game components. We then describe some potential ways CI can be used to augment the TD experience. Finally, a prototype TD game based on experience driven procedural content generation is presented.
Reinforcement learning (RL) problems come in many flavours, as do the algorithms for solving them... more Reinforcement learning (RL) problems come in many flavours, as do the algorithms for solving them. It is currently not clear which of the commonly used RL benchmarks best measure an algorithm's capacity for solving real-world problems. Similarly, it is not clear which types of RL algorithms are best suited to solve which kinds of RL problems. Here we present some dimensions along the axes o which RL problems and algorithms can be varied to help distinguish them from each other. Based on results and arguments in the literature, we present some conjectures as to what algorithms should work best for particular types of problems, and argue that tunable RL benchmarks are needed in order to further understand the capabilities of RL algorithms.
An approach to robotics called layered evolution and merging features from the subsumption archit... more An approach to robotics called layered evolution and merging features from the subsumption architecture into evolutionary robotics is presented, its advantages and its relevance for science and engineering are discussed. This approach is used to construct a layered controller for a simulated robot that learns which light source to approach in an environment with obstacles. The evolvability and performance of layered evolution on this task is compared to (standard) monolithic evolution, incremental and modularised evolution. To test the optimality of the evolved solutions the evolved controller is merged back into a single network. On the grounds of the test results, it is argued that layered evolution provides a superior approach for many tasks, and future research projects involving this approach are suggested.
Applied to certain problems, neuroevolution frequently gets stuck in local optima with very low f... more Applied to certain problems, neuroevolution frequently gets stuck in local optima with very low fitness; in particular, this is true for some reinforcement learning problems where the input to the controller is a high-dimensional and/or ill-chosen state description. Evidently, some controller inputs are "poisonous", and their inclusion induce such local optima. Previously, we proposed the memetic climber, which evolves neural network topology and weights at different timescales, as a solution to this problem. In this paper, we further explore the memetic climber, and introduce its population-based counterpart: the memetic ES. We also explore which types of inputs are poisonous for two different reinforcement learning problems.
This paper presents a reliable and efficient approach to procedurally generating level maps based... more This paper presents a reliable and efficient approach to procedurally generating level maps based on the self-organization capabilities of cellular automata (CA). A simple CA-based algorithm is evaluated on an infinite cave game, generating playable and well-designed tunnel-based maps. The algorithm has very low computational cost, permitting realtime content generation, and the proposed map representation provides sufficient flexibility with respect to level design.
Computing Research Repository, 2006
This paper describes the evolution of controllers for racing a simulated radio-controlled car aro... more This paper describes the evolution of controllers for racing a simulated radio-controlled car around a track, modelled on a real physical track. Five different controller architectures were compared, based on neural networks, force fields and action sequences. The controllers use either egocentric (first person), Newtonian (third person) or no information about the state of the car (open-loop controller). The only controller that able to evolve good racing behaviour was based on a neural network acting on egocentric inputs.
Multi-objective optimisation is applied to encourage the effective use of state variables in car ... more Multi-objective optimisation is applied to encourage the effective use of state variables in car controlling programs evolved using Genetic Programming. Three different metrics for evaluating the use of state within a program are introduced. Comparisons are performed among multi-and single-objective fitness functions with respect to learning speed and final fitness of evolved individuals, and attempts are made at understanding whether there is a trade-off between good performance and stateful controllers in this problem domain.
Geometric particle swarm optimization (GPSO) is a recently introduced formal generalization of tr... more Geometric particle swarm optimization (GPSO) is a recently introduced formal generalization of traditional particle swarm optimization (PSO) that applies naturally to both continuous and combinatorial spaces. In this paper we apply GPSO to the space of genetic programs represented as expression trees, uniting the paradigms of genetic programming and particle swarm optimization. The result is a particle swarm flying through the space of genetic programs. We present initial experimental results for our new algorithm.
Geometric Particle Swarm Optimization (GPSO) is a recently introduced formal generalization of tr... more Geometric Particle Swarm Optimization (GPSO) is a recently introduced formal generalization of traditional Particle Swarm Optimization (PSO) that applies naturally to both continuous and combinatorial spaces. Differential Evolution (DE) is similar to PSO but it uses different equations governing the motion of the particles. This paper generalizes the DE algorithm to combinatorial search spaces extending its geometric interpretation to these spaces, analogously as what was done for the traditional PSO algorithm. Using this formal algorithm, Geometric Differential Evolution (GDE), we formally derive the specific GDE for the Hamming space associated with binary strings and present experimental results on a standard benchmark of problems.
The problem of how to acquire a model of a physical robot, which is fit for evolution of controll... more The problem of how to acquire a model of a physical robot, which is fit for evolution of controllers that can subsequently be used to control that robot, is considered in the context of racing a radio-controlled toy car around a randomised track. Several modelling techniques are compared, and the specific properties of the acquired models that influence the quality of the evolved controller are discussed. As we aim to minimise the amount of domain knowledge used, we further investigate the relation between the assumptions about the modelled system made by particular modelling techniques and the suitability of the acquired models as bases for controller evolution. We find that none of the models acquired is good enough on its own, and that a key to evolving robust behaviour is to evaluate controllers simultaneously on multiple models during evolution. Examples of successfully evolved racing control for the physical car are analysed.
We demonstrate an approach to modelling the effects of certain parameters of platform game levels... more We demonstrate an approach to modelling the effects of certain parameters of platform game levels on the players' experience of the game. A version of Super Mario Bros has been adapted for generation of parameterized levels, and experiments are conducted over the web to collect data on the relationship between level design parameters and aspects of player experience. These relationships have been learned using preference learning of neural networks. The acquired models will form the basis for artificial evolution of game levels that elicit desired player emotions.
Multi-population competitive co-evolution is explored as a way of developing controllers for a si... more Multi-population competitive co-evolution is explored as a way of developing controllers for a simple (but definitely not trivial) car racing game. The three main uses we see for this method are to evolve more complex general intelligence than would be possible with other methods, to compare different evolvable architectures for controllers, and to develop behaviourally diverse populations of agents for computer games. Nine-population co-evolution is compared with single-population co-evolution and standard evolution strategies, steady-state and generational versions of the algorithm are compared, and a number of different controller architectures are compared with each other.
Uploads
Papers by Julian Togelius