Neuroevolution for General Video Game Playing

Machine Learning, Optimization, and Big Data, 2016

The proposed model represents an original approach to general game playing, and aims at creating a player able to develop a strategy using as few requirements as possible, in order to achieve the maximum generality. The main idea is to modify the known minimax search algorithm removing its task-specific component, namely the heuristic function: this is replaced by a neural network trained to evaluate the game states using results from previous simulated matches. A method for simulating matches and extracting training examples from them is also proposed, completing the automatic procedure for the setup and improvement of the model. Part of the algorithm for extracting training examples is the Backward Iterative Deepening Search, a new original search algorithm which aims at finding, in a limited time, a high number of leaves along with their common ancestors.

2017 IEEE Conference on Computational Intelligence and Games (CIG), 2017

In game artificial intelligence (AI), two common directions for developing non-human computer players are strong AI and human-like AI. Human-like AI aims at making computer agents behave like humans. In this direction, NeuroEvolution (NE), which is a combination of an artificial neural network (ANN) and an evolutionary algorithm (EA), had been frequently used to a make computer agent to behave like a human. Our research introduces a novel approach to create human-like computer agents in a platform game Super Mario Bros. (SMB)-we called it a 2D action game in this research. The approach utilizes statistical penalties to evaluate candidates created by NE algorithm. The penalties help in reducing mechanical actions of computer agents based on human data statistics, and the effects of statistical penalties are analyzed by asking human subjects to rate the human-likeness of agents. Experiments show that our method improves the human-likeness in the behavior of a computer agent.

Proceedings of the 11th annual conference companion on Genetic and evolutionary computation conference - GECCO '09, 2009

We apply CMA-ES, an evolution strategy with covariance matrix adaptation, and TDL (Temporal Difference Learning) to reinforcement learning tasks. In both cases these algorithms seek to optimize a neural network which provides the policy for playing a simple game (TicTacToe). Our contribution is to study the effect of varying learning conditions on learning speed and quality. Certain initial failures with wrong fitness functions lead to the development of new fitness functions, which allow fast learning. These new fitness functions in combination with CMA-ES reduce the number of required games needed for training to the same order of magnitude as TDL.

Conference of the Spanish Association for Videogames Sciences, 2016

Simulating human behaviour when playing computer games has been recently proposed as a challenge for researchers on Artificial Intelligence. As part of our exploration of approaches that perform well both in terms of the instrumental similarity measure and the phenomenological evaluation by human spectators, we are developing virtual players using Neuroevolution. This is a form of Machine Learning that employs Evolutionary Algorithms to train Artificial Neural Networks by considering the weights of these networks as chromosomes for the application of genetic algorithms. Results strongly depend on the fitness function that is used, which tries to characterise the human-likeness of a machinecontrolled player. Designing this function is complex and it must be implemented for specific games. In this work we use the classic game Ms. Pac-Man as an scenario for comparing two different methodologies. The first one uses raw data extracted directly from human traces, i.e. the set of movements executed by a real player and their corresponding time stamps. The second methodology adds more elaborated game-level parameters as the final score, the average distance to the closest ghost, and the number of changes in the player's route. We assess the importance of these features for imitating human-like play, aiming to obtain findings that would be useful for many other games.

Neural Computing and Applications, 2020

Real-time strategy (RTS) games differ as they persist in varying scenarios and states. These games enable an integrated correspondence of non-player characters (NPCs) to appear as an autodidact in a dynamic environment, thereby resulting in a combined attack of NPCs on human-controlled character (HCC) with maximal damage. This research aims to empower NPCs with intelligent traits. Therefore, we instigate an assortment of ant colony optimization (ACO) with genetic algorithm (GA)-based approach to first-person shooter (FPS) game, i.e., Zombies Redemption (ZR). Eminent NPCs with bestfit genes are elected to spawn NPCs over generations and game levels as yielded by GA. Moreover, NPCs empower ACO to elect an optimal path with diverse incentives and less likelihood of getting shot. The proposed technique ZR is novel as it integrates ACO and GA in FPS games where NPC will use ACO to exploit and optimize its current strategy. GA will be used to share and explore strategy among NPCs. Moreover, it involves an elaboration of the mechanism of evolution through parameter utilization and updation over the generations. ZR is played by 450 players with varying levels having the evolving traits of NPCs and environmental constraints in order to accumulate experimental results. Results revealed improvement in NPCs performance as the game proceeds.

Artificial Intelligence Review, 2008

Artificial intelligence for digital games constitutes the implementation of a set of algorithms and techniques from both traditional and modern artificial intelligence in order to provide solutions to a range of game dependent problems. However, the majority of current approaches lead to predefined, static and predictable game agent responses, with no ability to adjust during game-play to the behaviour or playing style of the player. Machine learning techniques provide a way to improve the behavioural dynamics of computer controlled game agents by facilitating the automated generation and selection of behaviours, thus enhancing the capabilities of digital game artificial intelligence and providing the opportunity to create more engaging and entertaining game-play experiences. This paper provides a survey of the current state of academic machine learning research for digital game environments, with respect to the use of techniques from neural networks, evolutionary computation and reinforcement learning for game agent control.

2020 IEEE Conference on Games (CoG), 2020

The General Video Game Artificial Intelligence (GVGAI) competition has been running for several years with various tracks. This paper focuses on the challenge of the GVGAI learning track in which 3 games are selected and 2 levels are given for training, while 3 hidden levels are left for evaluation. This setup poses a difficult challenge for current Reinforcement Learning (RL) algorithms, as they typically require much more data. This work investigates 3 versions of the Advantage Actor-Critic (A2C) algorithm trained on a maximum of 2 levels from the available 5 from the GVGAI framework and compares their performance on all levels. The selected subset of games have different characteristics, like stochasticity, reward distribution and objectives. We found that stochasticity improves the generalisation, but too much can cause the algorithms to fail to learn the training levels. The quality of the training levels also matters, different sets of training levels can boost generalisation over all levels. In the GVGAI competition agents are scored based on their win rates and then their scores achieved in the games. We found that solely using the rewards provided by the game might not encourage winning.

2020

The General Video Game Artificial Intelligence (GVGAI) competition has been running for several years with various tracks. This paper focuses on the challenge of the GVGAI learning track in which 3 games are selected and 2 levels are given for training, while 3 hidden levels are left for evaluation. This setup poses a difficult challenge for current Reinforcement Learning (RL) algorithms, as they typically require much more data. This work investigates 3 versions of the Advantage Actor-Critic (A2C) algorithm trained on a maximum of 2 levels from the available 5 from the GVGAI framework and compares their performance on all levels. The selected subset of games have different characteristics, like stochasticity, reward distribution and objectives. We found that stochasticity improves the generalisation, but too much can cause the algorithms to fail to learn the training levels. The quality of the training levels also matters, different sets of training levels can boost generalisation over all levels. In the GVGAI competition agents are scored based on their win rates and then their scores achieved in the games. We found that solely using the rewards provided by the game might not encourage winning.

—This paper presents a way to evolve a non-playing character by controlling it through a neural network, then using a genetic algorithm to alter the weights. Previous neural network data provides a population from which to select individuals, and the simulated results allow relative rankings that we use as a fitness function. Treating the selected sets of neural network weights as a binary sequence, the program uses cross-over and mutation to create a new set of weights, which are subsequently evaluated. Through this process, we find that the program creates better sets of neural network weights, and that the relative rankings allow the computer-controlled character to have a range of difficulty levels. We implement this experiment as part of a game.

An artificial neural network is a system that tries in various degrees to emulate a human brain in order to perform tasks that other computer systems are usually not fit to handle. Artificial neural networks are used in many different areas due to their ability to learn and adapt to many different tasks and make complex predictions. In gaming, computer controlled opponent behavior is usually rule based and dependent on specific conditions and can thus be predictable to a certain degree. As the field of AI and learning systems using artificial neural networks is being developed and expanded, it is inevitable that its use in gaming will be explored thoroughly. This short survey looks at the attempts of using artificial neural networks for opponents in board games and modern computer games, as well as other uses in gaming throughout the last 20 years.

IEEE Transactions on Systems, Man, and Cybernetics, 2007

We have recently shown that genetically programming game players, after having imbued the evolutionary process with human intelligence, produces human-competitive strategies for three games: backgammon, chess endgames, and robocode (tank-fight simulation). Evolved game players are able to hold their own-and often win-against human or human-based competitors. This paper has a twofold objective: first, to review our recent results of applying genetic programming in the domain of games; second, to formulate the merits of genetic programming in acting as a tool for developing strategies in general, and to discuss the possible design of a strategizing machine.

2004

Neural Networks, 2002. …, 2002

2009

1 Abstract In our earlier research, we looked into the need for and use of AI in video games. Our survey on the existing literature on game artificial intelligence and our hands-on experience with some of the games which were developed through 1990s up to today have shown that the Artificial Intelligence in commercially available video games has made significant progress over the decades, but one area which commercial games have largely ignored is the use of learning AI. Meanwhile, game artificial intelligence research continues to look into and create examples of using such artificial intelligence techniques, e.g. reinforcement learning, evolutionary algorithms, in academic games. At the moment these techniques are largely employed only by game artificial intelligence research; however, considering that game environments in commercial games are becoming more dynamic and unpredictable, one would think that these techniques will be more capable of handling such environments and as su...

2007 IEEE Symposium on Computational Intelligence and Games, 2007

This paper describes the EvoTanks research project, a continuing attempt to develop strong AI players for a primitive 'Combat' style video game using evolutionary computational methods with artificial neural networks. A small but challenging feat due to the necessity for agent's actions to rely heavily on opponent behaviour. Previous investigation has shown the agents are capable of developing high performance behaviours by evolving against scripted opponents; however these are local to the trained opponent. The focus of this paper shows results from the use of coevolution on the same population. Results show agents no longer succumb to trappings of local maxima within the search space and are capable of converging on high fitness behaviours local to their population without the use of scripted opponents.