The Evolving Actors Project at Purdue Polytechnic
The evolving actors project at Purdue Polytechnic is exploring the use of online near-real time learning techniques to generate behavior for computer-controlled actors in games. These techniques are distinct from deep learning techniques in the following ways:
• They need no pre-existing data to train
• They are light weight and executable in real-time, scalable to massive numbers of actors simultaneously
• They adapt on the fly rather than needing extensive pre-training
See below for definitions and the Research Projects page for information on specific projects
Definitions
Actors in a videogame are software entities that the player of a game identifies as independent decision makers. They could be monsters, competing race cars, or even allied characters. The key thing that makes them actors is that they have a behavior based on the state of the world and the actions of the player. Evolving Actors are actors who learn and adapt their behavior according to their success or failure at achieving programmer defined goals.
Online learning algorithms are a class of machine learning (ML) algorithms that collect data and update their learned behavior models during their execution. This is distinct from offline learning algorithms where the AI is trained prior to its use in the application. We are focused on online learning algorithms because our goal is a continuously learning application.
Genetic Algorithms are a class of learning algorithms that search a parameter space for good parameter values in a manner analogous to (limited) biological evolution. Genetic algorithms are applicable to any parametrized decision mechanism and are often applied to fuzzy logic and artificial neural network based systems. The Evolving Actors project is an attempt to apply genetic algorithms to the control of actors such that they can learn dynamically within the environment and adapt to player driven changes in their environment.
Artificial Neural Networks (ANN) are a class of mathematical decision making techniques that are (again limitedly) based on the idea of interacting neurons. There are many kinds of ANN and approaches to their use but, at their core, they are all fundamentally a programmable mapping function that is programmed by a set of node weights. A given input sent to an ANN with a given design and weights, will return a repeatable set of outputs. As the mapping is controlled by the node weights, training a neural network consists of modifying its weights until it provides the desired mappings. Finding those weights is generally referred to as "training" the ANN.
Two major methods today for training neural networks are back-propagation and genetic algorithms. Back-propagation produces more exact solutions but requires a large database of inputs and the desired resulting outputs. Thus, it is most useful in offline learning where the target behavior is already known.
By contrast, genetic algorithms are experimental in the sense that the computer starts with a set of random experiments and then iterates in the spaces around and between them to find the best set of weights. All that is necessary for a genetic algorithm to learn is a "scoring" function that evaluates the success of that weight set. Such learning is referred to as "reinforcement learning" because random positive traits are encouraged and negative traits suppressed according to their success or failure at solving the posed problem. It is our hypothesis that the need for little to no starting data, and their iterative nature, will make them ideally suited to environment sensitive applications such as challenging a player. As the player learns different strategies, so too will the actor.
Fuzzy And Machines (FAM) are another decision making structure. The inter relationships of inputs to outputs are sepcified as fuzzy and-statements. Weights are then used to inform the machine of how much to consider each input statement in calculating the output. As FAM are also coded with weights, they too benefit from evolutionary algorithms. The major difference being that in the FAM the relationships between inputs and output are coded by the programmer whereas the ANN figures out its own relationships between the data. Reaching a usable solution therefor can often be faster with a FAM, but is limited to the solution strategy that was implemented as and-statements by the programmer.