Game Development Reference
In-Depth Information
Reinforcement learning, by contrast, emphasizes learning by interacting with the
environment. The system starts with some basic knowledge and the need to
maximize a reward of some kind. In operation, the system must balance between
exploitation and exploration. At any given point in time, the system might pick
the action it thinks is best, exploiting the knowledge it already has. Or it can pick
a different action to better explore the possibilities. Reinforcement learning uses
exploration to improve future decision making by making changes to the basic
knowledge the system starts with.
A good introduction to reinforcement learning is given in the first chapter of
[Sutton98], which is available in its entirety online. Advanced readers will want
to study the method of temporal differences (TD); TD(l) is of particular interest.
Reinforcement learning has produced a few notable success stories; in com-
mercial video games, Black & White used reinforcement learning. TD-Gammon
began as a research project at IBM's Watson Research Center using TD(l). It
plays backgammon at slightly below the skill level of the top human players, and
some of its plays have proven superior to the prior conventional wisdom
[Tesauro95]. Reinforcement learning creeps into games in small ways as well
Why Don't These Methods Get Used in Games?
Many experienced game AI programmers attribute characteristics of undead
vampires to neural networks and genetic algorithms; they are enticing, parti-
cularly to the na¨ve. They keep returning, at least as a topic of conversation,
despite heroic efforts to lay them to rest. They cannot be trusted to do what you
want them to do, and it takes arcane knowledge and skills to get them to do
anything at all. At the end of the project, there is great fear that they will turn to
dust should the game ever see the light of day. All kidding aside, what are the real
There are three basic concerns: control over the AI, lack of past history of
achieving a successful AI with these methods in games, and control over the
Many game designers demand arbitrary control over what the game AI does. The
smarter the AI gets, the more it wants to think for itself. In and of itself, that is not
a problem; the problem comes when a designer wants to override the AI, usually
for dramatic impact or playability reasons. As we shall see, neural networks
effectively are a black box. They might as well carry the standard warning label,
Search Nedrilad ::

Custom Search