Game Development Reference
In previous chapters we discussed how accelerometers have changed the way that people
interact with video games. The same sort of innovation is occurring with optical sensors.
Cameras, both in visual and infrared spectrums, are being used to generate input for
games. This chapter will focus on the Microsoft Kinect for Windows SDK and give an
overview of how to make a simple game that combines optical tracking with physics.
First we'll give a short introduction on the technologies these systems use to turn a
camera into a tracking device.
Without getting too detailed, we should start by discussing a few things about digital
cameras. First, most of us are familiar with the “megapixel” metric used to describe
digital cameras. This number is a measure of how many pixels of information the camera
records in a single frame. It is equal to the height of the frame in pixels multiplied by
the width of the frame in pixels. A pixel, or picture element, contains information on
intensity, color, and the location of the pixel relative to some origin. The amount of
information depends on the bits per pixel and corresponds to the amount of color
variation a particular pixel can display. Perhaps you've seen your graphics set to 16-bit
or 24-bit modes. This describes how many colors a particular pixel can display. A 24-
bit pixel can be one of 16.8 million different colors at any instant. It is commonly held
that the human eye can differentiate among about 10 million colors; 24-bit color is called
“true color,” as it can display more colors than your eye can recognize. You might also
see 32-bit color modes; these include an extra 8 bits for a transparency channel that tells
the computer what to do if this image were put on top of another image. This is some‐
times referred to as opacity or alpha .
Optical tracking and computer vision, in general, work by detecting patterns in this
wealth of pixel data. Pattern recognition is a mature field of computer science research.
The human brain is an excellent pattern recognizer. For instance, look at Figure 25-1 .
Most of us can't help but see a face in what is in reality a collection of three random