Game Development Reference
shapes. Our brains are so primed to recognize the basic pattern of a human face that we
can do it even when we don't want to!
Figure 25-1. Four unrelated geometric entities
Computers, on the other hand, have a harder time looking at two circles and a few lines
and saying, “Hey, this is a smiling face.”
Sensors and SDKs
The modern interest in computer vision as a consumer input for computer games has
led to the development of several SDKs for performing computer-vision pattern rec‐
ognition. One such system is Kinect for Windows. Although Microsoft provides a very
high-level API with the Kinect, the downside is that you are locked into its hardware.
The popular open source alternative is OpenCV, a library of computer-vision algo‐
rithms. Its advantage is that it can use a wide variety of camera hardware and not just
the Kinect sensor.
The Kinect was originally developed for the Xbox 360 but has recently been rebranded
to include Kinect for Windows. As console game design has high entrance requirements,
the Kinect for Windows allows more casual developers to try their hand at creating
games with optical input. The system has a hardware component, called the Kinect
sensor, and the previously mentioned Kinect SDK that does a lot of the heavy lifting for
us in terms of pattern recognition and depth sensing. The hardware component consists
of an infrared projector, infrared camera, visible light camera, and an array of micro‐
phones. The two cameras and the projector form the basis of the optical tracking system.
The projector sends out infrared light that is invisible to humans. This light bounces off
objects and is reflected back to the Kinect. The infrared camera records the reflected
light pattern, and based on how it has been distorted, calculates how far the object is
from the sensor. This exact method is carried out in the hardware of the sensor and is
proprietary. However, the patent applications reveal that a special lens projects circles
that, when reflected, become ellipses of varying shapes. The shape of the ellipse depends
on the depth of the object reflecting it. In general, this is a much-improved version of