Game Development Reference
depth from focus , in which the computer assumes that blurry objects are farther away
than objects in focus.
As for object detection, the Kinect comes with a great set of algorithms for skeleton
direction. It can also be trained to detect other objects, but skeleton detection is really
its forte. The skeleton detection is good because of the massive amount of training
Microsoft used for the algorithms when creating the SDK. If you were to use an average
computer to run the Kinect skeleton training program, it would take about three years.
Luckily, Microsoft had 1,000 computers lying around, so it takes them only a day to run
the training simulation. This gives you an idea of the amount of training you need to
provide for consumer-level tracking in your own algorithms. The Kinect can track up
to six people with two of them being in “active” mode. For these 2 people, 20 individual
joints are tracked. The sensor can also track people while standing or sitting.
The OpenCV method for 3D reconstruction is, well, more open! The library is designed
to work with any common webcam or other camera you can get connected to your
computer. OpenCV works well with stereoscopic cameras and is also capable of at‐
tempting to map depth with a single camera. However, those results would not be ac‐
curate enough to control a game, so we suggest you stick with two cameras if you're
trying to use regular webcams.
Indeed, finding depth is relatively straightforward using OpenCV. The built-in function
ReprojectImageTo3D calculates a vector for each pixel ( x , y ) based on a disparity map .
A disparity map is a data set that describes how pixels have changed from one image to
the next; if you have stereoscopic cameras, this essentially is the reverse of the technique
we use in Chapter 24 when dealing with 3D displays. To create a disparity map, OpenCV
provides the handy function FindStereoCorrespondenceGC() . This function takes a
set of images, assumes them to be from a sterescopic source, and generates a disparity
map by systematically comparing them. The documentation is very complete, and there
are several topics on the subject of OpenCV, including Learning OpenCV by Gary Brad‐
ski and Adrian Kaehler (O'Reilly), so we again will save the details for independent
Object detection is also possible with OpenCV. The common example in the OpenCV
project uses Harr-like features to recognize objects. These features are rectangles whose
mathematical structure allows for very fast computation. By developing patterns of these
rectangles for a given object, a program can detect objects out of the background. For
example, one such pattern could be if a selection rectangle includes an edge. The pro‐
gram would detect an edge in the pixel data by finding a sharp gradient between color
and/or other attributes. If you detect the right number of edges in the right position,
you have detected your object.