Game Development Reference
In-Depth Information
Kinect Programming
with Direct3D 11
Jason Zink
The Microsoft Kinect is a sensor peripheral originally released for use with the
Xbox 360 and later on the PC. The sensor includes a variety of different inputs,
including a microphone array, color image acquisition, and a special depth image-
acquisition system. With these diverse inputs, Microsoft has developed a number
of algorithms that can be used to sense and track the position, pose, and voice
status of one or more users—which can subsequently be used by a game or ap-
plication as input. This allows for a number of new ways for a user to interact
with their computers—instead of using a gamepad, the user interacts with the
application in a very natural way.
Applications that wish to utilize the Kinect and the data streams that it
produces can do so with the Kinect for Windows software development kit (SDK)
[Microsoft 12]. To properly obtain and interpret these data streams, a developer
must understand the mechanics of how the device operates and also have a clear
understanding of what can and can't be done with it. This article seeks to provide
the theoretical underpinnings needed to use the visual and skeletal data streams
of the Kinect, and it also provides practical methods for processing and using
this data with the Direct3D 11 API. In addition, we will explore how this data
can be used in real-time rendering scenarios to provide novel interaction systems.
Meet the Kinect
We will begin our investigation of the Kinect by examining how its camera systems
function. The camera system is essentially composed of a color (RGB) camera,
an infrared (IR) projector, and an IR camera. These three elements are shown
in Figure 2.1.
These sensors, when used together, provide a form of a vision
Search Nedrilad ::

Custom Search