Enabling the real-time capture and reproduction of human motion in 3D
The 3D Content Component (3DCC) supports experimenters in acquiring and manipulating 3D information from depth sensing devices (e.g. the Kinect). Various types of information can be acquired: raw depth images, human skeleton, as well as low resolution (640 x 480 pixels) RGB images registered to the depth images. Moreover the 3DCC can provide functionalities to manipulate this information to suit the experimenters’ particular needs. Finally, avatar editing and avatar interactive motion is provided by the 3DCC.In more detail, 3DCC comprise the following functionalities which are partitioned in three layers (low, mid and high):
Low Level Functionalities
Depth Acquisition: This provides the experimenter with the raw depth information of a scene. It is the middle layer between the Kinect device and the experimenter. Simple and easy to use functions provide the experimenter with per pixel information about the depth of the scene.
Skeleton Acquisition: In the case where humans are involved in a scene, 3DCC can provide robust skeleton extraction for up to 15 joints. Moreover, skeleton tracking can be performed for human motion analysis.
RGB Acquisition: Images coming from Kinect, along with their registration to depth pixel transformation, can be provided to the experimenter. This is important, since texturing of a post produced 3D model can be made possible through this information.
Mid Level Functionalities
Depth Enhancement: Since raw depth data is noisy we provide several filtering algorithm to smooth and de-noise the raw information so that more accurate depth measurement can be made possible.
Skeleton Enhancement: Jerky (noisy) skeleton joints are detected and tracked and therefore corrected through a sophisticated tailored filtering framework to provide a more realistic skeleton.
Biomechanical Measurements: The 3DCC can provide several biomechanical measurements that are inferred from both depth and skeleton information. The most important being: angles between bones, human joints and calibrated objects’ velocities, human body parts surface areas and calibrated objects’ surface area. These measurements can be used by the experimenters in a multitude of ways to infer high level information that suits their needs.
High Level Functionalities
Avatar Creation: The 3DCC provides an avatar authoring tool so that experimenters can create their own avatars that can be easily integrated into a virtual world. Other than a simple database of several features that can create artificial avatars, the 3DCC avatar creation tool can provide custom authoring capabilities that provide functionalities such as avatar personalization (so that the user’s facial image can appear on the avatar).
Avatar Motion: The 3DCC can also interactively move the avatar using a Kinect alone. This functionality can be used from the experimenter in a multitude of ways to animate his avatar and interact into a virtual world.