Interactive manipulation of 3D objects using Kinect for visualization tools in education

초록

This study develops and implements a Kinect-based 3D gesture recognition system for interactive manipulation of 3D objects in educational visualization softwares. The developed system detects and tracks human hands from the RGBD images captured by a Kinect sensor and recognizes human gestures by counting the number of open fingers of each fist and tracking 3D motion of both hands. The status of fists and gestures of hands are recognized as control commands of manipulating 3D structures visualized in 2D monitor by Molecule Viewer. The developed system is implemented on a Windows 7 laptop PC using C# and Emgu CV 2.3.0 library, and tested in ordinary classroom environment. Its performance demonstrates the overall average accuracy of around 90% in recognizing status of hands and gesture commands under various ambient lighting conditions.