A Low-Cost Motion Capture System using Synchronized Azure Kinect Systems (https://andyj1.github.io/kinect)
by Andy Jeong, Yue Wang, Professor Mili Shah (Advisor)
Abstract
Body joint estimation from a single vision system poses limitations in the cases of occlusion and illumination changes, while current motion capture (MOCAP) systems may be expensive. This synchronized Azure Kinect-based MOCAP system serves as a remedy to both of these problems by creating a low-cost, portable, accurate body-tracking system.
Keywords: Motion capture (MOCAP) system, synchronization, Kinect, body-tracking
See Azure.com/Kinect for device info and available documentation.
Link to submitted poster to ACM SIGGRAPH’20: Poster
Link to submitted abstract to ACM SIGGRAPH’20: Abstract
(Received 4 feedback; 3 neutral, 1 slight negative)
Link to documentation: Documentation
Check out the outcomes on some various movements! Note: this demo experiences some offset due to a parallax problem (devices are at a lower height than the human).
Flowchart
-lk4a
)-lk4abt
)`pkg-config --cflags --libs opencv`
)g++ file.cpp -lk4a -lk4abt `pkg-config --cflags --libs opencv` -o program // compile
./program // execute
// running on a single device
make one && make onerun
// running on two synchronized devices
sudo bash ./script/increaseusbmb.sh // change USB memory bandwidth size
make two && make tworun
// running on three synchronized devices
sudo bash ./script/increaseusbmb.sh // change USB memory bandwidth size
make sync && make syncrun
Configuration
Testing Environment
Camera Calibration to capture synchronous images
- reference: green screen example from Azure Kinect SDK examples on its GitHub repository
With multiple devices in place, joint estimation is still performed as if there is no occlusion or lighting effect. The following videos and images are tested in the test setup shown above.
Videos Samples
2-Device 3 -Device Systems
- on the right: joint angles for angles designated as below
Occlusion at Subordinate Device 0 Occlusion at Subordinate Device 1 Varying Illumination at Master Device
Example of selection of data streams by confidence levels per joint
Azure Kinect SDK is a cross platform (Linux and Windows) user mode SDK to read data from your Azure Kinect device.
The Azure Kinect SDK enables you to get the most out of your Azure Kinect camera. Features include:
OpenPose, AlphPose, Kinect, Vicon MOCAP system
Media art collaboration
Control of drone system (crazyflie)