Sim2Real GAN for End-to-End robotic prosthesis control.
In collaboration with the Exoskeleton and Prosthetic Intelligent Controls lab at Georgia Tech.
Motion capture data collection, processing, and sensor simulation.
Biomechanical data only tracks position and ground reaction forces. What we needed were sensor signals: joint angles, angular velocities, and accelerations. Using OpenSim in junction with MATLAB, we used inverse kinematics and inverse dynamics to generate simulated sensor data from motion capture recordings. The best part: the same pipeline can be applied to able-bodied subjects too, turning recordings into learning material for the controller.
Early U-net performance
I have impemented a U-net TCN to map perfect, simulated sensor values to real, noisy sensor values. The model showed great performance on evaluations, but an important conceptual detail had to be taken into account. Deleting noise is a better approach than adding artificial noise, as information is deleted and not created. Thus, the model was retrained to map real signals into simulated signals.
Experiment
Before, the controller learned to walk like a robot, taught on data from other controllers. Now, it learned to walk like a human.