Deep Learning-Driven State Estimation for Nonlinear Systems

ESP32 hardware setup

The first stage of the project was setting up the Kalman Filter on ESP32. After researching quaternion kinematics and learning how to apply a Kalman filter, I fused gyro with accelerometer readings, getting clean, smooth attitude estimation.

TCN architecture

Then I developed a compact, 4-layer, 4,274 parameter TCN to perform IMU-to-attitude mapping from ESKF-generated training data. The biggest challenge was the compromise between model size and accuracy, as the ESP32's strict memory constrains were of essence. My experience with the EPIC Lab implementing a large TCN proved to be useful in understanding the design choices to be made under constrains.

Deployment results

Finally, I quantized the model and deployed it using ESP-DL. One funny thing that happened at first is that the model found a shortcut: just take the nominal quaternion from the next frame and inject it as the corrected quaternion prediction for the current frame. This look-ahead bias was solved by making the model predict not the corrected quaternion, but the dq correction factor applied to the nominal quaternion.