DeepUKF-VIN: Adaptively-Tuned Deep Learning Unscented Kalman Filter for 3D Visual-Inertial Navigation Based on IMU-Vision-Net

Abstract

Three-dimensional visual-inertial navigation requires reliable fusion of camera and IMU data under nonlinear dynamics and challenging visual conditions. This work introduces DeepUKF-VIN, which couples an adaptively tuned deep unscented Kalman filter with IMU-Vision-Net—a learning-based module that links inertial and visual information to drive filter adaptation. The approach retains the uncertainty-aware structure of the unscented Kalman filter while using deep learning where it improves robustness and accuracy. The method is evaluated on visual-inertial benchmarks against classical filtering and learning-based alternatives, demonstrating strong navigation performance.

For a walkthrough of the method and results, watch the embedded YouTube video on this page. Use the links below for the full journal article (Expert Systems with Applications) and for the PyTorch reproduction code in the DeepUKF-VIN repository on GitHub.