robotics//navigation//visual-inertial odometry

Visual-inertial odometry (VIO) is the estimation of a vehicle's motion by fusing a camera with an IMU, and it is how drones hold position indoors without GPS and how phones keep virtual objects pinned to a table in augmented reality. The two sensors cover each other's weaknesses. The IMU is fast and smooth but drifts within seconds (dead reckoning); the camera sees features of the world that do not move, which bound the drift, but a single camera cannot tell size (a room or a dollhouse look the same) and loses track in fast motion. The accelerometer supplies the metric scale and the direction of gravity, the camera supplies the slow, drift-free anchoring, and a filter or a small sliding-window optimization ties them together (extended Kalman filter, nonlinear least squares).

Visual-inertial odometry (VIO) is the estimation of a vehicle's motion by fusing a camera with an IMU, and it is how drones hold position indoors without GPS and how phones keep virtual objects pinned to a table in augmented reality. The two sensors cover each other's weaknesses. The IMU is fast and smooth but drifts within seconds (dead reckoning); the camera sees features of the world that do not move, which bound the drift, but a single camera cannot tell size (a room or a dollhouse look the same) and loses track in fast motion. The accelerometer supplies the metric scale and the direction of gravity, the camera supplies the slow, drift-free anchoring, and a filter or a small sliding-window optimization ties them together (extended Kalman filter, nonlinear least squares).

Even perfectly fused, VIO has four directions it can never see: the absolute position (three) and the rotation about gravity (one). It knows how far it has travelled and which way is down, and it cannot know where it started or which way was north. Those four drift freely, slowly, and only an outside reference (GNSS, a magnetometer, a known map, a recognised place) pins them (observability).

Timing is half the calibration. A camera frame matched with the IMU sample of 10 ms earlier puts a fast turn in the wrong place, so the time offset between the two sensors is estimated as carefully as their relative pose, and a hardware trigger is better still (time synchronization, sensor calibration).

It fails where the camera fails: low texture (a white wall, open sky, water), darkness, motion blur and the skew of a camera with a rolling shutter. A well-built system notices the camera degrading and falls back on the IMU for the seconds it can bridge.

It is odometry, so it forgets: it keeps no lasting map and never recognises a place it has seen before. Adding that recognition and a persistent map turns it into visual-inertial SLAM, which is industry in augmented reality and drones, while maps that last for years in changing places are still research.