DeepEVO: Towards Robust Visual Odometry via Attention-Based Events and Video Fusion

Visual odometry (VO) is one of the fundamental tasks for vision-based navigation on mobile robots. It estimates the position and rotation of the camera by investigating the translation between consecutive video frames or other visual signals. However, traditional video-based odometry suffers from the issues of blur and overexposure under fast movement or challenging light conditions, which weaken the robustness of the visual odometry approach. Event-based cameras are new bio-inspired sensors that capture changes in image intensity asynchronously across all pixels on the camera, which introduce better performance in capturing scenes of fast-moving and high dynamic range over the standard cameras. These advantages allow event cameras to complement standard RGB cameras for robust visual odometry. In this paper, We propose an end-to-end learning approach, DeepEVO, which adopts a mixed attention module to fuse events and video for robust motion estimation. The video cameras provide rich semantic information while the events complement the visual odometry on solving the problem of motion blur and lighting conditions. Through extensive experiments, the results show that the DeepEVO can greatly improve the pose estimation accuracy compared with the approaches that only use a standard camera in a variety of scenarios by automatically adopting dominant information from different sources.