Semantic SLAM building based on instance segmentation and optical flow in dynamic scenes

The visual simultaneous localization and mapping technique is commonly used for indoor intelligent robot navigation, but its poses are estimated with static environment in mind. In order to improve the robustness and real-time performance of visual Simultaneous Localization And Mapping(SLAM ) for localization and mapping in dynamic scenes, we add dynamic region detection threads and semantic point cloud threads to the original ORB-SLAM2. The dynamic region detection thread consists of the instance segmentation network and the optical flow estimation network. The instance segmentation gives semantic information to the dynamic scene while generating a priori dynamic object masks, and in order to solve the under-segmentation problem of the instance segmentation network, the lightweight optical flow estimation network is used to assist the detection of dynamic regions and generate dynamic region masks with higher accuracy. The generated dynamic region masks are passed into the tracking thread for real-time rejection of dynamic region feature points, and then the remaining static feature points in the map are used for the camera's positional estimation and to build a semantic point cloud map. Experimental results on the publicly available TUM dataset show that the improved SLAM system improves the robustness of its localization and map building in dynamic scenes while ensuring real-time performance.