Visual gaze target tracking method based on spatiotemporal attention mechanism and joint attention

The current visual tracking technology tends to ignore the connection between the figure and the scene graph, as well as the lack of analysis and detection of joint attention, which results in unsatisfactory detection performance. In response to these problems, this paper proposed a visual gaze target tracking method based on spatiotemporal attention mechanism and joint attention. For any given image, the method extracts the head features of a person by using a deep neural network, and then adds extra-interaction between the scene and the head to enhance the saliency of images. Lots of interference information on the depth and field of view can be filtered out by the enhanced attention module. In addition, the attention of the remaining characters in the scene is considered into the area of interest to improve the standard saliency model. After adding the spatiotemporal attention mechanism, the candidate target, target gaze direction and time frame number constraints can be effectively combined to identify the shared location, and the saliency information can be used to detect and locate joint attention better. Finally, the image is visualized as a heat map. Experiments show that the model can effectively infer dynamic attention and joint attention in videos with good results.