Abstract: Hand is a highly variable organ and hand features are easily affected by environmental factors.Considering the characteristics of hand gesture, a novel hand gesture recognition algorithm based on fusion contour moments is presented.First, According to the color cue the hand is available to extract from the complicated background, then the contour moment invariant and Fourier Descriptor are extracted and fused into a mixed feature, finally the mixed feature are put into BP network to identity.The experimental results show that the method proposed has better robustness and higher recognition rate.