Abstract: In the process of crowd counting based on the amplitude of channel state information(CSI), poor robustness of classification model remains. To resolve the above problem, an incremental learning algorithm based on SVM for the phase difference expanded matrix of CSI between different receiving antennas is proposed. Firstly, triple processes are performed on the CSI raw phase data so that the environment interferences and phase errors can be eliminated effectively. Then, a phase difference expansion matrix is established, which adds dynamic characteristics of different number of people to improve the accuracy of crowd counting. In addition, after the new scenario is added, the original training data and the new data are retrained. Massive training data will cause the high computational overhead. To solve this problem, we propose an incremental learning classification algorithm based on SVM algorithm with designing an iterative loop process, which realizes the function of online learning of incremental data, and it has a good effect on the accuracy improvement of crowd counting and the computational complexity reduction. Experimental results show that this proposed model can realize crowd counting in real-time and achieve over 95% counting accuracy with only a maximum of 1-count-error, and the training time for the people classifier model is greatly reduced with more scenes.