Abstract: The results of most feature selection based on greedy strategies are suboptimal solutions. So, a two-stage dimension reduction method is proposed in this paper.A conditional product Kullback-Leibler divergence algorithm for feature selection is designed to get the feature subset at the first stage and then the adaptive Local Linear Embedding (ALLE) for feature extraction in the feature subset is given out. Experimental results show the novel method can significantly reduce the dimension of features and improve performance of text mining.