Abstract: Aiming at the problems of insufficient utilization of multi-scale information and losses of detailed features in the semantic segmentation model SegFormer, an improved lightweight semantic segmentation algorithm is proposed, and a novel decoder is designed to enhance multi-scale semantic feature representation. A novel bottleneck with spatial pyramid pooling is adopted to obtain more accurate multi-scale information; and a Laplacian Pyramid is used to obtain high-resolution detail features in the encoding stage, and it is applied to the decoding stage to solve the problem of loss of details. Finally, the features are progressively fused in to avoid the loss of details caused by the excessive upsampling rate, and greatly retain richer details to enhance the final semantic segmentation effect. The experimental results based on the ADE20K dataset show that using the improved decoder for semantic segmentation improves both the accuracy and the reduction of computation. Taking the experiment using the MiT-B0 encoder as an example, its mIoU index is 1.36% higher than that of the original network, and the amount of floating-point operations is only 51% of the original network. According to the above experimental results, the proposed model can improve the segmentation accuracy of the model without increasing a large amount of computational cost, and the amount of floating-point operations is reduced, which proves that the enhanced semantic segmentation model is better than the original model. It has better effect on multi-scale feature representation and boundary details of feature.