Accuracy optimizing for fast inverse square root algorithm
-
Abstract
The fast inverse square root algorithm is a common method in computer numerical calculation. Based on the principle analysis of the fast inverse square root algorithm, a new algorithm constant is proposed to improve the statistical accuracy of the calculation results. The experimental results show that the algorithm constant provided by this paper can increase the average precision of batch data processing by about 30%~40% compared with the original algorithm constant. For the calculation of single data, the significance of the method in this paper is to obtain a more accurate result than the original method on a larger probability (average of about 77%).
-
-