ZHOU Quan, YANG Liang, HE Wei-qiang. Accuracy optimizing for fast inverse square root algorithm[J]. Microelectronics & Computer, 2019, 36(5): 64-69.
Citation: ZHOU Quan, YANG Liang, HE Wei-qiang. Accuracy optimizing for fast inverse square root algorithm[J]. Microelectronics & Computer, 2019, 36(5): 64-69.

Accuracy optimizing for fast inverse square root algorithm

  • The fast inverse square root algorithm is a common method in computer numerical calculation. Based on the principle analysis of the fast inverse square root algorithm, a new algorithm constant is proposed to improve the statistical accuracy of the calculation results. The experimental results show that the algorithm constant provided by this paper can increase the average precision of batch data processing by about 30%~40% compared with the original algorithm constant. For the calculation of single data, the significance of the method in this paper is to obtain a more accurate result than the original method on a larger probability (average of about 77%).
  • loading

Catalog

    Turn off MathJax
    Article Contents

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return