Shulou(Shulou.com)06/02 Report--

What this article shares to you is about the solution to the problem of numerical calculation accuracy in the design of JavaScript strategy. The editor thinks it is very practical, so I share it with you to learn. I hope you can get something after reading this article.

When writing JavaScript strategy, there are some problems in scripting language itself, which often lead to the problem of numerical accuracy in calculation. It has a certain influence on some calculations and logical judgments in the program. For example, when calculating 1-0.8 or 0.33-1.1, the data with errors is calculated:

So how to solve this kind of problem? The root cause of the problem lies in: the highest progress of floating-point values is 17 decimal places, but the accuracy of the operation is far less than that of integers; integers are converted to decimal when performing operations; while decimal operations in Java and JavaScript will first convert decimal to the corresponding binary, part of the decimal can not be completely converted to binary, here is the first error. After all the decimals are converted to binary, the operation between binary is carried out to get the binary result. Then convert the binary result to decimal, where there is usually a second error.

In order to solve this problem, some solutions are searched on the Internet. after testing, the following solutions solve this problem well:

The principle is to convert the two operands to be calculated into integers to avoid the problem of accuracy, and restore the calculation results after calculation (according to the magnification when converted to integers) to get accurate results. In this way, when we want the program to hang up the order at the opening price plus the minimum price precision of one hop, we don't have to worry about the numerical accuracy.

The above is the solution to the problem of numerical accuracy in the design of JavaScript strategy. The editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.