Shulou(Shulou.com)05/31 Report--

This article is about examples of bug problems caused by MySQL-JDBC drivers. The editor thinks it is very practical, so share it with you as a reference and follow the editor to have a look.

Problem background

The company does e-commerce system, and the whole system is built on Huawei cloud. When designing the system, considering the large number of follow-up users and orders, we need to use some components of large database. Relational database, considering the rapid growth of subsequent data, is not written directly to MySQL, but uses Huawei Cloud's distributed database middleware DDM. After using DDM, you can directly increase the number of MySQL read instances without business awareness, and linearly improve the read performance. It also supports sub-database and sub-table at the middleware level, providing the operation of a large number of relational databases. It is simply customized for e-commerce systems.

DDM itself provides services in the form of clusters, and multiple connected IP addresses are open to business. A layer of load balancing is required. If you use the traditional form of adding LB to do load balancing, there will be one more layer of transfer, resulting in performance loss. Therefore, the client load balancing capability provided by MySQL-JDBC is directly used.

The logical structure is shown in the following figure:

▲ services can access multiple DDM nodes through the Loadbalance of MySQL-JDBC. MySQL-JDBC provides load balancing capabilities.

Problem description

MySQL JDBC-driven client load balancing ability, has been running well, the performance is outrageous. But some time ago, the business request failed for no reason. I am in charge of the e-commerce order module, involving the real Money, this problem can scare the baby in a cold sweat.

So I quickly checked the backend log and found that there was an exception in accessing DDM. Without saying a word, I directly submitted a work order to Huawei Cloud DDM service.

I have to say, Huawei Cloud's service is still very good, less than half an hour a special staff contacted me, but also with me to troubleshoot the problem.

Take down the log of our business, analyze it with the support staff of DDM, and find that the error is as follows: the root cause is MySQL-driven bug, which leads to StackOverflow local stack overflow. It turned out that it was a murder caused by Bug, which misunderstood the DDM service. I'm really sorry.

As you can see from the stack, an exception triggers the bug of MySQL-JDBC, causing a circular call until the stack overflows. At the suggestion of Huawei DDM support staff, the driver code is decompiled, from the case of decompilation, we can see that there is indeed the possibility of loop nesting.

Loadbalance polling connection-> synchronizing the status of new and old connections-> sending sql to the server-> Loadbalance polling connection.

The related code is as follows:

With such an obvious bug, I don't believe that MySQL will not find out. At present, we are using the 5.1.44 driver. After looking at the latest 5.1.66 code, we found that this problem has indeed been fixed, as follows:

Loop nesting is avoided by filtering out SET and SHOW statements.

But 5.1.66 introduces a new bug, and since there is not a SQL in every place where postProcess is called, the code here will empty the pointer exception. Don't developers of MySQL JDBC do testing?

No way, after analyzing the code of 5.1.44 below, it is found that the occurrence of loop nesting can be avoided by properly adjusting the value of the parameter loadBalanceAutoCommitStatementThreshold. Our environment has been changed to 5, and after the revision, it has been running smoothly for a week, and there have been no more problems.

Modify the scheme

LoadBalanceAutoCommitStatementThreshold has been modified to 5, but the problem introduced is that if the business contains some time-consuming SQL, it may result in an imbalance in the load of the DDM. However, according to the current situation, the performance of DDM is still relatively strong.

Thank you for reading! This is the end of this article on "examples of bug problems caused by MySQL-JDBC drivers". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, you can share it for more people to see!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.