Shulou(Shulou.com)06/03 Report--

preface

We all know that DDM is Huawei Cloud's excellent distributed database middleware, which is far ahead in terms of performance and ease of use. His maturity is not only reflected in the rapid horizontal smooth expansion, support for a variety of distributed things types and so on, but also reflected in many subtle aspects of DDM. Today, I share with you an insignificant subtle problem (small problem, big disaster, repeated in the history of IT industry, we have to alarm) that exists on mycat for many years, but does not exist in DDM. This problem is that after I played a lot of sql on DDM, I found that DDM is an immortal cockroach. Out of curiosity, I also put some sql played on DDM on mycat. Unfortunately, after the first sql was put on mycat, there was a magical scene. Let's take a look at my investigation process.

process of investigation

First of all, my test code is as follows. My sql seems to be ordinary except for a comment. However, as soon as the execution was carried out, it seemed to be stuck.

So quickly use jstack to view the thread stack information of the test application, as follows:

From the above figure, it is not difficult to find that it is stuck on line 25 of the test code TestJDBC, that is, stuck in the "ps.executeQuery()" line of code. Of course, it was also found that it was finally stuck in communication with mycat. So why doesn't mycat return results? I immediately went to the test machine where mycat was deployed, checked the logs, and found no abnormal information. So he looked at the resource usage of the test machine and found something unexpected, as follows:

I found a java process with very high cpu usage. In this case, it must be mycat process, because mycat is the only java application I deployed on this machine. One wonders what mycat is doing, and which thread has the problem. So I executed a command like ps -mp 3403 -o THREAD,tid,time to see what happened. The 3403 in the command is the id of the mycat process, so I found the following information.

It is found that thread 3420 in the 3403 process consumes a very high CPU. It is not difficult to think of using the jstack command to call up the thread stack information of the mycat process. Before executing jstack, we first convert the thread id(nid) 3420 to a hexadecimal number, because the nid (native thread id) seen in jstack is a hexadecimal display. The conversion method is as follows:

Well, let's take a look at the stack snapshot of the mycat process for that thread, as follows:

The original problem is in DruidMycatRouteStrategy class line 724, so take a look at mycat source code, as follows:

After seeing the code, is it enlightened? The problem lies in the processing logic of this while loop. Of course, the bug fix is not complicated. Friends who are interested in solving this problem can leave a message under this article, and we can communicate.

conclusion

Through such a problem above, it is not difficult to find that the distributed database middleware in the industry can basically be aligned on large features, so why Huawei cloud distributed database middleware DDM, as a rising star, appears to be more mature and leaves a better impression on users? I think there may be a reason why DDM has done a very good job in many subtle aspects like this.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.