Shulou(Shulou.com)05/31 Report--

This article mainly introduces "what are the ways to make the micro-service in the production environment more fluent". In the daily operation, I believe that many people have doubts about the way to make the micro-service in the production environment more fluent. I have consulted all kinds of materials and sorted out the simple and easy-to-use operation methods. I hope it will be helpful for you to answer the doubts about "what are the ways to make the micro-services in the production environment more fluent?" Next, please follow the editor to study!

The other side of microservices

Separation of concerns is not a new concept, nor is distributed computing. The advantages are obvious, but their price is usually higher operation and maintenance costs in terms of time and money. Mix the two and you will encounter all types of problems. If you put it in the production environment, the problem will be quadrupled. Debug and fix, but wait a minute--

Debugging does not make the problem disappear.

As Bryan Cantrill pointed out in his QCon speech, "debugging has degenerated into an oral legend, hoping that the problem will disappear mythically." In fact, debugging is more like a science, understanding how the system works than how we think it works.

Debugging is not just a trivial marginal task, but a fundamental problem. Sir Maurice Wilkes, the debugger who created one of the first programs, realized that debugging would be the primary responsibility of the developer.

"in 1949, when we started programming, we were surprised to find that it was not easy to make the program work as we thought. We have to find a way to debug. I can still remember the moment when I realized that a large part of my life would be spent looking for errors in the programs I wrote from now on."

We used to think it was to solve the problem. But the real challenge is to understand how the system works.

Question # 1: if monitoring monomer applications is not difficult enough

Whether you are gradually breaking down a single application into micro-services or building a new system from scratch, you now have more services to monitor. Every one of them is possible:

Use different technologies / languages

In different machines / containers

Have your own version.

The key point is intelligent monitoring, the system is highly fragmented, and there is an urgent need for centralized monitoring and logging to understand what is going on.

For example, in a scenario described in the recent ongoing discussion of podcasts, there is a poor version that needs to be rolled back. This is the most direct way of monomer application. But-- now we have microservices. So you need to determine which service needs to be rolled back, what impact such a rollback will have on other services, or it may just need to add some functionality, but it may simply push the problem to another service.

Point # 1: if you think it's difficult to monitor a single architecture, microservices will be 10 times harder and more investment needs to be planned in advance.

Question # 2: logs are distributed among services

Log. The server produces several GB of unstructured text every day. The equivalent of IT and carbon dioxide emissions are overflowed hard drives and crazy Splunk bills / ELK storage charges. In addition, if you want to learn Splunk or ELK, take a look at our latest ebook, "Splunk vs ELK: decision guidance for Log Management tools."

Under the monolithic architecture, the log is likely to be scattered in different places, so you have to use several layers of design in which the log is recorded in different places. In microservices-logs are more fragmented. Now when studying some scenarios related to user transactions, you have to collect all the different logs from all the services that may be used to understand what went wrong.

In Takipi, our team solves this problem by using Takipi. For all log errors or alarms from the production JVM, we have injected links into the log to guide event analysis. Includes the complete stack trace and variable states for each frame, even if they are distributed over a certain number of services / machines.

Point # 2: microservices refer to breaking things down into individual components. The side effect of this is that operations and monitoring is broken down into each service and loses the power of the system as a whole. The challenge here is to re-centralize these with the right tools.

Question # 3: problems caused by one service can cause problems elsewhere

If you track failed transactions for a particular service, you can't guarantee that the service you're looking at is what went wrong. Assume that there are some messaging mechanisms between services, such as RabbitMQ, ActiveMQ, or possibly Akka.

Even if the service behaves normally and no problems are found, the following scenarios may occur:

The input it receives is wrong, so you need to understand what caused the abnormal behavior of the previous service.

The recipient of the result returns some abnormal responses, so you need to understand how the next service works

What if these dependencies were more complex than 1:1? Or do multiple services benefit from the problem?

No matter what the problem is, the first step in microservices is to know where to start looking for answers. The data is completely scattered and probably not in the log or dashboard at all.

Point # 3: in a single application, you can usually know that the inspection is going in the right direction, and microservices make it more difficult to understand where the problem is coming from and where to get the data.

Question # 4: find the root cause of the problem

All right, let's keep digging. The starting point now is that we have found the problematic service, got the data we need, and found stack trace and some variable values from the log. If you are using APM (like New Relic, AppDynamics, or Dynatrace, we also have articles, here and here), you may also get some other data that makes some basic assessments about the high-speed processing time of some methods / the severity of a pair of problems.

But-- what is the real problem? The real root cause? To find the code that actually went wrong.

In most cases, the variable data first obtained from the log is not the data that is really needed. They usually point to the next clue, asking you to discover more truth and add more log statements. Deploy changes and expect the problem to reappear or not, because-- sometimes just adding a log declaration seems to solve the problem.

Point # 4: when the root cause of a microservice affects multiple services, it is important to have a centralized root cause detection tool available. If you use Java/ in other JVM languages, be sure to come and see how we Takipi do it.

Takipi's error analysis dashboard-the variable value of each frame is placed on top of the actual code

Question # 5: circular dependencies between version management and services

Another issue mentioned in the continuous discussion blog is from the single-tier model of a typical monolithic architecture to the graph model of micro-services.

Two possible problems here have to do with dependency.

If there is a circular dependency between services, an overflow error occurs when a transaction may be deadlocked in the loop.

If two services share a dependency and you update the API of other services in a way that affects them, you need to update all three services at once. This leads to such a question, which one should you update first? How can we make such an update transition smoothly?

More services mean that each service has a different release cycle, which greatly increases the complexity here. When the problem disappears in one version and reappears in the new version, the recurrence problem is very complicated.

Point # 5: in a micro-service architecture, you are more likely to encounter dependency-related problems.

At this point, the study on "what are the ways to make micro-services in the production environment more fluent" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.