Shulou(Shulou.com)06/01 Report--

Switch CPU load up to more than 90% (1)

one。 Preface

Since working, I have come into contact with a lot of projects and encountered countless problems, some of which seem very strange, but in fact, they are solved in theory.

Shidu makes sense, of course, we rule out the bug problem of the device or the software itself, because such a problem is disgusting, and we must be in the same boat.

It is also a deep feeling; summing up my work in the past few years, I have submitted a lot of bug information for H3C MagneCSCO; there were many in my blog before.

The number is to explain the theoretical knowledge related to the network, and such articles can be found everywhere on the Internet. Some time ago, we suddenly lamented that the actual cases in China were shared.

There are very few, so in my blog, I will bring you my work experience in the past six years, and share it with you.

While providing cases, I try my best to restore the "scene" and share practical experience with more articles, so bloggers want to learn from

If we can get more benefits from the next blog post, we can only read the text carefully and patiently, so that bloggers can feel the situation at that time and hope to be big.

Home brings more benefits!

two。 The first case

The cpu load of sharing cases among snowy people is more than 90% (1)

What happened

Since this is the real network environment of a company group, I can't reflect the whole network environment in the blog post. I can only take it out of context.

But rest assured, this does not affect us to present the problem at that time, let's move on, now we are starting to tell stories, I hope you can

To listen!

This is a brand-new project. At that time, the customer used his own computer room and purchased four 12510 sets (two cores, two converged sets), as shown in the picture.

There are two converging switches, more than 40 access switches and 495 server, all of which are stacked in pairs (as shown in the figure).

I was in charge of the project, and I was also mainly in charge of the network, as well as mainframe and database, and, of course, people from H3C manufacturers, the night of the incident.

We were all in the computer room. At that time, the two cabinet machines often failed to communicate, and the packet loss was serious. I didn't pay attention to this area at first, because I was mainly responsible for planning and

To coordinate this area as a whole, the specific technical implementation and command configuration were completed by the manufacturer; at that time, they were ready to go back from work, and the business people began to react.

It is said that the packet loss of the machine is getting more and more serious. I asked the people from the manufacturer to check first. After a period of time, they asked them how they were doing, and they said it was no different.

Often, at this time, I began to wonder how it could be normal. At that time, I went to a switch to check, and there was really nothing different.

Often, including cpu and memory usage, try rebooting the switch if you can't, so I restart the switch, because it's a new environment, so there won't be anything.

It has too much influence, and it will be normal after the restart. It will be no problem for us to observe for more than 10 minutes. We will all be off duty.

The next morning, the people in the business began to lose their packets again, and I went to the computer room to have a look. If there was anything wrong with the switch, it should not be restored.

Again, the switch traffic was not that large at that time, so it should not be caused by the traffic. I was busy with other things at that time, so that the manufacturer

Ten percent of the people went to investigate, but the people of the manufacturers seemed to be at the end of their skills and didn't know what to do. It was really difficult to troubleshoot the problems of lost bags and impassable problems.

There was a big difference. When I finished the task at hand, I went to look at the problem and told them not to be afraid and that everything could be done. First of all, I checked.

For the traffic of each interface, I found that there was an interface with very large traffic, so I checked the packet changes of the interface with a single command, and later found the benefits of the interface.

The rate has been growing slowly, but CPU is really normal, about 30 minutes, and then the utilization of the interface has reached 100%; at that time, I was straight.

Then I went to troubleshoot the line problem and found that the construction team connected it wrong when wiring. It turned out that it was going to transfer the switch on one cabinet to the server on another cabinet.

Connected, as a result, he mistakenly connected the cable to the server to the switch, resulting in a loop in my layer 2 topology (STP is off in the whole network.

Closed), as shown in the following wiring diagram:

The cpu load of sharing cases among snowy people is more than 90% (1)

two。 Summarize the conclusion

1. For switches, generally speaking, if there is a loop in the network, the CPU of the switch will quickly rise to 100%, but H3C is not.

Like this, there was no problem checking the CPU at that time, so I didn't think about it on the ring road, so this is a pit, so we can't just look at the loop.

CPU and memory of the switch

two。 The loop is caused by the wrong insertion of the line by the construction team, and the lines deployed on the site are indeed many and very complex, so the physical line one

It must be straightened out.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.