Shulou(Shulou.com)06/01 Report--

How to analyze the zabbix alarm system, I believe that many inexperienced people are at a loss about it. Therefore, this paper summarizes the causes and solutions of the problem. Through this article, I hope you can solve this problem.

In zabbix, the configuration of alarm channel is too tedious, so let's analyze it separately:

Mail alarm: if you need to configure mail alarm, you have to configure the mail call interface in Zabbix. And the Zabbix email alarm is often delayed (Zabbix itself delay + email service provider delay), and more sadly, the alarm email is often left unread. Since it is inconvenient to read email, we will consider sending text messages to the police.

SMS alarm: if you configure SMS alarm, you need to purchase SMS service from the service provider. There are indeed a lot of SMS services that are very cheap. And if we use a cloud service like Aliyun, SMS can be launched at any time (13 years).

You need to set the last shift of SMS, configure the SMS calling API in zabbix, and avoid the delay and omission of a SMS service provider.

Call the police: this is the worst way to wake up the operators from their sleep in the middle of the night and solve the emergency failure in time. But the phone call is not good is really a kind of harassment, love half of the phone calls, answer or not answer is a problem. Our previous company, the use of telephone alarm, all need to do secondary development. Python is useful at this time, of course, before the brain short circuit, with java.

Wechat alarm, is it another upgrade? if you follow the business form and configure Wechat alarm, you need to apply to activate Wechat-related services, and then schedule for targeted development.

First of all, Wechat's Subscription account, service number, enterprise number, its message push restrictions, message content restrictions, restrictions on the number of people before and after authentication are all different, can it really be so clear?

Second, if there is an alarm storm, make sure your little heart can handle hundreds of alarm alerts that suddenly pop up on Wechat. I'm used to it anyway.

However, it was only after I had been abused a thousand times that I remembered that as long as the operation and maintenance personnel were involved in maintenance, I would no longer send messages, otherwise the general assembly would go crazy.

The above is the alarm scheme that we often use, but for zabbix, the configuration of either scheme is very complex, and there are no Chinese references. I added to a new company, their operation and maintenance is to configure a User group, configure a Trigger.

Whether it is email alarm, SMS alarm or Wechat alarm, there will be the problem of alarm storm: if large-scale failures such as network disconnection are found, and there is no alarm protection and convergence mechanism, it is common for the phone to ring for an hour. not to mention unclear alarm emails and crazy text messages.

Later, we made adjustments to the management of the monitoring system to solve the malpractice of immediately notifying all staff in the event of an alarm, and the alarm level was strictly divided. Severity polling upgrade notification processing, using java for secondary development, and later refactoring with zabbix when doing automated operation and maintenance platform.

During this period, we have also changed monitoring treasures, New Relic, and other charging tools. Although charging tools are easy to use, the guarantee based on enterprise information security is relatively poor. Of course, these may not be recognized as monitoring service providers, but we make comprehensive considerations on our own enterprise costs, labor costs, and information security.

When it comes to security, there is another very low-level ridiculous thing. A company that used to test the service used Tencent Cloud. When I intervened, Tencent Cloud was still in public trial. When the server was about to expire, I repeatedly stressed that Tencent Cloud recycled resources immediately when the server was about to expire, unlike Aliyun, which had a week's buffer, but unfortunately, it expired in the end and did not recharge immediately, and all the efforts came to nothing. Fortunately, it is a testing machine, or it may be a testing machine, so it does not attract the attention of the company. Of course, this is the time of Tencent Cloud's public testing, and it seems that resources will not be released immediately when it expires.

Share a Wechat alarm here-Enterprise account (shell) version:

#! / bin/bash

# SCRIPT_NAME:TFedu###

# send message from weixin for zabbix monitor###

# develop###

# V7-2014-05-12 #

CropID='wx4a5706c1941562f0'

Secret='wWddGauAGusDEkETib9PEwnDacTQ9Ie-w-Fh7HhSUD3rmsQ_ro980vun4jbOAVrS'

GURL= "https://qyapi.weixin.qq.com/cgi-bin/gettoken?corpid=$CropID&corpsecret=$Secret"

Gtoken=$ (/ usr/bin/curl-s-G $GURL | awk-F "'{print $4}')

PURL= "https://qyapi.weixin.qq.com/cgi-bin/message/send?access_token=$Gtoken"

Function body () {

Application id in local int AppID=1 # Enterprise account

Wechat recipients defined in local UserID=$1 # Department member id,zabbix

Local PartyID=2 # Department id, which defines the scope, and all members of the group can receive messages

Local Msg=$ (echo "$@" | cut-d ""-f3 -)

# filter out the third parameter passed in zabbix

Printf'{'

Printf'"touser": "'" $User "", "

Printf'"toparty": "'" $PartyID "", "

Printf'"msgtype": "text",'

Printf'"agentid": "'" $AppID "", "

Printf'"text": {'

Printf'"content": "" $Msg "

Printf'},'

Printf'"safe": "0"'

Printf'}'

}

/ usr/bin/curl-- data-ascii "$(body $1 $2 $3)" $PURL

Of course, this script is not the only answer, you can rewrite it as a python script.

After reading the above, have you mastered the method of how to analyze the zabbix alarm system? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.