Shulou(Shulou.com)06/01 Report--

Problem description:

Multiple projects, some projects sub-library sub-table distributed in multiple nodes, therefore, the cutting and collection of slow logs is extremely necessary for slow log analysis.

Implementation plan:

① environment preparation

Host1: ftp server (192.168.xx.xxx) for centralized storage of daily (scriptable) logs for each project

Host2: mysql server (192.168.xx.xxx), which stores configuration-related parameters (such as log host,user,pwd, log location, and so on) that are then used for query invocation of scripts.

Host3: the machine that performs the operation (192.168.xx.xxx), that is, the machine that executes the scheduled tasks set by the script

② script preparation

Involves two scripts, logrotate.sh and job.sh

Where job.sh is used for batch execution of related host operations stored in mysql, and job.sh calls logrotate.sh

Logrotate.sh is used to actually perform operations on each machine that needs to collect logs.

The structure and data of the mysql configuration table are as follows (please add an account where all machines can query the table):

The job.sh script is as follows:

Click (here) to collapse or open

#! / bin/bash

Mysql_host=192.168.xx.xxx

Mysql_port=3456

Mysql_user=xxx

Mysql_pwd=xxxxxx

Function get_info {

Cmd= `which mysql`

$cmd-h$mysql_host-P$mysql_port-u$mysql_user-p$mysql_pwd-BNe "select * from DB_SPLIT_LOG.t_log_info;" | while read line

Do

Host_ip= `echo $line | awk'{print $3}'`

Ssh_user= `echo $line | awk'{print $9}'`

Ssh_pwd= `echo $line | awk'{print $10}'`

/ usr/bin/sshpass-p $ssh_pwd ssh-o StrictHostKeyChecking=no $ssh_user@$host_ip "bash" $tran_log

Exec 2 > $tran_log

}

Function create_rotate_conf {

Echo ""

$slow_log {

Daily

Copytruncate

Dateext

Create 0660 mysql mysql

Olddir $old_dir

Rotate 1

}

"" > $tmp_conf

}

Function start_logrotate {

Echo ""

/ usr/sbin/logrotate-vf $tmp_conf

}

Function change_logname {

Echo ""

For file in `find $old_dir-maxdepth 1-type f-name "* slow*$ {date} *" `

Do

Echo $file

Done

Echo "change log name $file to ${old_dir} / ${log_name}"

Mv $file ${old_dir} / ${log_name}

}

Function ftp_put {

Echo ""

Echo "start ftp put $log_name"

Ftp-in

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.