In addition to Weibo, there is also WeChat
Please pay attention
WeChat public account
Shulou
2025-04-10 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Database >
Share
Shulou(Shulou.com)06/01 Report--
File name: add.sh
Path: $HOME/ggadd
Function: an optimized version of the batch table addition script for a small number of table addition requirements, including repeated configuration table filtering and automatic addition of additional logs
The script is written to the account system configuration file through alias. Profile and .bash _ profile are used by command
Log path: $HOME/ggscript/ggadd
Specific script content
#! / bin/bash# condition judgment to determine whether the parameters are complete when calling # format must be if [$#-eq 0] for the process table domain name; then echo "info EXTRACT TABLE AREA user" exit 2fiif [$#-eq 1]; then echo "info EXTRACT TABLE AREA user" exit 2fiif [$#-eq 2] Then echo "info EXTRACT TABLE AREA user" exit 2fiif [$#-eq 3] Then echo "info EXTRACT TABLE AREA user" exit 2fibackuptime = `date +% Y% m% d valcat AddTemp#echo% H% M`datenow = `date +% Y% m% d% H`echo $backuptime >'$HOME'/gglog/ggadd/InsertLog-'$backuptime'.log' cd $HOME/ggscript/ggaddecho $backuptimeecho "This script can only insert one table (by RenYi)" val=$1echo $2 > AddTemparea=$3name=$4#echo $valcat AddTemp#echo $areaecho "- insert process name-" > >'$HOME'/gglog/ggadd/InsertLog- '$backuptime'.log'echo $val > >' $HOME'/gglog/ggadd/InsertLog-'$backuptime'.log'val= `echo $val | process type determination of tr Amurz Amurz `# If it starts with E, it is classified as an extraction process, and if the beginning of P is classified as a sending process if [`echo $val | grep ^ E`] Then IType=EXTRACTelif [`echo $val | grep ^ P`]; then IType=POSTelif [`echo $val | grep ^ R`] Then echo "can not operate the REPLICAT process" echo "can not operate the REPLICAT process" >'$HOME'/gglog/ggadd/InsertLog-'$backuptime'.log' echo "process break off" echo "process break off" >'$HOME'/gglog/ggadd/InsertLog-'$backuptime'.log' exit 2else echo "only can operate the EXTRACT and the POST process" echo "can not operate the REPLICAT process" > > '' $HOME'/gglog/ggadd/InsertLog-'$backuptime'.log' echo "process break off" echo "process break off" > >'$HOME'/gglog/ggadd/InsertLog-'$backuptime'.log' exit 2fi# locates the process file and determines whether val= `echo $val.PRM exists in the parameter input process | tr Akoz Z-z`echo $valecho "- insert file name-" > >'$HOME'/gglog/ Ggadd/InsertLog-'$backuptime'.log'echo $val > >'$HOME'/gglog/ggadd/InsertLog-'$backuptime'.log'cd $HOME/ggserver/dirprmif [!-e $val] Then echo "the EXTRACT is not exist" echo "the EXTRACT is not exist" > >'> $HOME'/gglog/ggadd/InsertLog-'$backuptime'.log' exit 2fi# locates the region and determines whether cd $HOME/ggscript/ggaddecho $valcat $HOME/ggserver/dirprm/$val exists in the region entered by the parameter | grep Begin | sed's Uniqis Universe GOOG'| sed's Universe Mergin Universe g'> arealistcat arealistecho "- area Check- "> >'$HOME'/gglog/ggadd/InsertLog-'$backuptime'.log'check= `grep-w $area arealist`if [" $check "=" $area "] Then echo "area is exist" echo "area is exist" >'$HOME'/gglog/ggadd/InsertLog-'$backuptime'.log'else echo "area is not exist" # echo "area is not exist" > >'$HOME'/gglog/ggadd/InsertLog-'$backuptime'.log' # exit 2fi# duplicate table judgment Filter out the configured table cat $HOME/ggserver/dirprm/$val | awk'/ -'$area'-Begin/,/--'$area'-End/ {if (I > 1) print x (1) {print $1} 'temp > temp2rm-rf tempsed-e' / ^ $/ d 'temp2 > temp3rm-rf temp2awk-F','{print $1} 'temp3 > temp4rm-rf temp3awk-F' '{print $1}' temp4 > temp5rm-rf temp4echo "Confirm the tables that have increased" cat temp5 | awk-F 'TABLE''{print $2}'> AddedTemprm-rf temp5sed'/ ^ $/ d 'AddedTemp > AddedTemp2sed' / ^ $/ d 'AddTemp > AddTemp2cat AddedTemp2 | sort-n > AddTablecomm-12 AddTable AddTemp > Duplicateecho "- Table Check-" cat AddTemp > >' $HOME'/gglog/ggadd/InsertLog-'$backuptime'.log'echo " -Table Check- "> >'$HOME'/gglog/ggadd/InsertLog-'$backuptime'.log'check= `grep-w $2 Duplicate`if [" $check "=" $2 "] Then echo "The table has being added" echo "The table has being added" >'$HOME'/gglog/ggadd/InsertLog-'$backuptime'.log' echo "process break off" exit 2else echo "The table not be add" echo "The table not be add" > >'$HOME'/gglog/ggadd/InsertLog-'$backuptime'.log'fi# configuration file backup cp $HOME/ggserver/dirprm/$val $HOME/ggserver / dirprm/bak/$val.$backuptime.bakecho "backup complete" echo "backup complete" >'$HOME'/gglog/ggadd/InsertLog-'$backuptime'.log'# table insert operation sed's / ^ / TABLE / 'AddTemp > table1sed's insert operation / 'table1 > table2rm-f table1echo "- The Operator is' $name'-" > >'$HOME'/gglog/ggadd/InsertLog-'$backuptime'.log'sed'1I\--'$datenow'-'$name'-add' table2 > table3rm-f table2 mv table3 table2 echo "inserting table" sed'/'$area'-Insert/r table2 '$HOME/ggserver/dirprm/$val > $HOME/ggserver/dirprm/tem3 rm-f $HOME/ggserver/dirprm/$val mv $HOME/ggserver/dirprm/tem3 $HOME/ggserver/dirprm/$val# judgment Write additional logs if it is an EXTRACT process, skip if ["$IType" = = "EXTRACT"] if it is a sending process Then sed's / ^ / addtrandata / 'AddTemp > addtrandata cd $HOME/ggserver echo dblogin `more $HOME/ggserver/dirprm/$val | grep USERID` > dirdat/addtrandata cat $HOME/ggscript/ggadd/addtrandata > > dirdat/addtrandata echo OBEY dirdat/addtrandata |. / ggsci echo "process complete" Echo "process complete" > >'$HOME'/gglog/ggadd/InsertLog-'$backuptime'.log' exit 2elif ["$IType" = = "POST"] Then echo "process complete" echo "process complete" >'$HOME'/gglog/ggadd/InsertLog-'$backuptime'.log' exit 2else echo "process complete" echo "process complete" > >'$HOME'/gglog/ggadd/InsertLog-'$backuptime'.log'fi
Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.
Views: 0
*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.
Continue with the installation of the previous hadoop.First, install zookooper1. Decompress zookoope
"Every 5-10 years, there's a rare product, a really special, very unusual product that's the most un
© 2024 shulou.com SLNews company. All rights reserved.