Shulou(Shulou.com)05/31 Report--

Most people do not understand the knowledge points of this article "python how to combine shell to automatically create a kafka connector", so the editor summarizes the following contents, detailed contents, clear steps, and has a certain reference value. I hope you can get something after reading this article. Let's take a look at this "how to combine python with shell to automatically create a kafka connector" article.

Environment cat / etc/redhat-release CentOS Linux release 7.5.1804 (Core) [root@localhost ~] # uname-aLinux localhost.localdomain 3.10.0-862.el7.x86_64 # 1 SMP Fri Apr 20 16:44:24 UTC 2018 x 86 "64 GNU/Linuxpython-VPython 2.7.5 install python package connected to oracle pip install cx_Oracle==7.3 get oracle table information cat query_oracle.py #! / usr/bin/env pythonimport cx_Oracleimport sysimport osimport csvimport tracebackfile = open (" oracle.txt ") Close () user = "test" passwd = "test" listener = '10.0.2.15:1521/orcl'conn = cx_Oracle.connect (user, passwd) Listener) cursor = conn.cursor () sql = "select table_name from user_tables" cursor.execute (sql) LIST1= [] while True: row = cursor.fetchone () if row = = None: break for table in row: # print table LIST1.append (table) LIST2= [] for i in LIST1: sql3 = "select COLUMN_NAME,DATA_TYPE,DATA_PRECISION DATA_SCALE from cols WHERE TABLE_name=upper ('% s') "I cursor.execute (sql3) cursor.execute (sql3) row3 = cursor.fetchall () for data in row3: # LIST2.append (I) LIST2.extend (list (data)) LIST2.append (I) f=open ('oracle.txt','a+') print > > fame LIST2 LIST2= [] # f=open (' test.txt','a+') # select table_name,column_name DATA_TYPE from cols WHERE TABLE_name=upper ('student') # select column_name,DATA_TYPE from cols WHERE TABLE_name=upper ('student') Remove the redundant cat auto.sh #! / bin/bash#python query_oracle.py | tr ","'| tr "'"'| tr "["| tr"] ""# > oracle.txt > oracle_tables.txtcat oracle.txt | tr" [] '"" | sed "s # [] [] * # # g" | sed's / ^ [\ t] * / / g' > > oracle_tables.txtcat oracle_tables.txt SNO NUMBER 19 0 SNAME VARCHAR2 None None SSEX VARCHAR2 None None SBIRTHDAY DATE None None SCLASS VARCHAR2 None None STUDENT DATE_DATE SNO2 NUMBER 19 0 SNAME VARCHAR2 None None SSEX VARCHAR2 None None SBIRTHDAY DATE None None SCLASS VARCHAR2 None None STUDENT2 INPUT_TIMESNO3 NUMBER 19 2 SNAME VARCHAR2 None None SSEX VARCHAR2 None None SBIRTHDAY DATE None None SCLASS VARCHAR2 None None STUDENT3 DATA_DATEshell foot This processing table information file cat connect.sh#! / bin/bash# gets the number of lines of temporary files FILE_NUM=$ (cat oracle_tables.txt | egrep-v'# | ^ $'| wc-l) # clear the script for automatically creating connectors > create-connect.sh# circular temporary files each line for i in `seq $FILE_ NUM`do FILE_LINE=$ (sed-n ${I} p oracle_tables.txt) TABLE_NAME=$ (echo ${FILE_LINE} | sed's / [\ t] * $/ / g' | awk'{print $(NF-1)}') COL_NUM=$ (echo ${FILE_LINE} | sed's / [\ t] * $/ / g' | awk-F "[]"'{print NF}') REAL_COL_NUM= `expr $COL_NUM-2` # clear temporary files > ${TABLE_NAME} .txt > ${TABLE_NAME} .sql # looping temporary files per line Column for j in `expr $j + 1`m = `expr $j + 2`n = `expr $j + 3` COL_NAME=$ (echo $FILE_LINE | cut-d ""-f ${j}) COL_DATA_TYPE=$ (echo $FILE_LINE | cut-d "- f ${k}) COL_DATA_PRECISION=$ (echo $FILE_LINE | cut-d"-f ${m}) COL_DATA_SCALE=$ (echo $FILE_LINE | cut-d ""-f ${n}) # determines whether the data type of the column is NUMBER if ["$COL_DATA_TYPE" = "NUMBER"] then # CAST (* AS *) AS * part of the circular stitching SQL query Append to the temporary file echo "CAST ($COL_NAME AS $COL_DATA_TYPE ($COL_DATA_PRECISION,$COL_DATA_SCALE)) AS $COL_NAME" > > ${TABLE_NAME} .txt else # column names in the circular stitching SQL query Append to temporary file echo "$COL_NAME" > > ${TABLE_NAME} .txt fi done # to assemble the complete SQL statement, append to temporary file echo "select $(cat ${TABLE_NAME} .txt | tr"\ n "," | sed-e's / ) from $TABLE_NAME where $(sed-n ${I} p oracle_tables.txt | cut-d''- f$COL_NUM) > = trunc (sysdate-2) and $(sed-n ${I} p oracle_tables.txt | cut-d'- f$COL_NUM) = trunc (sysdate-2) and $(sed-n ${I} p oracle_tables.txt | cut-d'- f$COL_NUM) = trunc (sysdate-2) and $(cat. / TABLE_TIME/) ${TABLE_NAME} _ TIME_COL.txt)

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.