Shulou(Shulou.com)06/02 Report--

This article mainly introduces "the processing method of python io file data". In the daily operation, I believe that many people have doubts about the processing method of python io file data. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the doubts of "python io file data processing method". Next, please follow the editor to study!

The specific code is as follows:

The format may be messy. It is recommended to download the file directly:

Io read and write

Mac path path

Path ='/ Users/Public/odpscmd_public/odps_download/'

Path ='/ Users/Public'

Csv document

Csv document reading

Import pandas as pd

From datetime import datetime

Import time

Def read_csv (from_path):

Table = pd.read_csv (from_path,encoding='gbk') # # Select table

Return table

Def write_csv (to_path,data):

Data.to_csv (to_path,index=False)

If _ name__ = = "_ _ main__":

Start_time = time.time () # start time

Path ='/ Users/Public'

Path ='/ Users/Public/odpscmd_public/odps_download/'

File_name = 'activity three behavior data .csv'

From_path = path+ "/" + file_name

Print (from_path)

Df = read_csv (from_path)

Df.head ()

End_time = time.time () # end time

Print ("the program takes% f seconds." >

Print (df.head ())

Save csv documents

Import re

Import pandas as pd

From datetime import datetime

Import time

Def write_csv (to_path,data):

Data.to_csv (to_path,index=False)

Return 'successful write'

If _ name__ = = "_ _ main__":

Start_time = time.time () # start time

Path ='/ Users/Public'

File_name = 'new user first registration order schedule .csv'

To_path = path+ "/" + file_name

Print (to_path)

Write_csv (to_path,df.head ())

End_time = time.time () # end time

Print ("program takes% f seconds."% (end_time-start_time))

Excel file IO

Excel file reading

Import re

Import pandas as pd

From datetime import datetime

Import time

Def read_xlsx (path,sheet_name):

Xlsx_file = pd.ExcelFile (path) # # path

Table = xlsx_file.parse (sheet_name) # # Select table

Return table

If _ name__ = = "_ _ main__":

Start_time = time.time () # start time

Path ='/ Users/Public'

File_name = 'source Gemini uvpv.xlsx'

Sheet_name_list = {

'hive':'Sheet'

'mysql':'Sheet4'

}

Path = path+ "/" + file_name

Sheet_name = sheet_name_list ['hive']

# sheet_name = sheet_name_list ['email']

Df = read_xlsx (path,sheet_name)

Print (df.head ())

End_time = time.time () # end time

Print ("program takes% f seconds."% (end_time-start_time))

Excel save

Def write_xlsx (to_path,data,sheet_name):

Writer = pd.ExcelWriter (to_path, engine='xlsxwriter')

Data.to_excel (writer,'Sheet1',index=False)

Writer.save ()

If _ name__ = = "_ _ main__":

Start_time = time.time () # start time

Path ='/ Users/Public'

File_name = 'remove the point .xlsx'

Sheet_name = 'Sheet1'

To_path = path+ "\\" + file_name

Data = df

Try:

Write_xlsx (to_path,data,sheet_name)

Except:

Print ('Excel save failed')

End_time = time.time () # end time

Print ("program takes% f seconds."% (end_time-start_time))

Mysql data iO

Mysql save

Import pymysql

From sqlalchemy import create_engine

# import mysql.connector as sql

Def py_mysql (table,table_name):

Try:

Engine = create_engine ("mysql+pymysql://root:password@127.0.0.1:3306/db_name?charset=utf8mb4")

Table.to_sql (name=table_name,con=engine,if_exists='replace',index=False,chunksize=10000

)

Print ('database written successfully')

Except:

Print ('database write failed')

If _ name__ = = "_ _ main__":

Start_time = time.time () # start time

Try:

Py_mysql (df_trade_im,'tmp_news_im_v3')

Except:

Print ('sql query failed')

End_time = time.time () # end time

Print ("program takes% f seconds."% (end_time-start_time))

Mysql read

Import pymysql

Import time

Import pandas as pd

Def read_mysql (sql):

Db_connection= pymysql.connect (host= "127.0.0.1", port=3306,user= "root"

, passwd= "password", db= "db_name")

Df = pd.read_sql (sql, con=db_connection)

# Open database connection

Df = pd.read_sql (sql, con=db_connection)

# create a cursor object cursor using the cursor () method

# cursor = db.cursor ()

# execute SQL using the execute () method, and delete if the table exists

# create tables using preprocessing statements

# data = cursor.execute (sql)

# close database connection

Db_connection.close ()

Return df

If _ name__ = = "_ _ main__":

Start_time = time.time () # start time

Sql = ""

Select * from table_name

"

Try:

Df = read_mysql (sql)

Print (df.head ())

Except:

Print ('sql query failed')

End_time = time.time () # end time

Print ("program takes% f seconds."% (end_time-start_time))

At this point, on the "python io file data processing methods" study is over, I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.