Shulou(Shulou.com)06/01 Report--

This article mainly explains "Python how to batch merge tables", the article explains the content is simple and clear, easy to learn and understand, the following please follow the editor's ideas slowly in depth, together to study and learn "Python how to batch merge tables" bar!

I. menu method

This method is suitable for small amount of data, when the amount of data is large, it takes a lot of time to open an EXCEL table, for example, the data folder of this article is a month of data in September, you can create a new EXCEL table, click data > get data > from file > from folder, click group and, you can merge the data table in batches.

II. Python batch merge forms

Another method is to write code, which is suitable for batch data with a large amount of data. With the help of the two standard libraries in Python, pandas and os, let's first understand the role of these two libraries.

Pandas

Pandas is a data analysis package of python

Pandas is a tool for solving data analysis tasks based on NumPy.

Pandas incorporates a large number of libraries and some standard data models

Pandas provides the tools needed to manipulate large datasets efficiently

Os library

The os library provides general and basic operating system interaction functions.

The os library contains processing functions such as path operation, process management, environment parameters, etc.

The following is the actual use of code for batch processing of table data.

The first step is to import the Python standard library needed for batch processing data.

The second step of importing the library import pandas as pdimport os is to set the folder to be read, the folder to be saved, and the name of the table to be saved. # folder to read batch csv read_path ='C:\ Users\\ shangtianqiang\\ Desktop\\ data'# folder of merged csv to be saved, create a data_merge folder save_path ='C:\\ Users\\ shangtianqiang\\ Desktop\\ data_merge'# to be saved step 3, modify the current working directory with the help of os library. The fourth step of modifying the current working directory os.chdir (read_path) is to store all the file names under this folder in a list. # put all the file names under this folder into a list file_list = os.listdir () step 5, read the first CSV file and include the header, pay attention to the encoding of the data. # read the first CSV file and include the header df = pd.read_csv (read_path +'\\'+ file_list [0], encoding = 'gbk') # Encoding the default UTF-8. If the garbled code is changed to step 6 of gbk, write the first read CSV file to the merged file and save it.

# write the first CSV file read into the merged file to save df.to_csv (save_path+'\'+ save_name,encoding= "utf_8_sig", index=False) step 7, loop through the file name of each table in the list, and append it to the merged folder one by one, where the data of the first table has been imported, so here FileStart=1, not FileStart=0.

# iterate through each CSV file name in the list and append it to the merged file FileStart = 1 FileEnd = len (file_list) for i in range (FileStart,FileEnd): df = pd.read_csv (read_path +'\\'+ file_ list [I], encoding = 'gbk') # Encoding default UTF-8 If the garbled code is changed to gbk df.to_csv (save_path+'\'+ save_name,encoding= "utf_8_sig", index=False, header=False, mode='a+') Thank you for your reading, these are the contents of "Python how to merge tables in batches". After the study of this article, I believe you have a deeper understanding of how Python merges tables in batches, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.