Shulou(Shulou.com)06/01 Report--

In this article Xiaobian for you to introduce in detail "python how to read the file designated line", detailed content, clear steps, details handled properly, I hope that this "python how to read the file designated line" article can help you solve doubts, the following follow the editor's ideas slowly in-depth, together to learn new knowledge.

Line traversal implementation

In python, if you want to load a file completely into memory, you can use file.readlines (), but when the file occupies a high amount, we cannot completely load the file into memory, so you need to use python's file.readline () to read iteratively line by line:

Filename = 'hello.txt'with open (filename,' r') as file: line = file.readline () counts = 1 while line: if counts > = 50000000: break line = file.readline () counts + = 1

Here our implementation is to first use a with statement to open a file, and then use the readline () function with while loop to load line by line, and finally through an ordinal tag to end the loop traversal, outputting the contents of line 5000000 of the file. The execution of this code is as follows:

Dechin@ubuntu2004:~/projects/gitlab/dechin/$ time python3 get_line.py real 0m10.359suser 0m10.062ssys 0m0.296s

You can see that the time here is a little more than 10 seconds.

Linecache implementation

Although the scheme for reading the contents of the specified line is not implemented in the readline function of python, it is implemented in another library, linecache. Because the method is relatively simple, a code example is provided here for reference:

Filename = 'hello.txt'import linecachetext = linecache.getline (filename, 50000000)

The execution result of this code is as follows:

Dechin@ubuntu2004:~/projects/gitlab/dechin/$ time python3 get_line.py real 0m11.904suser 0m5.672ssys 0m6.231s

Although the implementation is much simplified, we find that this implementation takes more than 11s, which is not as good as the loop traversal scheme we implemented manually. Therefore, if it is a scenario with certain performance requirements, this scheme is not recommended.

Command line sed get

We know that the data of the specified line or the specified line range of the file can also be obtained by using the sed instruction of the Linux system itself, and its execution instruction is: sed-n 50000000p filename means to read the contents of the 500000000 line of the file. At the same time, with python, we can execute system instructions in python code and get the output:

Filename = 'hello.txt'import osresult = os.popen (' sed-n {} p {} '.format (50000000, filename)) .read ()

It should be noted that if you run os.system () directly, there is no return value, only os.popen () has a return value, and you need to add an option of read () to the tail. The execution result of this code is as follows:

Dechin@ubuntu2004:~/projects/gitlab/dechin/$ time python3 get_line.py real 0m2.532suser 0m0.032ssys 0m0.020s

You can see that the execution speed of using sed instructions directly is very fast, but it is not profitable to use this method, such as the following example:

Filename = 'hello.txt'import osresult = os.popen (' sed-n {} p {} '.format (500, filename)) .read ()

Let's change reading line 50000000 to read line 500,000, and run the program again:

Dechin@ubuntu2004:~/projects/gitlab/dechin/$ time python3 get_line.py real 0m2.540suser 0m0.037ssys 0m0.013s

However, we find that this speed is not reduced by the decrease in the number of rows to be read, but remains almost constant.

Read here, this article "how to read python file designated line" article has been introduced, want to master the knowledge of this article also need to practice and use in order to understand, if you want to know more related articles, welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.