Shulou(Shulou.com)06/03 Report--

This article mainly introduces "how to save multiple python objects efficiently". In daily operation, I believe that many people have doubts about how to save multiple python objects efficiently. Xiaobian consulted all kinds of materials and sorted out simple and easy-to-use methods of operation. I hope it will be helpful to answer the questions of "how to save multiple python objects efficiently". Next, please follow the editor to study!

Preface

Json files are very popular on the Internet because of their simplicity and refinement. When we write crawlers, we often encounter websites that use json format to transmit data. But if the data to be stored is 1G, then reading a json file needs to be read at once, which takes up a lot of memory and puts too much pressure on the computer. So we need to store the data as many objects to reduce the memory footprint by reading row by line. So today I'm going to talk about the jsonlines library. I hope you can get something.

Json files are very popular on the Internet because of their simplicity and refinement. When we write crawlers, we often encounter websites that use json format to transmit data. But if the data to be stored is 1G, then reading a json file needs to be read at once, which takes up a lot of memory and puts too much pressure on the computer. So we need to store the data as many objects to reduce the memory footprint by reading row by line. So today I'm going to talk about the jsonlines library. I hope you can get something.

Jsonlines

Each row is a json or python object

Using utf-8 coding

Features of the jsonlines Library

Convenient open () function

Easy to write

Pass in the file name and write method mode

Return Reader or Writer instance

Can be used as a context manager

Installation

Pip install jsonlines

Usage

Jsonlines can hold various data type objects of python, such as lists, dictionaries, numbers, tuples, collections, and so on.

For example, write to the jsonl file line by line as a python dictionary type

Import jsonlineswith jsonlines.open ('data.jsonl', mode='w') as writer: writer.write ({' aisle as writer 5}) writer.write ({'afrol6}) writer.write ({' afrol7})

It is found that a jsonl file has been generated under the current work path.

Read multiple dictionaries in a jsonl file line by line

With jsonlines.open ('data.jsonl', mode='r') as reader: for row in reader: print (type (row), row)

Running

At this point, the study on "how to save multiple python objects efficiently" is over. I hope to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.