Shulou(Shulou.com)06/01 Report--

This article mainly introduces "how to create Celery". In daily operation, I believe many people have doubts about how to create Celery. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the doubts about "how to create Celery". Next, please follow the editor to study!

What is Celery?

Celery (Chinese means celery) is a distributed queuing service implemented in Python. It supports not only immediate tasks but also scheduled tasks. Celery has five core roles.

Task

Task is what you need to do. For example, there are many tasks in a registration process. Sending verification emails to users is a task. This time-consuming task can be handed over to Celery, and another task is timed tasks, such as counting the number of website registrations regularly every day. This can also be handed over to Celery to deal with periodically.

Broker

Broker, which means broker in Chinese, refers to a person who provides intermediary services for buyers and sellers in the market. In Celery, this role is equivalent to a queue in a data structure, a broker between the producer and the consumer. For example, in a Web system, the producer is the main program, it produces the task and sends the task to the Broker. The consumer is Worker, which is a background service dedicated to the execution of the task. Celery itself does not provide queue service, generally using Redis or RabbitMQ to implement queue service.

Worker

Worker is the person who has been performing tasks in the background and becomes the consumer of tasks. It monitors the queue in real time for tasks, and if so, immediately takes them out and executes them.

Beat

Beat is a scheduled task scheduler that sends tasks to Broker at a fixed time according to the configuration, waiting for Worker to consume.

Backend

Backend is used to save the execution result of a task. Each task has a return value. For example, the service that sends email will tell us whether it has been sent successfully. The result is stored in Backend. Of course, we do not always care about the execution result of the task.

It's easy to understand Celery after remembering these five roles.

Getting started

When you come into contact with anything new, nothing can be learned faster than hands-on. Suppose we choose Redis as the broker, you need to install redis and have started the redis service (please borrow your own search engine to solve this step)

Pip install-U "celery [redis]" 1. Create Celery instance # tasks.py

From celery import Celery

App = Celery ('tasks', broker='redis://localhost:6379/0') 2, create task

Suppose the task of sending mail takes 5 seconds to complete.

# tasks.py

@ app.task

Def send_mail (email):

Print ("send mail to", email)

Import time

Time.sleep (5)

Return "success"

In the absence of Celery, the program executes sequentially, and each step needs to wait for the previous step to complete.

1. Insert records into database 2. Send email 3. Registered successfully

We can put 2 in a task and give it to celery to execute, so we don't have to wait for the email to finish, you just need to arrange for celery to handle it and help me finish it. The code becomes

1. Insert records into database 2. Celery to help me send email 3. Registered successfully

The second step is very fast, it just needs to put the task in the queue, and will not wait for the task to be actually executed. This is completely relevant to life. For example, we don't experience and do many things ourselves, but hand over to others something that is not so important or immediate.

3. Start Worker

Start Worker and listen for tasks in Broker. Command: celery worker, you may need to specify parameters

Celery-A tasks worker-- loglevel=info

-A: specify the module in which the celery instance resides. In the example, the celery instance is in the tasks.py file. After starting successfully, you can see the information.

Once the function is decorated with the app.task decorator, it becomes a Task in Celery.

4. Call the task

Call the task in the main program and send the task to Broker instead of actually executing it

# user.py

From tasks import send_mail

Def register ():

Import time

Start = time.time ()

Print ("1. Insert records into the database")

Print ("2. Celery email for me")

Send_mail.delay ("xx@gmail.com")

Print ("3. Tell the user to register successfully")

Print ("time:% s seconds"% (time.time ()-start))

If _ _ name__ = ='_ _ main__':

Register ()

In the main program, call the .delay method of the function

Directory structure:

── celery_test ├── tasks.py └── user.py

Run python user.py to start the application

1. Insert records into database 2. Celery send me email 3. Tell users that successful registration takes 0.22688984870910645 seconds.

The program takes less than 0.23 seconds to complete, and if you follow the normal synchronization logic, it takes at least 5 seconds, because the e-mail task takes 5 seconds.

View the log information in the worker service window

Note:

1. When celery worker starts, if you are a root user, you need to set the environment variable:

$export paid for rootstrapping truth'

2. Celery4.x no longer supports Windows platform. If you need to develop in Windows, please use 3.x version.

3. Use RabbitMQ or Redis as Broker, never use relational database in production environment

4. Do not use complex objects as parameters of task functions

# Good

@ app.task

Def my_task (user_id):

User = User.objects.get (id=user_id)

Print (user.name)

#... # Bad

@ app.task

Def my_task (user):

Print (user.name)

#... At this point, the study on "how to create Celery" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.