Shulou(Shulou.com)06/01 Report--

This article introduces the relevant knowledge of "the installation method of Python Celery distributed task queue". Many people will encounter such a dilemma in the operation of actual cases, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Celery is a distributed task queue (Distributed Task Queue) written based on Python. Asynchronous processing of tasks (time-consuming tasks, scheduled tasks) can be realized by simple operation of Celery.

Since Celery4.0 version, windows platform is not supported.

1.1 install celerypip install-U "Celery [redis]" through pip

Note:

After installation on windows, the following error may occur:

ValueError:'_ _ name__' in _ _ slots__ conflicts with class variable

Uninstall celery at this point, and then try to reinstall it with the following command

Pip install-U https://github.com/celery/py-amqp/zipball/masterpip install-U https://github.com/celery/billiard/zipball/masterpip install-U https://github.com/celery/kombu/zipball/masterpip install-U https://github.com/celery/celery/zipball/masterpip install-U "Celery [redis]" 1.2 create a soft connection for celery ln-s ~ / .venv/project_dj/bin/celery / usr/bin/celery1.3 implementation Line celery command [root@localhost ~] $celery-- helpOptions:-A -- app APPLICATION-b,-- broker TEXT-- result-backend TEXT-- loader TEXT-- config TEXT-- workdir PATH-C,-- no-color-Q,-- quiet-- version-- help Show this message and exit.Commands: amqp AMQP Administration Shell. Beat Start the beat periodic task scheduler. Call Call a task by name. Control Workers remote control. Events Event-stream utilities. Graph The ``celery graph``command. Inspect Inspect the worker at runtime. List Get info from broker. Logtool The ``celery logtool``command. Migrate Migrate tasks from one broker to another. Multi Start multiple worker instances. Purge Erase all messages from all known task queues. Report Shows information useful to include in bug-reports. Result Print the return value for a given task id. Shell Start shell session with convenient access to celery symbols. Status Show list of workers that are online. Upgrade Perform upgrade between versions. Worker Start worker instance.2.1 creates the celery application and defines the task #-*-coding: utf-8-*-# @ Time: 2021-5-24 11 coding @ Author: chinablue# @ File: task.pyfrom celery import Celery# creates an app (Celery instance) As the entry point for all celery operations broker_url = f "redis://:123456@127.0.0.1:6379/5" backend_url = f "redis://:123456@127.0.0.1:6379/6" app = Celery ("tasks", broker=broker_url, backend=backend_url) # define a task @ app.taskdef add (x, y): return x + y

Event description:

1) when creating an Celery instance, you need to specify a message broker (broker) to receive and send task messages. This article uses Redis (docker redis build)

2) format of broker and backend parameters: redis://:password@hostname:port/db_number

2.2 launch celery worker server celery-A tasks worker-loglevel=INFO

Event description:

1) in a production environment, the celery service is run in the background as a daemon using the supervisor tool

2.3 invoke Task

Open the terminal and enter python command line mode:

> result = add.delay (4,4) > result = add.apply_async ((4,4), countdown=5)

Event description:

1) add.apply_async ((4,4)) can be abbreviated as add.delay (4,4).

2) add.apply_async ((4,4), countdown=5) indicates that the task is issued for 5 seconds before execution

2.4 tracking task information

To get the execution information of each task, you need to specify a backend when creating a Celery instance. This article uses Redis (docker redis build)

Result = add.delay (4,4) result.ready () # Task status: in progress, completed result.failed () # Task completed, Task failed result.successful () # Task completed, Task successful result.state # Task status: PENDING, STARTED SUCCESSresult.get () # get the return value of the task result.get (timeout=10) result.get (propagate=False) # if the task throws an exception, propagate=False means that the exception will not be thrown (default will be thrown) result.id # Task id

Note:

1) in celery, if you want to configure the backend parameter, there are three ways

#-*-coding: utf-8-*-# @ Time: 2021-5-24 11 chinablue# @ Author: chinablue# @ File: task.pyfrom celery import Celery# create an app (Celery instance) as the entry point for all celery operations broker_url = f "redis://:123456@127.0.0.1:6379/5" backend_url = f "redis://:123456@127.0.0.1:6379/6" app = Celery ("tasks") Broker=broker_url, backend=backend_url) # define a task @ app.taskdef add (x, y): return x + y

Method 1: pass in when instantiating Celery

#-*-coding: utf-8-*-# @ Time: 2021-5-24 11 Time @ Author: chinablue# @ File: task.pyfrom celery import Celerybroker_url = f "redis://:123456@127.0.0.1:6379/5" backend_url = f "redis://:123456@127.0.0.1:6379/6" app = Celery ("tasks") app.conf.update ({"broker_url": broker_url, "result_backend": backend_url }) # define a task @ app.taskdef add (x, y): return x + y

Method 2: through the update method of conf

#-*-coding: utf-8-*-# @ Time: 2021-5-24 11 banks @ Author: chinablue# @ File: task.pyfrom celery import Celerybroker_url = f "redis://:123456@127.0.0.1:6379/5" backend_url = f "redis://:123456@127.0.0.1:6379/6" app = Celery ("tasks") app.conf.broker_url = broker_urlapp.conf.result _ backend = backend_url# defines a task @ app.taskdef add (x Y): return x + y, "installation method of Python Celery distributed task queue", that's all. Thank you for your reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.