Shulou(Shulou.com)06/01 Report--

This article shows you how to conduct Django Celery analysis, the content is concise and easy to understand, can definitely brighten your eyes, through the detailed introduction of this article, I hope you can gain something.

For the implementation of task scheduling, in fact, there are many open source projects.

Let me first talk about the understanding of task scheduling, if from the database level, task scheduling is scheduler, which is more detailed in Oracle.

Creating a scheduler in Oracle and running JOB in the background to complete data processing will basically split a task into different dimensional attributes.

If there are many tasks, there are a large number of tasks to deal with, and the tasks are located in different server environments, then this complexity will greatly increase, so introducing message queues is a natural way.

At present, there are many options for message queuing, such as Redis,RabbitMQ, etc., which can be met according to your own needs.

First, we need to make sure that celery is installed properly.

> pip list | grep celery

Celery (3.1.20)

Celery-with-redis (3.0)

Django-celery (3.2.2)

If it comes with Django in a newer version, let's take a quick look at the functionality of Django Celery.

Create a project

Django-admin startproject django_celery

Initialize an application

Cd django_celery

Django-admin startapp celery_app

We modify the configuration of settings.py.

It is important to note here that if we do not use Redis,RabbitMQ, it is also possible to test using our own broker service.

If you enable your own configuration, the configuration of settings.py is as follows:

INSTALLED_APPS = (

'django.contrib.admin'

'django.contrib.auth'

'django.contrib.contenttypes'

'django.contrib.sessions'

'django.contrib.messages'

'django.contrib.staticfiles'

'celery_app'

'djcelery'

'kombu.transport.django'

)

BROKER_URL = 'django://localhost:8000//'

If you are using RabbitMQ, we need to deploy and install this message queue separately, available

Yum install rabbitmq-server is fine, the project itself is developed in erlang, so a large number of erlang-related packages will be installed.

The configuration of settings.py is as follows:

Import djcelery

Djcelery.setup_loader ()

BROKER_URL= 'amqp://guest@localhost//'

CELERY_RESULT_BACKEND = 'amqp://guest@localhost//'

# Application definition

INSTALLED_APPS = (

'django.contrib.admin'

'django.contrib.auth'

'django.contrib.contenttypes'

'django.contrib.sessions'

'django.contrib.messages'

'django.contrib.staticfiles'

'celery_app'

'djcelery'

'kombu.transport.django'

)

Then we configure the information for the task and create the file celery.py under the django-celery project directory

From _ _ future__ import absolute_import

Import os

From celery import Celery

From django.conf import settings

# set the default Django settings module for the 'celery' program.

Os.environ.setdefault ('DJANGO_SETTINGS_MODULE',' django_celery.settings')

App = Celery ('django_celery')

# Using a string here means the worker will not have to

# pickle the object when using Windows.

App.config_from_object ('django.conf:settings')

App.autodiscover_tasks (lambda: settings.INSTALLED_APPS)

@ app.task (bind=True)

Def debug_task (self):

Print ('Request: {0roomr}' .format (self.request))

Under the directory where the celery_app is applied, create the task tasks.py, and we define several methods to call.

From _ _ future__ import absolute_import

From celery import shared_task

Import time

@ shared_task

Def add (x, y):

Return x + y

@ shared_task

Def mul (x, y):

Time.sleep (10)

Return x * y

@ shared_task

Def xsum (numbers):

Time.sleep (10)

Return sum (numbers)

After the basic configuration is complete, let's try it out.

Then configure the information for DB, using the command

Python manage.py syncdb

This process will prompt you to create a superuser, just do it.

Start the service

Python manage.py runserver

Then open another window and start the celery service

Python manage.py celery worker-l info

This process is likely to lead to warnings:

Root@localhost django_celery] # python manage.py celery worker-l info

Running a worker with superuser privileges when the

Worker accepts messages serialized with pickle is a very bad idea!

If you really want to continue then you have to set the C_FORCE_ROOT

Environment variable (but please think about this before you do).

User information: uid=0 euid=0 gid=0 egid=0

In fact, the meaning is very clear, if you need to confirm, to set the variable C_FORCE_ROOT, the style is very similar to sandbox.

Export C_FORCE_ROOT=test

> python manage.py celery worker-l info

You can see the information about task from the startup log:

[tasks]

. Celery_app.tasks.add

. Celery_app.tasks.mul

. Celery_app.tasks.xsum

. Django_celery.celery.debug_task

Start a new session again, which is session 3, and we open the shell interaction window.

> from celery_app.tasks import *

> > dir ()

['_ _ builtins__', 'absolute_import',' add', 'mul',' shared_task', 'xsum']

> mul (5pm 2)

ten

At this time, if you use delay,add, you will enter the message queue.

> mul.delay (5pm 2)

> > >

> add.delay (2Pol. 3)

> > >

View the log information of worker as follows:

Received task: celery_ app.tasks.add.bac53d49-24cf-4d07-8515-8eff8083cab9]

Task celery_ app.tasks.add.bac53d49-24cf-4d07-8515-8eff8083cab9] succeeded in 0.0008037839998s: 6

Logs that use RabbitMQ are similar.

If you want to enable the flower interface, it will be done in minutes.

Install flower:

Pip install flower

Start the service

Python manage.py celery flower

Access Port:

Http://127.0.0.1:5555/

The above content is how to conduct Django Celery analysis. Have you learned any knowledge or skills? If you want to learn more skills or enrich your knowledge reserve, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.