This is aimed at those people who already have a python project with django in active heroku, the steps to follow to upload a project will be ignored.

Previous Requirements
First we need have install in our project celery and django-celery so let’s go see:

  1. installl celery and django-celery in your virtualenv:
    “pip install Celery django-celery”

Note: remember each new dependency install have to added in the file requiremenst.txt for when you upload the changes to heroku he install this dependencies. you can see all dependencies instaled runing the comad pip freeze

 2. After they are installed all dependency, in our setting.py have to added:

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'django.contrib.sites',
    'djcelery'
]

besides importing

import djcelery
djcelery.setup_loader()

After performing this configuration execute the command:
“python manage.py migrate djcelery”
with this command have to create the tables neceary to configure the djcelery in django admin. If you want to expand more you can visit the following link https://github.com/celery/django-celery y http://www.celeryproject.org/

Note: For to realize local test is necesary have to install a handler for the message queue between django and celery which can be Redis https://redis.io/downloador Rabbitmq https://www.rabbitmq.com/download.html

  • If you decide utilice redis have to install one aditional dependency in our virutalenv “pip install -U “celery[redis]”, as well as adding it to the requirements.txt file. affter added in the file setting.py CELERY_BROKER_URL = ‘redis://localhost:6379/0’
  • if you decide utilice rabbitmq addd in the file setting.py: CELERY_BROKER_URL = ‘amqp://localhost’

Remember that the redis or the rabbitmq must be turned on by the console to work

Configuration of celery
we have to create a new file whose name is celery.py in the configuration folder in our project

  • Navigate to the root project configuration module (where are the URL modules and settings):
    yourproject
    -app
    -migrations
    __init__.py
    0001_initial.py
    admin.py
    app.ys
    models.py
                     tasks.py
    tests.py
    view.py
    -yourproject
                     __init__.py
                     celery.py
    setting.py
    urls.py
    wsgi.py
  • Inside of the file write this:
    from __future__ import absolute_import, unicode_literals
    import os
    from celery import Celery
    from django.conf import settings
    # set the default Django settings module for the 'celery' program.
    os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'yourproject.settings')
    app = Celery('yourproject')
    # Using a string here means the worker don't have to serialize
    # the configuration object to child processes.
    # - namespace='CELERY' means all celery-related configuration keys
    # should have a `CELERY_` prefix.
    app.config_from_object('django.conf:settings')
    
    # Load task modules from all registered Django app configs.
    app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)
  • then update the file __init__.py in the configuration folder in our project:
    from __future__ import absolute_import, unicode_literals
    	
    	# This will make sure the app is always imported when
    	# Django starts so that shared_task will use this app.
    	from .celery import app as celery_app
    
    	__all__ = ['celery_app']

with this have the celery correctily configured in our project. now we will proceed to realice the tasks

Creating asynchronous and periodic tasks

  • we need create a file tasks.py in the Django aplication you want (remember this aplication it must be valid in INSTALLED_APPS):
    from __future__ import absolute_import, unicode_literals
    from celery import shared_task
    from account import models as account_models
    
    
    @shared_task
    def disableCodeRecoveryPassword(instance):
        instance = account_models.User.objects.get(id=instance)
        instance.recovery = ''
        instance.save()
        return True
  • to finish we will activated the worker with the next command: “celery -A yourproject worker -l info”. but before having your server running in another terminal staying like this:

now we have all configuration to need to iniciate a tasks asynchronous in django, now we show to you how to run in heroku

Project in heroku

Note: To be able to use celery in heroku we have need a pay account, because when we try to install the redis or rabbitmq addons we will request a credit card

  • in the procfile file we must added two new line being that way:
    web: gunicorn yourproject.wsgi --log-file -
    worker: python manage.py celery worker --without-gossip --without-mingle --loglevel=info
    beat: python manage.py celery beat –loglevel=info
  • then we must added in the file setting.py:
    BROKER_URL = os.environ.get("CLOUDAMQP_URL", "django://")
    
    BROKER_POOL_LIMIT = 1
    BROKER_CONNECTION_MAX_RETRIES = None
    BROKER_HEARTBEAT = None 
    RESULT_BACKEND = None EVENT_QUEUE_EXPIRES = 60 
    WORKER_PREFETCH_MULTIPLIER = 1 
    WORKER_CONCURRENCY = 50 
    CELERYBEAT_SCHEDULER ='djcelery.schedulers.DatabaseScheduler'

After to upload all changes in heroku we must added a new addons in this example used the addons cloudAMQP writting the next command in console
“$ heroku addons:add cloudamqp
or you can go to heroku in the resource part and look for the addons and install it:

Note: the previus configuration only work with cloudamqp I invite you to do your tests with redis install the addonshttps://elements.heroku.com/addons/rediscloud

Then we have all ready upload the project to heroku, now to activate the worker in heroku we must execute next command “$ heroku ps:scale worker=1”

if you need to activate the beat, then do it using the same command, only changing the worker for the beat “$ heroku ps:scale beat=1”

to simplify things you can activate both at the same time
“$ heroku ps:scale worker=1 beat=1”

only in the case that you are using the asynchronous and periodic tasks

In doing so, this will be shown in the console
“heroku-cli: update available from 6.14.36-15f8a25 to 6.15.25-635e5c6
Scaling dynos… done, now running beat at 1:Hobby, worker at 1:Hobby”
for all cases if only active the worker will exit: worker at 1:Hobby

to off the worker execute the next command:
$ heroku ps:scale worker=0 beat=0

to verify that the worker is working well execute the command: “heroku ps” 

inally show the logs the heroku executing the command
heroku logs -t -p worker

heroku logs -t -p beat


to see if we have an error when uploading it or it was done correctly

to finist execute the command:
“heroku restart”

if you followed all the steps to the letter you can perform leave asynchronous tasks in heroku
I hope this guide has been useful, share and leave your comments 🙂