================================= celery – Distributed Task Queue =================================<\/p>\n
\u2026 image:: http:\/\/cloud.github.com\/downloads\/celery\/celery\/celery_128.png<\/p>\n
|build-status| |coverage-status|<\/p>\n
:Version: 3.2.0a1 (Cipater) :Web: http:\/\/celeryproject.org\/ :Download: http:\/\/pypi.python.org\/pypi\/celery\/ :Source: http:\/\/github.com\/celery\/celery\/ :Keywords: task queue, job queue, asynchronous, async, rabbitmq, amqp, redis, python, webhooks, queue, distributed<\/p>\n
\u2013<\/p>\n
Task queues are used as a mechanism to distribute work across threads or machines.<\/p>\n
A task queue\u2019s input is a unit of work, called a task, dedicated worker processes then constantly monitor the queue for new work to perform.<\/p>\n
Celery communicates via messages, usually using a broker to mediate between clients and workers. To initiate a task a client puts a message on the queue, the broker then delivers the message to a worker.<\/p>\n
A Celery system can consist of multiple workers and brokers, giving way to high availability and horizontal scaling.<\/p>\n
Celery is a library written in Python, but the protocol can be implemented in any language. So far there\u2019s RCelery_ for the Ruby programming language, and a \u2026 _RCelery: https:\/\/github.com\/leapfrogonline\/rcelery \u2026 _ Celery version 3.0 runs on,<\/p>\n This is the last version to support Python 2.5, and from Celery 3.1, Python 2.6 or later is required. The last version to support Python 2.4 was Celery series 2.2.<\/p>\n Celery<\/em> is usually used with a message broker to send and receive messages. The RabbitMQ, Redis transports are feature complete, but there\u2019s also experimental support for a myriad of other solutions, including using SQLite for local development.<\/p>\n Celery<\/em> can run on a single machine, on multiple machines, or even across datacenters.<\/p>\n If this is the first time you\u2019re trying to use Celery, or you are new to Celery 3.0 coming from previous versions then you should read our getting started tutorials:<\/p>\n Tutorial teaching you the bare minimum needed to get started with Celery.<\/p>\n<\/li>\n A more complete overview, showing more features.<\/p>\n<\/li>\n<\/ul>\n \u2026 _ \u2026 _ Simple<\/strong><\/p>\n Celery is easy to use and maintain, and does not need configuration files<\/em>.<\/p>\n It has an active, friendly community you can talk to for support, including a Here\u2019s one of the simplest applications you can make::<\/p>\n Highly Available<\/strong><\/p>\n Workers and clients will automatically retry in the event of connection loss or failure, and some brokers support HA in way of Master\/Master<\/em> or Master\/Slave<\/em> replication.<\/p>\n<\/li>\n Fast<\/strong><\/p>\n A single Celery process can process millions of tasks a minute, with sub-millisecond round-trip latency (using RabbitMQ, py-librabbitmq, and optimized settings).<\/p>\n<\/li>\n Flexible<\/strong><\/p>\n Almost every part of Celery<\/em> can be extended or used on its own, Custom pool implementations, serializers, compression schemes, logging, schedulers, consumers, producers, autoscalers, broker transports and much more.<\/p>\n<\/li>\n<\/ul>\n \u2026 _ \u2026 _RabbitMQ: http:\/\/rabbitmq.com \u2026 _Redis: http:\/\/redis.io \u2026 _MongoDB: http:\/\/mongodb.org \u2026 _Beanstalk: http:\/\/kr.github.com\/beanstalkd \u2026 _CouchDB: http:\/\/couchdb.apache.org \u2026 _SQLAlchemy: http:\/\/sqlalchemy.org \u2026 _ Celery is easy to integrate with web frameworks, some of which even have integration packages:<\/p>\n The integration packages are not strictly necessary, but they can make development easier, and sometimes they add important hooks like closing database connections at \u2026 _ \u2026 _celery-documentation:<\/p>\n The \u2026 _ \u2026 _celery-installation:<\/p>\n You can install Celery either via the Python Package Index (PyPI) or from source.<\/p>\n To install using To install using \u2026 _bundles:<\/p>\n Celery also defines a group of bundles that can be used to install Celery and the dependencies for a given feature.<\/p>\n You can specify these in your requirements or on the The following bundles are available:<\/p>\n Serializers<\/p>\n :celery[eventlet]: for using the eventlet pool.<\/p>\n :celery[gevent]: for using the gevent pool.<\/p>\n :celery[threads]: for using the thread pool.<\/p>\n Transports and Backends<\/p>\n ================================= celery – Distributed Task Queue ================================= \u2026 image:: http:\/\/cloud.github.com\/downloads\/celery\/celery\/celery_128.png |build-status| |coverage-status| :Version: 3.2.0a1 (Cipater) :Web: http:\/\/celeryproject.org\/ :Download: http:\/\/pypi.python.org\/pypi\/celery\/ :Source: http:\/\/github.com\/celery\/celery\/ :Keywords: task queue, job queue, asynchronous, async, rabbitmq, amqp, redis, python, webhooks, queue, distributed \u2013 What is a Task Queue? Task queues are used as a mechanism to distribute work across threads or machines. […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[9,1],"tags":[],"class_list":["post-7591","post","type-post","status-publish","format-standard","hentry","category-php-memcached","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/posts\/7591","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/comments?post=7591"}],"version-history":[{"count":1,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/posts\/7591\/revisions"}],"predecessor-version":[{"id":8998,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/posts\/7591\/revisions\/8998"}],"wp:attachment":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/media?parent=7591"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/categories?post=7591"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/tags?post=7591"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}PHP client<\/code>, but language interoperability can also be achieved by using webhooks.<\/p>\nPHP client<\/code>: https:\/\/github.com\/gjedeer\/celery-php \u2026 _using webhooks<\/code>: http:\/\/docs.celeryproject.org\/en\/latest\/userguide\/remote-tasks.html<\/p>\nWhat do I need?<\/h2>\n
\n
Get Started<\/h2>\n
\n
First steps with Celery<\/code>_<\/p>\nNext steps<\/code>_<\/p>\nFirst steps with Celery<\/code>: http:\/\/docs.celeryproject.org\/en\/latest\/getting-started\/first-steps-with-celery.html<\/p>\nNext steps<\/code>: http:\/\/docs.celeryproject.org\/en\/latest\/getting-started\/next-steps.html<\/p>\nCelery is\u2026<\/h2>\n
\n
mailing-list<\/code>_ and and an IRC channel.<\/p>\n from celery import Celery\n\n app = Celery('hello', broker='amqp:\/\/guest@localhost\/\/')\n\n @app.task\n def hello():\n return 'hello world'\n<\/code><\/pre>\n<\/li>\n- **Message Transports**\n\n - RabbitMQ_, Redis_,\n - MongoDB_ (experimental), Amazon SQS (experimental),\n - CouchDB_ (experimental), SQLAlchemy_ (experimental),\n - Django ORM (experimental), `IronMQ`_\n - and more...\n\n- **Concurrency**\n\n - Prefork, Eventlet_, gevent_, threads\/single threaded\n\n- **Result Stores**\n\n - AMQP, Redis\n - memcached, MongoDB\n - SQLAlchemy, Django ORM\n - Apache Cassandra, IronCache\n\n- **Serialization**\n\n - *pickle*, *json*, *yaml*, *msgpack*.\n - *zlib*, *bzip2* compression.\n - Cryptographic message signing.\n<\/code><\/pre>\nEventlet<\/code>: http:\/\/eventlet.net\/ \u2026 _gevent<\/code>: http:\/\/gevent.org\/<\/p>\nIronMQ<\/code>: http:\/\/iron.io<\/p>\nFramework Integration<\/h2>\n
+--------------------+------------------------+\n| `Django`_ | not needed |\n+--------------------+------------------------+\n| `Pyramid`_ | `pyramid_celery`_ |\n+--------------------+------------------------+\n| `Pylons`_ | `celery-pylons`_ |\n+--------------------+------------------------+\n| `Flask`_ | not needed |\n+--------------------+------------------------+\n| `web2py`_ | `web2py-celery`_ |\n+--------------------+------------------------+\n| `Tornado`_ | `tornado-celery`_ |\n+--------------------+------------------------+\n<\/code><\/pre>\nfork<\/code>.<\/p>\nDjango<\/code>: http:\/\/djangoproject.com\/ \u2026 _Pylons<\/code>: http:\/\/www.pylonsproject.org\/ \u2026 _Flask<\/code>: http:\/\/flask.pocoo.org\/ \u2026 _web2py<\/code>: http:\/\/web2py.com\/ \u2026 _Bottle<\/code>: http:\/\/bottlepy.org\/ \u2026 _Pyramid<\/code>: http:\/\/docs.pylonsproject.org\/en\/latest\/docs\/pyramid.html \u2026 _pyramid_celery<\/code>: http:\/\/pypi.python.org\/pypi\/pyramid_celery\/ \u2026 _django-celery<\/code>: http:\/\/pypi.python.org\/pypi\/django-celery \u2026 _celery-pylons<\/code>: http:\/\/pypi.python.org\/pypi\/celery-pylons \u2026 _web2py-celery<\/code>: http:\/\/code.google.com\/p\/web2py-celery\/ \u2026 _Tornado<\/code>: http:\/\/www.tornadoweb.org\/ \u2026 _tornado-celery<\/code>: http:\/\/github.com\/mher\/tornado-celery\/<\/p>\nDocumentation<\/h2>\n
latest documentation<\/code>_ with user guides, tutorials and API reference is hosted at Read The Docs.<\/p>\nlatest documentation<\/code>: http:\/\/docs.celeryproject.org\/en\/latest\/<\/p>\nInstallation<\/h2>\n
pip<\/code>,::<\/p>\n$ pip install -U Celery\n<\/code><\/pre>\neasy_install<\/code>,::<\/p>\n$ easy_install -U Celery\n<\/code><\/pre>\nBundles<\/h2>\n
pip<\/code> comand-line by using brackets. Multiple bundles can be specified by separating them by commas. ::<\/p>\n$ pip install \"celery[librabbitmq]\"\n\n$ pip install \"celery[librabbitmq,redis,auth,msgpack]\"\n<\/code><\/pre>\n\n:celery[auth]:\n for using the auth serializer.\n\n:celery[msgpack]:\n for using the msgpack serializer.\n\n:celery[yaml]:\n for using the yaml serializer.\n\nConcurrency\n<\/code><\/pre>\n\n:celery[librabbitmq]:\n for using the librabbitmq C library.\n\n:celery[redis]:\n for using Redis as a message transport or as a result backend.\n\n:celery[mongodb]:\n for using MongoDB as a message transport (*experimental*),\n or as a result backend (*supported*).\n\n:celery[sqs]:\n for using Amazon SQS as a message transport (*experimental*).\n\n:celery[memcache]:\n for using memcached as a result backend.\n\n:celery[cassandra]:\n for using Apache Cassandra as a result backend.\n\n:celery[couchdb]:\n for using CouchDB as a message transport (*experimental*).\n\n:celery[couchbase]:\n for using CouchBase as a result backend.\n\n:celery[beanstalk]:\n for using Beanstalk as a message transport (*experimental*).\n\n:celery[zookeeper]:\n for using Zookeeper as a message transport.\n\n:celery[zeromq]:\n for using ZeroMQ as a message transport (*experimental*).\n\n:celery[sqlalchemy]:\n for using SQLAlchemy as a message transport (*experimental*),\n or as a result backend (*supported*).\n\n:celery[pyro]:\n for using the Pyro4 message transport (*experimental*).\n\n:celery[slmq]:\n for using the SoftLayer Message Queue transport (*experimental*).\n\n.. _celery-installing-from-source:\n\nDownloading and installing from source\n--------------------------------------\n\nDownload the latest version of Celery from\nhttp:\/\/pypi.python.org\/pypi\/celery\/\n\nYou can install it by doing the following,::\n\n $ tar xvfz celery-0.0.0.tar.gz\n $ cd celery-0.0.0\n $ python setup.py build\n # python setup.py install\n\nThe last command must be executed as a privileged user if\nyou are not currently using a virtualenv.\n\n.. _celery-installing-from-git:\n\nUsing the development version\n-----------------------------\n\nWith pip\n~~~~~~~~\n\nThe Celery development version also requires the development\nversions of ``kombu``, ``amqp`` and ``billiard``.\n\nYou can install the latest snapshot of these using the following\npip commands::\n\n $ pip install https:\/\/github.com\/celery\/celery\/zipball\/master#egg=celery\n $ pip install https:\/\/github.com\/celery\/billiard\/zipball\/master#egg=billiard\n $ pip install https:\/\/github.com\/celery\/py-amqp\/zipball\/master#egg=amqp\n $ pip install https:\/\/github.com\/celery\/kombu\/zipball\/master#egg=kombu\n\nWith git\n~~~~~~~~\n\nPlease the Contributing section.\n\n.. _getting-help:\n\nGetting Help\n============\n\n.. _mailing-list:\n\nMailing list\n------------\n\nFor discussions about the usage, development, and future of celery,\nplease join the `celery-users`_ mailing list.\n\n.. _`celery-users`: http:\/\/groups.google.com\/group\/celery-users\/\n\n.. _irc-channel:\n\nIRC\n---\n\nCome chat with us on IRC. The **#celery** channel is located at the `Freenode`_\nnetwork.\n\n.. _`Freenode`: http:\/\/freenode.net\n\n.. _bug-tracker:\n\nBug tracker\n===========\n\nIf you have any suggestions, bug reports or annoyances please report them\nto our issue tracker at http:\/\/github.com\/celery\/celery\/issues\/\n\n.. _wiki:\n\nWiki\n====\n\nhttp:\/\/wiki.github.com\/celery\/celery\/\n\n\n.. _maintainers:\n\nMaintainers\n===========\n\n- `@ask`_ (primary maintainer)\n- `@thedrow`_\n- `@chrisgogreen`_\n- `@PMickael`_\n- `@malinoff`_\n- And you? We really need more: https:\/\/github.com\/celery\/celery\/issues\/2534 \n\n.. _`@ask`: http:\/\/github.com\/ask\n.. _`@thedrow`: http:\/\/github.com\/thedrow\n.. _`@chrisgogreen`: http:\/\/github.com\/chrisgogreen\n.. _`@PMickael`: http:\/\/github.com\/PMickael\n.. _`@malinoff`: http:\/\/github.com\/malinoff\n\n\n.. _contributing-short:\n\nContributing\n============\n\nDevelopment of `celery` happens at Github: http:\/\/github.com\/celery\/celery\n\nYou are highly encouraged to participate in the development\nof `celery`. If you don't like Github (for some reason) you're welcome\nto send regular patches.\n\nBe sure to also read the `Contributing to Celery`_ section in the\ndocumentation.\n\n.. _`Contributing to Celery`:\n http:\/\/docs.celeryproject.org\/en\/master\/contributing.html\n\n.. _license:\n\nLicense\n=======\n\nThis software is licensed under the `New BSD License`. See the ``LICENSE``\nfile in the top distribution directory for the full license text.\n\n.. # vim: syntax=rst expandtab tabstop=4 shiftwidth=4 shiftround\n\n\n.. image:: https:\/\/d2weczhvl823v0.cloudfront.net\/celery\/celery\/trend.png\n :alt: Bitdeli badge\n :target: https:\/\/bitdeli.com\/free\n\n.. |build-status| image:: https:\/\/travis-ci.org\/celery\/celery.svg?branch=master\n :target: https:\/\/travis-ci.org\/celery\/celery\n.. |coverage-status| image:: https:\/\/coveralls.io\/repos\/celery\/celery\/badge.svg\n :target: https:\/\/coveralls.io\/r\/celery\/celery\n<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"