Multiprocessing Worker - celery.worker¶

Utility functions

celery.utils.chunks(it, n)¶

Split an iterator into chunks with n elements each.

Examples

# n == 2 >>> x = chunks(iter([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]), 2) >>> list(x) [[0, 1], [2, 3], [4, 5], [6, 7], [8, 9], [10]]

# n == 3 >>> x = chunks(iter([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]), 3) >>> list(x) [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9, 10]]

celery.utils.fun_takes_kwargs(fun, kwlist=[])¶

With a function, and a list of keyword arguments, returns arguments in the list which the function takes.

Parameters:	fun – The function to inspect arguments of. kwlist – The list of keyword arguments.

Examples

>>> def foo(self, x, y, logfile=None, loglevel=None):
...     return x * y
>>> fun_takes_kwargs(foo, ["logfile", "loglevel", "task_id"])
["logfile", "loglevel"]

>>> def foo(self, x, y, **kwargs):
>>> fun_takes_kwargs(foo, ["logfile", "loglevel", "task_id"])
["logfile", "loglevel", "task_id"]

celery.utils.gen_unique_id()¶

Generate a unique id, having - hopefully - a very small chance of collission.

For now this is provided by uuid.uuid4().

celery.utils.get_full_cls_name(cls)¶

With a class, get its full module and class name.

celery.utils.mitemgetter(*keys)¶

Like operator.itemgetter() but returns None on missing keys instead of raising KeyError.

celery.utils.repeatlast(it)¶

Iterate over all elements in the iterator, and when its exhausted yield the last value infinitely.

celery.utils.retry_over_time(fun, catch, args=[], kwargs={}, errback=<function <lambda> at 0x10301f6e0>, max_retries=None, interval_start=2, interval_step=2, interval_max=30)¶

Retry the function over and over until max retries is exceeded.

For each retry we sleep a for a while before we try again, this interval is increased for every retry until the max seconds is reached.

Parameters:

fun – The function to try catch – Exceptions to catch, can be either tuple or a single exception class. args – Positional arguments passed on to the function. kwargs – Keyword arguments passed on to the function. errback – Callback for when an exception in catch is raised. The callback must take two arguments: exc and interval, where exc is the exception instance, and interval is the time in seconds to sleep next.. max_retries – Maximum number of retries before we give up. If this is not set, we will retry forever. interval_start – How long (in seconds) we start sleeping between retries. interval_step – By how much the interval is increased for each retry. interval_max – Maximum number of seconds to sleep between retries.