Task Pool - celery.pool¶

Process Pools.

class celery.pool.DynamicPool(processes=None, initializer=None, initargs=())¶

Version of multiprocessing.Pool that can dynamically grow in size.

add_worker()¶

Add another worker to the pool.

grow(size=1)¶

Add workers to the pool.

Parameter:	size – Number of workers to add (default: 1)

replace_dead_workers()¶

Replace dead workers in the pool by spawning new ones.

Returns:	number of dead processes replaced, or None if all processes are alive and running.

class celery.pool.TaskPool(limit, logger=None)¶

Process Pool for processing tasks in parallel.

Parameters:	limit – see limit attribute. logger – see logger attribute.

limit¶

The number of processes that can run simultaneously.

logger¶

The logger used for debugging.

apply_async(target, args=None, kwargs=None, callbacks=None, errbacks=None, on_ack=<function <lambda> at 0x10301f6e0>)¶

Equivalent of the :func:apply built-in function.

All callbacks and errbacks should complete immediately since otherwise the thread which handles the result will get blocked.

on_ready(callbacks, errbacks, on_ack, ret_value)¶

What to do when a worker task is ready and its return value has been collected.

start()¶

Run the task pool.

Will pre-fork all workers so they’re ready to accept tasks.

stop()¶

Terminate the pool.

celery.pool.pid_is_dead(pid)¶

Check if a process is not running by PID.

Rtype bool:

celery.pool.process_is_dead(process)¶

Check if process is not running anymore.

First it finds out if the process is running by sending signal 0. Then if the process is a child process, and is running it finds out if it’s a zombie process and reaps it. If the process is running and is not a zombie it tries to send a ping through the process pipe.

Parameter:	process – A multiprocessing.Process instance.
Returns:	True if the process is not running, False otherwise.

celery.pool.reap_process(pid)¶

Reap process if the process is a zombie.

Returns:	True if process was reaped or is not running, False otherwise.