apalis.GroupHandler

class apalis.GroupHandler(objs, threads=None, affinity=False, params=None)

The GroupHandler class saves a list of objects into a forked processes and provides an interface to interact with them. Depending on how many CPU cores your machine has it will save several objects into one child process. This reduces overhead.

Parameters

objslist

List of objects to be send to forked processes.

threadsint, optional

Number of child processes to be created, by default CPU cre number

affinitybool, optional

If affinity is True it will set the affinity of the child processes to one core. This can lead to a faster execution time in some instances, by default False

paramslist, by default None

If this argument is given the object will be called after the child process has been forked. A list of list with the structre [[args, kwargs], …] should be given.

Examples

>>> gh = apalis.GroupHandler([A() for _ in range(2)])
>>> tasks = [h.f(5) for h in gh]
>>> token = gh.multiple_run(tasks)
>>> token()
[5, 5]

Methods

multiple_run(tasks)	Executes several tasks.
run(tasks)	Runs the tasks and directly returns the ouputs.
single_run(task)	Executes a single task.

multiple_run(tasks)

Executes several tasks.

Parameters

taskdict

Dictionary contatining the required fields to execute a function, get an attribute, etc.

Returns

Token

A token is returned. If it is called the main process will wait until the child process has returned the outputs of the function.

Examples

>>> tasks = [{'i': i, name': '<function_name>', 'args': args, 'kwargs': kwargs, 'mode': 'run'} for i in range(16)]
>>> gh.multiple_run(tasks)

single_run(task)

Executes a single task.

Parameters

taskdict

Dictionary contatining the required fields to execute a function, get an attribute, etc.

Returns

Token

A token is returned. If it is called the main process will wait until the child process has returned the outputs of the function.

Examples

>>> gh.single_run([{'i': i, 'name': '<function_name>', 'args': args, 'kwargs': kwargs, 'mode': 'run'}])

run(tasks)

Runs the tasks and directly returns the ouputs. It is faster than multiple_run since it does not need to deal with Tokens.

Parameters

taskslist(dict)

List of tasks.

Returns

list

List with the output of the tasks.

Examples

>>> tasks = [{'i': i, name': '<function_name>', 'args': args, 'kwargs': kwargs, 'mode': 'run'} for i in range(16)]
>>> gh.run(tasks)