Functions

apalis.get(tokens)

Returns the result of tasks executed in parallel by apalis.

Parameters

tokenslist

List of objects of the type Token.

Returns

list

List of results

Examples

>>> handler_list = [Handler(ClassName(i)) for i in range(10)]
>>> apalis.get([g.f() for g in handler_list])

apalis.put(obj)

Loads object into Plasma. Will only work if plasma has been initialized with init_plasma()

Parameters

obj :

Object that will be shared through Plasma.

Returns

ObjectId

A ObjectId generated by Plasma that can be be passed to functions that are executed in a child process by apalis.

Examples

>>> apalis.put(1)
ObjectID(bad02ba2c0f59e5f55033298520668cbf0ad1102)

apalis.init_plasma(mem=1000)

Initializes a Plasma object store.

Args:

mem (int, optional): The argument specifies the size of the store in megabytes. Defaults to 1000.

Returns:

(PlasmaClient): Plasma client object

apalis.patch_pickeling()

Exchanges the pickle protocol from multiprocessing to the one used by cloudpickle.

From the cloudpickle README: cloudpickle is especially useful for cluster computing where Python code is shipped over the network to execute on remote hosts, possibly close to the data. Among other things, cloudpickle supports pickling for lambda functions along with functions and classes defined interactively in the __main__ module (for instance in a script, a shell or a Jupyter notebook).