datacube.utils.dask.pmap#

datacube.utils.dask.pmap(func, its, client, lump=1, max_in_flight=3, name='compute')[source]#

Parallel map with back pressure.

Equivalent to this:

(func(x) for x in its)

Except that func(x) runs concurrently on dask cluster.

Parameters

  • func (Any) – Method that will be applied concurrently to data from its

  • its (Iterable[Any]) – Iterator of input values

  • client (Client) – Connected dask client

  • lump (int) – Group this many datasets into one task

  • max_in_flight (int) – Maximum number of active tasks to submit

  • name (str) – Dask name for computation

Return type

Iterable[Any]