datacube.utils.dask.compute_tasks

datacube.utils.dask.compute_tasks(tasks, client, max_in_flight=3)

Parallel compute stream with back pressure.

Equivalent to:

(client.compute(task).result()
  for task in tasks)

but with up to max_in_flight tasks being processed at the same time. Input/Output order is preserved, so there is a possibility of head of line blocking.

Note

lower limit is 3 concurrent tasks to simplify implementation, there is no point calling this function if you want one active task and supporting exactly 2 active tasks is not worth the complexity, for now. We might special-case 2 at some point.

Return type

Iterable[Any]