datacube.utils.dask.compute_tasks(tasks, client, max_in_flight=3)[source]#

Parallel compute stream with back pressure.

Equivalent to:

  for task in tasks)

but with up to max_in_flight tasks being processed at the same time. Input/Output order is preserved, so there is a possibility of head of line blocking.


lower limit is 3 concurrent tasks to simplify implementation, there is no point calling this function if you want one active task and supporting exactly 2 active tasks is not worth the complexity, for now. We might special-case 2 at some point.

Return type