Masking#

Bit Masking#

Masking No Data Values#

datacube.utils.masking.mask_invalid_data(data, keep_attrs=True)[source]#

Sets all nodata values to nan.

This will convert numeric data to type float.

Parameters:

  • data (Dataset or DataArray)

  • keep_attrs (bool) – If the attributes of the data should be included in the returned .

Returns:

Dataset or DataArray

Masking with Bit-Flag Measurements#

One of the common types of data used with the Data Cube contains discrete values stored within a numeric value. These values are often classifications and outputs from tests, and need to be interpreted in specific ways, not as simple scalar values. They are often used as a mask to exclude observations which are deemed unsuitable for a given analysis. For example, we want to exclude observations of clouds when we are interested in what is on the ground.

Several methods are used when encoding these types of variables:

  • On-off bit flags, for a particular binary bit

  • Collections of bits that can indicate more than two possible values

  • Looking for a specific value stored using all available bits in the variable

From prior work, it is very easy to make mistakes when using these types of variables, which can lead to processing the wrong set of observations, and also making it quite difficult to read the code using them, and ensuring that they are used in a consistent way in different places.

Open Data Cube provides a way of describing the meanings that can be encoded in variables, which can then be used to give a readable method when using that variable.

datacube.utils.masking.describe_variable_flags(variable, with_pandas=True)[source]#

Returns either a Pandas Dataframe (with_pandas=True - default) or a string (with_pandas=False) describing the available flags for a masking variable

Interprets the flags_definition attribute on the provided variable and returns a Pandas Dataframe or string like:

Bits are listed from the MSB (bit 13) to the LSB (bit 0)
Bit     Value   Flag Name            Description
13      0       cloud_shadow_fmask   Cloud Shadow (Fmask)
12      0       cloud_shadow_acca    Cloud Shadow (ACCA)
11      0       cloud_fmask          Cloud (Fmask)
10      0       cloud_acca           Cloud (ACCA)
Parameters:

variable – Masking xarray.Dataset or xarray.DataArray

Returns:

Pandas Dataframe or str

datacube.utils.masking.make_mask(variable, **flags)[source]#

Returns a mask array, based on provided flags

When multiple flags are provided, they will be combined in a logical AND fashion.

For example:

>>> make_mask(pqa, cloud_acca=False, cloud_fmask=False, land_obs=True)

OR

>>> make_mask(pqa, **GOOD_PIXEL_FLAGS)

where GOOD_PIXEL_FLAGS is a dict of flag_name to True/False

Parameters:
Returns:

boolean xarray.DataArray or xarray.Dataset

How to Define Meanings on Measurements#

# TODO

How to Create Masks within code#

# TODO