Products#

Products are collections of datasets that share the same set of measurements and some subset of metadata.

Product Definition#

A product definition document describes the measurements and common metadata for a collection of datasets.

Example Product Definition#

name: landsat8_example_product
description: Landsat 8 example product
metadata_type: eo3

metadata:
    product:
        name: landsat8_example_product
    # Alternatively, include specific items to match
    # properties:
        # eo:instrument: OLI_TIRS
        # eo:platform: landsat-8

measurements:
    - name: 'red'
      aliases: [band_4, sr_band4]
      dtype: int16
      nodata: -9999
      units: 'reflectance'
    - name: 'blue'
      aliases: [band_2, sr_band2]
      dtype: int16
      nodata: -9999
      units: 'reflectance'
    - name: 'pixel_qa'
      aliases: [pixel_quality, level2_qa]
      dtype: uint16
      nodata: 1
      units: 'bit_index'
      flags_definition:
        pixel_qa:
          bits: [0,1,2,3,4,5,6,7,8,9,10,11]
          description: Level 2 pixel quality band 
          values:
            1: Fill
            2: Clear
            4: Water
            8: Cloud shadow
            16: Snow
            32: Cloud
            64: Cloud Confidence Low Bit
            128: Cloud Confidence High Bit
            256: Cirrus Confidence Low Bit
            512: Cirrus Confidence High Bit
            1024: Terrain Occlusion
            2048: Unused  # Be careful of repeated names which could confuse the masking code
        # Alternatively or additionally, use the bit on/off method
        fill:
          bits: 0
          description: No data
          values: {0: false, 1: true}
        clear:
          bits: 1
          description: Clear
          values: {0: no_clear_land, 1: clear_land}
        # ...
        cloud_confidence:
          bits: [6, 7]
          description: Cloud confidence
          values: {0: none, 1: low, 2: medium, 3: high}
        # ...
            

Product Definition API#

name

Product name

description

Product description

metadata_type

Name of the metadata-type-definition

license

The license of the data.

This is either a SPDX License identifier (eg ‘CC-BY-SA-4.0’) or ‘various’ or ‘proprietary’

metadata

Dictionary containing bits of metadata common to all the datasets in the product.

It is used during indexing to match datasets to their products. That is, the keys and values defined here must also be in the dataset-metadata-doc.

In the above example, product: name would match a specific product.

load (optional)

Define default projections and resolution to use when loading data from this product. User supplied load options take precedence over settings configured here.

crs

Coordinate reference system to use as a fallback when loading data from this product. 'EPSG:<code>' or WKT string.

resolution.{x,y}

Default resolution to use during dc.load(..) specified in projection units. Use latitude, longitude if the projection is geographic and x, y otherwise.

align.{x,y} (optional)

By default the pixel grid is aligned such that pixel boundaries fall on x,y axis. This option allows us to translate the pixel grid. For example, to ensure that the pixel center of a 30m pixel grid is coincident with 0,0 use align:{x:15,y:15}.

storage (optional)

Describes some of common storage attributes of all the datasets. While optional, defining this will make product data easier to access and use. This only applies to products that have data arranged on a regular grid, for example ingested products are arranged like that.

crs

Coordinate reference system common to all the datasets in the product. 'EPSG:<code>' or WKT string.

resolution

Resolution of the data of all the datasets in the product specified in projection units. Use latitude, longitude if the projection is geographic and x, y otherwise.

tile_size

Size of the tiles for the data to be stored in specified in projection units. Use latitude and longitude if the projection is geographic, otherwise use x and y.

origin

Coordinates of the bottom-left or top-left corner of the (0,0) tile specified in projection units. If coordinates are for top-left corner, ensure that the latitude or y dimension of tile_size is negative so tile indexes count downward. Use latitude and longitude if the projection is geographic, otherwise use x and y.

measurements

List of measurements in this product. The measurement names defined here need to match 1:1 with the measurement key names defined in the dataset-metadata-doc.

name

Name of the measurement

units

Units of the measurement

dtype

Data type. One of ``(u)int(8,16,32,64), float(16, 32, 64), complex(64,128) ``

The allowed type can be found in schema file: datacube/model/schema/dataset-type-schema.yaml

nodata

No data value. One of number or NaN, Inf, -Inf.

The allowed type can be found in schema file: datacube/model/schema/dataset-type-schema.yaml

Note

for floats you should always choose nan or an inf - as direct comparison of float values is not reliable.

scale_factor,add_offset (optional)

Mapping from pixel value to real value real = scale_factor*pixel_value + add_offset.

spectral_definition (optional)

Spectral response of the reflectance measurement.

spectral_definition:
   wavelength: [410, 411, 412]
   response: [0.0261, 0.029, 0.0318]

For 3D datasets spectral_definition should be a list of the same length as the extra_dimensions coordinate it applies to.

spectral_definition:
   - wavelength: [410, 411, 412]
     response: [0.0261, 0.029, 0.0318]
   - wavelength: [410, 411, 412]
     response: [0.0261, 0.029, 0.0318]
   ...
flags_definition (optional)

Bit flag meanings of the bitset ‘measurement’

flags_definition:
    platform:
      bits: [0,1,2,3]
      description: Platform name
      values:
        0: terra
        1: aqua_terra
        2: aqua
    contiguous:
      bits: 8
      description: All bands for this pixel contain non-null values
      values: {0: false, 1: true}
extra_dim (required for 3D datasets)

Name of the extra dimension supported by this measurement. Must match a name from the extra_dimensions definition (see below).

extra_dim: z
extra_dimensions (required for 3D datasets)

Definition of the extra dimensions.

name

Name of the dimension.

values

List of coordinate values of the dimension.

dtype

Data type. One of (u)int(8,16,32,64), float32, float64

extra_dimensions:
  - name: z
    values: [5, 10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85,
             90, 95, 100, 105, 110, 115, 120, 125, 130, 135, 140, 145, 150]
    dtype: float64

3D product definition#

Example 3D product definition for GEDI L2B cover_z:

name: gedi_l2b_cover_z
description:
  GEDI - Cumulative canopy cover from height (z) to ground (z=0) with a
  vertical step size of dZ, where cover(z > z_max) = 0
metadata_type: eo3
metadata:
  product:
    name: gedi_l2b_cover_z
extra_dimensions:
  - name: z
    values: [5, 10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 95, 100, 105, 110, 115, 120, 125, 130, 135, 140, 145, 150]
    dtype: float64
measurements:
  - name: cover_z
    extra_dim: z
    dtype: float32
    nodata: -9999.0
    units: na

Note

extra_dimensions is defined in the yaml