Products#
Products are collections of datasets
that share the same set of measurements and some subset of metadata.
Product Definition#
A product definition document describes the measurements and common metadata for a collection of datasets.
Example Product Definition#
name: landsat8_example_product
description: Landsat 8 example product
metadata_type: eo3
metadata:
product:
name: landsat8_example_product
# Alternatively, include specific items to match
# properties:
# eo:instrument: OLI_TIRS
# eo:platform: landsat-8
measurements:
- name: 'red'
aliases: [band_4, sr_band4]
dtype: int16
nodata: -9999
units: 'reflectance'
- name: 'blue'
aliases: [band_2, sr_band2]
dtype: int16
nodata: -9999
units: 'reflectance'
- name: 'pixel_qa'
aliases: [pixel_quality, level2_qa]
dtype: uint16
nodata: 1
units: 'bit_index'
flags_definition:
pixel_qa:
bits: [0,1,2,3,4,5,6,7,8,9,10,11]
description: Level 2 pixel quality band
values:
1: Fill
2: Clear
4: Water
8: Cloud shadow
16: Snow
32: Cloud
64: Cloud Confidence Low Bit
128: Cloud Confidence High Bit
256: Cirrus Confidence Low Bit
512: Cirrus Confidence High Bit
1024: Terrain Occlusion
2048: Unused # Be careful of repeated names which could confuse the masking code
# Alternatively or additionally, use the bit on/off method
fill:
bits: 0
description: No data
values: {0: false, 1: true}
clear:
bits: 1
description: Clear
values: {0: no_clear_land, 1: clear_land}
# ...
cloud_confidence:
bits: [6, 7]
description: Cloud confidence
values: {0: none, 1: low, 2: medium, 3: high}
# ...
Product Definition API#
- name
Product name
- description
Product description
- metadata_type
Name of the metadata-type-definition
- license
The license of the data.
This is either a SPDX License identifier (eg ‘CC-BY-SA-4.0’) or ‘various’ or ‘proprietary’
- metadata
Dictionary containing bits of metadata common to all the datasets in the product.
It is used during indexing to match datasets to their products. That is, the keys and values defined here must also be in the dataset-metadata-doc.
In the above example,
product: name
would match a specific product.- load (optional)
Define default projections and resolution to use when loading data from this product. User supplied load options take precedence over settings configured here.
- crs
Coordinate reference system to use as a fallback when loading data from this product.
'EPSG:<code>'
or WKT string.- resolution.{x,y}
Default resolution to use during
dc.load(..)
specified in projection units. Uselatitude
,longitude
if the projection is geographic andx
,y
otherwise.- align.{x,y} (optional)
By default the pixel grid is aligned such that pixel boundaries fall on
x,y
axis. This option allows us to translate the pixel grid. For example, to ensure that the pixel center of a 30m pixel grid is coincident with0,0
usealign:{x:15,y:15}
.
- storage (optional)
Describes some of common storage attributes of all the datasets. While optional, defining this will make product data easier to access and use. This only applies to products that have data arranged on a regular grid, for example ingested products are arranged like that.
- crs
Coordinate reference system common to all the datasets in the product.
'EPSG:<code>'
or WKT string.- resolution
Resolution of the data of all the datasets in the product specified in projection units. Use
latitude
,longitude
if the projection is geographic andx
,y
otherwise.- tile_size
Size of the tiles for the data to be stored in specified in projection units. Use
latitude
andlongitude
if the projection is geographic, otherwise usex
andy
.- origin
Coordinates of the bottom-left or top-left corner of the
(0,0)
tile specified in projection units. If coordinates are for top-left corner, ensure that thelatitude
ory
dimension oftile_size
is negative so tile indexes count downward. Uselatitude
andlongitude
if the projection is geographic, otherwise usex
andy
.
- measurements
List of measurements in this product. The measurement names defined here need to match 1:1 with the measurement key names defined in the dataset-metadata-doc.
- name
Name of the measurement
- units
Units of the measurement
- dtype
Data type. One of ``(u)int(8,16,32,64), float(16, 32, 64), complex(64,128) ``
The allowed type can be found in schema file:
datacube/model/schema/dataset-type-schema.yaml
- nodata
No data value. One of
number
orNaN
,Inf
,-Inf
.The allowed type can be found in schema file:
datacube/model/schema/dataset-type-schema.yaml
Note
for floats you should always choose nan or an inf - as direct comparison of float values is not reliable.
- scale_factor,add_offset (optional)
Mapping from pixel value to real value
real = scale_factor*pixel_value + add_offset
.- spectral_definition (optional)
Spectral response of the reflectance measurement.
spectral_definition: wavelength: [410, 411, 412] response: [0.0261, 0.029, 0.0318]
For 3D datasets spectral_definition should be a list of the same length as the extra_dimensions coordinate it applies to.
spectral_definition: - wavelength: [410, 411, 412] response: [0.0261, 0.029, 0.0318] - wavelength: [410, 411, 412] response: [0.0261, 0.029, 0.0318] ...
- flags_definition (optional)
Bit flag meanings of the bitset ‘measurement’
flags_definition: platform: bits: [0,1,2,3] description: Platform name values: 0: terra 1: aqua_terra 2: aqua contiguous: bits: 8 description: All bands for this pixel contain non-null values values: {0: false, 1: true}
- extra_dim (required for 3D datasets)
Name of the extra dimension supported by this measurement. Must match a name from the extra_dimensions definition (see below).
extra_dim: z
- extra_dimensions (required for 3D datasets)
Definition of the extra dimensions.
- name
Name of the dimension.
- values
List of coordinate values of the dimension.
- dtype
Data type. One of
(u)int(8,16,32,64), float32, float64
extra_dimensions: - name: z values: [5, 10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 95, 100, 105, 110, 115, 120, 125, 130, 135, 140, 145, 150] dtype: float64
3D product definition#
Example 3D product definition for GEDI L2B cover_z:
name: gedi_l2b_cover_z
description:
GEDI - Cumulative canopy cover from height (z) to ground (z=0) with a
vertical step size of dZ, where cover(z > z_max) = 0
metadata_type: eo3
metadata:
product:
name: gedi_l2b_cover_z
extra_dimensions:
- name: z
values: [5, 10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 95, 100, 105, 110, 115, 120, 125, 130, 135, 140, 145, 150]
dtype: float64
measurements:
- name: cover_z
extra_dim: z
dtype: float32
nodata: -9999.0
units: na
Note
extra_dimensions is defined in the yaml