Ubuntu Developer Setup#

Base OS: Ubuntu 20.04 LTS

This guide will setup an ODC core development environment and includes:

  • Anaconda python using conda environments to isolate the odc development environment

  • installation of required software and useful developer manuals for those libraries

  • Postgres database installation with a local user configuration

  • Integration tests to confirm both successful development setup and for ongoing testing

  • Build configuration for local ODC documentation

Required software#

GDAL, HDF5, and netCDF4:

sudo apt-get install libgdal-dev libhdf5-serial-dev libnetcdf-dev

Install the latest Postgres version available <https://packages.ubuntu.com/search?keywords=postgresql> for your Ubuntu distribution, eg:

sudo apt-get install postgresql-12

# Optionally, Postgis too
sudo apt-get install postgresql-12-postgis-3

Ubuntu’s official repositories usually ship older versions of Postgres. You can alternatively get the most recent version from the official PostgreSQL repository.

Optional packages (useful utilities, docs):

sudo apt-get install libhdf5-doc netcdf-doc libgdal-doc
sudo apt-get install hdf5-tools netcdf-bin gdal-bin pgadmin3

Python and packages#

Python 3.8+ is required.

Anaconda Python#

Install Anaconda Python

Add conda-forge to package channels:

conda config --append channels conda-forge

Conda environments are recommended for use in isolating your ODC development environment from your system installation and other Python environments.

Install required Python packages and create a conda environment named odc_env.


conda create --name odc_env python=3.8 datacube

Activate the odc_env conda environment:

conda activate odc_env

Find out more about conda environments here.

Install other packages:

conda install jupyter matplotlib scipy pytest-cov hypothesis

Postgres database configuration#

This configuration supports local development using your login name.

If this is a new installation of Postgres on your system it is probably wise to set the postgres user password. As the local “postgres” Linux user, we are allowed to connect and manipulate the server using the psql command.

In a terminal, type:

sudo -u postgres psql postgres

Set a password for the “postgres” database role using the command:

\password postgres

and set the password when prompted. The password text will be hidden from the console for security purposes.

Type Control+D or q to exit the posgreSQL prompt.

By default in Ubuntu, Postgresql is configured to use ident sameuser authentication for any connections from the same machine which is useful for development. Check out the excellent Postgresql documentation for more information, but essentially this means that if your Ubuntu username is foo and you add foo as a Postgresql user then you can connect to a database without requiring a password for many functions.

Since the only user who can connect to a fresh install is the postgres user, here is how to create yourself a database account (which is in this case also a database superuser) with the same name as your login name and then create a password for the user:

sudo -u postgres createuser --superuser $USER
sudo -u postgres psql

postgres=# \password $USER

Now we can create an agdcintegration database for testing:

createdb agdcintegration

Connecting to your own database to try out some SQL should now be as easy as:

psql -d agdcintegration

Open Data Cube source and development configuration#

Download the latest version of the software from the repository

git clone https://github.com/opendatacube/datacube-core
cd datacube-core

We need to specify the database user and password for the ODC integration testing. To do this:

cp integration_tests/agdcintegration.conf ~/.datacube_integration.conf

Then edit the ~/.datacube_integration.conf with a text editor and add the following lines replacing <foo> with your username and <foobar> with the database user password you set above (not the postgres one, your <foo> one):

db_hostname: localhost
db_database: agdcintegration
db_username: <foo>
db_password: <foobar>

Note: For Ubuntu Setup the db_hostname should be set to “/var/run/postgresql”. For more refer: https://github.com/opendatacube/datacube-core/issues/1329

Verify it all works#

Run the integration tests:

cd datacube-core
./check-code.sh integration_tests

Build the documentation:

cd datacube-core/docs
pip install -r requirements.txt
make html

Then open _build/html/index.html in your browser to view the Documentation.