The AODN Portal is recommended for discovering and downloading IMOS data, however, when large or un-subsetted dataset collections are required web services enable direct access to the infrastructure.
Three new Jupyter notebooks using web services for data access, data manipulation and plotting IMOS data were debuted recently at a DataTas workshop. Adding to the data-tools and applications publicly and freely available for IMOS data access.
AODN Project Officer Laurent Besnard authored three new python notebooks, when the opportunity arose to host a workshop as part of the DataTas series. These are in addition to the five (R and Python) notebooks already available in the IMOS-User-Code Library Github Repository. The new notebooks differ as they are highly compatible with Google Colab making sharing and online use a streamline process.
Jupyter Notebook is an open-source web application that allows the creation and sharing of notebook documents that contain live code, equations, visualisations and narrative text.
A notebook document contains all the content from the Jupyter Notebook web application session, which includes the inputs and outputs of computations, mathematics, images, and explanatory text.
Google Colaboratory (Colab) allows writing and execution of Python Jupyter notebooks within a Browser https://colab.research.google.com/notebooks/intro.ipynb which makes it a perfect tool for online workshops.
The workshop conducted in early May provided over 20 attendees with an overview of how IMOS works and the data collected, information on the various AODN services for accessing data, and through the use of google colab they were able to use the Python notebooks to:
- Read NetCDF stored on a THREDDS server
- Aggregate NetCDF’s via Xarray
- Query WMS/WFS to find and plot various datasets.
The imos_acorn_torq_dm.ipynb notebook, demonstrates how to access and plot AODN NetCDF files via THREDDS, as well as how to use NetCDF Quality Control flags and understanding the impact they have. In this example the Turquoise Coast High Frequency Radar (HF RADAR) Delayed Mode dataset collection was used.
The IMOS_Ocean_Colour_Chl_OC3_access_via_S3.ipynb notebook can be used to mount the AWS S3 bucket as a filesystem in Python to access the NetCDF files, it also shows how to use Xarray to combine many similarly structured NetCDF files. The IMOS SRS MODIS Chlorophyll a (OC3 model) dataset collection was used to illustrate the functionality.
The imos_geoserver_access.ipynb notebook provides data retrieval bypassing the AODN Portal by using the OWSLib python package to query the AODN Geoserver. This notebook enables plotting of different instrument data together. The dataset collections used were:
- IMOS SOOP Expendable Bathythermographs (XBT) Research Group – XBT real-time data
- IMOS – Argo Profiles
- IMOS Australian National Facility for Ocean Gliders (ANFOG) delayed mode glider deployments.
Although the most complex of the three additions, when executed will combine and plot the data within seconds.
For any questions about the notebooks or any other AODN related queries please email info(at)aodn.org.au