The IMOS Remote Sensing Collections is a data management activity that was developed to improve access to satellite remote sensed marine products.
The baseline data sets created and managed by the IMOS Satellite Remote Sensing facility comprise approximately 2 million data files occupying nearly 30TB of storage. The data are formatted and organised for efficient storage and access using community-adopted standards such as netCDF. Remote sensing instruments are also complex sensors and their data often require specialised knowledge to extract, manipulate and use. A core activity of the Remote Sensing Collections sub-facility is to support users by providing guidance in selecting and using data, by:
- Providing documentation and tools (often via eMII)
- Assisting eMII with managing and maintaining the core data sets
- Enabling specialised tools to access some data sets that are not readily delivered via the IMOS portal (for example, ungridded data)
- Reorganising data to improve usability in response to community demand (such as aggregation and subsetting)
- Providing enhanced access to data sets where practical constraints (primarily volume) dictate bypassing the IMOS portal
- Opportunistically delivering products beyond the core remits of the data production sub-facilities (such as sea surface temperatures from the Ocean Colour MODIS sensor).
In addition to the data sets produced by IMOS, there are also many global data sets available from overseas. Rather than many researchers downloading and curating these individually, Remote Sensing Collections seeks (through community consultation) to identify data sets of value and, where appropriate, to play a role in maintaining local archives within Australia. A parallel activity is the management and storage of locally produced data sets of interest to the wider Australian research community. Respective examples (of each type) are global sea surface salinity products and locally tuned chlorophyll products for the southern ocean.
The key infrastructures employed by Remote Sensing Collections are large scale computing and digital storage facilities. The National Compute Infrastructure (NCI, www.nci.org.au) is Australia’s peak eResearch facility located in Canberra. Remote Sensing Collections, with other Satellite Remote Sensing sub-facilities, works to process and store national data sets at the NCI. These data sets are available to researchers with NCI projects directly, or via a variety of web-based access mechanisms. A subset of the data are located on a second data store operated by CSIRO in Melbourne. This storage is used as a pre-production facility to prepare data sets for serving via the IMOS portal. A range of value adding and quality control steps are undertaken at this site. Additional datasets, often created by third parties, are also compiled on this system, together with some derived products that are computed directly. Again, all data sets are available via the web-based access methods supported at the NCI.
The historical ocean colour and sea surface temperature data sets are both created at the NCI by their respective Satellite Remote Sensing sub-facilities. The contemporary near real time data sets are produced within the CSIRO and the Bureau of Meteorology respectively. Examples of third party data sets managed by Remote Sensing Collections are remotely sensed salinity from the Aquarius sensor and improved southern ocean chlorophyll data sets provided by researchers at the University of Tasmania. In many cases the role of the Remote Sensing Collections is to ensure conformance with netCDF metadata conventions, provision of metadata, and maintaining data set currency.
Remote Sensing Collections also plays a collaborative role in ensuring IMOS works effectively with other providers and managers of remote sensing data sets of common interest. Examples include:
- Liaising with TERN to collaboratively manage a shared whole-of-mission Australian archive for the MODIS and VIIRS sensors
- Operating a data merging service for Australian NOAA reception stations to provide a national archive.
The Remote Sensing Collections staff also maintain a suite of tools (not readily included within the IMOS Portal) to assist users in finding and selecting the data that they need from the large data sets. These range from web pages of quick-look imagery through web catalogue engines such as ERDDAP and even customised tools for special tasks such as matchup extraction or time series analysis.
Dr Edward King
Your primary access to the IMOS Remote Sensing Collection data is through the Australian Ocean Data Network (AODN) Portal.