eMarine Information Infrastructure (eMII)
Welcome to eMII!
Your main access to IMOS data is through the IMOS Ocean Portal.
Marine data and information are the main products of IMOS, and data management is therefore a central element to the project's success. eMII provides a single integrative framework for data and information management that will allow discovery and access of the data by scientists, managers and the public.
IMOS data streams can be categorised in four ways:
· gridded data from satellites and HF radar systems;
· time series data from moorings, Argo floats, gliders and ships of opportunity;
· image data from Autonomous Underwater Vehicles;
· biological data from continuous plankton recorders and acoustic tagging.
The first two provide real-time and delayed-mode data sets whereas the latter are delayed-mode delivery only.
The IMOS data management infrastructure employs OGC standards wherever possible. The main components of the system are: OPeNDAP / THREDDS servers hosting CF-compliant NetCDF, HDF or GeoTIFF data; the open-source GeoNetwork Metadata Entry and Search Tool (the IMOS MEST) for metadata cataloguing; SensorML, which provides standard models and an XML encoding for describing sensors and measurement processes; the open-source DataTurbine, data streaming middleware providing the foundation for reliable data acquisition and instrument management services; a web portal (the IMOS Ocean Portal) using the open-source ZK Ajax framework and the OpenLayers geospatial framework incorporating access to Web Services.
IMOS data utilises the Australian Research Collaboration Service (ARCS) Data Fabric, a simple production service allowing researchers, research groups, research organizations and research communities to store their data. A distributed network of OPeNDAP / THREDDS servers around Australia forms the primary data storage.
This complements the regional nodal structure of IMOS (i.e. WAIMOS, SAIMOS, NSWIMOS, GBROOS, Blue Water & Climate, see http://imos.org.au/nodes.html) and allows rapid access to data by the local research community and the public. Each local server also supports the GeoNetwork catalog (the IMOS MEST) with, wherever possible, automatic harvesting of metadata from the OPeNDAP / THREDDS system. An IMOS NetCDF standard ensures that all necessary metadata to comply with ISO 19115 can be automatically extracted from the NetCDF files. Automation of metadata creation from non-NetCDF datasets is also being investigated. A master IMOS MEST at the University of Tasmania routinely harvests new metadata records from the regional catalogs to maintain a central registry.
Data flows from the IMOS facilities into the ARCS Data Fabric and from there to publically accessible locations. Typical lifecycle of data within IMOS is shown in the figure below:
This shows the workflow for real-time data passing through DataTurbine with limited or no Quality Control (QC) as well as delayed mode data passing through rigorous QC procedures before storage, creation of the metadata record for the IMOS MEST and subsequent viewing through the IMOS Ocean Portal.
The initial strategy has focussed on defining specific data streams, and then developing end-to-end protocols, standards and systems to join the related observing systems into a unified data storage and access framework. A set of data products combining data across observations on regional and national scales will ultimately be developed, as well as tools that facilitate the integration and analysis of data.
The link between datasets and the metadata is critical to the performance of the search and discovery of IMOS data. The metadata hierarchy of IMOS records allows complete retrieval of all levels of information about datasets, sensors, platforms, facilities and IMOS itself.
Data storage and retrieval in IMOS is designed to be interoperable with other national and international programs. Thus, it will be possible to integrate data from sources outside IMOS into IMOS data products, and IMOS data is exported to international programs such as Argo.
In the future, IMOS data will be archived within the Australian Ocean Data Network (AODN), which is a distributed data storage and discovery network based at leading Australian marine research facilities.