The AODN is currently undertaking a review of its data systems.
The goal is to identify solutions that:
- provide an improved user experience in the discovery and access of datasets, including those described in the data catalogue regardless of where they are hosted,
- provides better support for data science workflows,
- makes it easier to produce ‘ready-to-use’, ‘analysis-ready’ datasets,
- solve existing data extraction performance issues,
- provide easier and more flexible ways for organisations to share data via the AODN,
- can scale in terms of cost and effort to support future growth in the number of datasets,
- reduce the maintenance burden on the AODN team,
- solves other relevant issues documented in the AODN External Review (2021).
We have investigated the technical challenges and problems of the existing systems in order to understand the drivers and barriers of contributing data to the AODN.
We have reached out to other organisations to learn about their solutions. Evaluations of various technologies have been conducted to understand their suitability with a major focus on finding alternatives to streamline data ingestion and processing and data formats that enable faster data sub-setting.
While a significant part of the work to date has been focused on existing internal architectural challenges, we have begun to engage the various communities to seek input, including:
- AODN data contributors via the AODN-Technical Advisory Group
- Data science users
We are currently engaging all AODN Portal users via a survey.
Once the technical evaluations are complete, proposals for two or three solution options will be produced and circulated for comment.
Development of the new solution would begin in 2023.
If you wish to reach out, please contact Mark Rehbein – Project Leader at info(at)aodn.org.au