The Australian Ocean Data Network (AODN) has commenced using Amazon Web Services (AWS) Batch; a service that is optimising gridded data aggregation ready for downloading, by enabling the computing workload to be run on the AWS Cloud. AWS Batch offers reduced operational complexities, time saving and reduced costs and we certainly do not disagree. With faster downloads, download size options increased, and parallelised job processing it is so far meeting all expectations.
Once a download is up and running, something that may have taken a day in the past will most likely be completed in a few hours, there is now no restriction on the size of the request. The largest download may still take a couple of days though as it’s a lot of data! Downloads now run concurrently, in the past only one job ran at a time, delivering requested downloads to our users much quicker.
Improvements have been immediately quantifiable however, we are still tidying up small details and polishing off the administration side of the system, and therefore interested in any feedback. Try it in the AODN Portal.
For more information on AWS and why AODN use it see the story Australian Ocean Data Network (AODN) using Amazon Web Services (AWS).