NIWA partners with VAST Data to upgrade climate data systems
The National Institute of Water and Atmospheric Research (NIWA) in New Zealand has selected VAST Data's platform to enhance the management of its critical climate data infrastructure.
NIWA, recognised for its research in atmospheric science, oceanography, and climate modelling, holds a substantial archive of over 20 petabytes of historical weather data. The organisation aims to sync its production workloads and research using VAST's AI-driven data platform to optimise storage efficiency and prepare for emerging GPU-based scientific workloads.
Warrick Johnston, General Manager of Technology and Innovation at NIWA, remarked, "The VAST Data Platform will provide the backbone for NIWA's advanced technology and intelligent data capture, empowering not only our storage and archival needs but also being able to support other research and agencies across New Zealand."
The capacity to manage massive datasets is crucial for NIWA, given the incorporation of AI and machine learning technologies into its operations. Jeff Zais, HPC Senior Science Advisor and Platform Architect at NIWA, said, "VAST Data stood out in a crowded field for its ability to unify our sites and prepare us for the future of AI-driven research. The platform not only supports our current CPU-based workloads but also gives us the flexibility and confidence to integrate GPU capabilities as we transition to new weather and climate modelling codes. VAST's synchronisation features, density advantages, and vision for the future were critical in our decision to partner with them."
The VAST Data Platform is noted for its unique active-active clustering through a global namespace, allowing seamless data synchronisation and utilisation across multiple sites. Such features are designed to reduce operational complexity when compared to legacy systems, facilitating advanced functionalities like write-read access across these sites.
According to NIWA, the platform's integrated compression, deduplication, and multi-tenancy capabilities help optimise storage space utilisation and reduce management burdens, thereby allowing the institute to concentrate on its research priorities.
NIWA selected VAST Data's Disaggregated Shared-Everything (DASE) architecture as a strategic move to overcome challenges associated with legacy systems, reduce costs, and cut down operational complexity. The high-density design of the platform has reportedly enabled NIWA to shrink its physical infrastructure footprint, ultimately reducing costs over an extended period while enhancing data accessibility and reliability.
Commenting on the partnership, Sunil Chavan, Senior Vice President, APAC at VAST Data, stated, "NIWA is a pioneer in climate and environmental research, and it's critical that it has the tools to access and analyse its immense datasets in real time. The VAST Data Platform provides the scalability, performance, and integration needed to address the complexities of modern scientific workloads. We're proud to support NIWA as it continues to innovate and lead on a global scale."
Through this deployment, NIWA can efficiently utilise resources at its secondary site, enabling the simultaneous running of more climate models and improving disaster preparedness. The infrastructure upgrade is expected to deliver faster and more detailed insights into environmental patterns, supporting both New Zealand and global scientific efforts.
As NIWA edges towards its full production capacity by May 2025, the deployment positions the institute at the forefront of using AI-driven data platforms in environmental science, setting a standard for other institutions and underscoring the important role of data solutions in addressing global environmental challenges.
