Introduction
As with the IT world in general, in 2023 the major hyperscalers focused heavily on AI. That focus showed up prominently in public cloud storage launches. Hyperscalers also forged more partnerships with major on-premises storage vendors to facilitate hybrid cloud offerings.
AWS and Google Cloud enhanced and expanded their file and object storage options for unstructured data. Unstructured data includes audio, video, image, and document content frequently utilized for AI. To deal with growing ransomware threats, Amazon Web Services (AWS) and Google also beefed up data protection and resiliency features. AWS, Google, and Microsoft Azure all initiated or expanded partnerships with on-premises storage companies.
AWS Highlights
The big storage launch at AWS re:Invent 2023 was Amazon S3 Express One Zone, a new storage class that AWS claims gives a 10x performance boost for smaller objects than S3 standard storage. AWS also claims it can handle hundreds of thousands of requests per second with single-digit millisecond latency. The objects are stored and replicated within one AWS Availability Zone to reduce the latency between compute and storage. The Express One Zone class is especially beneficial for smaller objects because there is less data to read. By keeping storage and compute closer together, S3 Express One Zone helps generative AI by placing data closer to training nodes. It also speeds monitoring and inferencing for AI.
At re:Invent, AWS also added a new file storage class called Elastic File System (EFS) Archive, designed to keep cold data always available at a lower cost than EFS Standard and EFS Infrequent Access storage. AWS enhanced FSx for NetApp ONTAP, an enterprise-class file system co-engineered with NetApp – adding a new scale-out option, virtual private cloud (VPC) support, and FlexGroup volume management. A new feature for FSx for OpenZFS, on-demand replication, enables customers to send snapshots from one file system to another file system in their FSx for OpenZFS accounts.
These file enhancements can be of great benefit to customers that have petabytes of data in AWS that they want to use for generative AI. Even before re:Invent, AWS made improvements to FSx services and the OS-agnostic AWS Elastic File Storage (EFS) delivered at AWS Storage Day. Those included performance and scalability improvements for FSx for Windows File Server, and multi-Availability Zone deployment of FSx for OpenZFS with automatic failover and failback for high availability.
On the data protection front, AWS added SnapLock for Amazon FSx for NetApp ONTAP, which creates immutable snapshots that cannot be altered or deleted for a designated period. Additionally, an AWS Backup logically air-gapped storage vault was launched in preview featuring immutable and encrypted backup copies with AWS-owned keys.
Google Cloud Highlights
Google also improved its cloud file storage, with the help of a partnership with NetApp.
Google Cloud (GC) NetApp Volumes is a first-party cloud service that runs NetApp technology natively in Google Cloud. It is sold and managed by Google. This partnership gave NetApp first-party relationships with the three major hyperscalers, following Azure NetApp Files (ANF) and AWS FSx for NetApp ONTAP.
GC NetApp Volumes combines NetApp’s ONTAP multiprotocol file storage and data management (snapshots, volume copies, volume replication, backup, etc.) with GC services such as identity access management, the GC API, managed Active Directory for Windows file servers, and encryption for data in transit and at rest. Customers can use Google Cloud credits to buy NetApp Volumes.
Storage Partnerships
NetApp was not the only major storage vendor to tightly integrate with public clouds in 2023. Dell followed NetApp’s lead in the public cloud by expanding the APEX Cloud Platforms that are part of its storage-as-a-service program; Pure Storage began a multiyear strategic product and go-to-market partnership with Microsoft Azure; and HPE added GreenLake services for AWS.
Dell launched APEX File Storage and APEX Block for AWS and Microsoft Azure. APEX File Storage is based on Dell PowerScale (Isilon) storage, and APEX Block Storage uses Dell PowerFlex software-defined storage.
Dell also launched APEX Cloud Platforms for Microsoft, Red Hat, and VMware. Cloud Platform for Microsoft Azure is focused on Azure Arc on-premises services, with Dell software plugging into the Microsoft UI. APEX Cloud Platform for Red Hat OpenShift is aimed at speeding development of cloud-native apps. APEX Cloud Platform for VMware is designed to manage VMware’s enterprise software-defined storage and Kubernetes platforms, all through vCenter.
Flash storage vendor Pure introduced Pure Cloud Block Store for Azure VMware Solution (AVS). The solution is an external storage option for AVS. It decouples Pure’s storage software from its FlashArray hardware by running its Purity storage management software natively on Azure Premium SSD v2 back-end storage. The partnership also adds Pure’s Safemode ransomware data protection and Purity’s zero-RPO ActiveCluster, near-zero RPO ActiveDR, and Purity CloudSnap disaster recovery capabilities to AVS.
HPE expanded its GreenLake as-a-service program’s capabilities for AWS. HPE made Amazon Elastic Kubernetes Service (EKS) Anywhere in GreenLake’s private cloud generally available and added support for EKS and Amazon Relational Database Service in GreenLake for backup and recovery.
Looking Ahead
I expect these trends to accelerate in 2024, as all storage companies continue to position their products and services as storage for AI. AWS, Google Cloud, and Microsoft Azure are all heavily invested in AI, and AI deployments often now reside in the public cloud.
AI is also one of the major drivers that will produce more competition between on-premises storage vendors and public cloud providers. To gain the scale and cost-effective compute required for AI and machine learning, companies need to migrate workloads across clouds and from on-premises to public clouds. They also need to bring compute to where data is that they want to run AI models on, and that data can be on-premises, in public clouds, and/or at the edge.
A hybrid cloud can be the best way to quickly scale the compute and storage needed to train large generative AI models. Many companies will find it most efficient to use a public cloud for the most demanding AI tasks and on-premises resources for other AI workloads. The leading hyperscalers and on-premises storage vendors realize this and tailor their services accordingly.
Disclosure: The Futurum Group is a research and advisory firm that engages or has engaged in research, analysis, and advisory services with many technology companies, including those mentioned in this article. The author does not hold any equity positions with any company mentioned in this article.
Analysis and opinions expressed herein are specific to the analyst individually and data and other information that might have been provided for validation, not those of The Futurum Group as a whole.
Other Insights from The Futurum Group:
With an Eye on AI, AWS Adds, Enhances Storage Services – The Futurum Group
Driving Growth and Adoption of AWS Marketplace – The Futurum Group
2024 Trends and Predictions for Data Storage – The Futurum Group
Author Information
Dave focuses on the rapidly evolving integrated infrastructure and cloud storage markets.
