The News: AWS launched a series of enhancements at AWS Storage Day aimed at improving storage for AI and machine learning (ML), data protection, and cyber-resiliency. You can see all the new rollouts on the AWS News Blog.
AWS Storage Day 2023: AWS Tackles AI/ML, Cyber Resiliency in the Cloud
Analyst Take: AWS focused its Storage Day releases on two hot button areas: providing infrastructure and a data strategy for high-performance AI and ML in the cloud, and enhancing cyber-resiliency. Here are the highlights:
Artificial Intelligence and Machine Learning
AWS is taking steps to provide the massive amount of compute, storage, and network resources required for generative AI and ML.
Although not brand new, AWS highlighted the recent July 26 launch of its EC2 P5 instances that can be deployed in EC2 UltraClusters. These UltraClusters combine high-performance compute, storage, and networking. Amazon EC2 P5 instances use NVIDIA H100 Tensor Core GPUs, and each EC2 UltraCluster provides supercomputing capability to cut AI model training time. AWS customers generally use FSx for Lustre as the storage infrastructure with UltraClusters. The FSx Lustre file system delivers the storage requirements for hundreds of GBs per second of throughput and millions of IOPS, keeping GPUs saturated to meet requirements of large data foundation models.
Now AWS has added a deeper integration of AWS FSx for Lustre and AWS S3 object storage with a file release feature. Using FSx for Lustre with S3 allows customers using AI and ML to store massive datasets on S3, but work on them from their file interface. File release lets customers automatically release data from a Lustre file system that has been synched to S3, freeing up space on the file system for cost savings. Metadata for released files stays on the FSx for Lustre file system, so that data can still be retrieved if needed.
Data Protection and Cyber-Resiliency
Cyber-resiliency is a critical topic, as cyber-attacks continue to become more frequent and have a greater impact on enterprises, and as enterprises increasingly use cloud-hosted storage. In fact, 50% of respondents in The Futurum Group’s Trends in Enterprise Data Protection 2023 study indicated that they are using public cloud storage as a backup target.
In July, AWS added SnapLock for Amazon FSx for NetApp ONTAP. SnapLock is a NetApp technology for creating data snapshots that are immutable – that is, they cannot be altered or deleted for a designated period. Per AWS, this marks the first and only fully managed file offering with support for write once read many (WORM) data retention holds. This includes support for both Enterprise Mode (authorized users can delete the SnapLock snapshot if necessary) and Compliance Mode (no user is allowed to delete the SnapLock snapshot) to address growingly stringent cybersecurity requirements, in addition to compliance mandates. This is notable because The Futurum Group’s Trends in Enterprise Data Protection 2023 study indicated approximately 30% of enterprises use managed backup storage infrastructure and managed backup and operational recovery processes.
Additionally, an AWS Backup logically air-gapped storage vault is now in preview. These backup copies are immutable and encrypted by default using AWS-owned keys. The Futurum Group’s research indicates growing popularity for using public cloud storage as an air-gapped environment, though notes that specific controls such as network isolation ideally should be present. We would look for additional details regarding the solution’s isolation and access control capabilities. AWS is investing in additional capabilities such as data integrity validation to further enhance its air gap vault.
AWS has also added up to 85% faster data retrieval from Amazon S3 Glacier Flexible Retrieval. This will help enterprises to address legal discovery requests more quickly, and in the event of a cyber-attack, to mitigate the amount of business downtime if longer-term retention data needs to be recovered.
Elastic Block Storage (EBS)
AWS recently launched a new EC2 instance called M7i, powered by the fourth generation Intel Xeon scalable processor. Customers can now attach 128 EBS volumes per instance – up from 28 volumes on M6i. It also can host 127 containers and up to 128 multi-volume crash-consistent EBS snapshots per instance, enabling backup of EBS storage as an instance with one API call.
File Storage – FSx and Elastic File Storage (EFS)
AWS Storage Day included improvements for several FSx file systems plus the OS-agnostic AWS EFS.
These include performance and scalability improvements for FSx for Windows File Server. AWS is increasing maximum throughput from 2 GB/s to 12 GB/s, maximum disk IOPS from 80,000 to 350,000, and allowing customers to provision up to 500 IOPS per GB, up from 3 IOPS per GB. These new performance and scaling levels for FSx for Windows File Server are similar to those available on FSx for NetApp ONTAP and FSx for OpenZFS.
AWS also added multi-Availability Zone deployment of FSx for OpenZFS, with automatic failover and failback for high availability. FSx for OpenZFS previously could deploy only on one Availability Zone. The high availability allows customers to run critical apps such as databases, core banking and electronic design automation (EDA).
EFS now supports up to 55,000 read IOPS (1.6x improvement) and 25,000 write IOPS (3.6x improvement) per file system, which helps run data intensive workloads such as ML research, financial simulations, and big data analytics.
Other Enhancements
AWS also made it easier to move file and object data to and from other clouds through its DataSync service. DataSync already supported Microsoft Azure Files and Azure Blob, and Google Cloud Storage. It now also supports Oracle, Wasabi, Digital Ocean, Cloudflare and Backblaze clouds.
AWS announced GA of Mountpoint, an open-source file client that connects Linux-based applications directly to AWS S3 buckets to make contents in buckets appear as local files.
How AWS Approaches Storage
While AWS Storage Day 2023 lacks any one blockbuster addition, the new launches reflect Amazon’s solid three-pillar approach to cloud storage. Those pillars are:
- Creating a comprehensive set of services that can cover a diverse set of customer needs
- Allowing customers to integrate their data sources so they can easily access them and run analysis wherever the data sources reside
- Providing the proper tools to support customers’ governance strategy so they can feel confident about using their data
These pillars should help AWS stand out during this time when organizations are looking to gain more value and improve costs whether their data is stored on premises or in public clouds.
Disclosure: The Futurum Group is a research and advisory firm that engages or has engaged in research, analysis, and advisory services with many technology companies, including those mentioned in this article. The author does not hold any equity positions with any company mentioned in this article.
Analysis and opinions expressed herein are specific to the analyst individually and data and other information that might have been provided for validation, not those of The Futurum Group as a whole.
Other Insights from The Futurum Group:
AWS Expands File Storage Performance
Using AWS Services for Cyber-Resiliency
Amazon’s Ongoing Commitments to Sustainability and DEI
Author Information
Krista Case brings over 15 years of experience providing research and advisory services and creating thought leadership content. Her vantage point spans technology and vendor portfolio developments; customer buying behavior trends; and vendor ecosystems, go-to-market positioning, and business models. Her work has appeared in major publications including eWeek, TechTarget and The Register.
Dave focuses on the rapidly evolving integrated infrastructure and cloud storage markets.
