What is Covered in this Article:

IBM Storage Scale and some of the details
How RHEL and Nvidia play in the announcement
Perspective on parallel file systems versus pNFS for AI

The News: IBM announced content-aware Storage Scale for AI workloads, addressing requirements for faster response to data changes and data silos with the AI data pipeline.

IBM’s Content-Aware – Storage Scale for AI Workloads

Analyst Take: IBM announced their latest for IBM Scale, their parallel file system used heavily in the HPC and research markets. This release is focused on AI workloads and addressing the needs of the AI data pipeline, including what they call Content Aware Storage Scale, which incorporates semantics for metadata and near real-time detection for data changes. The new capabilities will initially be released through the IBM Fusion software and later released with the Fusion appliance.

IBM Storage Scale some of the details

IBM Storage Scale, which for some is also known as GPFS, has been well-known and used in the HPC clusters. Recently, its usability has improved significantly, including integrated appliance delivery. Now, with Generative AI, IBM is adding key functionalities that address the AI data pipeline needs with their Content Aware Storage (CAS). Using Active File Management (AFM), a function typically used for migration, Scale can abstract non-IBM NFS and S3 file systems. For instance, NetApp Ontap systems or Dell PowerScale (Isilon) can now be accessed through IBM Scale. This includes auto-detection of file changes that can be processed in real-time with the AI system.

IBM states that it will be “encapsulating vector database and data ingest pipeline within storage,” meaning it will be within the storage layer and available for other data derivatives within the vectorization process. Scale will preserve the data source ACL permissions for security purposes through the vector database. IBM is not developing a vector database but rather keeping the integration at a level so that clients can migrate if needed.

RHEL and Nvidia

IBM is providing two routes to inferencing, both of which include the vector database with 1+ billion vectors: one, integration of Nvidia multi-model PDF data extraction and NIMS, and the second, Red Hat’s RHEL AI for RAG.

We will also note here that IBM is supporting Nvidia GPU Direct using Nvidia BlueField-3 DPUs and Spectrum-X networking.

Where this is headed

As previously noted, the data infrastructure market is ramping up and extending into integrations with Nvidia, AI software components, vector databases, and real-time streaming databases, all to speed up the processing and action time of new and existing data. By now, most AI initiatives have been faced with data silos performance and access control security issues. IBM seeks to address these issues with Storage Scale and its CAS capabilities. Current clients will likely be attracted to the abstraction of multiple data sources. New clients will likely be interested in the promises of high speed at high-scale performance. All will be interested and require the integration aspects for vector databases plus Nvidia.

What to Watch:

The AI battleground for data at large scale: nPFS vs parallel file systems
Integration of vector databases with the data infrastructure
RHEL AI as an alternative to Nvidia software platforms

Disclosure: The Futurum Group is a research and advisory firm that engages or has engaged in research, analysis, and advisory services with many technology companies, including those mentioned in this article. The author does not hold any equity positions with any company mentioned in this article.

Analysis and opinions expressed herein are specific to the analyst individually and data and other information that might have been provided for validation, not those of The Futurum Group as a whole.

Other insights from The Futurum Group:

SuperComputing 2024: A Playground for the Future of Technology

VAST Data Takes on Agentic AI with a Major Platform Update

NetApp Insight 2024: Making Waves with AI

Author Information

Camberley Bates

Camberley brings over 25 years of executive experience leading sales and marketing teams at Fortune 500 firms. Before joining The Futurum Group, she led the Evaluator Group, an information technology analyst firm as Managing Director.

Her career has spanned all elements of sales and marketing including a 360-degree view of addressing challenges and delivering solutions was achieved from crossing the boundary of sales and channel engagement with large enterprise vendors and her own 100-person IT services firm.

Camberley has provided Global 250 startups with go-to-market strategies, creating a new market category “MAID” as Vice President of Marketing at COPAN and led a worldwide marketing team including channels as a VP at VERITAS. At GE Access, a $2B distribution company, she served as VP of a new division and succeeded in growing the company from $14 to $500 million and built a successful 100-person IT services firm. Camberley began her career at IBM in sales and management.

She holds a Bachelor of Science in International Business from California State University – Long Beach and executive certificates from Wellesley and Wharton School of Business.

IBM’s Content-Aware – Storage Scale for AI Workloads

What is Covered in this Article: