The News: On November 13, Hewlett Packard Enterprise (HPE) announced a turnkey supercomputing solution for generative AI designed to accelerate the training and tuning of AI models using private data sets. The solution will be available in December.
Here are the key details of the solution:
- Features NVIDIA’s newest and most powerful graphics processing units (GPUs), the Grace Hopper GH200 Superchips.
- Includes software tools to build AI applications, customize pre-built models, and develop and modify code. The software is integrated with HPE Cray supercomputing technology.
- Software: HPE Machine Learning Development Environment, NVIDIA AI Enterprise, HPE Cray Programming Environment
- Performance gains:
- Using HPE Machine Learning Development Environment on the system, the 70 billion parameter Llama 2 model was fine-tuned in less than 3 minutes.
- Improved system performance by 2x-3x when compared with running similar performance tests on a NVIDIA A100 based system. No word on how the new Grace Hopper system performed against NVIDIA H100 systems.
- Performance, power efficiency:
- Energy efficiency is core to HPE’s computing initiatives, which deliver solutions with liquid-cooling capabilities that can drive up to 20% performance improvement per kilowatt over air-cooled solutions and consume 15% less power.
Read the press release from HPE on the new AI turnkey solution here
HPE Offers Turnkey Solution for AI Training
Analyst Take: The scale of generative AI is intimidating particularly when you consider the compute power it takes to train AI models. Conventional thinking in these early days of generative AI was that public cloud-based compute could offer the scale and efficiency to run AI training workloads. HPE, and other on-premises compute providers, are proving that generative AI can be handled on-premises by enterprises in their own environments. Here is what HPE’s new turnkey AI training solution means for on-premises AI.
AI Training at Scale On-Premises
HPE’s new solution tests the theory that AI training is too expensive to be done on-premises. HPE will have to prove to its prospects that AI training can be done economically. If the company succeeds in doing so, it totally changes the dynamic regarding the market barriers for generative AI. How? It breaks a stranglehold the public cloud hyperscalers have on AI training and likely will drive down AI training costs.
Addressing AI Compute Costs
There is no question that AI workloads, the compute costs to run generative AI, are formidable. HPE’s commitment to liquid-cooled systems could be a major factor in driving down costs. Although it is true that this approach is not necessarily unique to HPE, the company does have the expertise and experience to deliver improving performance and power efficiencies for compute systems.
Conclusion
HPE is highly motivated to drive on-premises AI. It is interesting that NVIDIA has committed allotting some of its latest and greatest GPUs to HPE’s on-premises solution, a good sign that HPE not only believes in the viability of on-premises AI but the company’s significant partner does as well. Prospects will likely look for proof points of the cycles it takes for the new systems to run AI training, as to gauge costs, which means HPE has likely already run such scenarios and is confident it can sell the solution.
Disclosure: The Futurum Group is a research and advisory firm that engages or has engaged in research, analysis, and advisory services with many technology companies, including those mentioned in this article. The author does not hold any equity positions with any company mentioned in this article.
Analysis and opinions expressed herein are specific to the analyst individually and data and other information that might have been provided for validation, not those of The Futurum Group as a whole.
Other insights from The Futurum Group:
Empowering AI Innovation with HPE’s Advanced Supercomputing Solution
Powering Your Future Business with AI Inference – Futurum Tech Webcast
Author Information
Based in Tampa, Florida, Mark is a veteran market research analyst with 25 years of experience interpreting technology business and holds a Bachelor of Science from the University of Florida.
