The News: Groq confirmed its $640 million Series D round funding and Cerebras Systems is planning an IPO. Intel Corporation plans to reduce costs by $10 billion in 2025 by laying off 15% of its staff and otherwise decreasing expenses. NVIDIA is the target of a US antitrust investigation and has delayed volume deliveries of its Blackwell GPU by a quarter.
AI Chipsets in Context: Good and Bad News for Vendors
Analyst Take: According to the soon-to-be-released Futurum Intelligence report on vendor revenue for processor and accelerator chipsets used in data centers for AI training and inference applications, we estimate vendor revenue of $38B in 2023, growing to $138B by 2028. Good and bad strategic moves by vendors are amplified by the sheer size of this rapidly growing market. Put another way, there is a lot of money to be made or lost.
Up-and-coming companies such as Cerebras Systems and Groq have capitalized on the need for and advantages of specialized hardware for AI but must get large influxes of cash to fuel their manufacturing and cloud deployment growth. NVIDIA is the GPU juggernaut, but has it grown too fast in ways that might stifle competition? The US government wants to know, and the stakes are high, given the recent monopoly case judgment against Google. Intel, the first semiconductor company many of us learned about, has evidently faltered in getting traction for its AI accelerators.
No company is too big or growing too fast to fail, or to get restructured voluntarily or by decree. Still, though the market is growing fast, it will mature over many years, allowing ample time for corrections.
Cerebras Systems Wants to be a Public AI Chipset Company
Cerebras takes a different approach from most vendors by consolidating AI processing power that would usually be spread over a multi-rack cluster onto wafer-scale AI-optimized chips in a single rack. Their latest product, the CS-3, contains 900,000 cores and 44 GB of memory in that rack, with an interconnect bandwidth of 214 PB/s.
Compared to GPU clusters, the company claims that the CS-3 delivers orders of magnitudes more processing power in a far smaller footprint, using much less energy.
Sustainability is an important but nuanced issue with AI accelerators and the large data centers that contain them. On one hand, if I can do the same or much more work and use significantly less energy, I can produce much less heat and other byproducts. On the other hand, if I use less power in one place or for one Generative AI application, for example, I can use that power to do even more processing. In considering such power use, I recommend you start with the assumption that your total energy use is capped and then understand what you can do and trade off within that limit. In addition to accelerator semiconductors, you must look at other infrastructure elements, such as the sustainability factors of HDDs versus SDDs for storage. Cost is always a variable in the equation as well.
Cerebras has not yet given specifics on the proposed IPO. Almost three years ago, its $250 million Series F funding round valued the company at $4 billion, so we will likely not see an IPO valuation less than that.
Groq Gets Twice the Money as Last Time
A simple view of AI processing is that it has two processes: training a model from millions to billions of inputs and then using it for inferencing to produce answers to queries or prompts. Rather than using a general-purpose CPU or GPU, users can employ AI accelerators designed and manufactured specifically for training and inferencing. For example, Amazon created the Tranium and Inferentia chipsets for these purposes and made them available on AWS. I’m guessing you know which task which chip is for!
Groq focuses on inferencing. While training a deep learning or generative AI model can take weeks, consumers of the model expect fast answers when they use it. Groq has built what it calls a Language Processing Unit, or LPU. It focuses on accelerating the mathematical operations from a discipline called linear algebra. This area looks at transformations on vectors of numeric information and involves matrices that can involve thousands of dimensions. The Groq LPU design differs from that of a GPU by eliminating bottlenecks on one or more chips caused by delays in getting instructions or access to required data in memory.
On August 5, Groq announced that it raised $640 million in a Series D investment round, with a company valuation of $2.8 billion. Their Series C brought in $300 million in April 2021, with a valuation of $1.1 billion. In total, they have raised slightly more than $1 billion.
What Groq needs now is revenue. In our Futurum Intelligence report, Groq ranked 17 in overall revenue out of the 18 vendors we considered. Their LPU architecture is compelling, and they now need to go full speed at market leader NVIDIA for inferencing applications share.
Intel Is Reckoning with Its Poor Performance in the AI Chipset Market
Intel has been feeling the pressure from NVIDIA, AMD, the Arm vendors, and the specialty “XPU” AI accelerator manufacturers. It announced it will lay off 15% of its staff and pause its dividend. At the market close on August 8, the stock sat at $20.49/share, down $27.31 since the beginning of the year. Obviously, things have to change, and many of them.
In a letter to employees on August 1, Intel CEO Pat Gelsinger stated:
Simply put, we must align our cost structure with our new operating model and fundamentally change the way we operate. Our revenues have not grown as expected – and we’ve yet to fully benefit from powerful trends, like AI. Our costs are too high, our margins are too low. We need bolder actions to address both – particularly given our financial results and outlook for the second half of 2024, which is tougher than previously expected.
Later in the letter, Gelsinger laid out the general actions the company is taking: reducing operational costs, simplifying the portfolio, eliminating complexity, reducing capital and other costs, and suspending the dividend. Most of these are straight from a standard business school text, though the dividend action indicates that the financial situation warrants robust responses.
Intel can and will bounce back, but it will be a different company. In March, Intel and the US Department of Congress announced that the company would receive up to $8.5 billion under the 2022 Chips and Science Act. This is a vote of confidence, though moderated by necessity, that Intel can design, manufacture, and sell in volume advanced semiconductors, AI and otherwise, in the United States.
When companies find themselves in this situation, it is not simply a failure of the rank-and-file to execute a perfect strategy from the leadership. The strategy must change, and Intel has outlined how it will do it. The 15,000 leaving the company will feel the change and the sacrifice. Leadership must feel it as well in their own ways.
NVIDIA Faces Federal Probe and a Shipping Delay
NVIDIA has been exuberantly successful over the last several years, making it the number three company in market cap after Apple and Microsoft. Has it grown too fast into a dominant market position? Is it taking anti-competitive steps? After NVIDIA announced that it planned to buy Israeli company Run:ai, the US Department of Justice began an antitrust investigation of the company.
On August 6, a US judge determined that Google is a monopoly because of its search business and deals with companies such as Apple. Spurred by this success, the DOJ will feel more confident going after large and highly profitable companies such as NVIDIA, trying to understand precisely how they got there and if everything was proper. Ultimately, this is a legal issue, not a matter of opinion for pundits. Whatever happens, we can be sure it will take a long time for the investigation and any litigation that might follow.
On a final note, NVIDIA introduced its Blackwell GPU at the company’s conference in March, promising up to 25 times less energy use and cost than its predecessor model. Who would want one? Evidently, many people do, from the hyperscalers, to the data center managers, to the enterprise IT managers. Unfortunately, many of them will have to wait a few months for the new chipset and packaging to ship in volume.
I think this will make very little difference. At the moment, we have significant cloud and on-premises accelerator capacity for AI applications. Most of these are in development and have not yet been deployed in production, especially for Generative AI use cases. The delay is a minor blip and not something that will adversely affect innovation in the use of AI.
Key Takeaway
The Futurum Group predicts that the market for AI accelerators for cloud and data center applications will grow to $138 billion in 2028 with a CAGR of almost 30%. Strategic moves and decisions today are unusually sensitive because of the enormous amount of investment. Lose or gain one hyperscaler or a few big enterprises, and a vendor can find itself struggling or winning big. A vendor in this market must catch its breath and think through decisions with competitive market intelligence across the breadth of the industry. Moreover, the computer industry is cyclic, so a vendor must draw upon the experience of its leaders and third parties to get a clear and honest analysis of where it is and what it must do to reach its goals. Said otherwise, it needs Human Intelligence for predictive and prescriptive analytics.
Disclosure: The Futurum Group is a research and advisory firm that engages or has engaged in research, analysis, and advisory services with many technology companies, including those mentioned in this article. The author does not hold an equity position in any company mentioned in this article.
Analysis and opinions expressed herein are specific to the analyst individually and data and other information that might have been provided for validation, not those of The Futurum Group as a whole.
Other Insights from The Futurum Group:
Intel’s Q2 2024 Earnings Release: Navigating Challenges and Strategic Shifts
Talking Qualcomm, Intel, AMD, Arm, Lattice, Microsoft, Amazon
Cerebras CS-3: Bring on the NVIDIA Blackwell Competition
GroqDay: Groq Sets its Sights on Accelerating the Speed of Artificial Intelligence
Author Information
Dr. Bob Sutor has been a technical leader and executive in the IT industry for over 40 years. Bob’s industry role is to advance quantum and AI technologies by building strong business, partner, technical, and educational ecosystems. The singular goal is to evolve quantum and AI to help solve some of the critical computational problems facing society today. Bob is widely quoted in the press, delivers conference keynotes, and works with industry analysts and investors to accelerate understanding and adoption of quantum technologies. Bob is the Vice President and Practice Lead for Emerging Technologies at The Futurum Group. He helps clients understand sophisticated technologies in order to make the best use of them for success in their organizations and industries. He is also an Adjunct Professor in the Department of Computer Science and Engineering at the University at Buffalo, New York, USA. More than two decades of Bob’s career were spent in IBM Research in New York. During his time there, he worked on or led efforts in symbolic mathematical computation, optimization, AI, blockchain, and quantum computing. He was also an executive on the software side of the IBM business in areas including middleware, software on Linux, mobile, open source, and emerging industry standards. He was the Vice President of Corporate Development and, later, Chief Quantum Advocate, at Infleqtion, a quantum computing and quantum sensing company based in Boulder, Colorado USA. Bob is a theoretical mathematician by training, has a Ph.D. from Princeton University, and an undergraduate degree from Harvard College.
He’s the author of a book about quantum computing called Dancing with Qubits, which was published in 2019, with the Second Edition released in March 2024. He is also the author of the 2021 book Dancing with Python, an introduction to Python coding for classical and quantum computing. Areas in which he’s worked: quantum computing, AI, blockchain, mathematics and mathematical software, Linux, open source, standards management, product management and marketing, computer algebra, and web standards.
