Analyst(s): Steven Dickens
Publication Date: August 29, 2024

The News: As IBM prepares to launch its next mainframe system in 2025, Telum II will be a cornerstone of its strategy to drive the next wave of innovation in enterprise computing. Whether deployed in traditional mainframes or LinuxONE systems, Telum II’s advanced capabilities are poised to deliver significant value to businesses seeking to enhance operational efficiency and security through AI.

IBM’s New Telum II Processor – A Strategic Advancement in Transactional AI

Analyst Take: In a recent interview at IBM, I had the pleasure of sitting down with two key figures behind IBM’s mainframe processor developments: Susan Eickhoff, Director of IBM Z Processor Development, and Chris Berry, a Distinguished Engineer specializing in processor design. Our conversation delved into the intricacies of IBM’s Telum II, the latest iteration in their line of mainframe processors, and IBM’s broader role in the semiconductor industry. Below, I’ll share the insights and key takeaways from our discussion.

IBM and the Semiconductor Industry

While companies such as NVIDIA, Intel, and AMD dominate headlines in the semiconductor industry, IBM’s contribution is often overlooked. However, as Chris Berry pointed out, IBM has been a pivotal player since the inception of semiconductors. Perhaps surprisingly to some, IBM has a global team of 800 engineers at the forefront of advancing semiconductor technology, working on cutting-edge nodes such as the two-nanometer process.

IBM’s role extends beyond research, encompassing the design and development of processors that form the backbone of its mainframe systems. The Telum II processor, for instance, is a marvel of engineering, featuring 32 billion transistors, 24 miles of wire, and an operating frequency of over 5.5 gigahertz. Such specifications underscore IBM’s deep integration and influence within the semiconductor industry despite not being as publicly recognized as some of its competitors.

Tellum II – Key Technical Takeaways:

IBM Telum II Processor – The IBM Telum II processor represents a significant advancement in mainframe technology, particularly in its ability to handle AI-driven workloads. Built on Samsung’s 5nm process technology, Telum II features eight high-performance cores running at 5.5GHz, with a 40% increase in on-chip cache capacity compared to its predecessor. The processor integrates a new Data Processing Unit (DPU) for IO acceleration, which optimizes data flow and reduces latency in complex transaction processing environments. The 360MB L2 cache and the expanded virtual L3 and L4 caches, reaching 2.88GB, ensure that large datasets can be managed more efficiently, providing low-latency access to critical data during transactions.
IBM Spyre Accelerator – The IBM Spyre Accelerator, previewed at Hot Chips 2024, is designed to complement the Telum II processor by providing additional AI compute capability. Spyre is built with 32 AI accelerator cores, leveraging 25.6 billion transistors and 14 miles of wiring, all within a 5nm process node. This architecture allows for dense packing of compute power, enabling the Spyre to perform complex AI tasks efficiently. Each Spyre card, mounted on a PCIe interface, can be clustered to scale AI workloads across the IBM Z system. A cluster of eight Spyre cards can add 256 accelerator cores to the system, significantly enhancing the mainframe’s ability to process large AI models, including those required for generative AI and ensemble AI methods. The efficiency of Spyre’s architecture, particularly in handling matrix and vector multiplications, makes it well suited for the demanding AI workloads typical in enterprise environments.
IO Accelerator – The IO Accelerator, integrated into the Telum II processor as a DPU, marks a transformative leap in IO subsystem design. Each DPU features four processing clusters, with eight programmable micro-controllers per cluster, managing IO operations across two IO drawer domains. This design reduces the overhead associated with data transfers and enhances power efficiency. The DPU connects to both the PCIe interface and the cache fabric, which prevents the L2 cache from being overwhelmed by bulk data transfers, routing data directly to its destination. Using custom IO protocols within the DPU further improves availability, error checking, and virtualization, ensuring the system can handle massive bandwidth requirements with high reliability. The ability to support up to 192 PCIe cards in future IBM Z systems highlights the scalability of this architecture, making it well equipped to meet the demands of next-generation AI-driven applications.

AI Capabilities in Telum II

Artificial intelligence is undoubtedly the hot topic of 2024, and IBM is not sitting on the sidelines. Telum II represents a significant leap forward in integrating AI capabilities directly into mainframe processors. The journey began with the first Telum processor, which featured an on-chip AI accelerator. Telum II builds on this foundation by enhancing support for various data types and increasing processing power.

The introduction of Spire, a PCIe-attached dedicated AI accelerator, is a game-changer. Spire allows IBM to host larger models and provides the necessary compute power for generative AI, enabling tasks such as running the Watson Code Assistant on IBM’s granite models. This integration of AI within the mainframe infrastructure is particularly relevant for real-time processing tasks, such as fraud detection, where speed and accuracy are paramount.

The Power of Cache – Especially for Linux Workloads

One of the most critical features of the Telum II processor is its advanced cache architecture, which plays a pivotal role in optimizing database and transaction processing workloads, especially within Linux environments. With 360 megabytes of cache spread across eight cores, Telum II is engineered to ensure rapid data access. This capability is essential for handling large-scale database operations such as those deployed at Citi Bank with MongoDB. The substantial cache size is crucial for maintaining the high-performance standards required by enterprise workloads, where minimizing latency and maximizing throughput are key.

This advanced cache architecture is equally vital for IBM’s LinuxONE systems, designed to handle extreme workload consolidation. Linux workloads, whether deployed directly on Z systems or on LinuxONE, benefit significantly from Telum II’s caching capabilities. The large cache allows for sustained high utilization rates, enabling efficient processing of large datasets and complex queries without the performance degradation typically seen in x86-based systems. For organizations such as Citi Bank where massive MongoDB workloads are commonplace, leveraging such a robust cache architecture means faster processing times, reduced latency, and, ultimately, lower total cost of ownership. This translates to significant savings in software licensing and energy costs, further enhancing the value proposition of deploying Linux workloads on IBM’s mainframe and LinuxONE platforms.

DPU Market Growth

Futurum Research forecasts substantial growth in the Data Processing Unit (DPU) market, projecting it to reach $4.01 billion by 2028, driven by a compound annual growth rate (CAGR) of 32.8% from 2023 levels. This rapid expansion underscores the increasing importance of specialized processors in managing complex data tasks, particularly as AI workloads become more demanding and pervasive across industries. Within the broader AI processor market, DPUs are emerging as a critical component, providing the necessary infrastructure to handle data-intensive operations more efficiently than traditional CPUs or GPUs. As AI continues to evolve and scale, the role of DPUs is expected to become even more prominent, fueling further innovation and investment in this market segment.

What to Watch:

As IBM prepares to introduce the Telum II processor, Spyre Accelerator, and integrated IO Accelerator, clients should closely monitor how these technologies will impact AI deployment strategies. Telum II’s advanced capabilities, particularly its enhanced cache architecture and AI accelerator, are designed to support high-performance transactional workloads and complex AI models. The integration of the Spyre Accelerator further amplifies AI compute power, making it essential for organizations to evaluate how these improvements can be leveraged within their existing infrastructure.
IBM is actively developing over 200 use cases for AI deployment, focusing on how these new processors can enhance applications ranging from fraud detection to real-time analytics. This breadth of use cases highlights the versatility of the Telum II and Spyre systems, suggesting that businesses across various industries will find tailored solutions that meet their specific needs. Clients should consider how these advancements align with their AI strategies, particularly as IBM continues expanding its AI ecosystem to deliver more accurate and efficient outcomes.
With the next-generation IBM Z mainframe system expected to ship in 2025, clients should start planning now to integrate these technologies into their IT environments. The Telum II processor and its associated accelerators are poised to set new benchmarks in enterprise computing, offering enhanced performance, security, and scalability. Organizations anticipating their future AI and transactional workload demands will be best positioned to take full advantage of these innovations as they come to market.
Add competitors, technologies, market moves, and/or enterprise preferences and direction to predict how this news can be enhanced, limited, or derailed by these other factors.

Final Thoughts

IBM’s Telum II processor represents a strategic advancement in the company’s ongoing efforts to innovate within the semiconductor and mainframe sectors, particularly with a focus on transactional AI. As AI becomes increasingly central to business operations, IBM’s approach to embedding AI inferencing directly on the mainframe—co-located with the data—offers a significant competitive advantage. This design minimizes latency and enhances data security by ensuring that sensitive information remains within the secure confines of the system. For enterprises managing vast amounts of transactional data, this capability is particularly critical.

The introduction of Telum II is timely, as IBM continues to deliver strong revenue performance even late into the z16 cycle, evidenced by an 8% growth in Q2 earnings. This sustained growth highlights the ongoing demand for high-performance, reliable, and secure computing environments, especially in sectors where transaction processing is mission-critical. Telum II is engineered to meet these demands while positioning IBM to capitalize on future AI-driven opportunities.

By bringing AI capabilities directly to the data, IBM enables real-time analytics and decision-making at the speed of transactions. This integration of AI within the hardware architecture ensures that IBM’s mainframes will continue to be at the cutting edge of enterprise technology, even as AI evolves and becomes more sophisticated.

As IBM prepares to launch its next mainframe system in 2025, Telum II will be a cornerstone of its strategy to drive the next wave of innovation in enterprise computing. Whether deployed in traditional mainframes or LinuxONE systems, Telum II’s advanced capabilities are poised to deliver significant value to businesses seeking to enhance operational efficiency and security through AI. IBM’s ability to sustain revenue growth late into the z16 cycle underscores its strong market position, and the introduction of Telum II further solidifies its leadership in the enterprise computing space.

Read the IBM press release here.

Disclosure: The Futurum Group is a research and advisory firm that engages or has engaged in research, analysis, and advisory services with many technology companies, including those mentioned in this article. The author does not hold any equity positions with any company mentioned in this article.

Analysis and opinions expressed herein are specific to the analyst individually and data and other information that might have been provided for validation, not those of The Futurum Group as a whole.