IBM ML Inference Card

The Six Five team discusses IBM’s ML Inference Card.

If you are interested in watching the full episode you can check it out here.

Disclaimer: The Six Five Webcast is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors and we do not ask that you treat us as such.

Transcript:

Patrick Moorhead: IBM comes out with an ML Inference Card. So, we should be surprised but we shouldn’t be surprised, Daniel, at this and that we saw when the Z16 ship came out, it had an integrated AI block called Tellem. Okay.

So essentially, what the company has done is they have taken that block which is very scalable and they talked about this.

They made a much bigger chip and then they put it on a PCIE Express card that you could put in basically any server out there.

There wasn’t as much information about the chip and the card as I would have liked. And I actually had to ask IBM a couple questions. I never saw the word inference in the blog. And man, I looked at it, but I did see training in there twice. So, I was wondering, “Hey, wait a second, Tellem was inference, real-time inference, a low-latency inference. Is this their training play that you could run in a power or in an X86 system?” But the answer is no. This is absolutely an inference card.

And the key here is as we’ve seen and sometimes I think it’s better to be later than first, the industry has gone from a very high degree of precision, 32-bit to 16-bit to 8-bit to 4-bit where you don’t need all of the accuracy to do good inference.

So, this is a low bit-rate inference card. I don’t know how many watts it’s at so I don’t know how small the form factor. It could be at the edge. I don’t know if it needs passive or active cooling. So, they left a lot of questions out there. But I think the big story here is that IBM Research is doing things that are surprising us all.

You and I spent, gosh, three days between Yorktown and where was it? Yorktown, you and I both have been to Poughkeepsie for Z. What was the third city we went to?

Daniel Newman: Albany.

Patrick Moorhead: Yeah, Albany NanoTech. So, this came out from the research group, not the product groups. So, I think we’re going to have to see exactly how this is productized in the future. But I think it shows the very high capabilities of the IBM Research team that you and I have spent a lot of time with. Why didn’t they tell us about this when we were on site?

Man, they even stealth us, Daniel.

But listen, you can’t tell everybody everything. But we’re going to have to get these folks on the Six Five Pod to tell us more about this and what they’re going to do with it.

Daniel Newman: I know we’re talking to Rob Thomas next week about some Watson ML. Maybe we could sneak in an AIU ask, but I’m not sure he’d be the right person because it’s in research.

Pat, I’m going to give you a paragraph out of my impending research note. When will the AIU chip ready for enterprise use? How much will it cost? Is it a work in progress already in mass production? Those questions weren’t clear from the initial AIU announcement. What is clear is that IBM recognizes it’s beyond time to change the way AI computing happens.

Now, a little bit market-y maybe when I say that, but what I walked away like you was there’s a lot of questions yet. But I do like very much that IBM is continuing to plan its flag. It’s leaning into semiconductor manufacturing design research. And by the way, with the recent passage of the CHIPS and Technology Act, we know that IBM has its hand up in saying, “Hey, we’re another company with really tremendous engineering talent, manufacturing capabilities or research to support those capabilities, intellectual property.”

So, over the year, Pat, we’ve seen the two nanometer announcement come out from IBM. We’re seeing the AIU which we’ve needed another acronym, by the way. I feel like this is important that we add this to the GPU, VPU, DPU, CPU.

Patrick Moorhead: One more.

Daniel Newman: What?

Patrick Moorhead: QPU.

Daniel Newman: Quantum Processing Unit, nice. But like I said, I look at this more as IBM. Really like I said, planning a flag, raising its hand, clearly articulating its intent to participate in a more meaningful way with its intellectual property democratizing and making it available to the market. And in an era of US-based semiconductor, design and manufacture being more in demand than ever before and IBM’s obvious improved performance based on our third topic. It’s not a terrible time for IBM to make sure the world knows it’s also making big contributions in semiconductor technology.

So, that’s where I saw it. It’s interesting. It’s exciting. There’s so many more questions than answers right now. But Pat, it wouldn’t be the Six Five if we didn’t put a little speculation and an analysis around what was sort of a loose but exciting and interesting press release.

Patrick Moorhead: Yeah, probably the most important which I forgot in my diatribe was software. What is the middleware that it’s going to be using, right? Does it use CUDA? Does it use oneAPI? Is it going to use what AMD is creating with its combination with Xilinx? We don’t even know what middleware that this runs. But no, a lot of questions and look at us, we just fell right in. the planet. Ciao. to invest in more technology?

Author Information

Daniel is the CEO of The Futurum Group. Living his life at the intersection of people and technology, Daniel works with the world’s largest technology brands exploring Digital Transformation and how it is influencing the enterprise.

From the leading edge of AI to global technology policy, Daniel makes the connections between business, people and tech that are required for companies to benefit most from their technology investments. Daniel is a top 5 globally ranked industry analyst and his ideas are regularly cited or shared in television appearances by CNBC, Bloomberg, Wall Street Journal and hundreds of other sites around the world.

A 7x Best-Selling Author including his most recent book “Human/Machine.” Daniel is also a Forbes and MarketWatch (Dow Jones) contributor.

An MBA and Former Graduate Adjunct Faculty, Daniel is an Austin Texas transplant after 40 years in Chicago. His speaking takes him around the world each year as he shares his vision of the role technology will play in our future.

Related Insights
Foxconn's Record October Revenue Signals AI Infrastructure Peak Capture
November 17, 2026

Foxconn’s Record October Revenue Signals AI Infrastructure Peak Capture

Hon Hai Precision Industry posted record October 2026 revenue of NT$1.3 trillion, driven by 131% AI hardware growth, validating Foxconn's transformation into a full-stack AI infrastructure provider....
Applied Materials Q4 earnings
November 16, 2026

Applied Materials Q4 FY 2026: AI Chip Demand Lifts Equipment Sales

Applied Materials reported Q4 FY 2026 revenue of $10.41 billion, up 53% YoY, as AI chip demand drives semiconductor systems revenue to $8 billion. Strong margins and earnings growth signal...
Cisco Q1 FY 2027 Silicon One Wins Expand AI Networking Reach
November 16, 2026

Cisco Q1 FY 2027: Silicon One Wins Expand AI Networking Reach

Futurum Research analyzes Cisco’s Q1 FY 2027 earnings, focusing on hyperscaler networking wins, the campus refresh, and Splunk’s expansion into identity monitoring....
FPT IS Bets on Silicon-Software Convergence at VKAISS 2026
November 14, 2026

FPT IS Bets on Silicon-Software Convergence at VKAISS 2026

FPT IS signals strategic pivot toward silicon-software convergence, signing four MOUs with Korean semiconductor partners at VKAISS 2026 to capitalize on Vietnam's semiconductor buildout and Asia-Pacific AI growth....
NXP Unveils Unified Robotics Platform to Take the Neural Axis Into Production
November 13, 2026

NXP Unveils Unified Robotics Platform to Take the Neural Axis Into Production

Brendan Burke and Olivier Blanchard of Futurum examine the NXP Robotics Platform launch, where an open-source SDK and flight-validated drone reference designs convert the neural axis architecture into shipping hardware....
MHI's ¥67.5B IGCC Loss Flags Execution Risk in AI's Power Bet
November 13, 2026

MHI’s ¥67.5B IGCC Loss Flags Execution Risk in AI’s Power Bet

Mitsubishi Heavy Industries' ¥67.5 billion warranty loss on IGCC plants signals critical execution risks in the advanced power infrastructure that AI data centers depend on to meet projected 945 TWh...

Book a Demo

Welcome

The vision behind everything in Futurum’s Custom Research practice is this: research should show you what is happening, what comes next, and what to do about it. It should be personal to each audience, easy for people to grasp, and structured so LLMs can reason over it accurately. And it should be fast and turnkey; you want answers now, not another project to carry for quarters.

Whether you are defining business, channel, or go-to-market strategy; evaluating vendors or justifying ROI; or commissioning research to fill an emerging market need, we have your back, with a program that answers your questions with the objectivity and credibility to drive real decisions.

To do it, we bring unmatched data to bear: Futurum research, surveys, and market projections; validated market feeds; ETR’s 15 years of insight from 10,000 technology decision-makers; G2’s buyer and user data; and what our analysts hear every day. Add leading primary collection, from AI-moderated voice interviews to surveys and analyst-led interviews, all turnkey, and every project comes out credible, nuanced, and actionable.

And we don’t just drop the results in your lap. For internal work, we provide analyst-led sessions, interactive dashboards, and a range of formats. For market-facing work, Futurum delivers turnkey activation and amplification that actually gets seen, by people and by LLMs, through our media and share of voice. This is research that moves decisions and markets.

We will meet you wherever you are, from a fast-turn brief to a multi-year program, and shape the work to your goals, timeline, and budget. The right program for your moment.

If any of this is useful, I would love to talk.

Benjamin Brown, VP Custom Research, Futurum Research

Benjamin Brown

VP, Custom Research · The Futurum Group

Newsletter Sign-up Form

Get important insights straight to your inbox, receive first looks at eBooks, exclusive event invitations, custom content, and more. We promise not to spam you or sell your name to anyone. You can always unsubscribe at any time.

All fields are required






Thank you, we received your request, a member of our team will be in contact with you.