The Six Five team discusses Broadcom “Enabling AI Infrastructure” Investor Meeting 2024.
If you are interested in watching the full episode you can check it out here.
Disclaimer: The Six Five Webcast is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors and we ask that you do not treat us as such.
Transcript:
Daniel Newman: When Broadcom hosted its enabling AI infrastructure investor meeting you and I, Pat, were there and I think we both walked away impressed.
Patrick Moorhead: It was an investor meeting and there were three other analyst firms there, which is a very tight-knit group. There were no press, no pictures were allowed. It was quite the event. And this is really a coming out party for Charlie Kawwas, president. He runs all of semiconductors, all 17 business units. And this focus was really on what is Broadcom doing in terms of AI connectivity and an AI accelerator. Coming right off the NVIDIA event people wanted to know, hey, what are all these accelerators that Broadcom is working on? I learned a lot. I’m not one of these analysts who says, oh, I know everything, I don’t, and it was a learning exercise for me. First off, they brought out a third design. All three of them are consumer. Rumor has it that the first two are Google and Meta. What’s the third one? Who knows?
These are also not networking. These are not AI ASICs. Sure, it includes a compute ASIC, but these are full up SoCs without a CPU. It has compute, memory, network IO, storage on there. By the way, the other brain explosion here was the one year from design beginning to ramp. And when you think about an ASIC, you’re typically looking at a three to four years. So, hypothetically they could crank out a new one every 18 months for a customer, and that is just absolutely absurdly fast. A lot of this comes to them doing this for the last decade. So, pretty amazing. I think the second thing was ethernet versus InfiniBand, connecting GPUs, custom accelerators and merchant accelerators. I got to tell you, if you’re a hyperscaler or an enterprise that wants to connect all three types of accelerators, ethernet is absolutely what you need for scale out.
I came convinced. 10% of performance advantage, a 30% reliability advantage, cost advantage, the standards, it’s just crazy. And let’s not forget Thor 2, not a smart NIC, but heavy duty RDMA for a single memory plane, pretty amazing. I did like the going through and showing how many clusters, Amazon, Oracle, Meta, ByteDance, I think 130,000 AI clusters with ethernet. So, I came away really just all in on ethernet. It’s not that fiber channel is bad, it’s just that PCI, sorry, ethernet is even better. So, in the rack, Jas Tremblay got up, did a nice discussion on inside the rack, and essentially that’s connecting CPU, GPU, NICs, AI accelerators and storage, and they dominate this market by the way. You can look at Dell, HPE, Lenovo, Supermicro systems and PCIe switch from Broadcom has been the staple of that, and I think I don’t see any reason why that’s not going to be in the future.
And a little flashback to AMD’s AI event was PCIe Gen7 switch. So, PCIe, low latency. Again, this company just dominates here. Co-packaged optics, right? So, instead of having 140 transceivers in a system, you have this super-duper cool CPO, co-packaged optics module. And this has been in discussion forever, but they’re actually shipping this now. So, if you want lower power, increased reliability, lowest cost per bit, CPO looks like an amazing option. It’s almost like a no-brainer. I left wondering, what am I missing here? What do I not understand from this? And then, finally I’ll end this. SerDes is the basis of all analog goodness, and they’ve got this 200 gig SerDes based on three nanometer that bodes well for the future of Broadcom quite frankly, and gets in the business of Marvell with PAM-4. So, great stuff.
Daniel Newman: Take a breath buddy. Listen, I’m going to make this, I’m going to really try to dumb down my thoughts here.
Patrick Moorhead: And by the way, I meant ethernet versus InfiniBand, sorry.
Daniel Newman: Absolutely. I’m just going to try to, what did I come away with here? I want to do an analogy. We’re going to talk about Apple later, but here’s my analogy is that, we’re going to start to see market shift to a normalization in a two space market. And by the way, there’s two space, two players, but player one is NVIDIA, end to end, everything. Player two is everyone else, literally the rest of the ecosystem. It’s all the other chip makers, it’s other infrastructure providers. It’s going to be a collaboration. This is kind of that open AI, open ethernet, all these consortiums. And so, NVIDIA is Apple in this case, and InfiniBand, NVLink the whole building of chips and infrastructure and hardware and connectors and cables, and it’s going to be this totally vertically integrated solution here. And then, on the other side of it, ethernet is going to be the open, it’s going to be Google and Android.
This is how I’m looking at this thing. And the reason I’m saying this is, in the end they’re both very viable solutions, and that’s really what I came away with. Jensen got on the stage and goes, it’s not viable. And we went to this thing and it comes back. It’s like, well, it seems viable. You get at least equal if not better performance in optimal settings. You actually do it for less money, which is economical, which is important to people, and there’s a couple of other things. I thought it was really prudent that they talked about the fragility of InfiniBand. They talked about that InfiniBand has a higher fail rate, having those cables sitting and ready at any given point versus ethernet, which is a little bit more stable. But, the way I kind of walked away from it is, Broadcom’s going to be at the center of Android. And so, this network back plane and this network plane that’s going to have to connect all these XPUs or all this compute is going to be done one of two ways.
And we’re actually seeing this war playing out by the way, in other places. We’re seeing it play out with the hyperscalers. The hyperscalers are playing out right now, because some are adopting the all in NVIDIA and reselling it and others are saying, well, we want to control the network. We’re not going to use their connectivity. We’ve got our own plane. And I think people could figure out who we’re talking about. But, in the end, I think the market ends up being split and I think it ends up being much closer to parody and NVIDIA is going to be really big. And by the way, this is not like, this is a multi-trillion dollar ultimate market with hundreds of billions of GPUs, and then all the peripherals and stuff , and I think that’s how it plays out. Now, a couple of just really quick things on the overall AI story.
Look, the semiconductor business, everything about Broadcom has been about VMware, but the semiconductor business is growing healthily. They were able to basically say now that in ’24, 25% of its revenue is AI related, and actually they’re accelerating that forecast now to 35% of about $10 billion. They’ve got three mega customers now that are planning to work on their XPUs. No one knows who they are. There is some suspicion of who they are. I think it might be a company that is fruit related. It could be this new big customer. But, again, no speculation, heard nothing. This is just my opinion. But, I want it on the record in case I end up being right about this later.
Author Information
Daniel is the CEO of The Futurum Group. Living his life at the intersection of people and technology, Daniel works with the world’s largest technology brands exploring Digital Transformation and how it is influencing the enterprise.
From the leading edge of AI to global technology policy, Daniel makes the connections between business, people and tech that are required for companies to benefit most from their technology investments. Daniel is a top 5 globally ranked industry analyst and his ideas are regularly cited or shared in television appearances by CNBC, Bloomberg, Wall Street Journal and hundreds of other sites around the world.
A 7x Best-Selling Author including his most recent book “Human/Machine.” Daniel is also a Forbes and MarketWatch (Dow Jones) contributor.
An MBA and Former Graduate Adjunct Faculty, Daniel is an Austin Texas transplant after 40 years in Chicago. His speaking takes him around the world each year as he shares his vision of the role technology will play in our future.
