The Six Five team discusses Broadcom’s announcement of Jericho3-AI.
If you are interested in watching the full episode you can check it out here.
Disclaimer: The Six Five Webcast is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors and we ask that you do not treat us as such.
Transcript:
Daniel Newman: Let’s talk about Broadcom and AI. Now, interesting, right Pat? We’ve been covering Broadcom a lot more closely over the past few months because of, a lot of it had to do with the VMware deal. You and I have been watching this deal, talking a lot about its stature, trying to help the market understand. I would say that we’ve both, after getting under the hood, looking at the innovation thesis, talking to a lot of executives, including your time with CEO Hock Tan, I think we’ve probably come to the conclusion that Broadcom’s fairly misunderstood.
Patrick Moorhead: Totally, totally. Yeah. And the company’s been a lot more communicative as well, which I mean it has to go both ways, right?
Daniel Newman: Yeah. And I’d say it’s more misunderstood by its customer slash the overall IT community that consumes its product than it is by its investors. I think it’s a bit of a darling among investors because Hock Tan and the leadership team run that business for the street. I mean, they definitely understand the return on investment capital, the return on the acquisitions, they run lean, they run mean, they know how to invest in products that are growth oriented and they know how to maximize profitability of products that are sunsetting. And a lot of people don’t love that. But you know what? In the end, you have to run a business to be successful and that success means businesses have to make money. Now, Broadcom well known for a lot of things like routing, switching chips, network. Doesn’t necessarily get a lot of conversation around AI though. And in their high end switch family, they do have the Jericho line.
And this past week they came out talking about Jericho3AI ethernet switch. And the company came out with a somewhat bold position that they have some advantages in their technology over NVIDIA and the InfiniBand product. And as a very, and Pat, I’m not as familiar with cost, but what I understand is they have a lower cost, higher performing option and they offer something like a 10% performance improvement enabling the network to pay for itself. So they talked about a number of fabric innovations including load balancing, congestion free operation, zero impact failover and ultra-high rates were the things that they focused on, but they basically looked at the workload against the throughputs of InfiniBand at a number of different message sizes and a number of different speeds. And basically, what they found was the fabric that they’ve created was more performant at all of the different message sizes than InfiniBand.
And so it’s a really interesting sort of inflection because the other thing they’re also claiming, Pat, is power reduction. Now, I put out an article this week on MarketWatch where I talked about some of the interesting competitors coming up against NVIDIA. And while I still think NVIDIA, the thesis that they have the most opportunity in AI because of their positioning, I did call out sustainability as a big focal area and something that they’re going to have to watch because GPUs are a bit of power hogs. But on the networking side, I didn’t really reflect on that very much. But with a Jericho3AI, Pat, they’re saying that their fabric offers something like a 40% power reduction. It’s saying lowest power and cost optical interconnect, 25% system power reduction against pluggable optics. So they’re coming out and making some claims of some very strong performances on that side.
So while Broadcom isn’t making GPUs or ASICs right now for AI training and inference, we can’t forget about the fabric, how fast data can move, as such a key and critical part of success for this technology. And Broadcom, if they can offer a more efficient lower price product, will be very attractive to a number of different companies that are going to be getting in the game. And this could be a really exciting product for Broadcom and another competitor that NVIDIA’s going to have to be watching out for.
Patrick Moorhead: Yeah, Broadcom is an absolute connectivity beast of a company. And if I’m looking at their wired, I mean, they go all the way from the front haul to the back haul, enterprise access, they do metro aggregation, part of the metro core, the core, the DCI and the data center, and inside of the data center, they connect fleets and they also connect racks together. So they really are incredible. One of the important parts about doing training at an inference is that you have to scale. You can’t do that on a single GPU or an ASIC. The workload has to share a common memory plane, right? We talk about these 65 billion parameter models. You have to have a common data plane, sorry, common memory plane to be able to do that. And the way you can do that, the only way you can do that, is by networking the racks together that have the GPUs in them.
And it’s very rarely that somebody goes after a NVIDIA, and this is to me which made this so exciting. And essentially what they said is, hey, use our much lower cost ethernet connectivity to connect your GPUs than this super expensive InfiniBand. And the case they made was, hey, we have superior port speed, reducing the tiers in cluster sizes, load balancing, congestion management, telemetry, job completion time, multi-vendor support, I.E. Ethernet, and TCO, which is related to cost. So definitely looking like a little mini skirmish or a mini war of skirmishes is coming up on the network side. And it makes perfect sense also why, reinforcing one of the reasons that NVIDIA bought Melanox, which is to essentially partake in the revenue and be on the edge of creating training and inference clusters.
We hear AWS talk when they’re talking about their AI solutions as much about their super clusters as they do about the actual chips themselves. So this is a big deal, something to watch. And yet another example of Broadcom innovating out there in the market, right? Higher performance, lower cost, and a lot of other better attributes than InfiniBand. And by the way, the InfiniBand ethernet debate is only 10 years old, but the increase in performance and capabilities of ethernet are kind of a new thing. So, fun stuff.
Daniel Newman: It is. And we couldn’t make it to the list today, Pat, but it’s been a bit of an, I think NVIDIA’s doing fine, but I got some press calls this week about Microsoft building an AI chip. We’re not going to talk about that here today, but I’m saying the competition’s on the rise here, companies are looking to do it more efficiently. They’re looking to find ways to network cheaper, to do it with lower power, to build ASICs to run certain models. So the era of just the all-encompassing big powerful GPU is also in a bit of a flux. We’ll come back to that one probably at some point in the future.
Author Information
Daniel is the CEO of The Futurum Group. Living his life at the intersection of people and technology, Daniel works with the world’s largest technology brands exploring Digital Transformation and how it is influencing the enterprise.
From the leading edge of AI to global technology policy, Daniel makes the connections between business, people and tech that are required for companies to benefit most from their technology investments. Daniel is a top 5 globally ranked industry analyst and his ideas are regularly cited or shared in television appearances by CNBC, Bloomberg, Wall Street Journal and hundreds of other sites around the world.
A 7x Best-Selling Author including his most recent book “Human/Machine.” Daniel is also a Forbes and MarketWatch (Dow Jones) contributor.
An MBA and Former Graduate Adjunct Faculty, Daniel is an Austin Texas transplant after 40 years in Chicago. His speaking takes him around the world each year as he shares his vision of the role technology will play in our future.
