Tech giants are increasingly turning to custom silicon to power their AI ambitions – a shift that is rapidly reshaping the semiconductor industry. Heavy-weights like AWS, Google, Microsoft, and Meta, are aiming to enhance performance, conserve power, and stay ahead in the AI race.
Host Patrick Moorhead is joined by guest Nigel Alvares, VP of Marketing at Marvell Technology, on this episode of Six Five On The Road at Marvell Industry Analyst Day to unpack the implications of this trend for the semi industry and how Marvell is navigating the future of custom silicon development.
Tune in for these key takeaways:
- Hyperscalers are driving the demand for custom chips to optimize performance and power efficiency as Moore’s Law slows
- Customization is happening across various devices, including CPUs, XPUs, CXL devices, and NICs
- Examples of chip customization in action, including projects from major players, showcasing the technological innovation driving the semiconductor market
- Marvell’s role and strategies within the custom chip sphere, highlighting the importance of IP foundation, process technology advancements, partnerships, and IP flexibility
- The emerging trend of leveraging AI in Electronic Design Automation (EDA) to expedite design processes, cut costs, and democratize access to chip customization
Learn more at Marvell Technology.
Watch the video at Six Five Media at Marvell Industry Analyst Day, and be sure to subscribe to our YouTube channel, so you never miss an episode.
Or listen to the audio here:
Disclaimer: Six Five On The Road is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors, and we ask that you do not treat us as such.
Transcript:
Patrick Moorhead: The Six Five is On The Road, here in Silicon Valley at Marvell Technology Headquarters. We are here talking about my favorite topic, and that is data center AI. So much investment what the hyperscalers are putting in theirs. We seem to be in this period where customization is important to get the performance, to get the speed, get the cost, and the power at the right place. I can’t imagine a better person to talk about this than Nigel with Marvell. How are you?
Nigel Alvares: I’m doing great, Patrick. Great to have you here.
Patrick Moorhead: Yeah. It’s great to be here. I love to visit and hear what you guys have to say. Hopefully we can give you some insights you can think about as well.
Nigel Alvares: Absolutely. All the time.
Patrick Moorhead: Yeah. I want to start out, I’d said in the prelude here, the industry always goes through … It’s like an accordion. We go through these mass customization, where things are custom. Even mainframes and minicomputers, everybody was doing their own thing. Then client server hit, where it was all about standards. We only built standard stuff. But we’re in now, this period where customizing chips is the thing to do. Can you tell me why people are doing this?
Nigel Alvares: Yeah. I think at the highest level, there’s two real drivers. One is the total cost of ownership. All these cloud operators are competing with one another.
Patrick Moorhead: Yeah.
Nigel Alvares: And they’re looking to deliver an optimal solution because they don’t want to be a commodity. That’s one, is the total cost of ownership. There’s many factors they have to customize. It’s not only silicon, it’s across the board. The other one is differentiation tied to the services they’re providing.
Patrick Moorhead: Yeah.
Nigel Alvares: That’s another key factor. Again, back to the commoditization, they need to really optimize the service they’re delivering to make it optimal at scale. The real problem they’re trying to solve is at scale. That drives the need to lower costs, get more out of the silicon, lower your power, as all these factors become bigger and bigger challenges for them to get to the next level of service and customer base.
Patrick Moorhead: Yeah, it makes total sense. The power is just such a driver that people are talking about, “Where do I put my nuclear reactor?” We’re talking about gigawatt facilities. It’s just mind-blowing. The only way that you can get the highest density in compute and networking in there is to go custom. I’m just amazed that so many people are doing this.
Nigel Alvares: Yeah. It’s amazing you bring up this gigawatt. Just put that in perspective. One gigawatt is the peak power for the whole city of San Francisco.
Patrick Moorhead: Yeah.
Nigel Alvares: One gigawatt for the whole city. That’s probably a million or so people. New York is 10 gigawatts in power, which is about 10, 11 million people. It just puts into perspective a whole city needs a gigawatt, and we’re talking about a data center needing a gigawatt. This is a huge problem. What compounds the problem is it’s a race to get more AI out there. They don’t even have time to plan, get the power. It’s a real huge challenge. Customized silicon or custom silicon enables them to get there faster.
Patrick Moorhead: Yeah. As an industry analyst, I think I was on the road 42 weeks this year. I go to all the hyperscaler shows. I also go to industry shows like this. It seems like a lot of the talk is about custom CPUs and XPUs, but it doesn’t stop there. What are the chips that are being customized today inside of the hyperscalers?
Nigel Alvares: It’s a great question because I think if you go back in time, cloud computing started maybe in the early 2000s. AWS was what, 2006 they started?
Patrick Moorhead: Yeah.
Nigel Alvares: They started just putting a data center off-prem and hosting. But then, they quickly realized to create more differentiation and not become a utility, they had to start customizing silicon or developing custom silicon. They acquired a company called Annapurna that started doing custom NICs. You’re starting to see that happen across the spectrum.
Patrick Moorhead: Right.
Nigel Alvares: All the hyperscalers are doing this. The NICs, the DPUs, the HSMs. They’re looking to get the most out of their platforms to differentiate and lower their TCO. It’s just continuing. Marvell, we see it continuing to where you see multiple of these pieces, the networking, the compute, the storage, starting to re-partition and remodel. Where you have networking going into compute devices, and vice versa with storage into compute elements. I think we’re just in the early, early innings of this. But to your point, it’s happened in the past and it’ll continue to accelerate as the prize gets bigger and bigger. This is a huge prize, as you know. This is like the internet in 2000, this is AI 2024. In 10 years, we’ll see where the winners are. This is how the game’s going to be played, using custom silicon to win that war.
Patrick Moorhead: Yeah. FOMO is a great driver and we’re seeing this today. What I’m fascinated with is what the architecture of the data centers are going to look like in five years. Today, they look a bit similar. I think you might see some bifurcation in the way that these hyperscalers lay them out in the future. Talk hyperscalers. Can you talk, name the hyperscalers that are doing things? I may know a few of them.
Nigel Alvares: Yeah. What I can say is what’s in the public domain.
Patrick Moorhead: Yeah.
Nigel Alvares: These companies have been pretty vocal about their custom silicon because they realize it is going to be a differentiator for them. I mentioned Annapurna with Amazon. They’ve been doing custom silicon from the networking side and on. Google has been doing TPUs.
Patrick Moorhead: That’s right.
Nigel Alvares: We announced with Meta a NIC device, the FBNIC at OCP this past year. Microsoft has been public about all the products they’ve been doing at their Ignite shows.
Patrick Moorhead: That’s right.
Nigel Alvares: Be it custom AI chips, be it HSMs. You are seeing every major hyperscaler have a big initiative in this. Again, it’s to make sure they can differentiate and optimize their TCO and not just become a commodity. I think that’s the biggest fear out there. Especially in this AI era, where today there is one huge leader driving the market. Which is great for the market, but longer term, there’s concerns for these companies on, “How do I compete with all these other folks?”
Patrick Moorhead: Obviously, you’re doing this. Who else is in this custom game?
Nigel Alvares: I think, given how complex these AI chips are, you’re talking about three-and-a-half-D, three-D types of solutions where you’re packaging multiple memory devices, chiplets, stacking technology. There’s only really two major companies in silicon who do this in a custom manner. That’s Broadcom and Marvell. We’re the two companies who have the IP portfolio, the experience, and the know-how to really execute this for custom silicon.
Patrick Moorhead: Nigel, what are the elements that … It’s a two-horse race, we’ve established this right now. What are the conditions for having the best partner? What does it include? What are the building blocks that are important to have? You talked about three-and-a-half-D and the packaging. But what else matters?
Nigel Alvares: Yeah. I think it’s a multi-faceted question with two sides. There’s the technology side, and then there’s the business model, partnership model.
Patrick Moorhead: Yeah.
Nigel Alvares: Let’s start first with the technology. Foundational technology for data centers, you’re operating at high, high bandwidth, so SerDes technology. SerDes technologies is the technology that connects all these chips together, and you want to serialize it. That’s what SerDes stands for, serializer/deserializer. You need to get to the highest speed to move the bits as quickly as possible in the quickest manner of time. SerDes technology is a foundational block for this AI era. Or what we like to call it at Marvell, accelerated infrastructure for artificial intelligence. AI for AI. For AI for AI, some of the key technologies outside of SerDes are the other key IP blocks around process node. Process node is the shrinking of semiconductor process.
Patrick Moorhead: Yeah.
Nigel Alvares: You want to be on the bleeding-edge for a couple of reasons. But the main ones is lower power and higher density. You need to have all this IP on those process nodes. You need someone whose investing in those process nodes ahead of the time to accelerate time to market to get more computer power at a lower power footprint. I would summarize with CMOS technology process node, IP with SerDes IP being the most critical. But there’s die-to-die technologies, where you want to connect all these chiplets together. That really pivots into this advanced packaging that we talked a little bit earlier. Where you have to really be able to stack horizontally and vertically. This is getting more and more challenging. That’s on the technology side. On the business side, it’s quite interesting because as we talked about, the cloud operators have different pieces of silicon.
Patrick Moorhead: Right.
Nigel Alvares: Some are hugely complex, some are a little bit more simple. Now they want partners that are going to satisfy their whole spectrum of needs. Not, “Oh, I only want the most juicy socket.” They want someone that they could trust and is going to be there for all their different needs. That’s on the business model of having the flexibility to take on even lower volume, not as complex devices. As well as the ability to do multiple different pieces, because some of these cloud operators have huge design teams that are designing specific IP that’s unique to that cloud operator. But sometimes, they don’t have the people to do that so they have a spec. They say, “Hey, do you have front end designers that can design the code RTL for that?” And then package it all together. Having the ability to do the spectrum and not saying, “Oh, I only do back end,” or, “I only do front end.” But being able to provide and offer the spectrum as a true partner.
Patrick Moorhead: Yeah. Does that also apply to, I hear a lot of terms, scale up and scale out? The scale out side, it could be multiple kilometers between data centers, and obviously clusters inside of those data centers. It can be on a campus. Or it can be just inside of your data center. Does that variability matter in this custom business?
Nigel Alvares: It absolutely does because what we’re seeing is these technologies starting to mesh. Networking, and computer, and storage all starting to gel together. When you look at scale up, that’s about building the biggest supercomputer you can do. Where you share all the memory resources, all the compute resources using a scale up fabric.
Patrick Moorhead: Right.
Nigel Alvares: That doesn’t go typically far distances. You’re really restricted today, mainly a rack, maybe moving to multi-racks now. Probably limited inside a data center over time. When you move to these more multi-rack architectures, you’ll need interconnect technology that can actually go further, which is typically optical technology. We see the need for CPO type of technology for that scale up architecture. While scale out is where you’re connecting all these supercomputers together to finish a training model in the most sufficient time. In those cases, as you pointed out, you might have one big football field data center that can do it.
But as we just talked about, power is becoming a bigger issue. What they’re starting to do is use multi-site data centers. The easiest model is to do a multi-campus. Within a region, you’ll have small data centers that are intertwined and connected together. You would need another optical which we call coherent light technology for that. Then the other one is across a metro area, which is 80 kilometers to 200 kilometers. Where Marvell is a pioneer with our COLORZ technology, which is our DCI product line. Where you’re able to cluster multiple of these regional data centers together. Every cloud is unique, meaning they will have different mixing of data centers, architectures, and that drives more customization.
Patrick Moorhead: Right.
Nigel Alvares: Because they are unique. That’s how these all come together. That’s why I mentioned that Marvell with Broadcom are the only ones who have all these pieces of technology to build a custom infrastructure.
Patrick Moorhead: I appreciate that. You have your merchant business, dare I call it the off the shelf. There’s nothing off the shelf, but it kind of is, particularly when you buttress it next to custom. How are you keeping pace to be able to do all these designs? Is it AI in EDA, is that one way you’re keeping up? I talked to Chris Koopmans today, and he talked about parallelizing teams. And also, that chiplets help. You can parallelize.
Nigel Alvares: That’s right. There’s a lot of things we’re doing to do more of these chips in parallel. There’s technologies like EDA in the cloud. We announced a partnership, a multi-generational agreement with AWS where we’re doing EDA in the cloud with them. That enable us really to have the compute resources and storage resources to do multiple products in parallel and deal with the spikes they go through when you’re developing a product. That’s the EDA in the cloud. We’re using AI technologies to accelerate development in many facets. Verifying chips, enabling the layouts of chips. That’s a spectrum of things that AI is going to help us complete things in a more efficient manner and increase productivity. Then the other side is having the scale. I think when you go back to this merchant or off the shelf versus custom, they actually feed off each other. You do a merchant product, that IP gets used in the custom and vice versa. You really want to have this yin and yang type of approach, where you develop this product. Then people are like, “Oh, that’s cool IP. Can you put it in my custom chip?” Maybe you’re doing something custom and they’re like, “Maybe we should put that in our merchant or off the shelf product.” It actually feeds of each other to build the scale and get us more opportunities going forward, so it’s really exciting.
Patrick Moorhead: This has been a great conversation, Nigel. I love the custom conversation. I really appreciate your time. Hopefully we can do this again, to get an update of where you are in six months to a year.
Nigel Alvares: Sounds good, Patrick. Thank you for the time.
Patrick Moorhead: Appreciate that. You heard it here at Marvell Technology Headquarters. We are talking custom networking, custom XPUs, HSM, DSPs, you name it. Everything is going custom with the hyperscalers. An exciting time to be here. Check out all of our interviews in and around Marvel Technology, and hit that subscribe button. Have a good one.
Author Information
Six Five Media is a joint venture of two top-ranked analyst firms, The Futurum Group and Moor Insights & Strategy. Six Five provides high-quality, insightful, and credible analyses of the tech landscape in video format. Our team of analysts sit with the world’s most respected leaders and professionals to discuss all things technology with a focus on digital transformation and innovation.
