Disclaimer: The Six Five webcast is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded, and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors, and we ask that you do not treat us as such.

Transcript:

Patrick Moorhead: The Six Five is on the road at AMD’s Advancing AI event. It’s been an amazing day. I mean, we have seen AI, big picture, we’ve seen AI in the data center, the client computer, and everything that’s in between. Daniel Newman, it’s been a great day.

Daniel Newman: Yeah, it’s been one of those days That’s really for the memory. We knew 2023 was going to be a big inflection. We saw it coming into last year. The next year is probably going to get even more exciting, Patrick Moorhead, but we’ve also seen a year where the market’s been really looking for who are the players that are going to step up and contribute, and really enable AI at scale. And this was an event that I think you, me, and a lot of our peers, vendors in the industry had marked on their calendar, and it didn’t disappoint.

Patrick Moorhead: Absolutely. And one thing that we always need to recognize is the deep technology that is required to pull this off. And some of these bets needed to be made a decade ago for AMD to bring out what it brought out today. So, I think two of the best people to talk about this is Victor and Mark. Welcome to The Six Five. Thanks for coming on the show.

Victor Peng: Thanks for having us.

Mark Papermaster: Thank you.

Daniel Newman: Yeah, I think you’re both first time, but hopefully this is the first of many, and there will be a lot more announcements. I have the feeling based on the proliferation of AI, there’s going to be more of these events, more announcements, more innovation, and I expect AMD will be bringing a lot of that. So, maybe we start there, Mark. We’re at this inflection. It’s all happened really, really quickly. The crowd at the event today, there was a lot of energy. You could hear the excitement, there was some great data up on the screens, comparative, competitive. But you’re also talking to customers, you’re having those conversations in the background. How are the customers feeling about what you’ve launched, what AMD is bringing to market and their ability to partner and get where they need to with AI, working alongside you at AMD?

Mark Papermaster: Well, Daniel, it is a great day, and Victor and I are so excited along with the rest of Team AMD because it has been a huge company effort to get today. And it did start, Pat, as you said, with a decade of innovation, starting with our whole design approaches and gearing up for chiplet, and being able to build this type of incredible compute capability that AI is so hungry to be able to consume. And that is exactly what I’m hearing from customers. When I talk to CTOs and CIOs across the industry, what they’re saying is, “My God, I am not losing any of the workloads I’ve already been running, and I’m having to add AI on top of it because I’ve got all this data and now I need to run analytics. I need to run gen AI. I need to put that data to use.”

And how do we do it in a more economic way? AMD, hurry up and bring competition so that we can move more quickly to address the needs that we have. And it’s really a consistent story that we hear across enterprise. And of course, data centers such a massive consumption because the huge LLMs for generative AI, it’s a massive economic issue if there’s not competition. So, huge milestone for us to bring competition to the marketplace with MI300.

Victor Peng: Yeah, And I think if you look at it from an innovation perspective, that’s the other thing, which is AI is just changing super rapidly. It is, believe it or not, still in its early stages. And so, it’s going to continue to evolve in addition to proliferate. And that also means not only do you need multiple suppliers that are viable, but you really want multiple sources of innovation. And I think that’s really critical, and I think that’s what we pride ourselves on in innovating at multiple levels. Technology level, architectural level, and software level.

Patrick Moorhead: Yeah. Software is super important, and I know software is eating the world. I’m like, “No, software can’t run on error. It has to have amazing hardware.” But the reality is, it has to have both. And at certain times in history, we needed the software to hurry up and support the hardware, and then vice versa. And software, as it relates to AI, is about as important as it gets. And I’ve seen AMD field some very competitive hardware in AI, and I sometimes get the, “Well, hey, what about the software?” You had some big software developers. You had CSPs who write a lot of software, you have hyperscalers like Meta who write a lot of software themselves. I’m curious, what is it that you’ve done? What changed? Because I didn’t hear the… I heard a lot of your partners talking about how they support ROCm, ROCm 6, and what you’ve done.

Victor Peng Peng: Yeah, look, I think we not only strengthened our development teams and got them really focused on AI, right? We did a re-org early in the year where we put all the software resources all into one place, and got them super focused on AI. But we also did some acquisitions as I shared. Mipsology and Nod AI, and we’re working with the ecosystem. So I think it’s, each one of those elements is really important, just from organic growth that just strengthening, bringing in more AI practitioners, doing some acquisitions of folks who really were contributing both to key technologies like MLIR, but also have been maintainers in the ecosystem and working with the community. And then, these standards bodies and these standard frameworks, they see AMD is going to be attracting a lot of developers. So, they’re also on board with supporting us as foundational partners. So I think each of those elements are really important. And by the way, again, it gets back to innovation too, right? We have deep engagement to the customers. They’re telling us the problems they’re solving, whether it’s their first party tools or whether they’re getting ready to stand up infrastructure, public instances. We have to be prepared to support all of that, and that really has pushed us to innovate on the software side.

Patrick Moorhead: As a quick follow up on that, what do you say to an enterprise software provider, a SaaS company that has written a lot of software to CUDA? Is that no longer the conversation, no longer a potential barrier?

Victor Peng: Yeah, the good thing about this is that, not just within SaaS, but in general, the industry is moving away from optimizing at a very low level. Why are they doing that? It’s a combination of that this is moving so fast. If you do that, and then it’s in innovation, you have to redo all of that before you could actually move on. The other thing is that things like compiler technology, MLIR, is getting much more powerful. So, the significance, not everybody’s familiar with OpenAI Triton, but the significance of that is that people can, it takes the backend from the frameworks like Python, PyTorch, and then it’ll still compile it and you still get excellent performance, right? So, everybody’s going to move to that because this is about productivity, right? And even the folks that still, the ninjas that still want to get that last bit, they’re being really careful about where they’re going to do it. They don’t want to do a lot of it, right? So I think I would say is that that’s just a general trend that we’re just getting behind. So, it’s not even about just a competition, it’s just that that’s the way the industry wants to go and we’re aligning ourselves what the customers and the partners want.

Mark Papermaster: And the thing I’ll add is if our customer did have legacy code that was written at a very low level of CUDA, we are a GPU. And that’s what, the fact is, it’s semantics. The language is actually quite alike, and so it’s a very straightforward process.

Victor Peng: Yeah, and we, actually we worked at that too. We worked at making sure that the libraries, we had some kind of equivalent libraries like Brickell and Nickel, and things like that, just to make it a low-friction thing. But the reality is that that’s even just what we need in the moment, but the trend is people are going to want to work at as a high level of abstraction as they can.

Daniel Newman: Yeah. It feels like that’s been a lot of the buzz. And Mark, I appreciate you kind of mentioning, because I think that compiling and moving has been the thing that’s been a hold up for a lot of companies. Of course, not having an option like an MI300X, it’s been another substantial hold up. But between having the hardware and then having the ability to compile and move and then going kind of forward, because that’s the thing. I think the world’s kind of acting like AI has done, and it’s one of the weirdest things the phenomenons of this year though was like, the market’s set, this is the winner. It’s like, are you kidding? If this is a baseball game, we’re in the first inning. If this is football, we’re in the first court. I can go on and on. Sports analogies are great for videos, by the way. But one of the things that I definitely did want to kind of touch on too is, you’re talking about software and you’re really spent a lot of time here talking about ecosystem. You had a lot of partners on stage. Just yesterday, while this wasn’t your event, there was a big alliance announced that you’re taking a substantial part in with, I think it was IBM? Meta were leading the way, but AMD had a big role to play. Talk a little bit about the ecosystem and the importance of it, and beyond just even the partners that are building just with AMD, but the ecosystem at large that needs to come together to democratize processing power, to enable collaboration software innovation. This is really big, Mark.

Mark Papermaster: Well, we’re very happy to be founding members with the AI Alliance, because when you think about what that AI Alliance is trying to do, it’s exactly to create choice and to create an ecosystem, a set of open source software solutions, models that you can build on and fine tune for your various business needs. Well, guess what? That’s exactly what we’re about. That’s what our announcement was today with MI300. It was about an open source software based on ROCm, as well as all of the open frameworks, as Victor mentioned. And it is about bringing choice with the hardware that we have with MI300. People had said that there was a moat that our competitor had, and I think we’ve shown that we’ve built a bridge right over that moat today.

Patrick Moorhead: No, it’s great. No, by the way, that’s a great analogy. I like that. And the market wants that. I mean, Daniel and I both advise almost all of your end customers, and they want choice. And Dan and I are very supportive of-

Daniel Newman: Peers, competitors, I think we talked about.

Patrick Moorhead: … Exactly. Yeah. Let’s talk a little bit about the future. It’s interesting, the ground source paper for foundational models and generative AI, I think it was written maybe three or four years ago, right? And then, here we are today. I mean, a year ago almost to the day, as Daniel said, ChatGPT kind of opened, right? And I ignored it. Didn’t really think it was anything at the time until you started seeing what it could do. So the question I have for you both is, how are you preparing for the future? Is there a seminal research paper that’s been written now, and research that’s been done and your research groups are involved in that show us the future? What does the future look like, I don’t want to say after LLMs, but yeah. After LLMs, what’s next? And maybe we’ll start with you, Mark.

Mark Papermaster: Well, I think it’s so early. I mean, one of the reasons that you see about all of our solutions is they’re highly programmable. The GPU is imminently reconfigurable and programmable. The XDNA AI engine that Victor brought into the company when Xilinx joined AMD is imminently programmable and imminently energy efficient, and has all the heritage of adaptable computing, right, with its FPGA heritage. Well, that’s fundamental because there is new approaches every day. We talked at our release today about flash attention and other accelerations that are going into the model builds, and these are all new within the last months, and that’s the rate and pace of innovation. And so, I don’t see any end in sight in terms of how you’re going to see algorithms changing. And it is a holistic design. If the algorithms change, you’re going to have to have programmable engines that move more quickly. But then, you’re going to also see underlying hardware where some of the accelerations stick, you’ll see us optimizing hardware.

Victor Peng: Yeah, I think, look, I don’t have a crystal ball either and I think, but what I do believe is going to happen is it’s going to be innovation across the whole breadth of things, right? What gets intention is the people developing the really, really large foundational models, and we’re working with folks that are working with them. But there’s lots of innovation happening in more moderate size, like 100, 200 billion parameters. And even on device, right? Tens of-

Daniel Newman: Just a hundred billion.

Patrick Moorhead: Exactly.

Victor Peng: … Yeah.

Daniel Newman: It’s tiny.

Victor Peng: Yeah, well, that’s technology, right?

Daniel Newman: We trained that in years.

Victor Peng: Yeah.

Daniel Newman: I’m just joking.

Victor Peng: So… But I do think, the other interesting thing to add to this is, now we have AI to help with the AI, and I think as many people know, it’s not just the infrastructure, it’s the data, and people in generating data through AI. I mean… So, this gets back to the level of innovation that’s going on is just phenomenal, and I don’t think that’s cornered in any one place. And I do think what makes AMD’s position unique to capture that is because we are in clients.

Patrick Moorhead: Right.

Victor Peng: We are in GPUs and infrastructure. We have two different architectures, a spatial data flow architecture as well as a mainstream, really good leading edge GPU architecture. And stay tuned, we’ll definitely be doing things in servers, right? And we’re in all kinds of use cases. I think that’s the other thing that doesn’t always get appreciation. We talk about the breadth of our product portfolio. We have an incredibly broad market portfolio, right? We’re in communications infrastructure, we’re in autonomous vehicles, we’re in healthcare, we’re in factory automation. And traditional data centers, client gaming, things like that. There’s very few companies on the planet that can really say that they have both, right? The platform architectures and the applications and use cases, and we work with the leaders in every one of those markets. So, I don’t know what the answer is, but I bet one of my customers in one of those markets has some of those answers, and it’s just going to be exciting. It’s a great time to be in technology, right?

Patrick Moorhead: Sure is.

Daniel Newman: Yeah, Victor, I’m going to ask you both. I’ll start with you the question that I’m being asked the most though, and hopefully you can give me the most you’ll be willing to give me. But is… I’m being hammered by press, media, customers, partners, the people that you supply, to kind of get a better understanding of the competition and cooperation that’s going on in the marketplace. So obviously, you’re innovating very fast, video’s innovating very fast, and you’ve got AWS, Microsoft launching a chip now. Google I think made the day for us. I’m joking. They announced, right, at the same time as you announced. I mean, it’s so frequent now though. But I think everyone’s kind of wondering, what’s going on there? Because you’re bringing those same hyperscalers up on stage and partnering with them. Kind of, what’s the landscape for that kind of cooperation and competition right now? And how does AMD position itself to be a great asset to those partners?

Victor Peng: Yeah, that’s great. And actually, the last place you let off, maybe I’ll lead off on, which is, look, at the end of the day, we’ve been in many different environments, and back when I was in Xilinx as well was, we supply to a key customer and they also have a group within them that competes. That’s not really you. But I think it’s how it comes down to, are you delivering value? Are you delivering enough value that it really makes sense? Even if they have some capability, are they’re going to work with you. We talk about this a lot. It’s really in our DNA, because we do think that we don’t necessarily have all the best answers to everything. However, we are confident about what we do. We really are world-class at what we do, and we have to keep showing that and delivering that to our customers. So, that’s thing one. I think the other thing is that the pie is large, not only from a market size perspective. We’re talking about every time you look at it, the TAM goes up, right? But also, back to the innovation, right? This is unlike, honestly, I think this is unlike anything I’ve seen in my career. I’ve been around for a while because I’m an old dude, right? But I mean, seriously, when do you have a new application come out and you’re fundamentally changing data types.

When social came out, you didn’t have to change data types. And that’s just quantization. We haven’t really tapped into sparsity, and there’s just so much innovation. So, I think the other thing that diversity tells you is that not only is it a big market, but there’s not one size fits all, right answer canonically across everything. There’s innovation on the model side, there’s innovation on algorithms, there’s innovation on the software stack, and they all kind of interact. Innovation on the architectures too. So, I think that’s going to go on for a while. So the key thing is it’s an and function, it’s not an or function. Now, 10 years from now, will there’ll be like 15 different things? Probably not. But there’s not going to be two. I honestly don’t think there’ll be two. Even for us, right, we have to have a different architecture for a client-based system versus what we’re going to put in a massive supercomputer for training the foundational models. You don’t have an architecture that could span multiple orders of magnitude and it tops per watt.

Mark Papermaster: Yeah. Daniel, let me add just one element of secret sauce we have that makes me very confident for the innovation that we’ll have going forward, and that is the culture at AMD. We have a culture of collaboration. We really listen to our customers. We’re hearing what is their problem set? And then, exactly, we innovate around that. And it’s very special. We collaborate externally, we collaborate internally. And that’s why I’m very, very confident that as algorithms change, as the use cases across the broad industries, as we understand the problems, we’re poised to listen, innovate, and solve.

Patrick Moorhead: Appreciate those comments, and we start to wrap this awesome conversation. I wanted to ask both of you on this rate of change and this acceleration. Daniel, you alluded to it in the opening. I mean, new technology is going new technology, but as the two of you are basically the shepherds of not only the building blocks, and you have been, but also, how do you do that in a way that’s economically viable? And if we’re shrinking time to market, if the expectations are higher, how are you getting more efficient? Are you using AI and design and test and validation and things like that to increase the speed? What’s the strategy to do things even quicker?

Mark Papermaster: Let me start, and I’m sure Victor’s got comments as well. But I will say, fundamentally, part of when you look at the innovative approach we took at AMD, we went with a modular approach, and that enabled us to be first in the industry to really deploy chiplets. And you don’t have to look further than our MI300 announcement today. We started with the design that would supply the world’s largest exascale computer coming up with Lawrence Livermore, that’s the MI300A. But very rapidly, we were able to pivot that, using chiplets, optimize it for generative AI training and inference, and that’s the MI300X. So, that speeds the development cycle. And then beyond that, definitely we are applying AI. We have over 100 internal AI projects on all key aspects of chip design, from a physical design, our verification, our test cycles, our supply chain, and even our non-engineering applications.

Victor Peng: Yeah, and I think the other thing too is that, look, you just have to look at long lead time things .you open up with saying some decisions we had to work on a decade ago. So, we’re still working on deep technology things like really advanced packaging, really advanced integration technologies, more than two levels of stacking. We’re working with TSMC and our entire supply chain and pushing the state of the art, and getting it ready for a high volume production at the right time. This is all about, one, is that technology ready to go? Not too early, a lot of people crash and burn on that, and you certainly don’t want to be too late. So I think, at every level of technology, and one more thing, it keeps coming back to software and like you said, because these are complex systems. Systems are hardware and software, right? So, the thing about the software is, it’s got to hide that complexity, right, but let you take advantage of it, right? And that’s actually a hard thing to do. But that’s exactly because if you hide the complexity, people could move fast and you could be asynchronous with how you’re lifting up the substrate and just giving a much more powerful engine. Back to your point, this software doesn’t run on air, but at the same time, you don’t want to have such tight coupling and that’s why you don’t want to be doing assembly code or whatever, right?

Daniel Newman: Sure.

Victor Peng: Because you just can’t move fast. It seems like you can get stuff. So, I think it’s the whole stack of the software solution, the microarchitecture, partitioning chiplets, and all the way down to base technology. And we really could do that, and really, you can tell we’re excited about that because we can keep talking about it.

Daniel Newman: Yep. And I listened to Victor, and I think to myself, it’s the old adage of the simpler it is to use, the harder it is to build. Mark and Victor, I want to thank you both so much for joining us here on The Six Five today.

Victor Peng: Thanks for having us.

Mark Papermaster: Thank you.

Victor Peng: This was great. Yeah.

Daniel Newman: All right, everybody. Hit that subscribe button, tune into all the coverage from Patrick and myself here at AMD’s Advancing AI event in San Jose, California. It’s been a heck of a day, Pat.

Patrick Moorhead: Been great.

Daniel Newman: Been great to have these conversations. We appreciate you tuning in. Stay with us, but come back later.