What strategic imperatives aid in the future of AI deployments from the data center to the edge?
At AMD Advancing AI 2025, hosts Daniel Newman and Patrick Moorhead are joined by Mark Papermaster, Executive Vice President, Chief Technology Officer at AMD for a conversation on AMD’s strategic direction in AI, including a look at their business strategies, customer and partner progress, and how AMD is evolving its product portfolio to meet the growing needs of AI deployments from the data center to the edge.
Key takeaways include:
🔹CTO Insights from AMD AI: Gain exclusive takeaways directly from AMD’s CTO on the recent AMD AI event, spotlighting the significant milestones that underscore tangible progress towards 2025.
🔹Navigating the “Shift to Inference”: Explore AMD’s acute observation of the market’s fundamental “shift to inference” and how its robust portfolio is strategically positioned to cater to the widespread adoption of AI across diverse sectors.
🔹Hyperscale Adoption and Strategic Evolution: Understand the profound implications of broad hyperscale adoption of AMD’s solutions, the invaluable learnings gleaned from these critical engagements, and how these insights are actively shaping AMD’s strategy to target an expanded market.
🔹Beyond Silicon – A Holistic Approach: Discover AMD’s strategic commitment to delivering comprehensive solutions that extend far beyond core silicon, specifically targeting customers who are meticulously building complex AI pipelines or engaging in sophisticated multi-cloud deployments.
🔹End-to-End AI Execution: Witness a comprehensive portrayal of AMD’s current AI strategy, demonstrating its seamless execution from the powerful data center to the agile edge, unequivocally underscoring the company’s holistic solutions-based approach to AI.
Learn more at AMD.
Watch the video below at Six Five Media, and be sure to subscribe to our YouTube channel, so you never miss an episode.
Or listen to the audio here:
Disclaimer: Six Five On The Road is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded, and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors, and we ask that you do not treat us as such.
Transcript:
Patrick Moorhead: The Six Five is On The Road here in San Jose, California. We’re at AMD’s Advancing AI event. Daniel, the walk ons from XAI and Sam Altman coming on from OpenAI were incredible. We heard people talking about training where last year was pretty much limited to inference. So what’s going on here?
Daniel Newman: Yeah, it was a big, big one. And by the way, I got this great photo of Lisa and Sam hugging on stage. I tweeted it, or X did or whatever we call it these days. One of my big banger posts. I got like 500-600 people have already engaged with this thing and it seems that everybody was really excited. I said something along the lines of really feels like an inflection in this moment right here. And it was like this really nice embrace.
Patrick Moorhead: Yeah. And behind all of these incredible innovations is our bets that had to be made years ago, five, six years in advance. And one guy that has been consistent in this with his team is AMD CTO Mark Papermaster. Mark, welcome back to the show.
Mark Papermaster: Thanks very much, Pat. Great to be here with you and Daniel for sure.
Daniel Newman: Yeah, it’s great to see you. I think last time I heard from you, you were doing a little fireside chat over at Oracle in Austin.
Mark Papermaster: That’s right, that’s right.
Daniel Newman: It was great listening to you. I was hearing some of the things you were saying that some of it kind of came into fruition today. You’re talking about this inference inflection, talking about open source and talking a little bit about, you know, building software. You were very self aware about some of the opportunities, challenges that existed around that. And it seems based on the comments and less from what AMD people were saying and more from what the customers that Pat, as you mentioned, were parading out there, they’re really seeming to make the progress, seeing the progress, making bigger commitments, not just going in on AMD for the inference side, but really starting to do it for more training. And there seems to be a lot of TAM expansion and opportunities. But let’s start off by just getting your CTO perspective in your role, you’re trying to do a lot of things for the company. What are the sort of big takeaways today and what’s the progress you’re kind of most proud of at this event?
Mark Papermaster: Yeah, Daniel, great question. I mean, it absolutely is an inflection point in the industry and it’s a real inflection point for AMD. And why do I say that and why is today sort of a showcase of that inflection it goes back to actually what Pat was saying earlier. So think about what discussion before is AMD real? Okay. They’re talking about inference. And of course we did have partners already that were running production inference, but it was the big hyperscalers. It’s been the partners that have been partnering with us to ensure that we knew what the requirements were, we knew what the kind of productivity gains that they needed and frankly handling their workloads at a production scale. And so today marked a change where it’s evidence of what we’ve been doing to expand beyond that handful of customers who were really our early adopters. And so that’s what we’ve done today when we can show that we have these additional partners who are running not only inferencing but starting to expand with us in training. And the other inflection point is really sharing more of the next generation of our roadmap. The takeaway you have is that we are pulled in our Mi350 by three months. We’re starting production shipments right away in Q3. We already have sampling with early customers and it’s designed for much better inferencing so you can take advantage of new approximation modes. You can get up to actually above a 35x throughput advantage of inferencing, but now really designed for mid scale training. Then we unveiled our plans beyond that, not the details yet, but where we’re going on our Mi 400 series which will be the following year and that is truly designed for up to actually large scale LLM massive clusters for training and inferencing. So very, very exciting on where our customers have progressed in adopting our roadmap. And the other key takeaway is software. So last year again, I had lots of questions. AMD, are you real? I see you’ve got these few hyperscales up and running in production. What about the rest of the world? And so it was great to have Vamsi go through and really share what we have done to support the community. What are we doing to support developers? And so several very, very exciting announcements in that domain. And then a third and final takeaway that I want to share is an incredibly clear message about our commitment to an open ecosystem and an open software stack. Look, history has just shown that competition is needed. People don’t like to be locked in with one provided from their IT infrastructure solutions and the way to provide that competition is through partnering as an ecosystem and with open source.
Patrick Moorhead: Yeah. So last year at your Advancing AI event, you made a ton of progress and the one that really stood out for me was what you did with ROCM and over a series of years, really turning that from a high performance computing solution into an AI solution. That is not easy. And what you did with rock M7, looking at the numbers is impressive. And we talked to Anush and just had a great conversation with him. He’s doing new code uploads every day. That’s Q8 every week. That is very, very different. It’s a very different AMD.
Mark Papermaster: Containers going out every other week to the world with these updates.
Patrick Moorhead: Yeah, it’s impressive. I want to talk. It’s funny we were congratulating you, your customer, talking more about training, but it’s clear like we saw seven or eight years ago, the shift in where the action is moving from training to inference. Inference is really a significant area because it shows that people are actually using AI as opposed to running, doing training runs on it. Can you talk about how you’re positioning the portfolio today as you’re getting wider deployment here? I mean we all saw the giant numbers on inference and the improvement, but maybe if you can just expand on this a little bit.
Daniel Newman: 80%.
Mark Papermaster: What’s that?
Daniel Newman: 80% growth?
Mark Papermaster: That’s right, 80% growth. It’s huge. So inferencing is going to go everywhere. So when you think about inferencing to date it’s been inferencing on generally very large language models. So it’s the LMS underneath the commonly used generative AI applications. And so that’s what’s driven and will continue to drive our high end GPU roadmap. So there we’re really excited about this growth because we designed to excel at inference. We designed our techniques using both lateral 2 1/2 deep chip connectivity and the only one in high volume production with 3D chip development. That gives us advantages, that gives us advantages to be able to stack more memory. And guess what that means you can have bigger context windows when you’re running inferencing. And so that will continue to leverage our roadmap going forward. But the key bet that we made that’s now bearing fruit is that we enabled inferencing across the entire rest of our portfolio and we have the broadest portfolio in the industry. So let’s not forget about CPUs. CPUs of course are getting more demand now just because as you’re running more GPU inferencing, you’re using CPUs to help you process those large context windows, you’re getting agentic AI sending more workload there. But moreover, CPUs have their own vector engine and you can actually very effectively run AI. And we’ve improved our AI and our CPUs every cycle. So enterprise is running small language models which are now starting to create more bespoke tasks that they’re trying to accelerate and they’re fine tuning small language models. Many of them run on the infrastructure they already have.
Patrick Moorhead: Sure.
Mark Papermaster: And if you look at our 5th gen Epyc, I mean it does very very well. And then across the board with our AI PCs, our embedded and adaptive compute, we’ve enabled inference acceleration. So we think we’re very well positioned for what we’re now at just the cusp of a massive inflection.
Daniel Newman: Yeah, I like to use the analogy that the pre-training era was kind of like the R and D of the industry and now we’re in the monetization and inference is all about the monetization where people are going to make money from AI.
Mark Papermaster: I think that’s right. And then driving tremendous productivity improvements.
Daniel Newman: Absolutely. And I like to say prune to grow Mark. We’re seeing a lot of efficiencies first and then we’re going to I think start to see that up and to the right in terms of productivity. Hyperscalers are all over the stage. Clearly very committed. But one of the things that’s interesting is and this has been something that’s been pretty consistent with AMD, you’ve done really well with hyperscalers, you’ve done really well on the CPU. You’re starting to do very well with instinct in that particular space. Some of the market might challenge and say do better with the enterprises for instance. And we heard a lot of indications here that you’ve got a path to the enterprise. You’ve had Red Hat on stage talking a little bit about that BLLM. Some of the things like talk a little bit about what you’ve learned in terms of gaining that market share with the hyperscalers and how you might be able to take this on to this next more commercial enterprise AI opportunity.
Mark Papermaster: It’s a great question and Daniel, we had to go in the order we did of getting the hyperscalers on board first or frankly we wouldn’t have had the credibility with enterprise. And enterprises even today are still running a lot of their training on clouds. And so now you can get at AMD on the cloud and kick the tires and run your proof of concept and get that confidence with our RocM software stack very, very easily at the cloud. But you’re absolutely right, as you start deploying an enterprise often it’s not going to be like a greenfield new data center like you see the hyperscales able to adopt and so they can accommodate more power because you get more AI efficiency. If you have legacy data centers, then you’re going to actually have power constraints per square foot of your data center. And so we’re very focused on that going forward. But we’re going in the right order. We’re going in the right order, as you heard today, of first getting the entire developer ecosystem out there, first ensuring that we have the ISVs on board. And the analogy I’ll make to you is look at our EPYC CPU ramp where we did the same thing. We started out really working with enterprise, understanding the applications that they were running, getting them certified and then we took the enterprise ramp. We’re actually accelerating that play we ran on our server market and we’re full bore focused on that right now, on our GPU products.
Patrick Moorhead: So Mark, we saw RackScale Solutions delivered CPU, GPU networking, scale up, scale out networking. Pretty exciting. What are you thinking about this and why are you doing this? Why are you building these?
Mark Papermaster: Yeah, Pat, it’s a necessity right now. When you look at what you need. If you’re committed to bringing competition as we are, the competition of where the innovation is going on and the edge of AI is with the largest of LLMs, it’s in that race for AGI and we’re committed that there’s competition in that race. It’s expensive, yes. But what we get out of it is we have a seat at the table to understand where the newest algorithms are going. And so to compete there, we made the investment in its rack scale design. And so that’s what drove the acquisition of ZT Systems. The Helios design that we shared today, it’s a marvel. I grew up at IBM, I did how many massive computers at IBM and I feel like I’m actually back to the future and looking at because that’s the way we’re designing like I did early in my career where we started the system and we figured out how to optimize to be able to get big compute done. But at AMD we’re doing it in a modular fashion.So when we develop that technology we can then take it down market and be able to have that absolute optimization for the cutting edge AI, but bring it into different form factors and lower power.
Daniel Newman: I know that the analogy of mainframe may or may not be cool, but in some ways you sort of want to build the mainframe of the AI era. It kind of is that because I
Mark Papermaster: Like scale up.
Daniel Newman: Well, yeah. And think like 60 years of being kind of run, every transaction of importance in the world still runs on those things. And so, you know, building that kind of system that becomes the epicenter of the world and you got to really optimize that thing to be, you know, reliable, scalable, secure, all the things that. Have to be done. And of course have software, a common language that everybody that’s involved can use. Yeah, let’s kind of close this off, Mark, really quickly and just say in the kind of a quick summary of how you characterize AMD’s AI strategy from data center to edge at this time?
Mark Papermaster: So one of the things that I would really highlight to you when you think about our AI strategy is that although all the news now is on the massive compute of GPUs and the huge training and inference that’s associated with it, we take the long view of our AI strategy as you fast forward and people running AI on most any application of their work in their daily lives. That’s how we’ve targeted our strategy. So we have made sure that all of the advancements that we do for the most demanding AI that we have with one software stack for AI that we have across our entire portfolio CPUs, what we call APUs, where we’ve combined the CPU, GPU and the Neural processing unit, and our adaptive compute and our GPUs, even for consumer graphics, all of that will be under the same software, the same RocM software umbrella that you see running our large scale data center GPUs. So our strategy is simply AI everywhere and a portfolio to support the breadth and range of the inference applications that are coming.
Daniel Newman: Mark Papermaster, thank you so much for joining us here.
Mark Papermaster: Daniel and Pat, thanks for having me.
Patrick Moorhead: You got it. Thanks.
Daniel Newman: And thank you everybody for being part of this Six Five On The Road. We are here in San Jose, California at AMD’s Advancing AI 2025. Check out all the other coverage we had here at the event and subscribe and be part of our community. But for this episode, it’s time to say goodbye. We’ll see you all later.
Author Information
Daniel is the CEO of The Futurum Group. Living his life at the intersection of people and technology, Daniel works with the world’s largest technology brands exploring Digital Transformation and how it is influencing the enterprise.
From the leading edge of AI to global technology policy, Daniel makes the connections between business, people and tech that are required for companies to benefit most from their technology investments. Daniel is a top 5 globally ranked industry analyst and his ideas are regularly cited or shared in television appearances by CNBC, Bloomberg, Wall Street Journal and hundreds of other sites around the world.
A 7x Best-Selling Author including his most recent book “Human/Machine.” Daniel is also a Forbes and MarketWatch (Dow Jones) contributor.
An MBA and Former Graduate Adjunct Faculty, Daniel is an Austin Texas transplant after 40 years in Chicago. His speaking takes him around the world each year as he shares his vision of the role technology will play in our future.
