Six Five Media had the pleasure of having AWS CEO Matt Garman on the show. Patrick Moorhead and Daniel Newman spoke with him on the current state of AI, AWS’s approach to supporting it, the importance of scalability, predictions for the future, and their commitment to innovation and democratizing AI.
Specifically, their discussion covers:
- The current state of AI and its implications for businesses and developers.
- AWS’s approach to supporting a diverse array of AI models and frameworks.
- The importance of scalability and flexibility in AI infrastructure.
- Predictions for the future of AI technologies and their integration into industry solutions.
- Insights into AWS’s commitment to fostering innovation and democratizing AI access for all users.
Learn more at AWS.
Watch the video below at https://sixfivemedia.com/tag/six-five-on-the-road/
Or listen to the audio here:
Disclaimer: Six Five On The Road is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors, and we ask that you do not treat us as such.
Transcript:
Patrick Moorhead: The Six Five is On the Road and we are talking about our favorite subject, and that is AI. There’s been huge investments in AI in the industry, on infrastructure, on services, and also from developers making their own programs, in addition to enterprise SaaS. Dan, it’s been an exciting couple of years here, hasn’t it?
Daniel Newman: Yeah, it’s been a big couple of years, and we’ve seen so many changes, shifts, inflections, Pat. We’re watching this massive boom of CapEx investment, and of course, everybody’s looking on the back end and saying, “How does all this investment turn into productivity gains and efficiency gains? And who are going to be the new companies that rise? And how are the companies that came out of the first computing era going to continue to compete?”
Patrick Moorhead: That’s right. The inventor of cloud computing, Amazon Web Services, has made huge investments in all parts of that chain to deliver it to their customers, and I can’t think of a better person to have this conversation with, to check in, than newly appointed AWS CEO, Matt Garman. Matt, welcome to The Six Five.
Matt Garman: Thank you for having me. Nice to see you guys.
Daniel Newman: Yeah, Matt, it’s great to have you on. As Patrick said, AWS really was at the forefront of this entire movement, and now we’re entering a new era. You heard me say that I call it the great reset of computers. What’s been built hasn’t completely changed or gone away, but we’ve seen really significant, fundamental changes in this kind of GPU and computing era. I’m curious from your take, there’s the traditional competitive set, and you, of course, being the leader in the market, there’s all these new entrants coming into this space. How do you view this market shift and AWS’s role and its continued market position?
Matt Garman: Yeah, I think it’s interesting. Look, it’s a fascinating and super exciting time to be in this space and in the technology world generally, and I agree with you. I think when we look at the impact that AI has on all of our customers, on all the industries that we work in, I fundamentally believe, and we at AWS fundamentally believe, that AI is going to change every single workload, every single industry, every single company, and people are going to do things differently and has a massive potential for really improving the effectiveness, and the efficiency, and the customer experience across every aspect of what almost every industry does, from healthcare to financial services, to consumer electronics, to the gamut of industries. And so, we’re incredibly excited about that.
I think the couple of things that we see, though, is that, as you mentioned, it’s not replacing the old way of computing. It’s very complementary to that world because it turns out that when you go do AI, you also need a lot of data to feed those AI models. You need a lot of traditional computer infrastructure, you need a lot of analytics tools, you need a lot of databases, et cetera. And so it’s very complementary to the business that we’ve built. And so you’re right, we’re at AWS making a huge investment in AI, but as a complement to a bunch of the capabilities that we already have in the cloud. Because think, when you talk to companies out there and they say, “How am I going to get real value out of AI?”
And that’s what a lot of companies are actually thinking about and wrestling with today, is how they get value out of it. They know that their workloads are going to live in the cloud, these AI models are all running in the cloud. Your data needs to be in the cloud. And so there is, from our viewpoint, a real positive flywheel there, that if customers and enterprises can get all those workloads moving into the cloud, that’s when they can really start to take advantage of AI and, I think, over the long term, get the most benefit out of it.
Patrick Moorhead: So Matt, being the inventor of anything is awesome in itself, and turning that into a massively scalable business is as hard or harder, and doing it year in and year out. And you’re putting up some just extraordinary revenue numbers out there. I do have to ask, growth has slowed to the 10 to 15%. I remember the days of 50, 60, 70% growth. Can you give us a little input, a little feedback on that, but more importantly, how do you intend to keep driving that growth and staying ahead?
Matt Garman: Yeah. Well, there’s a couple of things. One is, it’s actually a little bit more than that. It’s actually 17, 18% now, and it’s actually been accelerating each of the last four or five quarters. And I would say that there’s a lot of large numbers in there too. At a hundred billion dollar business, you’re not going to grow 70%, that’s just massive growth. And so in absolute terms, the business is growing actually faster than it ever has, and it’s growing in absolute terms faster than any of the competitors as well. So in absolute size, the business is actually growing at a faster rate than anyone else out there and than it ever has before.
And I would say there’s a couple of pieces to that, which is we look at the long term of this business, and so we’re happy with what the growth is. And I think we’ll continue on that path, and we see a ton of headroom to the business, where it can continue to grow for a really long period of time. When I talk to customers out there, less than 20% of workloads that we talk about are running in the cloud today. And so, that ignores even the massive amount of new gen AI workloads that are going to happen. Just existing workloads, only a fifth of them or less, have moved to the cloud. So enormous opportunity for customers to continue on that super important modernization and migration path that they’ve been on for a couple of decades now as the industry moves to the cloud.
So there’s a ton of potential there for us. We see a massive amount of growth forward for the business, and we think we’re very well positioned to take advantage of that. And part of that is because we invest in every layer of that stack. We invest in power, we invest in data centers, we invest in networks, we invest in our own silicon to make sure that we can deliver the best cost performance for customers. And so by innovating across all those layers of the stack and then building super innovative services on top of that, we feel really excited about the continued acceleration of growth that we will see for many years to come.
Patrick Moorhead: Thanks Matt.
Daniel Newman: Matt, we talked about AWS being first in cloud, and of course that comes in a few different respective numbers, first into the market, now, of course, first as the largest in the market. I’m really glad you pointed out the law of large numbers. It’s not inconsequential that, like you said, in overall growth, it’s so substantial. Of course, as analysts, we wrestle with this, we wrestle with the fact that people are saying, “Oh, they’re going 28%, but they’re going 28% over one-tenth of the size.” So it’s been impressive, but we’re always matching that up on our side.
Matt Garman: You got to find a more interesting headline of course. So it’s okay.
Daniel Newman: Yeah, exactly. But one of the things that I think did ring true, and gosh, so many conversations we’ve had, so many reporters I’ve pressed that we’ve been trying to smooth this concept over, is that AWS was late to the gen AI party. We know how methodical the company is, we know how it likes to do things, have a very, very strict sort of process for defining new products, rolling things out, making sure there’s market fit. A lot of credit to be given there, but Microsoft jumped out of ahead. They had this OpenAI partnership, they didn’t build anything per se, but they integrated someone else’s technology, got out fast. Now, AWS now for the first time was sort of chasing, it was sort of chasing, at least there was a perception of that. Talk a little bit about how the culture is dealing with that perception and where do you think you are now versus what’s in the market. Do you feel like you’ve closed the gap or even surpassed the competition at this point?
Matt Garman: Yeah, look, I think that from our perspective, a lot of that was much more perception than reality. We’ve been involved in AI for well over a decade publicly with AWS and even longer internally. The most popular AI platform to build on is SageMaker, out there in the market. And we’d been working on generative AI models ourselves internally, but as ChatGPT came out and the world realized what some of these models were capable of, everybody else rushed to go put a chatbot out there and be able to slap it onto whatever capabilities they had. And what we saw is that customers also got super excited about that and ran to do proof of concepts. Honestly, our take was we were going to take a step back and understand if this is really going to be a fundamental technology that every company is going to have at the core of everything that they do. It can’t be a rushed-on consumer application that just gets slapped onto the side, that maybe has security after the fact and maybe has protections of your data after the fact.
We felt like we had to build that core layer first, and so we went and said, “Customers are going to want to make sure that all of their data is secure and never leaves their VPC or their protected network environment.” We went in and said, we know that security is going to have to be first as customers there. And we had this idea, actually, we didn’t know, but we had the idea that if you look forward ahead, it’s unlikely that there’s just going to be one model for the world. There’s probably going to be lots of different models, and maybe in combination, where you’re going to need small models and big models, and you’re going to need one model that’s really good at doing financial analysis. And another model that’s really good at doing indexing, and that was our view. Because we had spent a lot of time internally working with these models, understanding how they worked, and so we actually spent that time to go and actually build that platform.
Because we knew that our customers are enterprises, not consumers, and so from an AWS perspective, we wanted a platform that everybody else could go build on. And so we knew that we had to have a lot of different models. We knew that we had to have an easy way for people to build RAG indexes. We knew that people were going to want more and more capabilities to go build AI applications in a secure way because, at some point, customer’s data is their differentiating factor. And so that’s what we did with Bedrock. We went and built that, and now you’re starting to see customers who, I would say, over the last 12 months or so, scrambled to launch as many proof of concepts as they can.
And you’d go talk to enterprises, I’m sure many of you have done this, where people launched hundreds of proof of concepts, and they came up with some really cool ideas, and they’re really neat. And now they’re starting to look at it, and one, the bill’s really big for all those proof of concepts. Two, they’re not really sure which of those are going to deliver real enterprise value. And three, as they move from proof of concept to production, they’re realizing that actually, I need all of those things. That I need to figure out how when I integrate this with my enterprise data, is it secure? How do I have the right access controls, et cetera?
And so, we’re actually now seeing just a really, really, really rapid acceleration of customers adopting and actually building real production applications on Bedrock in AWS because it’s where their data is, it’s where their applications live, it has the right security controls around it. And customers are loving that model. I think there was the perception that we didn’t get something out there quickly, but it’s because we were building that platform for the long term. And now I’m delighted with where we are, and I think our enterprise customers really like where we are too.
Patrick Moorhead: So Matt, two things there. So first of all, we’re also seeing in our research in the enterprise people moving from what I will call experiments to POCs and then, at some point, to be able to scale. There are some fast movers, some rabbits who are currently deriving benefit today in big areas. And the second thing we recognized, my company has recognized your approach, and I think Dan’s as well, which was to build something long-term, build something that’s multi-model, build something that really addresses the concerns, which is security and data management. And you have done that, in our view, an effective job at doing this. I do want to hit on something that you mentioned before, and that was silicon. So I’ve been chronicling your journey of first-person silicon, and of course merchant silicon for a long time…
Matt Garman: We appreciate that too.
Patrick Moorhead: No, you –
Matt Garman: You were early into Graviton and we appreciate that.
Patrick Moorhead: Oh, I didn’t know you knew that, but thank you. Dan and I both, we were inside of one of your key development hubs in Austin, Texas, where we saw it first and foremost. And you have a first-party silicon strategy plus merchant silicon. Can you talk a little bit about how you balance those two? What’s the strategy behind and maybe what’s the benefit to AWS customers?
Matt Garman: Yeah. Well, it’s all about the benefit to AWS customers. There’s a couple of things, our view is I just want our customers to have the best selection of all of the technologies available to them. And so they can trade off what they think is the right technology. We have the right cost point for the right use case that they have. And so our view is, I mean, this is a massive space, and so there is room for a lot of these players. And if you take even the general purpose compute space to start with, we’ve been partners with Intel and AMD for multiple decades, and we saw a space for Graviton, which is our own custom silicon ARM-based processors. And we saw an opportunity where we could go build a processor that was lower power, higher or lower cost, and we thought we could get too much higher performance, which I think we’ve now proven with the Graviton, Graviton four and three and the last couple of generations that we’ve had.
And so from that perspective, we still can see, by the way, we still have a massive business of Intel and AMD processors of people running inside of AWS. And Graviton is a really, really large business where many customers get a huge cost-performance benefit, actually an absolute performance benefit at much lower costs. And so, we like that, and customers love that choice, and we think that’s great. About five years ago, we saw this trend of accelerators being a key part of that compute platform, and we’ve been partners with NVIDIA again for well over a decade plus, running in AWS with the first NVIDIA instance. I can’t even remember which year it is now, but when we launched many years ago. And so NVIDIA is an incredibly important partner for us. Today, the vast majority of AI workloads run on NVIDIA, and they will for a long time. They make a really great offering.
And AWS is the best place to run NVIDIA workloads. That’s why NVIDIA chose AWS as the place where they’re running all of their research workloads, and we’re building a massive gen AI cluster together for their next generation of NVIDIA products. And so we’re super excited about that partnership, and we think that it delivers a lot of value for customers. However, five years ago, we also saw that there’s going to be room for more, and we thought we could do the same thing for accelerators that we did for general purpose. And so we went and we built Inferentia and Tranium, which are inference and training platforms, and we’re incredibly excited about that. Tranium2 is going to be coming out at the end of this year, and I think that platform is going to be really, really impactful for many of these workloads, where I feel confident that it’s going to be the best price-performance offering for many workloads. Not all of them, by the way. Many workloads are going to be optimized, and they’re going to run the best on NVIDIA, and some of them we think can be on Tranium. And so we think that mix where customers can pick between a lot of different options, just makes AWS a better place to run every single workload that people want to run. And so we think that choice is super important.
Patrick Moorhead: Matt, just as a follow-up, even with your first party silicon, at least from my vantage point, you’ve found a way to differentiate with merchant silicon providers like NVIDIA, like AMD, EFA was an example of that. Can you talk a little bit about it? Is that the Go-forward plan, to differentiate coming up with ways to differentiate in merchant as well?
Matt Garman: Absolutely. Look, part of the benefit that we get is we get a lot of view into what these compute workloads look like and how they operate in our environment. And we have that benefit where, when you put a chip into an AWS data center, it doesn’t have to go into a unknown data center, which lots of folks have to deal with a lot of different environments. We have to deal with one environment with just ours, and so we can bring some of that learning to optimize that environment around our partners as well. And so we do oftentimes custom parts with Intel, as an example, where we’ll make custom modifications to make sure that the Intel processor actually gives better performance for customers in an AWS environment than they can get anywhere else because we really know how it’s going to operate.
You mentioned EFA. We do that with NVIDIA as well. I think for a long time everyone was viewing InfiniBand as the best networking capability for these clusters, which, by the way, InfiniBand works really well for small clusters that are relatively static. But as you start to expand them and they’re growing up and down, and scaling, and they’re getting much larger, InfiniBand becomes actually a little fragile and is actually a big source of errors and faults inside of these really large clusters. And it actually doesn’t scale to the scale that these training clusters need. And so, working with NVIDIA EFA, which is our custom networking that gives you very low latency, high throughput performance via an Ethernet network, that’s a thing that we’ve been partnering together with them on, and it’s part of why they’re excited about partnering with us because we can deliver that differentiated performance at the scale and flexibility that we need.
Daniel Newman: Well, Matt, our research actually validates both points on the EFA and use of Ethernet. We’re finding that to be true and more scalable and a bigger opportunity. Also, what you said about the custom AI silicon, we’re seeing a growth rate. Our intelligence is seeing something about 20% faster growth than these broader purpose GPUs in the years to come. And I think that’s why there’s so much excitement, enthusiasm, and demand. And we’ll certainly be tracking closely Tranium2 Inferentia and all the work. And I just want to thank you on behalf of both Patrick, myself, and The Six Five for taking some time to join us here. I know you have the big event, a lot of analysts and others in town to hear about what you’re doing in generative AI. We look forward to catching up with you again soon.
Matt Garman: Awesome. Thank you both for having me.
Daniel Newman: Thanks. And thank you, everybody, for tuning into The Six Five. We are on the road. We were joined here today by AWS CEO, Matt Garman. Thanks for being part of our community. Hit that subscribe button. Join us for all of our episodes for Patrick and myself. Time to say goodbye. See you all later.
Author Information
Daniel is the CEO of The Futurum Group. Living his life at the intersection of people and technology, Daniel works with the world’s largest technology brands exploring Digital Transformation and how it is influencing the enterprise.
From the leading edge of AI to global technology policy, Daniel makes the connections between business, people and tech that are required for companies to benefit most from their technology investments. Daniel is a top 5 globally ranked industry analyst and his ideas are regularly cited or shared in television appearances by CNBC, Bloomberg, Wall Street Journal and hundreds of other sites around the world.
A 7x Best-Selling Author including his most recent book “Human/Machine.” Daniel is also a Forbes and MarketWatch (Dow Jones) contributor.
An MBA and Former Graduate Adjunct Faculty, Daniel is an Austin Texas transplant after 40 years in Chicago. His speaking takes him around the world each year as he shares his vision of the role technology will play in our future.
