On this episode of Six Five On The Road at VMware Explore Las Vegas, hosts Daniel Newman and Patrick Moorhead are joined by Chris Wolf, Global Head of AI and Advanced Services, from the VMware Cloud Foundation Division of Broadcom. Chris brings an in-depth perspective on the evolving landscape of private AI and its implications for modern businesses. Our conversation circles around the pivotal shifts in AI deployment strategies and the growing importance of maintaining privacy and security in AI integrations.
Their discussion covers:
- The current state and future prospects of private AI within the enterprise environment.
- Strategies VMware is implementing to enhance privacy and security in AI solutions.
- Insights into Broadcom’s approaches to fostering innovation in Cloud and AI technologies.
- The challenges and opportunities businesses face in integrating private AI into their operations.
- Predictions for the next wave of AI advancements and their impact on the tech industry.
Learn more at Broadcom and VMware.
Watch the video below, and be sure to subscribe to our YouTube channel, so you never miss an episode.
Or listen to the audio here:
Disclaimer: The Six Five Webcast is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors and we ask that you do not treat us as such.
Transcript:
Patrick Moorhead: The Six Five is On the Road here in Las Vegas for VMware Explore 2024. I cannot believe, Dan, it’s been a full year since the last Explore, but here we are, we talking on-prem cloud, we’re talking private AI, we’re talking about advanced services that sit on top of the stacks. This is some great tech, but more importantly, it’s really great tech that solves real problems that enterprises have, and they have had for many years.
Daniel Newman: And Pat, it’s a big year, too. If you think about where we were with VMware a year ago, it was a major moment of transition. There was a big deal that we…
Patrick Moorhead: Nine months ago we did the deal close videos.
Daniel Newman: We’ve been involved in and we’ve been part of these conversations for some time, and now we’re starting to see that product that everybody was wondering what would this look like on the other end? And I think this is that marquee moment right now. And it’s great to be here, Pat.
Patrick Moorhead: It really is. And it’s so funny, even though the first AI algorithms came out in the 1960s, some pretend that AI started in the last 18 or 24 months. The reality is that just got the conversation going, particularly wrapped around generative AI. We saw the consumer instantiation of it. And I think, more importantly in better context for this conversation, is enterprise value. And, wait a second, there’s all this public cloud action, but all this data is sitting on-prem in either traditional infrastructure or on-prem cloud. And I can’t think of a better guy to bring in. Chris Wolf, great to see you. Welcome to the show.
Chris Wolf: Oh, thanks, Pat. Thanks, Dan. Happy to be here.
Daniel Newman: Chris, you’ve been with us a few times. I believe on The Six Five, actually one of the earliest.
Patrick Moorhead: Exactly.
Daniel Newman: You’ve been around, so you’ve been part of the experience. We really appreciate you having the conversation, and here we are. So you heard the preamble, we’re setting the tone. There is a lot going on. And, Pat, you’re coming up, business value is really, really important. CIOs right now, weighing up the complexity of IT. And then of course you have boards and CEOs raining down on teams saying, “AI me!”
Chris Wolf: Exactly.
Daniel Newman: “Move it to the cloud.” And it’s like, “Well, hold on a second.” There’s a lot going on. But we talked about last year you had this big private AI initiative. You unveiled it, I think it was last year.
Chris Wolf: Yeah.
Daniel Newman: How’s that been received today? I’d like to hear a little bit about how’s the reception and now that it’s GA, how’s that moving?
Chris Wolf: It is going really well. We thought even at the time that private AI was bigger than us, that it was an important market category that really hadn’t been in the conversation at the time. And what we’re seeing now is broad applicability. There was even a general manager of one of the largest hyperscalers that was talking to me a couple of weeks ago and said, ‘All of our customers keep asking us about private AI.” So it’s definitely started to take on a life of its own. And the reality is, for a lot of organizations, as Pat had said, they desire to bring the model adjacent to where their data is. That’s really important for them. It could be very difficult for a customer to abstract their data, put it into a different format, sometimes a proprietary data format just to gain the benefits of AI. They don’t want to do that.
And for some of the early use cases, like customer service, a lot of an organization’s support data is stored in repositories in their data centers. So they want to be able to unlock a very proven type of use case for where they can get business value right away, and that’s an area where we can absolutely help.
Patrick Moorhead: So there’s a lot of elements that come into play when it comes to AI. I always like to throw out, it takes a village to do anything big in an industry. And we’ve seen this in major inflection points, whether it… Even going from mini computers to client server, local, mobile, social, and then the public cloud and how everybody makes this happen. And with AI, everybody can’t do everything themselves, whether it’s the model builders, whether it’s the silicon, networking, and what ends up happening is people want choice. There might be rabbits that go out and they capture all this stuff immediately, but in the end, enterprise IT, they want choice, they want competition. Competition lowers costs, and it increases innovation. You can’t support everything in the ecosystem immediately, and you came out with a few partners last year. How are you expanding the private AI ecosystem?
Chris Wolf: That’s a good question. So our thesis was that the space was moving too fast. You had to take a platform approach. If I’m an enterprise, I don’t want to bet on one vertical stack, so I’m locked in at a particular service where something better might come along next week, and I’ll have buyer’s remorse. So our strategy from the beginning was start with an AI platform that allows you to share all of your AI infrastructure intelligently, pool it intelligently, manage it, bring automation around it, and then as additional services come around, you can very quickly onboard them because you already have the platform in place. It’s super simple.
One of the internal services we run, we have changed the foundation model three times in the last nine months. Like our data scientists can continually A-B test, maybe this has better accuracy. I have better inference response times. We’re always looking at things differently. We’re having really good success with open source, too, which is driving the cost down. And that’s led to new partnerships. Like a couple of the partners that we’ve added just this week are Codium and Tab-Nine, because we see a lot of traction in the code-assist use case where organizations desire to maintain their code repos on premises. We’re seeing more delivery partners emerge as well. These are companies like HCL and WWT standing up and really looking to offer that full service to the end customer. To your point, because of the complexity, but I’d say even one of our surprises when we first unveiled this solution as generally available at the Nvidia GTC conference this year, we take automation for granted sometimes. And the fact that we can take an AI service stack, sometimes these are dozens of microservices, and be able to deploy all of that in minutes, is something that the average enterprise, it’s taking them weeks to do something like this. So there’s just a ton of value in what we have and we’re building on top of that and there’s a lot more coming.
Patrick Moorhead: Chris, one thing, just as a follow-up here, supporting a partner ecosystem is also hardware. And there’s been a lot of talk about networking, right? Isn’t it funny? It was all about CPUs, and it was all about GPUs, and then now there’s as much discussion about networking goes into that. The reality is you need to have a balanced system. CPU, GPU, memory and networking. Ultra ethernet is the hottest thing out there. You get the benefit of ethernet, you get the benefits of lower latency, higher reliability. Are you supporting that coming up?
Chris Wolf: So we certainly have our solution with Nvidia that would leverage all of their hardware. We have a broad hardware compatibility list around networking and ethernet already. And I agree with you with your points on ethernet, it has ubiquitous ecosystem, and it also has, just from a recoverability perspective, we’re seeing an outage that’s ethernet based is going to recover on average 30% faster than if you’re looking at other architectures like InfiniBand. So you can get the performance, you can get the reliability, a massive amount of flexibility through the ecosystem. And that’s no surprise that you’re seeing a lot of uptake in ethernet as an AI fabric among the hyperscalers as well.
Patrick Moorhead: It is. It’s predominantly the fabric inside of the hyperscalers, AI training and even inference.
Daniel Newman: There’s a lot to be said for building on open. Chris alluded to that earlier about every day something new comes out, and there is, of course, vertical stack we all have. There’s one device maker that a lot of us use that has that vertically integrated stack, and there’s one that likes to focus on more open. And by the way, that market remain split for somewhat substantial reason. But in this world, it’s even more important. The costs are higher, the stakes are higher, the risks are higher, and of course longevity. Absolutely.
And so you alluded to another thing earlier, that aha moment, that aha moment sounds like it was all about customers coming to the realization that this doesn’t have to be that hard. And, Chris, when we kicked off this conversation, we talked about business value, because in the end, we can nerd out and we can talk about how cool it’s to build the stack tech. We can get in the rack and we can shed light, and go light and copper, and have a debate about that or different CPU structures, but customers want to get value from AI. So the ah-ha is they’re getting it up and running faster. What are some of the other things that customers are really telling you that they’re finding now that they’re running this private AI solution?
Chris Wolf: I’d say a couple of things. So one, definitely on the use case, like the contact center customer support, like document summation use case where folks are getting access to information, the right information, more quickly. It means your customer support technicians are closing more tickets in a given week. There are measurable efficiency gains there. It’s a great place to start. The other, I think probably the largest, ah-ha for us, certainly, I’d say to some degree, we took automation for granted. As Pat mentioned, AI is not just about the GPUs. I have to intelligently schedule memory networks, access to storage, I have to enforce quality of service. It’s extremely complex.
To that end, the surprise to us had been the companies that have been running AI services on our own bare-metal stacks for years, among our first wave of early customers, because they cannot do distributed resource management at scale. They have no good visibility into how to effectively share their GPUs, and understand utilization, and balance across them, and our distributed resource scheduler is almost twenty-year-old technology that we continue to grow and evolve. And when you look at even for on-prem AI, and there’s a lot of choice out there, there is nobody that has anything close to the sophistication we have today with DRS, and there are some of the largest companies in retail, largest companies in financial services, that are partnering with us for private AI because we can do things that nobody else can.
Daniel Newman: Well, Chris, I want to say this conversation is exactly what I think your partners, I think it’s what the customers out there need to hear. Everybody really wants to understand how to quickly derive meaningful value from AI, evolve their cloud ecosystems, and deal with, Pat, and you had many conversations with HACO, this continuum between prem and cloud, and there is this really serendipitous relationship between them, and of course where those workloads land. I know, Chris, VMware has an opinion and the public cloud has an opinion, but it seems like that’s starting to finally find its footing.
Chris Wolf: And I think it’s an and it’s our biggest focus has been on AI inferencing, because that’s where 96% of AI compute capacity is today. Build and train a model in the cloud, or you don’t even have to build a model in many cases, download a foundation model and start working with that right away. When you want to run the model, we’re going to give you a governance layer that allows you to very easily manage it using an enterprise container registry, which is Harbor. You can then even use Native NGC, you can use hugging-face CLI to interact with these models central. It can approve the models, and then you’re running these now on-prem.
We see dramatic cost savings and also predictable costs when you take this approach. We’ve had customers sell us our stack for them running on-premises and having control of all of their data control of their assets, flexibility to run any new service they want now or in the future. They’re getting that sometimes at a third, the cost of what they would get with a public AI service. So you get all the privacy and control and you get the cost benefits. So to me, it’s like for the runtime, our approach is a no-brainer.
Patrick Moorhead: Chris, there’s a lot of data that’s collected at the edge. That’s been true since the dawn of time. And I know like AI things edge gets popular, gets less popular, but when you look at a retail store, if you look at a bank branch, if you look at a satellite manufacturing or distribution center, that is the edge, industrial edge, things like that. Is everything we’ve talked about with private AI also applicable to that edge?
Chris Wolf: And it’s a good question, and we’re, to give you an example, one of our customers is a retailer. And what this retailer does is they have a fantastic computer vision use case. A lot of retailers are using computer vision, but they’re just trying to find people stealing from them at self-checkout, that’s the use case. But this company, what they’ve done is they’ve figured out that what they should be doing is monitoring people in their aisles, not like what they’re doing, but are they standing still? Because if I’m at a home improvement store, and I’m trying to find a plumbing adapter, I’m going to be standing in front of the same set of shelves for minutes, because I want to get it right, and I don’t want to go back.
So what a particular retailer is that we’re working with, what they’re doing is they’ll monitor customer idle time, they will automatically dispatch an associate to go to that aisle to help the customer, when they see somebody, say, standing there for two minutes, and what they’ve found is every store that they’ve deployed this technology, they have a measurable gain in sales performance in every single store. So there’s really good practical… That’s definitely an edge use case you may not think of, but it’s very practical. It makes a lot of sense. We’re seeing this in distribution centers, certainly manufacturers where there’s even a lot of air gap it’s another great use case because I can’t rely on a cloud for AI services, but there’s a lot of AI-driven event-driven automation that I can do as well.
Patrick Moorhead: I’m just envisioning the amount of cycles my favorite hardware store is spending on me, tracking me where I’m going back and forth and circling around, Dan.
Daniel Newman: I’m terribly unhappy. I’m just telling you, there’s not a handy type project that I’ve ever done well around my home. But in all seriousness, the edge, Pat, when we’ve talked about tech the edge created this huge opportunity. AI has really made that exponential. And so when you’re hearing what Chris had to say here, he’s basically saying that this is not limited to that core data center. It can obviously move out to the edge, which creates massive scale and opportunity for VMware and its customers. Chris, we do have to wrap up. I do want to thank you so much for spending some time with us here at VMware Explore 2024. Let’s have you back again soon.
Chris Wolf: Hope so. Thanks a lot.
Daniel Newman: Thanks, Chris. And thank you for tuning in with us. We are The Six Five On the Road here, VMware, Explore 2024 in Las Vegas. Subscribe, be part of our community. We appreciate you tuning in, and we’ll see you all later.
Author Information
Daniel is the CEO of The Futurum Group. Living his life at the intersection of people and technology, Daniel works with the world’s largest technology brands exploring Digital Transformation and how it is influencing the enterprise.
From the leading edge of AI to global technology policy, Daniel makes the connections between business, people and tech that are required for companies to benefit most from their technology investments. Daniel is a top 5 globally ranked industry analyst and his ideas are regularly cited or shared in television appearances by CNBC, Bloomberg, Wall Street Journal and hundreds of other sites around the world.
A 7x Best-Selling Author including his most recent book “Human/Machine.” Daniel is also a Forbes and MarketWatch (Dow Jones) contributor.
An MBA and Former Graduate Adjunct Faculty, Daniel is an Austin Texas transplant after 40 years in Chicago. His speaking takes him around the world each year as he shares his vision of the role technology will play in our future.
