On this episode of the Six Five – On the Road, hosts Patrick Moorhead and Daniel Newman are joined by Intel’s Justin Hotard, Executive Vice President and General Manager, Data Center and AI Group, and Sachin Katti, Senior Vice President and General Manager, Network and Edge Group. This episode offers an insightful discussion on Intel’s strategic direction regarding Enterprise AI, which was covered this week during Intel Vision 2024.
Their discussion covers:
- The Enterprise AI Opportunity
- Intel’s Vision for Enterprise AI and the necessary steps to realize it
- Intel’s scalable approach to AI and its impact
- Intel’s strategy on addressing software challenges in AI
- Key product announcements made by Intel this week
Learn more at Intel.
Watch the video below, and be sure to subscribe to our YouTube channel, so you never miss an episode.
This video was sponsored by Intel.
Or listen to the audio here:
Disclaimer: The Six Five Webcast is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors and we ask that you do not treat us as such.
Transcript:
Patrick Moorhead: The Six Five is on the road here at Intel Vision 2024 here in Phoenix, Arizona. Dan, can you believe we’re talking about AI again? AI PCs, AI on the edge and the data center. Pretty much AI everywhere.
Daniel Newman: It really is, Pat. And I got to say, the time we’ve spent here so far has completely eclipsed my expectations. For everybody out there, it is the solar eclipse day, and there have been a lot of cheesy eclipse jokes, but you know what? It’s funny as dads, we’re dads and we’re going to have some guests that may also be dads. Dad jokes work. I mean, that’s why they exist, isn’t it?
Patrick Moorhead: Yeah. I mean, I think about 64% of the eclipse jokes actually landed today.
Daniel Newman: It was pretty good. It was pretty good. It was about 67% landing here in Phoenix and 67% totality out here. But yeah, it’s been a great show so far, Pat. Look, Intel’s got a big opportunity here with AI. It also has a bit of a proverbial chip on its shoulder that it wants to prove to the world that it has a great AI story. And I think vision really is all about telling that story, communicating the value, and helping the market understand where they are today, where the company is going into the future, and how they intend to be really, really important in the future of this AI cycle that we’re entering.
Patrick Moorhead: That’s right. Let’s bring in our guests, Justin, Sachin. Great to see you. Sachin many times, Six Five. Justin, first time Six Five. Welcome to the show.
Justin Hotard: Thanks. It’s great to be a rookie.
Sachin Katti: Thank you. Great to be here.
Patrick Moorhead: I don’t know, you looked great on stage and look like a rookie. You look like you’ve done that a couple times.
Justin Hotard: Maybe once or twice.
Patrick Moorhead: Yeah.
Daniel Newman: It was great to hear from both of you up on the big stage. And so Sachin, let’s start with you. Enterprise AI is in focus. It was a big part of the presentations today from the stage. Talk to our audience, share a little bit about your vision both for Intel and just the overall enterprise AI opportunity.
Justin Hotard: Yeah, I mean I think enterprises are, obviously, they’re at the cusp of a big inflection point. As generative AI comes along, it’s going to transform the enterprise. If you are in the valley, everyone’s talking about the one-person, Unicom that’s coming, right? The modern enterprise where there’s going to be a single founder and nothing else. And AI is going to run all of the different parts of the enterprise. And the way we look at the enterprise is we are going to go through three generations very quickly. The first one is the era of copilots. We are all being surrounded by copilots. Windows. Copilot is coming, there’s copilot for programming. You name it, there’s going to be a copilot. And think of this as AI being a helper that is standing by and you call upon that AI to come help you with something.
Very quickly, and you saw many of those demos today in our keynotes, it’s going to be the age of agents, AI agents, where they’re going to take our entire complex workflows and automate it. So we showed a demo with a customer. It’s a production deployment today where when you do a drive-through, instead of some human taking an order, there’s an entire AI that’s just talking to you like a human suggesting things to eat based on your preferences and taking over that complex workflow. And then very soon after that, we’ll go to age of AI functions where multiple AI agents are interacting and collaborating with each other and entire enterprise functions.
Think finance, supply chain management, these kinds of functions are getting automated by AI. So you can imagine if you take a step back, this is transformative for the enterprise and it’s showing up in the numbers. I think the estimate is we’ll go from 40 billion of spend this year to more than 150 in two years. And frankly, these numbers could be a lot higher depending on the impact this thing is going to have. So tremendous change. More than 80% of enterprises are going to deploy Gen.AI in our opinion. And more than 50% of edge deployments are going to have Gen.AI in them by 2026. So a big option you had for us in Intel.
Patrick Moorhead: Yeah, it’s an incredible vision. And some people might say, “Hey, when we see something this big, how real is it?” And some people forget that A lot of the initial algorithms were created in the 1960’s. And remember seven years ago we had the chatbot revolution that was going to change everything. It changed very little. So the investment and the maturity as analysts, we have to call fad or trend. And this is absolutely a trend here. And then the challenge becomes how do you deliver on that and what are specific company’s vision? And I’ll hit you with this, Justin. So can you talk a little bit about Intel’s AI vision, how you are approaching it? And my second question would be how are you delivering on it? What’s your strategy to deliver it?
Justin Hotard: I think first of all, our Intel’s principles, our strategy has always been around enabling open ecosystems. We certainly saw that in the age of the internet, you talked about something that maybe we thought was a fad early on, but rapidly became a trend. We saw that because microservices drove new architectures, technologies like virtualization drove much greater scale and adoption. We got a lot of efficiency. And then we drove standards all the way down through the systems. And the other part of that is if you think about cloud adoption, Intel was really a spearhead with some of the major hyperscalers and driving standards, open compute platform, obviously the x86 architecture and the software tools.
And what we see with CNCF as an example, and if you look at this space, we see a lot of similar analogies, a lot of early innovation happening on closed stacks and we understand that that’s necessary-it’s pretty common for adoption. But we have a tremendous opportunity right now to enable an open ecosystem. The difference in this market and with GEN.AI in particular is we need to innovate and have an open stack from the silicon all the way up to the model to frameworks and models. And that’s a very different architecture than we’ve seen in some of these previous technology shifts. So our strategy to your second question is to be a leader in enabling that open ecosystem and providing interoperability at every level. And this is something Sachin and I are working on together across our businesses.
Patrick Moorhead: Yeah, it’s amazing the thought of having something that’s closed, having not my first job actually, my first job was pre-internet and seeing how that.
Daniel Newman: Long time ago
Patrick Moorhead: As Dan reminds me…
Justin Hotard: Pre-internet is like a prehistoric term…
Daniel Newman: That before 32 72 cluster controllers.
Patrick Moorhead: Yeah, there was something before this, but we’re definitely in that. We’re having this industry debate about closed and open, and unfortunately, those people who aren’t students of history or haven’t seen this before, aren’t seeing that. And I’m glad you’re doing what you’re doing in addition to some of the open organizations out here that are trying to do the same thing. Ultimately it comes down too about speed. The overall industry has to operate at a certain speed. There’s mass, there’s scale, there’s velocity, to actually get it there. Otherwise, closed systems will keep doing really well.
Daniel Newman: It’s really interesting. I was listening to Sachin. You talked about the one person billion dollar company. I think Sam Altman, he’s at least getting credit for it, saying it. Everything he says…
Sachin Katti: He gets credit for all of it.
Daniel Newman: It’s a meme. It’s inevitably a meme, but it is really interesting if you think about just volumes of productivity, the scale of how much companies can create efficiencies and grow. We have our summit coming up, The Six Five, and we’re at Bill McDermott, and I’ve had a number of conversations. He’s the CEO of ServiceNow for anyone that doesn’t know that. But he’s just talking about basically that AI is going to create a completely new look and feel to enterprise software. Basically enterprise software as we know it today, won’t look anything like it looks today.
And so we’re hearing about scale in healthcare, scale in financial services, small language models, mid-sized large language models, trillion parameter. And then you have certain companies now touting a hundred-million parameter models and saying, we can get higher fidelity with those. So it’s going to be all about scaling this. I’d love to get your sort of take, whether it’s about the hardware to software architecture, the enterprise to deployment challenges, how do we scale this? And Sachin, I’ll start with you, but I’d love to hear from both of you. How does this scale out and how does Intel help?
Sachin Katti: Yeah, I think if you look at an enterprise, they’re going to need AI at every point in the enterprise. Starting with PCs, every worker. How do they get access to co-pilots? How do they actually make use of this in their day-to-day tasks. If you look at IT and OT deployments, we show up at a venue like this. You saw a number of examples today where people wanted to automate crowd management. People wanted to automate line management, queue management, ordering, all these kinds of stuff. And then of course, the big models where you think about knowledge discovery and becoming co-pilots for software engineering or in Intel’s case becoming co-pilots for chip design. So at every point the enterprises want to leverage AI, and that tells you that enterprises need choice in the kind of form factors that they will deploy AI in. So they need to be able to deploy it on the PC, on the enterprise edge and then in the data center.
And our promise is we will bring AI everywhere. And the way we want to do that, as Justin said earlier, is open scalable systems where we work with our entire ecosystem to give enterprises choice, but we give consistent software abstractions that work from the PC to the data center so that the enterprises can sleep well at night, that they don’t have to worry about integrating specific software stacks for each piece of hardware. It’s one consistent software stack with one API and open. We know that works. And then of course, working with the ecosystem to build reference solutions and end business solutions that work well on top of these open systems so that enterprises can see the value rather than them having to figure out how to extract value from AI, which can be quite complex. So I think in a nutshell, the way I’d emphasize that we want to deliver the easy button, but with choice.
Daniel Newman: Hard to do.
Justin Hotard: Yeah, I think what I would add, Sachine summarized it really well. I think what I’d add is if you look at some of those things that were fads instead of trends, the friction was too high. And I think one of the things that’s easy to get excited about with AI because we see a great consumer application is we get excited about that, but then we realize, well, in the enterprise case, actually we can’t just start net new. We have existing data sets. We have customers that are still using Cobalt, right, still running mainframes. And in fact, some of their most valuable proprietary enterprise data are on systems in cobalt, right?
Daniel Newman: Transactions.
Justin Hotard: Exactly. That’s high value, high insights, something you’re going to monetize. And so it’s-how do we provide platforms and frameworks for adoption that leverage a lot of the investments that those enterprise already have and support the compliance issues they need to consider it the data locality issues, obviously data security. I think as we look at our experience in this space and where we believe we have great credibility, it’s what we’ve done to cultivate the x86 ecosystem where we believe as we bring some of those same principles that Sachin was talking about some of the developer tools that we have with OpenVINO and oneAPI, we see some of the same opportunities to provide that capability so that enterprises can innovate rapidly, but within the confines of what they need to do to be able to deliver their business impact within their requirements and business compliance constraints and of course their financial constraints.
Patrick Moorhead: So I want to drill down on software here. I mean, what’s becoming very rapidly understood, and I think we hit this inflection point probably six or seven years ago, was that the AI challenge is as big a software problem as a hardware problem. And I think I can confidently say the biggest challenge is software. And that’s not only from how do I scale this from a small little end point on the edge of the network to a PC to even a carrier to the big data center on-prem data center and everything in between.
But we seem to be in this situation where large groups can’t move quickly enough to have, let’s say, a magic abstraction layer and it seems to be going slower than the closed systems. And a lot slower closed systems might be going 5x the speed of the open systems. How, again, you talked a little bit about Sachin, but you drilled down on how you’re approaching it, maybe why is it going to be different this time? I remember just one example, OpenCL was going to be the magic interface that everybody was going to write to make all this computing off a GPU standard, but it just didn’t stick.
Sachin Katti: Right. I think it’s a great question. And the first thing I’d say is the entire ecosystem has an extremely strong incentive to figure out an open alternative. That’s understating it.
Patrick Moorhead: I’ve never seen more motivation today than the ecosystem…
Sachin Katti: And so I think the way we look at the world is we call them model developers and model consumers. Model developers are your data scientists who are training new models, and that interface is PyTorch. And so everything we do in Intel from a hardware, we want to make sure that it’s integrated into PyTorch frameworks and that developer never has to leave PyTorch. Underneath that, we are making sure that we have all of the one API kernels, but also the compiler tool chains working with the ecosystem like Triton, which is the open source MLIR compiler tool chain and enabling that for all of our platforms. So I think working with the biggest foundational model providers who also are super incentivized to build such an ecosystem, we want to make sure that we plug into the same frameworks that everyone’s using. The second one is model consumers, and that’s where enterprises are going to come in.
They’re unlikely to train a trillion parameter model on their own, but they’re likely to take a trained model and then tune it and then deploy it. And so they’re going to consume models and there they want abstraction. They want to make sure that they don’t have to worry about which GPU did I pick? Which SOC did I pick? Which CPU did I pick? It should just work. And there are strategies, OpenVINO giving you a consistent abstraction that works all the way from Atom-class CPUs to our highest-end CPUs and GPUs and accelerators in the cloud. One consistent abstraction for anyone who wants to consume a model. I think that’s how we are approaching the world. I think it’s very consistent and well-liked by the ecosystem, but we have a lot of work to do to make this happen.
Daniel Newman: Sachin’s also a professor at Stanford. I feel like you could just do the TLDR if you don’t want to come and be a student. There you go. Got the four years.
Justin Hotard: Exactly.
Daniel Newman: It doesn’t come with it.
Justin Hotard: I’m expecting a PhD after this conversation.
Daniel Newman: I’m just saying it’s like the courses are available online if you really want to learn some of that stuff. My son’s…
Sachin Katti: …Come on your podcast anytime for free. How about that?
Daniel Newman: Justin, anything you want to add on that one?
Justin Hotard: I think he covered it really well. Maybe one thing I’ll just add is in terms of the enabling ecosystem we have in the enterprise, we’ve got established virtualization platforms, OS’s enterprise applications where we’re looking for ways to accelerate development and deployment. And so there’s a whole industry ecosystem of how do we make this work within the existing frameworks, within the existing software stacks. And I think that’s the other part of this innovation. Tools like RAG makes sense because the other issue that enterprises have is they don’t have a ton of data scientists, so they’re constrained.
So they need simpler ways to deploy models. Let me fine tune a very focused model and then I’ll use a tool like RAG and I can go access the data in real time. By the way, that saves me money. Now I don’t have to become a model developer as well and get into retraining. I can just leverage some of that data. So I think there’s a lot of innovation coming in this space, all because there is such a significant amount of gravity with these existing enterprise stacks and applications.
Daniel Newman: It’s really interesting because it is a bit of an arms race. I mean, you only got maybe what a couple dozen companies that are going to be really in the big model game. And then everyone else, every other enterprise on the planet’s going to be trying to figure out how do we take small sets of proprietary data to use to differentiate and sit on top of, and even just how fast the industry has digested and almost made this the norm. What do I mean by that? Well, these massive language generating models, it’s just commonplace now. It’s table stakes. You go from one to another and sure, they’ll show fidelity data. This is one, but I’ll go from a palm thing to a GPT thing to a llama-based thing, and it’s just text and it’s all using the same data set. And until you take your data and fine-tune it on something proprietary to you, very, very hard to extract value.
Patrick Moorhead: Well, you get harder when you’re co-mingling the data. And that’s the big generative AI thing. It’s crossing streams the first time between a CRM, an ERP, a PLM, and every other M acronym…
Daniel Newman: Three letters.
Patrick Moorhead: Exactly,
Daniel Newman: And then all the obstruction that you have.
Justin Hotard: Well, and then remember, you’re ultimately trying to get either new offers, new services or productivity out of this, which means I have to be able to trust the output of that model. And so it really speaks to this idea of scoping down and focusing the model in very specific use cases and applications.
Daniel Newman: And I apologize, I went down a rabbit hole sometimes, we’re analysts. I like to listen to myself a lot. I’m a big fan, but it is serious. We are here.
Patrick Moorhead: He’s not lying,
Daniel Newman: He’s done a lot of shows. We are here at Intel Vision and you did have a keynote session, you had demos, you talked about some new product. Justin, first time just spill. What did you announce here at Vision? What products, innovations did you all announce?
Justin Hotard: Yeah, I mean, look, I think the big news from my session was we’re excited to announce Gaudi 3. We are sampling it with customers today. It’ll be in production deployment a little bit later this year. We’ll have it in the Intel Developer Cloud available in the early in the second half of the year in a large cluster. So we’re not talking about one GPU or one node of eight GPUs, but actually 512 XPUs. And remember Gaudi, to all the points Sachin was talking about Gaudi is not a GPU, it’s an XPU.
It’s an accelerator that abstracts a lot of this work. And so you can just operate it at the level of PyTorch and have the confidence to train train models using a lot of the open source models that are out there. So back to exactly what we were talking about and the performance on Gaudi 3 is going to be a significant jump over where we were at Gaudi 2 and even better than what’s commonly available on the market today. So we’re really excited about some of the data we’re seeing off of it.
Daniel Newman: Humble, you’re not going to spin it.
Justin Hotard: No. I mean look, I’ll tell you it’s 1.7 times faster time to train. 2.3 times more efficient in inferencing. And there’s, there’s a whole spec sheet. We could probably stream-stream as we’re talking here if you’d like. We’ve got lots of data sets across many of the open source models.
Patrick Moorhead: Would you, by the way, impressive gains. And when it comes to an XPU versus a GPU, I totally get it. We’re seeing a lot in the industry. Some of the questions I get are on TCO, right? GPU-based training. I get four iterations of this out of this. With minimal work, can I get that out of a Gaudi?
Justin Hotard: Yeah, actually, I think we’re still validating some of the data, but I think we’re going to see really, really good results on reliability too. The availability we’re seeing some of the knowledge and the learnings we have from building Xeon and supporting all of our great hyperscale customers and partners over the last couple of decades has taught us how to do this really well. That’s a big part of TCO, by the way, because if I’ve got a checkpoint and restart my runs, it’s a big impact. But also just the TCO of the platform. I mean by being a XPU, it’s a lot more cost-efficient and power-efficient over an equivalent amount of work.
Patrick Moorhead: Appreciate that.
Daniel Newman: Sachin, anything you want to add before we send you up, you both back onto your day here at Intel Vision?
Sachin Katti: Yeah, no, I think Justin covered a lot of the CPU and accelerator announcements. In addition, for these big data center deployments, as you try to build out these big AI fabrics, we announced a new product category called the AI NIC. It’s an open standard spaced UEC, other ethernet consortium based, AI-optimized Ethernet fabric. And so you can use this to build really big scale-out fabrics. That’s going to be available as a discrete product, but also interestingly as both a chiplet and soft and hard IP from Intel Foundry.
So as customers come to Intel Foundry to build their XPUs or GPUs, they can take our networking technology and integrate it into their chip designs as they build out those systems. So that is a big announcement on the networking side. In addition, we talked about Intel Edge platform, which we announced at MWC, but it’s really going to what I was saying earlier, how do we deliver an easy button to the enterprise? And the edge is even more complex than enterprise IT. So diverse, so distributed. So Edge Platform is a software platform to just make it easy to deploy, manage, and run AI at the edge.
Daniel Newman: I want to call it Intelli-Band.
Patrick Moorhead: I’m thankful he’s not your branding guy.
Daniel Newman: Sachin. Justin, I want to thank you both so much for joining us here on The Six Five. Let’s have you both back soon. I have a feeling, Pat, this topic is going to be important for sometimes.
Patrick Moorhead: Of course.
Justin Hotard: Absolutely.
Daniel Newman: All right, everybody hit that subscribe button. Join us here on The Six Five for all of our episodes. We are here at Intel Vision 2024 in Phoenix, Arizona on Eclipse Day. In case you missed the funny jokes at the beginning.
Patrick Moorhead: Great Dad jokes.
Daniel Newman: They were great. They were good. They were great. Great conversation here. We appreciate you tuning in. But for now, we got to go. We’ll see you all later.
Author Information
Daniel is the CEO of The Futurum Group. Living his life at the intersection of people and technology, Daniel works with the world’s largest technology brands exploring Digital Transformation and how it is influencing the enterprise.
From the leading edge of AI to global technology policy, Daniel makes the connections between business, people and tech that are required for companies to benefit most from their technology investments. Daniel is a top 5 globally ranked industry analyst and his ideas are regularly cited or shared in television appearances by CNBC, Bloomberg, Wall Street Journal and hundreds of other sites around the world.
A 7x Best-Selling Author including his most recent book “Human/Machine.” Daniel is also a Forbes and MarketWatch (Dow Jones) contributor.
An MBA and Former Graduate Adjunct Faculty, Daniel is an Austin Texas transplant after 40 years in Chicago. His speaking takes him around the world each year as he shares his vision of the role technology will play in our future.
