Pop quiz:
What is an AI factory? 🏭
David Nicholson is joined by Dell Technologies‘ Vice President of Product Management for Artificial Intelligence and Data Management, Chad Dunn on this episode of Six Five On The Road at SC24. They discuss Dell’s advancements in Generative AI (GenAI) Solutions and the unveiling of the Dell AI Factory.
Tune in for details 👇
- The emerging needs of enterprises in Generative AI and how Dell is addressing them through infrastructure, security, and sustainability
- The biggest roadblocks to Generative AI adoption, including data preparation and the complexity of coding for GenAI applications
- An overview of what’s new in the Dell AI Factory, highlighting advancements in servers, GPUs, and the expansion of the Dell Enterprise Hub
- Dell’s set of pre-built blueprints and solutions that accelerate customers’ journey to generative AI, from infrastructure to software and services
- Priority use cases for enterprises in GenAI and customer success stories showcasing the impact of these technologies
Learn more at Dell Technologies.
Watch the video below at Six Five Media at SC24 and be sure to subscribe to our YouTube channel, so you never miss an episode.
Or listen to the audio here:
Disclaimer: Six Five On The Road at SC24 is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors, and we ask that you do not treat us as such.
Transcript:
David Nicholson: Welcome to Six Five On The Road’s continuing coverage of Supercomputing 2024. I’m Dave Nicholson, and I’m here with Chad Dunn from Dell Technologies to talk about the world of generative AI, specifically from the perspective of the Dell AI Factory. Chad, welcome to the program. How are you?
Chad Dunn: Hey, Dave. I’m doing great. It’s great to be here with you.
David Nicholson: So what the heck is an AI factory? And then first define AI factory, and then let’s get into this discussion about what people are doing in the generative AI space.
Chad Dunn: Sure. Well, look, the AI factory is a set of configurations that we’ve cultivated that really accelerate the time to value for an enterprise that wants to adopt generative AI. So we go through all the hard work of qualifying the different hardware components, networking, storage servers, GPUs, including the software stacks. So when you decide to embark on your generative AI journey, you’ve got a blueprint that’s going to guide you to success when you go to deployment.
David Nicholson: So you just called out specifically generative AI. That’s good because for years Dell has been helping people with artificial intelligence.
Chad Dunn: Yep.
David Nicholson: It’s just that generative has taken all of the conversational bandwidth in the last year. But for all sorts of expert systems and machine learning, you guys have been doing this for years. What are the particular challenges or hurdles that you’re seeing folks need to overcome with generative AI specifically?
Chad Dunn: Oh man, there are a whole bunch. If you were to sort of compare this to AI that you just talked about, what we would call sort of classic AI, if you looked at the ecosystem of software partners that were out there, these were largely end to end use cases. That’s not the case with generative AI. We’ve got lots and lots of ecosystem partners that are out there adding value in all sorts of different ways, and the way that you assemble these components together to get to an outcome is incredibly varied and it allows for a lot of differentiation. But what we’re seeing in terms of challenges are enterprises who have sort of mandated that we need to find value out of AI, whether that’s specifying a use case or an outcome, a business outcome that they want to achieve. And a lot of that is happening in the lines of business within our biggest customers.
But then what we find is happening is as they get those pilots or those POCs to the point where they need to go to production, IT struggles to get them into production with a reasonable ROI. And so that’s where we try to come to the rescue with doing things like these pre-built blueprints that allow them to easily manage the infrastructure that’s going to run these AI workloads because you’ve got to satisfy that IT crowd, but you’ve also got to satisfy the AI practitioners and the data scientists that are going to be innovating inside of your company.
David Nicholson: So how do you deal with the dynamic where a customer says, hey, look Chad at, I don’t want to be doing this with someone who has never put something like this together before, when the reality is, yeah, let’s say 80% of it is stuff that you absolutely have experienced with, but to a degree we’re all in this journey together, kind of building an airplane in flight. How do you reassure folks that no matter what happens, you’ve got their back?
Chad Dunn: Hey, the good news and the bad news is none of us have done this before. This is really cutting edge technology. Now, the interesting thing that we’ve encountered here at Dell is we went all in on the infrastructure for generative AI from the beginning. So we were one of the first ones out there with the kinds of hardware that we have that can partner with people like Nvidia and AMD and Intel to support the accelerators that they built. But internally, we also had that same push to get return out of generative AI inside the four walls of Dell. And so we planted a garden of 1,000 flowers to see what would bloom in terms of pilots, and lots of them bloomed and lots of them didn’t, and lots of them couldn’t be transplanted. And so we learned a lot of the hard lessons about what it means to be able to take something from a good idea to a pilot to something that we can put into production.
And it’s sort of like crossing the chasm, right? Getting from pilot to production is always a big challenge. And that challenge could be in terms of data governance, that challenge could be in terms of IT support, ability, observability, it could be in terms of sustainability. So honestly, we took a few lumps as we did that, and we started to get pretty good at the way that we approached these POCs. We did things like saying, an AI POC should take 90 days. If we can’t do it in 90 days, it’s probably not something we should focus on right now. Right? We went through a lot of challenges and then developing a methodology for what we would do with AI, because you can do lots and lots of things, but if they don’t add to your core competency, if they don’t enhance your core competency or make you more differentiated, you probably shouldn’t do them. If you’re a software company, you probably shouldn’t bring in your own AI to do your HR system. You’re going to get that functionality for free. You should focus on building better code. And so we learned a lot of those lessons early on, and we’re really eager to share those with our customers.
David Nicholson: Okay. So a bit of Dell drinking its own champagne, which is a good thing. Then again, I hear a bit of experiments on Dell employees, the moral ambiguity there, we’ll just leave that aside, all that experimentation. But seriously, if someone is looking for a partner to move forward with these things, if they tell you today, oh yeah, we’ve got 100% of it figured out, they’re lying. So it’s refreshing to hear you say, yeah, there’s stuff we’re going to be figuring out along the way in a positive way. Because that kind of agility makes a lot of sense. Am I mischaracterizing it? I don’t want to paint you in a corner.
Chad Dunn: I would say there are probably just things that they haven’t learned yet.
David Nicholson: Yes, there you go. Yes, yes, yes, exactly. I know. Chad is like, no, Dave, no, we know, we have it figured out.
Chad Dunn: Yes.
David Nicholson: So what about from the AI factory perspective, this is a fast moving thing also. Coming out of the Supercomputing conference, and as we move forward into 2025, what’s new and what can you share with us about what’s under development from an AI factory perspective?
Chad Dunn: Oh, sure. Lots of stuff. So starting with the hardware roadmap, what we’re seeing in generative AI is the silicon roadmap and the systems roadmap is moving so fast that you’re seeing innovation on a pace that’s much faster than Moore’s Law ever predicted. So if you look at the AI factory that we have today, we have new servers like the XE-7740, the IR7000. We have new GPUs for Nvidia, the H200 NVL, the GB200. These are a lot of great letters and numbers. What they really mean is more horsepower, more iron, sometimes more power, sometimes-
David Nicholson: Sometimes more power?
Chad Dunn: Sometimes more power.
David Nicholson: Sometimes more power, sometimes more cooling. Now you’re hedging a little bit.
Chad Dunn: We’re now fully into liquid cooling is on the horizon and something we believe is going to come into the enterprise in the not too distant future. Silicon diversity, as I mentioned, we’re working with Nvidia, we’re working with AMD, we’re working with Intel. We’re working with, I couldn’t tell you how many software partners we’re working with in the ecosystem that accomplish different tasks in the AI workflow, whether it’s people like Hugging Face, where we have our enterprise hub for downloading models like Llama, Llama 3.2, like Mistral, like other partners that we’re cultivating.
But also vendors who are doing innovative things with AI. So as I mentioned, unlike sort of the classic AI use cases, there’s so many different ecosystem partners. The way that you assemble these ecosystem components lead to differentiation. So I think there’s a lot of room for our customers to really differentiate, really innovate around the solutions that they build, and we’re there to help them not only with the infrastructure and the software, but also with the services with the consulting to select the right use cases to put it together the right way and to get them up to speed on how to operate that infrastructure once they’ve got it stood up.
David Nicholson: Yeah. So 20 years ago, services from Dell would be plugging in an electrical cable, plugging in an ethernet cable, maybe wiring some storage up to a server. Here you go. I’m dramatically simplifying it, but seriously, think about it. All you need is a skew. If all I want… I want 500 terabytes of block-based storage. Okay, click these three boxes and we’ll get the crate out to you right away. It’s a little bit different when someone says, I want to put together an AI strategy. Those services are much more complex. Is that pretty much where the conversation starts? You must have teams where there’s a service person on board from the get-go, wouldn’t you?
Chad Dunn: Oh, yeah, yeah, absolutely. Yeah, we’re not crimping so many RJ45 connectors anymore. But what we are doing is getting engaged really early on with the customer and identifying what are the use cases. Because what we find is, at first, if you sort of drew the customer journey, you would say, I have data. I’m going to apply generative AI and I’m going to get to a great business outcome. It’s going to be amazing. And everybody in the C-suite goes, yes, go do it. Please report back your results. And there was a lot of messiness in the middle of infrastructure, of software, of services to put that all together. And so we’ve come in to really help pull that all together and make that a cohesive journey. But now what we’re finding… That’s sort of a last year thing. If I look at what’s happening now and what’s happening next year, we see customers who already have an idea of what that use case might be.
It might be retrieval augmented generative AI, so RAG for enterprise search or for customer support. It might be inferencing, but they have an idea of what it is, and they have an idea of what that outcome needs to be. Now, the trick is identifying those use cases that are going to be high business value and high feasibility. Because those are the ones that you want to target, right? That’s the low hanging fruit. And interestingly, what’s feasible today is going to be different than what’s feasible in six months with the rate at which this ecosystem and the technology is progressing. So starting out with a really robust consulting service to understand the results that you want to get and the path to get there is key to every single generative AI opportunity that we work on.
David Nicholson: Yeah, yeah, it makes sense. And I’m curious, when you’re engaging with customers today, do you feel like they have, at least on the IT side of things, have they gotten down into the weeds they might have been 10 or 20 years ago when it comes to specifying the bits and the bobs that the pieces of the puzzle? The reason why I ask that is I would say five years ago we were in this age where hyperscale cloud providers and even Dell offering hybrid cloud solutions, kind of the mantra was, don’t worry about the hardware, let’s talk about the outcomes. Let’s talk about the service that you want out of this. We’ll figure out on the back end which little Lego bricks need to be assembled. We’ll make sure that you get the outcome. You don’t need to worry about the brand names of the parts.
I guess the elephant in the room is we have Nvidia dominating this space, but you guys are the Switzerland when it comes to AI. I know that you’re doing work with Intel GPUs, AMD, and a whole host of other accelerator vendors for specific things. Do you have a challenge with customers who are basically telling you what they think they need as opposed to trusting your judgment on the back end? What does that look like?
Chad Dunn: It’s really funny that you asked that because I happen to be sitting at our office at One Penn Plaza in New York City and some of our most valuable customers and some of our most technically astute customers are within a one-mile radius of here. And some of them would mine their own silicon if they were given the opportunity. But certainly we’re seeing, as I mentioned, that chasm between getting from pilot to production. And there is a place there where IT does need to be involved because we can certainly give you the whole package of the infrastructure, the operating system, Kubernetes, the ML and LLM Ops layer, the applications for inferencing and RAG that sit on top, we can do all of that. But if IT can’t support it reliably in just the way that they support the rest of their IT infrastructure, it’s a non-starter. Right?
So if they can’t authenticate it, if they can’t protect it, if they can’t back it up, if they can’t monitor it, if they can’t audit it, then it’s not going to get into production. So they’re right. They need to understand that, yes, there’s some amazing magic that’s going to happen here at the generative AI application layer, but unless IT can support that full infrastructure, it’s useless to them. Now, the interesting thing is we’ve been very successful with customers like tier two cloud service providers who are largely greenfield opportunities. So they need hardware and they need some level of software, but they’re going to add their value add on top of that. But if I go into an existing enterprise, I all of a sudden have to worry about all the things that enterprises are supposed to worry about, security, auditability, observability. So we really do have to cater to that sort of a need within these customers. And I think that’s part of the challenges as we as vendors, and not just us, but other vendors move from service providers into the enterprise space.
David Nicholson: Yeah. I’m not sure if anyone told you beforehand, Chad, but we’re actually recording this. And so I would like to take the opportunity to ask you for what your predictions are for the year ahead. We’re in the waning weeks of 2024. What are your predictions for 2025? I mentioned that disclaimer that just to remind you that it is being recorded so we can come back and maybe humiliate or celebrate you a year from now depending upon what goes on. So give me a few legit predictions and then I want to hear your sort of craziest one that you almost cringe about when you say it, but I want you to say it anyway. If you’ve got some wild predictions like what the craziest thing might be. So hit us with what’s going to happen in 2025.
Chad Dunn: Well, look, I think for our customers, this is going to be the year of ROI for generative AI. Right? So companies are going to have to start to get serious just as Dell did about when they embark on a generative AI pilot. There needs to be an ROI there that needs to be proven and they need to be serious about it. So I think that the bar is going to be much higher for investment in generative AI pilots. I think that from a technology perspective, we’re going to see agents and agentic AI start to become mainstream. We’re starting to see the beginnings of that right now. But today, if I look at generative AI, it’s sort of a question response. And as we sit here a year from now, if we happen to have the good fortune to talk again, it’s going to be provide input into a series of agents and let them make decisions about what your intent is about the answer that you want to get. And so I think we’re going to see these systems become much, much more advanced.
I think we’re going to see models get smaller and more efficient. We’re already starting to see some of that. We see large language models of multi-billions of parameters, and that’s great. But in some use cases, you want a really small, really efficient model that’s really focused on the data that you want to process. And I think that’s going to be a big trend that we see in the coming year. And you asked me about something that was sort of crazy that I’ll cringe at if I hear a year from now.
David Nicholson: Take a chance.
Chad Dunn: I’m going to be really esoteric because this was a weird moment for me. If you’ve ever watched season three of Westworld when people could largely do anything by talking to their phone and saying, rent an apartment, purchase this, do these things, I think you’re going to start to see things like that. And I think that all of us vendors are wondering, when is AI going to start to negotiate prices between us and our customers and our suppliers, and really take all of that away from us and become a perfectly efficient economy? And we’ll all be perfect capitalists at that point.
David Nicholson: Yeah, no, it’s coming so quickly. I have to jump in with one. And that is, although I fully expect the McLaren Formula One team sponsored Dell Technologies to take the Constructors’ Championship next year, I still want Lewis Hamilton to win the Driver’s Championship driving for Ferrari. So sorry. So you got me half papaya-
Chad Dunn: I hope so. But for the record, my son Evan, is a huge Red Bull fan.
David Nicholson: Is he? Oh, so there you go. So you’re a house divided already.
Chad Dunn: We are a house divided.
David Nicholson: In a way. Chad, it’s been great to talk to you about AI, generative AI, Dell’s AI Factory solutions. We definitely will come back and chat about this next year and see where we’ve got it. Thanks for tuning in to this continuing coverage by Six Five On The Road of Supercomputing 2024.
Author Information
David Nicholson is Chief Research Officer at The Futurum Group, a host and contributor for Six Five Media, and an Instructor and Success Coach at Wharton’s CTO and Digital Transformation academies, out of the University of Pennsylvania’s Wharton School of Business’s Arresty Institute for Executive Education.
David interprets the world of Information Technology from the perspective of a Chief Technology Officer mindset, answering the question, “How is the latest technology best leveraged in service of an organization’s mission?” This is the subject of much of his advisory work with clients, as well as his academic focus.
Prior to joining The Futurum Group, David held technical leadership positions at EMC, Oracle, and Dell. He is also the founder of DNA Consulting, providing actionable insights to a wide variety of clients seeking to better understand the intersection of technology and business.
