The question of sustainability is the “elephant in the room” 🐘 as AI continues to revolutionize industry. On this episode of the Six Five On The Road at SC24, host David Nicholson is joined by Dell Technologies’ Seamus Jones, Director, Server Technical Marketing Engineering for a conversation on Sustainable AI and Dell’s initiatives towards eco-friendly technology solutions.
📻Tune in for these key takeaways:
- The importance of AI-powered efficiency & how Dell is using AI to optimize cooling and power consumption in their servers, creating a virtuous cycle of sustainability
- Liquid cooling and heat capture: Dell is exploring cutting-edge cooling technologies to minimize environmental impact and even potentially repurpose excess heat
- Right-sizing is key: Dell helps customers choose the right infrastructure for their needs, avoiding unnecessary energy waste
- The role of AI in enhancing sustainability efforts across industries
- Future trends and challenges in sustainable technology practices
Learn more at Dell Technologies.
Watch the video below at Six Five Media at SC24 and be sure to subscribe to our YouTube channel, so you never miss an episode.
Or listen to the audio here:
Disclaimer: Six Five On The Road is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors, and we ask that you do not treat us as such.
Transcript:
David Nicholson: Welcome to SuperCompute 2024, here in Atlanta, Georgia. That would be SC24. I’m here with Seamus Jones from Dell. I’m Dave Nicholson with Six Five On The Road. We’re going to talk about something that is critically important. It’s a foundational discussion underneath everything that we’re talking about here, from an AI, supercomputing, high performance computer perspective. It’s this idea of sustainability. Sustainability takes on all sorts of different angles. What I want to do, Seamus, is if you can, have us walk through Dell’s view of the sustainability lifecycle.
Seamus Jones: Yeah.
David Nicholson: What does that look like for you, starting with the design of the systems themselves?
Seamus Jones: Absolutely. From a sustainability perspective, Dell has a complete vision. It’s very important to us from the lifecycle of the products, even from inception, transport, usage, so the usage model. When we look at the usage model today, a lot of the emissions that are derived deliver greenhouse gases. Those gases are really the big concern for Scope 3 Cat 11 results. Those are based on the CSRD reporting that companies have to do for their-
David Nicholson: Okay, back up a second.
Seamus Jones: Sure.
David Nicholson: Scope what?
Seamus Jones: Scope 3 Cat 11.
David Nicholson: Okay.
Seamus Jones: It’s talking about emissions of all the products. So how much greenhouse gas emissions that are derived. When you look at the use cycle of each of the products, like your laptop, or your AI system that’s drawing 12,000-kilowatts.
David Nicholson: Right.
Seamus Jones: It’s a heck of a lot more than a laptop. On the AI platforms, they’re going to be drawing a lot more power and generating a lot more greenhouse gases. Roughly 20 times more. It’s critically important for us, from an engineering organization, to really develop platforms that are efficient, and use all the energy to power and not all the energy to cool. We bifurcate that usage. You think about the energy it takes to cool the system, because when we use compute it generates heat. We have to dissipate that heat in some way.
David Nicholson: Right.
Seamus Jones: That takes energy. One of the things that we’re trying to do in the design of the systems, in the air-cooled design, we have multi-vector cooling. We’re using telemetry data from the iDRAC to make sure that our fan cycles are spinning based on AI algorithms of the applications that are operating on the system at the time. It’s critical that they actually do that realtime, not just on a cycle. That’s why when Dell, as a system, when you spin up a Dell server, if you talk to any administrator, the first thing they do, they spike.
David Nicholson: Sure.
Seamus Jones: All the fans cycle on, so that way we know everything is operational. But, guess what? Then it comes down to a reasonable level until the workload starts to happen. As that increases and decreases, then the fan speeds and the power consumption change. We make it as efficient as possible for the usage model. One of the big things on the next generation power edge, so the 17th-generation platforms, has iDRAC10. iDRAC10 is the engine that’s driving all that telemetry data. It has double the amount of processing cores as the previous generation product.
David Nicholson: Okay.
Seamus Jones: It needs that because the AI inferencing that’s happening on the system, not necessarily the work it’s doing, but within the telemetry data, understanding, manipulating that telemetry data for the iDRAC means that we’re getting all these input points, thousands of inputs from the system criteria. Fans, temperature, thermal controls, inlet and exhaust temperatures. All that, it develops algorithms, things over time.
David Nicholson: Okay.
Seamus Jones: We can see what happens, so we know how to make those changes.
David Nicholson: Okay. It’s not just the intelligent design of the hardware itself.
Seamus Jones: Yeah.
David Nicholson: Engineering goes into that to make it more efficient. But, if I’m hearing what you’re saying correctly, you’re setting up a virtuous cycle where you’re leveraging the AI that has its own insatiable desire for power, you’re leveraging the AI to make all of this stuff more efficient-
Seamus Jones: Absolutely.
David Nicholson: At that level.
Seamus Jones: Yes.
David Nicholson: Once you’ve done the best you can to design in efficiency to the hardware, now you’re using this intelligence that the hardware is actually powering to make it more clever.
Seamus Jones: Yeah, you mentioned it earlier. Sustainable AI, it also comes from AI for sustainability.
David Nicholson: Okay, yeah.
Seamus Jones: You’re using AI to make our systems more intelligent, so that we can give customers better control. We do that through the iDRAC, we do that through open managed enterprise, we do that through AI ops, infrastructure observability, which has power managements within that cycle. As well as forecasting, to see what are your greenhouse gas emissions, and carbon footprint. I can forecast, looking at if I make this change in my data center, if I refresh these platforms. Let’s say, for example, the latest generation of PowerEdge, with the Turin CPUs. You can replace seven legacy platforms with one PowerEdge Turin-based platform. As a consequence, that generates a lot less heat. It takes up a lot less space in a rack. It’s a seven-to-one consolidation.
David Nicholson: Relatively.
Seamus Jones: Yeah.
David Nicholson: Because if you track the amount of power that we have to have going to each of these racks, that keeps going up.
Seamus Jones: Exactly.
David Nicholson: It’s this never ending … Seriously, at this point, we haven’t proven the end state of this, that it’s in fact a truly virtuous cycle. As a society, we’re having discussions about where this power’s all going to come from.
Seamus Jones: Yeah.
David Nicholson: It sounds like Dell is trying to be the best steward of everything in the middle.
Seamus Jones: Yeah.
David Nicholson: To make sure you get the best results coming out the end that are possible.
Seamus Jones: Yeah. It’s undeniable that racks that are driving up to a megawatt of power in the future.
David Nicholson: It’s insane.
Seamus Jones: We’re going that direction. Today, a lot of customers at 12-and-a-half, maybe 15-kilowatts.
David Nicholson: Right.
Seamus Jones: Even in our own data centers, some of them are 12-and-a-half, 15-kilowatts. We’re having to adapt those to be able to accommodate things like the XE9680. That’s why we’re developing platforms like the XE7740 and the XE7745. Those are PCIE-based GPU platforms, so we can fit eight GPUs in PCIE-base. It means that you can get more energy efficiency in those. If you don’t necessarily need the performance of the XE9680, with eight SXM GPUs. It’s PCIE versus SXM. The biggest thing for me is that an important feature with the Dell relationship with the customer is right-sizing the environment. We have a huge portfolio of products, and customers are still trying to figure out, “When can I do inferencing on a CPU only? When can I do inferencing on a GPU? When do I need multiple systems with multiple GPUs in a mesh to do distributed inferencing, training, or fine-tuning?”
David Nicholson: Do you think we’re at a point where people can have that rational conversation? I would argue that, a year ago, we were smack in the middle of fear of missing out.
Seamus Jones: Yeah.
David Nicholson: The only answer was, “I need the most expensive GPUs I can get my hands on. I’m going to go line up like it’s a sale at Walmart on Black Friday to try to get my hands on them. Then, I’ll figure out what I’ll do with them later.” Do you think people are willing to take a breath now, and have a rational conversation with Dell about, call it tiering their infrastructure, for lack of a better term?
Seamus Jones: We’re meeting people where their need lies. Those customers that were early adopters of that technology, they are definitely ones taking advantage of it and proposing it. Sometimes their customers, these CSPs, cloud service providers, things like that. What we’re seeing now is the midstream adoption from a standard enterprise, like a lawyer’s firm that’s saying, “Okay, I need to sort through all of these documents, and I’d like to do it on an LLM that’s an agentic RAG deployment. How best to do that in an efficient way for my state? Because I want to keep all those legal documents on-premise, and I want to a gen AI framework to be able to take advantage of all this goodness. How do I go about doing that?” That’s where we can come in, and give them options, because they don’t necessarily need the row rack and framework that the large players might be doing today.
David Nicholson: Yeah. I know you’ve been in this game for long enough to have been exposed to hot aisle, cold aisle, raised floor.
Seamus Jones: Yeah.
David Nicholson: We got to keep things cool. But come on, 10 years ago, we didn’t think in terms of thermal density as a primary consideration. I was curious maybe a year ago, and I randomly looked up how much heat do you have to dissipate when you pump a kilowatt of power, or a kilowatt-hour of power, into a GPU. The answer was basically all of it.
Seamus Jones: Yeah, yeah.
David Nicholson: With the exception of mechanical energy from the fans, you’re talking about a nearly perfect transfer of electrical energy into heat energy.
Seamus Jones: Yeah.
David Nicholson: That then has to be dissipated. It is like the wild frontier when it comes to direct liquid cooling technology, there’s all sorts of variants of it. Some come in at varying temperatures, and go out at hotter temperatures.
Seamus Jones: Yeah.
David Nicholson: How much is Dell looking at what can be done over the short term, longterm, to recover energy from now, that hot water or that hot liquid that’s coming out?
Seamus Jones: If you look at, there’s a lot of different strategies within our liquid cooling solutions. Everything from liquid air assisted cooling, everything from direct-to-chip liquid cooling.
David Nicholson: Okay.
Seamus Jones: There are different technologies, different implementations. There’s also rear-door heat exchangers, to try and understand that. You also have warm water heat capture cabinets, so that way you can capture that hot air, which is really 100% more efficient than exhausting air out of the facility. We’re actually capturing it, the hot air, recycling it back through the system. And cooling it in that process, and then recycling it. It means that what we’re able to do is … There’s a dynamic shift that’s happening in customers’ environments. These are choices that the customer has to make, not only in the data center, but in the facility. It makes a really big difference when they’re tied into facility planning. Because from a sustainability perspective, you don’t want to have a process in place that burns even more greenhouse gases. You don’t want to take that hot air or hot water, that cooling fluid that’s coming out of the back end of the server, and then taking it external to the building. Taking that with a secondary loop, going external to the building, and then having to cool that with an air conditioner. Think about that.
David Nicholson: Right.
Seamus Jones: Like your house.
David Nicholson: Right.
Seamus Jones: An air conditioner is generation more greenhouse gas emissions, so more carbon footprint. How much better would that be if the facility was able to do a cooling tower? If you’re based in Seattle, versus Austin, Texas, you can do cooling towers to just take it outside, use natural cooling of the environment, and bring it back in in inlet temperature.
David Nicholson: Yeah.
Seamus Jones: Any time you can do things like that, that are unique and different to the customer, and it’s very, very bespoke to each customer. That’s where we can help.
David Nicholson: Yeah. One thing that might be hard for Dell to test, being headquartered in and around Austin, is the idea of let’s find a day when it’s not 95-degrees and humid, so we can test these things. Did you ever think … Now everybody’s cool with the idea of liquids in the data center, which is crazy.
Seamus Jones: Yeah, yeah.
David Nicholson: Why not just have hot tubs in the data center? Why don’t we just directly heat hot tubs so IT people who have been toiling in obscurity can relax?
Seamus Jones: You laugh about that.
David Nicholson: I’m serious. I smile, but I’m serious.
Seamus Jones: In our lab, one row of our AI platforms generates the heat for the lab space that we all work in.
David Nicholson: Yeah.
Seamus Jones: Before we had installed our XE9680s, it was freezing in that room.
David Nicholson: Yeah.
Seamus Jones: It was about 15-degrees colder than it should have been. Once we installed it, it’s a comfortable space.
David Nicholson: Yeah. Well, there was a time when you would have been criticized for that being a luxury, hot tub in the data center.
Seamus Jones: Yeah, hot tub.
David Nicholson: Now, it is part of the sustainability-
Seamus Jones: It’s true.
David Nicholson: Efforts.
Seamus Jones: It would be relaxing at best.
David Nicholson: Yeah.
Seamus Jones: We’d never get engineers to leave, to be honest.
David Nicholson: I’m sure you’ve heard the phrase that if something can’t be measured, it didn’t happen.
Seamus Jones: Yeah.
David Nicholson: It’s like basic science. The difference between observation and science is writing it down. It’s all great to talk about Dell and sustainability, but if I’m a Dell customer, how do I quantify that? How do I prove to my board, who is very interested in ESG initiatives-
Seamus Jones: Absolutely.
David Nicholson: That we’re actually making progress? And progress from what? How do we baseline it, how do we measure it?
Seamus Jones: Especially at this time of year.
David Nicholson: Right.
Seamus Jones: Coming end of year, end of year results, that ESG initiative and approaching their 2030 goals, the environmental 2030 goals, a lot of companies need what’s called a PCF report, so product carbon footprint report.
David Nicholson: Okay.
Seamus Jones: Dell provides those for every product. Whether it’s a monitor, laptop, XE9680 with eight GPUs, or any of the other products within our portfolio. Those can be acquired at dell.com if they submit a ticket. Right up, based on the configuration of the system, it’ll be detailed. It’ll be following the product passport initiative. That’s a global initiative to understand what the greenhouse gas emissions on. Based on the components, the full lifecycle of the product, even right out to de-commissioning. It’s one of those things that’s understanding, okay, what’s going to be the greenhouse gas emission? That way, how are they approaching their 2030 goals or 2050 goals, depending on their current ESG initiatives.
David Nicholson: That makes an incredible amount of sense, not just for tracking things from an environmental perspective. But from a cost efficiency perspective, because ultimately these things, you’re going to be able to slice them and dice them into how much work per watt-hour of power are we generating and how much CO2. But within the data, you’re lost. I would think that that’s something that every customer should tap into, to at least get started with a baseline. What do you think the future holds? Where is this headed? Are you personally confident that this virtuous cycle is going to lead … I’ll tell you, in one of my classes that I teach, someone brought up the point that their friends who are not IT savvy were very skeptical about the idea that AI was going to deliver us from all evil. Because what’s being pitched here, we’ve got delusional optimism here at SC24. What’s being pitched is that, “Yeah, we might be doing something short term that’s not great, we’re going to have to dig a bunch of holes, and build a bunch of power plants, and do all this stuff.” But eventually, the intelligence, the insights we get from AI will drive us to a brighter, greener future.
Seamus Jones: Yeah.
David Nicholson: I believe in that.
Seamus Jones: I agree.
David Nicholson: Do you think we’re headed there?
Seamus Jones: I think today, what’s happening is that we have this requirement and need for more power. Any source of more power today to drive the AI platforms is where we need to look. If we can find longterm sustainable energy sources, then it doesn’t matter how much energy each of the system of using if it’s coming from a sustainable energy source. Nuclear, or wind. But today, we need that availability, and it’s just not there on our grids nationwide. And really, across the world. I think AI will help us deliver that. I think there’s some really unique implementations that we’ve never seen before. Things like the ability to drill into the crust of the Earth, and use evaporating air and evaporating water to then drive turbines. It’s some really interesting stuff that we’ve seen today at SC.
David Nicholson: Yeah.
Seamus Jones: The thing is, I don’t know which one’s going to take hold, which one’s going to be most impactful. It is undeniable that AI is driving this resource and a need. How we’re going to impact it, I’m not sure.
David Nicholson: Yeah. Yeah. Well, the good news is Dell typically has every horse in every race.
Seamus Jones: Yeah, true.
David Nicholson: Whichever ones do well, then Dell does well, and we all do well. I do have a final comment. I’m going to say this on your behalf, too, Seamus, so I hope you forgive me. I’m going to look directly in the camera and say ha! Infrastructure matters and is very cool.
Seamus Jones: Yeah.
David Nicholson: I know business outcomes sounds great. Yeah, we want business outcomes. Yeah, people shouldn’t care, shouldn’t care about how the service is provided.
Seamus Jones: Yeah.
David Nicholson: The fact is, correct me if I’m wrong, it’s increasingly important.
Seamus Jones: Absolutely.
David Nicholson: To get at the very foundation of these things when you’re deploying this stuff.
Seamus Jones: It’s becoming more important.
David Nicholson: That’s fun for guys like us, isn’t it?
Seamus Jones: Yeah, absolutely.
David Nicholson: Come on, come on.
Seamus Jones: It’s true.
David Nicholson: We’re a little gleeful about this.
Seamus Jones: Yeah. Yeah, this show this year’s become bigger than ever.
David Nicholson: Yeah. Yeah. Good stuff, Seamus. Seamus, Seamus Jones from Dell, thank you so much.
Seamus Jones: Thanks.
David Nicholson: Yeah. Sustainability in AI, sustainability in all things related to compute, it’s going to be front-and-center. You’re going to hear all sorts of philosophical, political, societal debates about where we get the power. I’m from California, I’m a big fan of the thermonuclear reactor in the sky that we call the sun. But I imagine it’s going to be a whole mix of other power sources.
Seamus Jones: Absolutely.
David Nicholson: For Seamus Jones from Dell, I’m Dave Nicholson, Six Five On The Road. Thanks for joining us here at SuperCompute 2024.
Author Information
David Nicholson is Chief Research Officer at The Futurum Group, a host and contributor for Six Five Media, and an Instructor and Success Coach at Wharton’s CTO and Digital Transformation academies, out of the University of Pennsylvania’s Wharton School of Business’s Arresty Institute for Executive Education.
David interprets the world of Information Technology from the perspective of a Chief Technology Officer mindset, answering the question, “How is the latest technology best leveraged in service of an organization’s mission?” This is the subject of much of his advisory work with clients, as well as his academic focus.
Prior to joining The Futurum Group, David held technical leadership positions at EMC, Oracle, and Dell. He is also the founder of DNA Consulting, providing actionable insights to a wide variety of clients seeking to better understand the intersection of technology and business.
