Discover how Solidigm’s solutions are revolutionizing AI infrastructure. On this episode of the Six Five On The Road at SC24, host Keith Townsend is joined by Avi Shetty, Sr. Director – AI Market Enablement & Partnerships at Solidigm for a conversation on the rapid advancements in artificial intelligence and the crucial role of high-density storage solutions. With the exponential increase in computing power necessitated by AI’s growth, this discussion looks at the importance of infrastructure enabling AI applications to operate efficiently and effectively.
Tune in for more on 👇
- The evolution of AI demands and the significance of high-throughput, low-latency networks and deep learning models
- The impact of scalability problems on AI development and how Solidigm’s high-density storage solutions address these challenges
- The benefits of high-capacity QLC SSDs, such as the 122TB Solidigm D5-P5336, in accelerating time-to-results and ensuring energy efficiency.
- The unique challenges posed by generative AI applications, including data access speed, volume, and format compatibility, and how high-density storage solutions can mitigate these issues
- An exploration of Dell PowerEdge XE servers and how they are optimized for Artificial Intelligence, Generative AI, and High Performance Computing, spotlighting Dell’s comprehensive approach to AI solutions
Learn more at Solidigm.
Watch the video below at Six Five Media at SC24 and be sure to subscribe to our YouTube channel, so you never miss an episode.
Or listen to the audio here:
Disclaimer: Six Five On The Road is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors, and we ask that you do not treat us as such.
Transcript:
Keith Townsend: All right, we’re live. Show floor of SuperCompute 2024. I’m pretty excited because you can’t see it from this vantage, but we’re on the rooftop of the Dell NVIDIA AI truck. They take this thing to 44 cities to tour the AI technologies that Dell and their partners use to actually change the world. We’re talking to one of the partners today, Avi Shetty. You’re in charge of ecosystem and partnerships over at Solidigm. Welcome to Six Five On The Road.
Avi Shetty: Thank you, Keith, and it’s a pleasure to be here. As you said, a great site, amazing companies setting up the whole AI ecosystem, and great to be here on the Dell AI Factory, which is essentially a demonstration of AI on wheels to give its partners a visual representation of what goes on in an AI data center. So I’m glad to be here.
Keith Townsend: So it takes a ecosystem to solve a lot of the challenges that we face when it comes to AI.
Avi Shetty: Yes.
Keith Townsend: Can you talk to me about what you’re excited about when it comes to not just the industry changes, but the social and world changes that AI promises?
Avi Shetty: Oh, AI is revolutionary, right? We’ve done things a certain way, and we are comfortable doing that over the last decade or two, and AI fundamentally changes those assumptions. Talk about any industry, any vertical financials, medical, corporate. We have to be open to those changes where certain assumptions will be pushed to change with AI and automation coming and replacing certain things which are mundane and things. And in fact, it’ll open up to new creativity as well as new solutions, which we’ve not even thought about, especially in the medical domain as well as in the computing domain.
We work with partners across the globe and we’ve seen it’s a beginning. From my perspective, I feel the ecosystem is kind of… I give this example to my wife and my kids on where the AI ecosystem is at currently. You can think of the whole ecosystem as more like it just graduated elementary school. We have a long way to go. We have to go graduate middle school, high school, and then go do a professional degree. And that’s where the AI models, everyone’s aware of or knowledgeable of the models, and the more data it generates and more data it trains on, it’s going to be more intelligent and help us solve world problems with that.
Keith Townsend: So this takes an immense amount of computing power, and you’re partnering with Dell to help solve some of these challenges. Before we get into the exciting stuff that Solidigm goes on, talk to me about the importance of the relationship with this particular ecosystem partner, Dell.
Avi Shetty: Oh, Dell’s been a strong partner for us for a long time, and we are at the Dell AI booth, and I’d want to iterate Dell’s AI mission. Dell’s AI mission is to provide the highest performing and the most power efficient AI servers to the market. So from that end, it’s absolutely critical. You talked about performance. So the AI PowerEdge servers, one of the highest performing servers, which require high bandwidth, low latency, SSD solutions, which we partner with them on, and power. Power is a very big deal. We’ll have a lot of discussion on power later on. But having highly efficient storage solutions, which give Dell AI platform power reduction and power emission reduction opportunities is very critical because you want to feed the GPU, which is your biggest investment. You don’t want to keep it inefficient. You want to keep it maximum utilization. So anytime other components in your system utilize more power, that’s less power to the GPU. So having an efficient storage solution with Solidigm SSDs allows Dell’s customers and partners to maximize their AI data center.
Keith Townsend: So Avi, talk to me about what you’re excited about when it comes to world changing innovation that AI is going to bring, not just from the industry, but society.
Avi Shetty: Oh, the more AI data centers get built around the world, that helps with technical job creation. It brings infrastructure goodness to the local economy, whether it’s fiber optics networks or good solid power grids. And once you have a data center, you have more of the ecosystem follow them to wherever they are, which essentially fuels more growth within the local ecosystem.
Keith Townsend: So you run ecosystems at Solidigm. Talk to me about the importance of the ecosystem. This is not just a problem that a company that creates storage can solve. Talk to me about the importance of folks like Dell in this solution.
Avi Shetty: An ecosystem is fundamental to any kind of technology evolution, right? You need all partners to be on the same kind of cadence in terms of technology releases, compatibility standards. A simple example is like, “Hey, you can’t have a team just with…” I play soccer a lot, right? You can’t just have a team with 11 strikers. You need to have an offense, a midfield, a defense, and a keeper all in sync, and you will need to play as a team. And that’s the whole essence of being an ecosystem partner as well as having an ecosystem community of technology companies working together to solve a greater problem.
Keith Townsend: Now, we’re in the Dell AI Factory, as mentioned in the opening. What is the Dell AI Factory? We’ve heard that term quite a bit, if you followed AI. How would you describe it?
Avi Shetty: Oh, Dell AI Factory is, I believe, this is my second iteration being at the Dell AI Factory. I was there at a previous conference as well, and Dell’s essentially taking a representation of different solutions and giving their customers and the attendees here a visual representation of what goes on in an actual AI data center. It’s pretty awesome, right? You walk through the different stations here at the AI booth, and you can get a visual of what’s the power domain, how PowerEdge servers play in the ecosystem, as well as in the data center. And different use cases of our customers can take advantage of Dell’s solution along with Solidigm’s here at the booth.
Keith Townsend: So I’ve seen some of the numbers here. Dell biggest PowerEdge server will hold four and eight GPUs depending on the configuration. These things take a tremendous amount of power. How is Solidigm helping to mitigate some of that issue around power? I’ve seen some people talking about one megawatt racks. That’s insane. The average, typical data center rack is about only 15 kilowatts. How are we going to heat, cool and power is a lot of what these folks talking about, but how are you contributing to solving this heat and power challenge?
Avi Shetty: Absolutely, Keith. I think this is a fundamental question, is how the industry solves data center AI power requirements. There’s one study which has essentially come out. By 2030, it’s expected that data center, because of AI, will consume up to 10% of worldwide power consumption. And that’s a lot. You’ve heard all major hyperscalers and CSPs talk about their investment into nuclear energy to fuel their sustainable power demands. And look at the show floor here. There are numerous cooling companies talking about, “Gone are the days of air cooling.”
Keith Townsend: Air cooling is not even an option anymore.
Avi Shetty: Right. Yeah. HVAC air cooling, who does that anymore? Right? We are now talking liquid cooling where the coolant goes over critical hot components, reduces their power. Are we talking immersion cooling where you put the entire system in a non-conductive liquid to essentially reduce heat and dissipate more power? The point being, power is very critical. And Solidigm, where we come in, is helping our customers have efficient data center storage solutions compared to legacy storage architectures, which are purely based on hard drive, and hard drive can consume, in a data center, up to 30% of system power. And this is not just Solidigm saying. Our partners, namely Meta and Stanford, released a white paper where they looked at Meta’s AI recommendation engine and data sensitivity of where were the key bottlenecks of power. And it came back with 35% of storage power was consumed, essentially, which meant that Meta could not reroute that 30% of power to its critical GPU systems, which are more power hungry. And if you just replace a hard drive with Solidigm high-density QLC SSDs, you win back 80% of power, which would’ve otherwise gone to a hard drive, which essentially means now that you can feed back to the GPU, which is your critical investment, and maximize GPU utilization.
Keith Townsend: So talking about the relationship of GPU and storage, Solidigm has a 61 terabyte drive, I believe, and that’s nice. The biggest HDD that I’ve seen is about 32 terabytes, but Solidigm has a 61 terabyte. I’m seeing the demand on the show floor for even more dense storage. How is Solidigm helping their customers reach that level of density that they’re searching for?
Avi Shetty: Yeah. So we released a 61 terabyte. We released that fall of last year, maybe Q3 of last year, and we’ve seen tremendous demand from our customers on that. And their one ask from us was, “Hey, give me more.” And Solidigm listened, and we just launched the world’s highest density SSD, which is double of what 61 is. This is a 122 terabyte, Keith, of SSDs capacity in a single U.2 unit. We are talking 122 terabytes. You pair eight of those together and you have one petabyte.
Keith Townsend: I’m going to slip this into my jacket pocket, and I promise you I’ll give it back to you by the end of the interview. So talk to me about some of that innovation that has occurred with the ecosystem, with this new level of density that we’re seeing.
Avi Shetty: But before I talk to the ecosystem, let me give you an example of what is in this 122 terabyte, right? It’s powered by our 192 layer 3D NAND technology QLC. And compared to the rest of the ecosystem we are on, we are in our fourth generation of QLC. We were the first ones to introduce QLC to the market back in 2018. And since then, every iteration, and this was on a 64 layer 3D NAND node, and since then we’ve introduced 96, 144, 192, and this is our fourth generation QLC. And every generation we’ve listened, learned, improved, made it more efficient, and that’s just making more high density as well as low latency. We’ve added hooks into our NAND. We are fully vertically integrated, which allows us to take advantage of that capability where we make our own NAND, we make our own SOC, which is the controller, which can talk to the NAND and do creative things with it, design our own hardware, and we make our own firmware.
So a fully integrated Solidigm engineering allows to reduce all the latency to the lowest. So this drive here has high bandwidth, low latency, and industry’s best leadership capacity. And this allows our customers, many on the show floor here, to demonstrate as well as solve customer problems, whether it’s in the data center or at the edge, by having high density in a single rack with low power consumption.
Keith Townsend: Yeah, I would love to talk about some of the edge use cases, kind of what’s possible, now that I can get 122 terabytes just in a single drive out to the edge. Any edge stories that you can think of?
Avi Shetty: Oh, we have many. Let me talk about one where one of our partners is a hospital and they had their MRI or radiology machines, which were using Legacy. Again, something which required low latency and high bandwidth, because those images are pretty big, and they used to send it to the cloud, which took a long time. So what these guys did was have one of the, I believe, the Dell systems located on-prem and paired with not the 122, we had just launched the 122 last week, but a 60 terabyte high-density SSDs. And as a result, their previous, whatever they were trying to do was closer to 30 minutes. Their whole flow, it reduced down to three minutes. So that was a big TCO improvement for their use case, where we are seeing companies intelligently define their on-prem plus on-cloud architecture and take advantage of GPU cycles as well as density local to where the data is generated.
Keith Townsend: Well, Avi, I really appreciate you stopping by The Six Five live on the show floor of the Dell AI Factory on the show floor. It’s a show floor within a show floor. We have a little of turtles all the way down, as we say in the industry. You know what, I’m going to give this back, but I have my backup server that has eight, like six terabyte HDDs, and I’m looking at this thing like, “Wow, I can replace that with a desktop.” What innovation that we’re seeing here at SuperCompute 2024 with Solidigm, Dell Technologies, NVIDIA, and an entire ecosystem trying to drive technical innovation for AI as we try to solve some of the toughest problems in society. I’m your host, Keith Townsend, and for my guest, Avi, continue to watch Six Five coverage here at SuperCompute 2024.
Author Information
Keith Townsend is a technology management consultant with more than 20 years of related experience in designing, implementing, and managing data center technologies. His areas of expertise include virtualization, networking, and storage solutions for Fortune 500 organizations. He holds a BA in computing and an MS in information technology from DePaul University. He is the President of the CTO Advisor, part of The Futurum Group.
