AI Democratization: CPU-Based AI for Retail – Futurum Tech Webcast

AI Democratization: CPU-Based AI for Retail - Futurum Tech Webcast

On this episode of the Futurum Tech Webcast, host David Nicholson is joined by Dell TechnologiesMohan Rokkam and Scalers AI‘s Steen Graham for a conversation on how CPU-based AI technologies are revolutionizing the retail sector.

Their discussion covers:

  • The current landscape of AI in the retail industry
  • Benefits of CPU-based AI over other AI models
  • Real-world applications of CPU-based AI in retail
  • Challenges in implementing AI at scale in retail settings
  • Future prospects of AI democratization in the retail sector

Learn more at Dell Technologies and Scalers AI. Download our related report, Dell CPU-Based AI PoC for Retail, here.

Watch the video below, and be sure to subscribe to our YouTube channel, so you never miss an episode.

Or listen to the audio here:

Or grab the audio on your streaming platform of choice here:

Disclaimer: The Futurum Tech Webcast is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded, and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors, and we ask that you do not treat us as such.


Dave Nicholson: Welcome to the Dell Experience Lounge. I’m Dave Nicholson, Chief Research Officer with The Futurum Group. I am joined today by Mohan Rokkam. Mohan, what do you do at Dell, specifically?

Mohan Rokkam: I’m a Technical Marketing Engineer.

Dave Nicholson: Fantastic. TME. Steen Graham, CEO of Scalers AI. Give us the 30 second elevator pitch on what Scalers AI does.

Steen Graham: Scalers AI was founded to help enterprises fast-track their transformation with AI. And so what we’re talking about in retail today is how do we actually transform retail, improve guest experiences, help retailers with their top line and their bottom line outcomes and so that’s near and dear to the heart of what we do at Scalers AI.

Dave Nicholson: So you mentioned it retail, AI in retail. Mohan, what did you put together here as a reference implementation that’s interesting for folks who might be in retail and what was the point? Why did you do this?

Mohan Rokkam: Yeah, no, so one of the things is Dell as a company, it works with a lot of large vendors and customers, right? And retail is a pretty big customer for us. We have a lot of servers across various large retailers, and one of the things we were thinking of is how can we make their lives better, especially with AI? With AI and it’s easy to talk about GPUs and CPUs. We were thinking, okay, how can we make sure it’s easily accessible? How can we solve a problem that they have today and come up with a use case that they can go change and put many more implementations around?

So the thing we came up with was around computer vision, recognizing images, doing work with those images that are now recognized. And one of the problems with retail was inventory, right? You want inventory to be fresh, you want your shelves to be fully stocked, but you don’t want too much inventory because then you have to worry about expiry dates. You worry about there’s too much cost stock on your shelf. So to solve that problem, we say, okay, there are cameras everywhere in your stores nowadays anyway. Can the cameras see what’s on the shelf and measure the stock situation? If a stock of something is going low, hey alert, go ahead and get that refilled, right? You don’t have to have somebody walk the shelves every day, 20 times a day to figure out what systems. So that’s the key thing.

The second part of it was how can we make it easier to access? Do we have to put this really complicated solution or can we put it on existing servers, leverage existing processors, really? And it’s a pretty fantastic solution. We can handle around 20, 30 streams of video easily.

Dave Nicholson: Okay. So that’s where I want to start is at the edge of this thing. We’re going to talk about kind of the infrastructure that’s running this, but out at the edge, you have cameras that are gathering information. What are the concerns and constraints associated with something like that? These sensors at the edge, they’ve got to pump a lot of data back, correct?

Steen Graham: Absolutely, yeah. You’ve got a full video pipeline workload. A lot of people when they think about AI at the edge, they think, okay, well the model’s going to be the biggest part of the workload. But actually the video pipeline in many cases is a much more sizable workload. So ingesting, decoding that video becomes really heavy workload for that equipment. And I think there’s a myth in the marketplace that the leading epic CPUs aren’t up to the task of running that full AI-based pipeline and workload. And I think what we wanted to do here, what Mohan really want to show is with that leading edge epic infrastructure, you can run full video pipelines with AI and you can do it to transform a business like a retail environment and solve… One of their biggest problems was having the right product at the right place at the right time.

Dave Nicholson: So we’ve been talking in other segments in this series about this idea of the democratization of AI and Dell’s role in that. One of your partners, Broadcom, Charlie Kawas talks about this idea that this concept of the XPU and the idea that it’s not just about GPUs, it’s about all of these different processor units that can be plugged in, sort of fit for function, along with of course the importance of networking in this solution. What was the networking that you used in this solution?

Mohan Rokkam: In this solution, we used 100 gigabit ethernet based off of Broadcom’s network infrastructure. The cameras will be plugged into a switch eventually, which is then into our servers. And within our own server infrastructure across the multiple machines, which are scalable by the way, you have a decoding machine, an AI machine, and of course a visualization machine. You can also take that same video stream because it’s a retail store for store footage is going to be saved off. You can save it off to your storage device in the back end for your disaster recovery or for any shoplifting concerns, anything else that’s being done. So for that whole solution, it’s easy to put in 100 gig ethernet, right now that’s with PCI Gen 4 and Gen 5 coming up. It’s very easy, very straightforward, and in the big grand scheme of things, relatively inexpensive even, to put this infrastructure together and get it running.

Dave Nicholson: In the retail space for an implementation like this, retailers have cameras, maybe the cameras don’t have 100% coverage of all inventory because they weren’t necessarily designed to look at the inventory, even though they kind of were. Because if it’s for security, they want to make sure they can see the things that people could steal, right? Is this a good example of the fact that this whole internet of things that we’re immersed in, these sensors are out there, they’re generating data, but the value of that data hasn’t been completely realized. Is AI going to take us there?

Steen Graham: Yeah, I think that’s a great example. I think really ultimately why AI is so important, or deploying deep learning at the edge is so important is because the half-life of that data is much shorter than the data that sits in the cloud. The cloud data is all kind of curated information and insights. And so knowing that a customer is there right now, when you have an item that’s unavailable, that it looks like they’re looking to buy, is very important data at that moment in time.

Knowing that the next day that they were there looking for that, lot less interesting. So the half-life of that data is particularly short. And so it’s very important to have deep learning at the edge, to be able to provide those insights as well. And I think what’s unique is Mohan identified is, we were able to build a highly scalable distributed infrastructure here via the 100 gig ethernet, so we can kind of execute multiple steps and have a kind of a purpose-built environment like you might see in the cloud, but really at the edge. And so that scalable cloud-like infrastructure at the edge is a nice hack to help that latency requirements and ensure uptime as well.

Dave Nicholson: So in getting ready to talk to you about this, my first thought was, well wait a minute, we have point-of-sale systems. We can keep track of inventory that way. And a colleague gave me an example that I thought was fantastic for a solution like this, and that is, the big box hardware store, where inventory says, we have six of them, and you walk over to the shelf and they’re not there. Maybe they need to know that not only is it in the store, but it’s actually on the shelf because there’s a difference between something that is on the shelf and in a crate, three stories high around the corner. I’ve heard that. Well, what’s surprised you about what you were able to do with this infrastructure?

Mohan Rokkam: Okay, yeah, you have GPUs that can handle 5, 10, 15 streams of video. How many streams do you think a CPU can really handle, and what is the impact of that, right? So in this particular case, we did this from a relatively mid-range, 32 core CPU. We got around 20 to 30 streams of video being handled at 25 frames per second. And then the question came up, do we really need to be doing 25 frames per second to just monitor location and presence of this stuff? That’s the part which I thought was really unique here, which surprised me how much you could scale with relatively low-end or mid-range infrastructure.

Steen Graham: Yeah, I think what’s important here too is as a retailer that’s looking to innovate, you might be interested in inventory management, but you might work with some big brands that really care about planet grant compliance. And so that’s another kind of highly scalable computer vision use case. Or you might want insights like, what’s my guest count? What’s my unique guest count or what are their wait times in different areas within the store? And so really the domain experts in retail know their challenges the best, but if we can show them that common off-the-shelf infrastructure, structured in the right scalable way across the Power Edge portfolio can allow them to unlock all these use cases that, as Mohan said, they might have this server in the back room anyway, just running their data warehousing infrastructure for their inventory or their building sales system. But now we can add all these great modern AI use cases to help them transform their business.

Mohan Rokkam: And one of the things which at Dell we’ve talked about is how can we leverage the power of data that you already have? How can we make that data work for you? Things like lost customer detection, right? Somebody’s walking in a store and they’re not going to find what they’re looking for. Okay, can we tag them and send somebody across to go find them, the inventory story. All these are video-based solutions that are easily implemented just on their existing infrastructure or with a mild upgrade.

Dave Nicholson: So is that one of the points here, this democratization of AI that we’ve been talking about?

Steen Graham: Yeah, and I think, I mean, retail margins not fantastic. So how would we show them that they can get off the shelf infrastructure and drive transformation in their business to help them optimize their margins and drive up their guest experiences? So we hope you don’t go to Amazon again next time you walk in and walk out of a store. So that’s really what we’re trying to help them with there.

Dave Nicholson: Yeah, and who but Dell to talk about optimizing margins and helping people get fit for function. This has been a great session, guys. Thanks. Thanks so much. Again, Dave Nicholson from the Futurum Group here in Dell’s Experience Lounge. I feel loungy and comfortable just hanging out here, just saying it, lovely.

Steen Graham: I want to get one of those alien-

Dave Nicholson: Round Rock, I know we’re going to go and we’re going to game later. Thanks for being with us. Tune in soon.

Author Information

David Nicholson is Chief Research Officer at The Futurum Group, a host and contributor for Six Five Media, and an Instructor and Success Coach at Wharton’s CTO and Digital Transformation academies, out of the University of Pennsylvania’s Wharton School of Business’s Arresty Institute for Executive Education.

David interprets the world of Information Technology from the perspective of a Chief Technology Officer mindset, answering the question, “How is the latest technology best leveraged in service of an organization’s mission?” This is the subject of much of his advisory work with clients, as well as his academic focus.

Prior to joining The Futurum Group, David held technical leadership positions at EMC, Oracle, and Dell. He is also the founder of DNA Consulting, providing actionable insights to a wide variety of clients seeking to better understand the intersection of technology and business.


Latest Insights:

Organization Seeing Success with RISE and GROW with SAP
Keith Kirkpatrick and Daniel Newman at The Futurum Group, share their insights on SAP’s Q2 2024 results fueled by strong cloud revenue growth, significant customer wins, AI integration, and strategic restructuring efforts.
Nivas Iyer, Sr. Principal Product Manager at Dell Technologies, joins Paul Nashawaty to discuss the transition from VMs to Kubernetes and the strategies to overcome emerging data storage challenges in modern IT infrastructures.
Shimon Ben David, CTO at WEKA, joins Dave Nicholson and Alastair Cooke to share his insights on how WEKA's innovative solutions, particularly the WEKApod Data Platform Appliance, are revolutionizing storage for AI workloads, setting a new benchmark for performance and efficiency.
The Futurum Group team assesses how the global impact of the recent CrowdStrike IT outage has underscored the critical dependency of various sectors on cybersecurity services, and how this incident highlights the vulnerabilities in digital infrastructure and emphasizes the necessity for robust cybersecurity measures and resilient deployment processes to prevent widespread disruptions in the future.