In this vignette of The 5G Factor, Ron Westfall and Olivier Blanchard discuss why they see Qualcomm’s portfolio as integral to optimizing generative AI models across the global digital ecosystem through edge device optimization. The discussion highlighted:
Qualcomm Spotlights Benefits of Running Gen AI on Devices. We see the cost of running generative AI models on device versus the cloud translates directly to the amount of power required to run these models. Edge devices with efficient AI processing can deliver leading performance per watt, especially when compared with the cloud. Edge devices run generative AI models at a fraction of the energy, especially when considering not only processing but also data transport. This difference is significant in energy costs as well as helping cloud providers offload data center energy consumption to meet their environmental and sustainability goals. We examine why on-device processing performance of mobile devices has increased by double-digits with each technology generation and is projected to continue this trend, and how Qualcomm’s portfolio is integral to optimizing generative AI models across the global digital ecosystem through edge device optimization.
Watch The 5G Factor show here:
Or, you can watch the full episode here, and while you’re there, subscribe to our YouTube channel.
Listen to the full episode here:
If you’ve not yet subscribed to The 5G Factor, hit the ‘subscribe’ button while you’re there and you won’t miss an episode.
Disclaimer: The Futurum Tech Webcast is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors and we ask that you do not treat us as such.
Transcript:
Ron Westfall: What are you seeing in terms of on-device generative AI developments? What do you see that’s, basically, standing out and jumping out at you?
Olivier Blanchard: Well, it’s going to be a huge push in the next year, for sure. And I think it’s going to be an opportunity for some the silicon manufacturers to differentiate themselves from the rest of the market, especially in the high ends. But I think I want to preface all this with just a reminder that AI, not necessarily generative AI, but AI, an extremely powerful and versatile AI has lived on-devices for quite some time. We’ve seen it with on-device translation, like live translation with Qualcomm, SoCs on mobile phones for the last few years. You’ve seen it, even if you haven’t necessarily understood that it was AI, pretty much every camera and image editing software, whether it does something automatically or helps you do it with manually added filters, all of that is controlled by AI. Even power consumption and optimization on your phone, on your devices has been handled by AI.
So AI and GPUs have been getting a lot more high performance on phones and other devices for quite some time. So the trend has been moving in that direction for the last three or four years, at least, where it’s been noticeable and where specific designs and functionality has been focused towards building out this on-device AI capability for a lot of things. So what we’re seeing now with generative AI is that we’ve essentially taken the next step in the evolution of AI. And generative AI means a lot of things to different people. On the one hand, you have the large language models, the GPTs, but there’s a lot of other stuff that falls into the category of generative AI, whether it’s creating images based on text prompts or enhancing images or essentially, creating emails out of just a few key parameters or tasks.
It can be a lot of different things. It can be personalization, it can be an onboard digital assistant that teaches itself to understand your needs and anticipate them. So you still have a need for cloud-based generative AI infrastructure, but a lot of the applications that involve generative AI, whether it’s just an element of the application or the core of the application, can live on devices, whether it’s your watch, your phone, your laptop, whatever else.
Ron Westfall: Right on. And that, I think, was very valuable, the fact that this has been going on for several years. So AI had this tremendous breakthrough earlier this year, naturally, when Microsoft, basically, came out and demonstrated, hey, generative AI can definitely transform the search experience, for example, and ChatGPT as being a prime example. But it’s more than that. Already, your device is supporting millions of parameters in terms of AI capabilities and it’s, potentially, going to hit a billion, if not already. So that’s just one example. But it’s also about enabling, for example, coding and other applications.
So this is something that is definitely going to be dynamic for quite some time, and it’s also very exciting. And so I think this is something that people need to, I guess, gain more context on. It’s not just about AI training, AI clusters, using massive GPU clusters, it’s also about its fact that it’s highly distributed. It’s relying on a fabric, basically, to be able to execute both the training, which is intensive and can be cloud centric, but also, it can definitely be offloaded or shared or distributed on devices. And that, I think, is going to be a major difference maker for the reasons we touched on, energy efficiency gains, flexibility, security and so forth. So-
Olivier Blanchard: One thing that you mentioned about security, I think it’s one of the… It’s not, necessarily, the sexiest aspect of this. People look for generative AI on-device in terms of productivity and creativity. They don’t necessarily think about security, but security is going to be a huge thing because depending on your query, depending on even just the type of generative AI that you’re talking about, for instance, your device has a digital assistance that is getting better and better and better at predicting your needs, at understanding your needs and understanding your purchasing habits, at making purchases for you, creating lists, for example, of a shopping list, whatever it is. A lot of that stuff doesn’t necessarily… first of all, doesn’t need to live in the cloud period because there’s no need to offload it to the cloud when the device can handle it itself. But also, by keeping a lot of this stuff on device, it protects it better.
And the devices themselves are getting much better at creating security layers for the data that’s being contained and processed on the device. So that’s, I think, a huge understated aspect of on-device generative AI and AI, in general. I think another one too is going to be personalization, which goes hand in hand with that. As you use generative AI on your device and through your device, having the ability to keep all this stuff contained, all of your queries, all of your preferences, all the things that it knows about you, that it understands about your process, about your work, about the type of work that you’re doing through generative AI, all of that stuff staying on the device is a huge plus. And then the last thing is lag, plain and simple. There are times when you’re not going to have access to the internet, for whatever reason, or it’s going to be choppy, you may have to go into airplane mode.
There’s so many different reasons why, if you’re in a busy airport that doesn’t have 5G or millimeter wave and it’s just, there’s a bottleneck, you need to be able to not lose productivity, not lose the ability to use generative AI with your devices. And this includes laptops, it’s not just phones. And so having the ability to have a lot of that processing on the device as opposed to exporting it to the cloud or talking to the cloud is a huge plus productivity-wise, because you’re not going to be sitting there waiting for something to happen. And even with some simple processes like chatbots, the ability to have a conversation at the pace of conversation, as opposed to waiting for the queries to be interpreted and just come back through a connection from the cloud, the user experience and the productivity gains are going to be massive.
So there’s a huge push to put this stuff on device or to put as much of it on device as possible, just because it’s more practical to do it. And thankfully, because chip makers have been working on putting a lot of AI on chip for years now, and they’ve become very good at it, the transition from traditional AI on-device to generative AI on-device isn’t particularly difficult, not in the way that it would’ve been three years ago. So it’s actually a natural transition for them.
Ron Westfall: Yes. And I mentioned Qualcomm, I believe it’s an opportunity for them to capture mindshare in this all important space. And I think we can wax eloquent the entire episode, just on gen AI on-device. And that, I think, is, clearly, a takeaway, that this is something that is already having tremendous impact. We’ll jump on this again, certainly, over the next month plus. And so that, I think, is something that will definitely be moving the needle in terms of what we’re going to be paying attention to.
Other insights from The Futurum Group:
5G Factor Video Research Note: Qualcomm: The Future of AI is Hybrid
Qualcomm’s Snapdragon Digital Chassis Will Power the 2025 Escalade IQ
MediaTek Announces On-Device Generative AI Powered by Meta’s Llama 2
Author Information
Ron is an experienced, customer-focused research expert and analyst, with over 20 years of experience in the digital and IT transformation markets, working with businesses to drive consistent revenue and sales growth.
Ron holds a Master of Arts in Public Policy from University of Nevada — Las Vegas and a Bachelor of Arts in political science/government from William and Mary.
Olivier Blanchard is Research Director, Intelligent Devices. He covers edge semiconductors and intelligent AI-capable devices for Futurum. In addition to having co-authored several books about digital transformation and AI with Futurum Group CEO Daniel Newman, Blanchard brings considerable experience demystifying new and emerging technologies, advising clients on how best to future-proof their organizations, and helping maximize the positive impacts of technology disruption while mitigating their potentially negative effects. Follow his extended analysis on X and LinkedIn.
