The News: At its annual Snapdragon Summit this week, Qualcomm welcomed in the era of on-device AI with the launch of its premium tier, next-generation platforms for Windows 11 PCs and mobile devices – both designed to enable on-device generative AI experiences.
Snapdragon Summit is Qualcomm Technologies’ annual premier event in Hawaii in which the company launches its next-generation platforms and showcases the technologies that will power upcoming consumer devices such as flagship PCs and mobile phones. Read more about Qualcomm’s on-device AI announcements on the Snapdragon Summit’s event hub.
Qualcomm Raises Bar for On-Device Generative AI at Snapdragon Summit
Analyst Take: One of the biggest tech megatrends to watch as we head into 2024 is the enablement of AI (and particularly generative AI) in devices such as PCs and mobile phones. On-device AI is not new. For years now, discrete machine learning (ML) and AI features have spread to virtually every critical feature: On-device AI already helps optimize power efficiency; manage connectivity; add depth and clarity to photos, videos, and gaming; improve noise cancellation and audio; help personalize experiences; and more. In the past few years, less discrete on-device AI features such as speech-to-text transcription, live translation, and voice-prompted digital assistants have also become standard user experience (UX) staples across a multitude of devices. But generative AI, given its significant compute requirements (especially when it comes to training large models), has become the next frontier in the evolution of on-device AI.
As Qualcomm has been leading the race toward bringing advanced on-device AI features across its Snapdragon ecosystem for years, it is not at all surprising that the new Snapdragon platforms introduced by Qualcomm at its annual Snapdragon Summit this week showcased impressive generative AI capabilities and processing speeds for mobile and PC applications.
Snapdragon 8 Gen 3 Brings On-Device Generative AI to Mobile
Starting with the obvious, Qualcomm’s new Snapdragon 8 Gen 3 mobile SOC features the most powerful neural processing unit (NPU) yet in a mobile device to leverage the power of AI to enhance not only camera, gaming, sound, and connectivity experiences but also content creation for premium smartphones. Broadly, Qualcomm’s new AI Engine was developed with on-device generative AI in mind. It enables generative AI-powered virtual assistants, text, voice, and image inputs to enhance creative content and intelligent responses and create a foundation for AI-based use cases that run on-device.
This functionality is a massive benefit to users on three critical fronts: speed, privacy, and reliability. Speed because the more AI processing can happen on-device, the faster the outcome because connecting to the cloud and sending packets back and forth is not necessary. Privacy because the more data and prompts remain on the device, the less chance there is that they will be intercepted by unauthorized third parties. Reliability because every process that runs on the device does not require a network connection, so connectivity interruptions will not affect those AI-powered tasks.
This feature is also Qualcomm’s first AI Engine to support multimodal generative AI models, including popular large language models (LLMs), language vision models (LVMs), and transformer network-based automatic speech recognition (ASR) up to 10B parameters—entirely on-device. (Snapdragon 8 Gen 3 can run Meta’s Llama 2 and Stable Diffusion.) LLMs can reportedly run up to 20 tokens per second, enabling on-device personal AI assistants to respond faster and more organically to prompts. Qualcomm reports up to 98% faster and 40% performance per watt (PPW) improvements to its Hexagon NPU performance, and up to 3.5x AI performance improvements to its Sensing Hub (based on 7B Llama 2). The new AI engine also delivers the world’s fastest stable diffusion on a mobile device, enabling users to generate an image in under a second. This speed will be useful for content creators looking to quickly generate multiple options for a visual concept or for a social media post. A bevy of other Snapdragon 8 Gen 3 AI features were also announced. Look for a more detailed overview in a separate article dedicated to Snapdragon 8 Gen 3.
Snapdragon X Elite Brings On-Device Generative AI to PCs
This year’s event also featured the launch of Snapdragon X Elite, a powerful new PC platform designed to outpace competing laptop CPUs and reset expectations for on-device AI performance on PCs. The Snapdragon X Elite platform, which features Qualcomm’s custom integrated Oryon CPU, reportedly delivers up to 2x faster CPU performance versus its closest competitor and matches its peak performance at only a third of the power. (Planting its flag in the benchmark sands, Qualcomm reports that it beats Intel’s i9-13980Hx in single threaded performance and matches competitor peak performance at 70% less power consumption. The Snapdragon Summit features benchmarking sessions before the end of the week, so attendees will likely be given the chance to verify this themselves.)
And because Snapdragon X Elite is built specifically to handle AI tasks on-device, it can run generative AI models with over 13 billion parameters by itself and boasts 4.5x faster AI processing power than competitors. Snapdragon X Elite is built from the ground up for on-device AI with an industry-leading NPU along with best-in-class CPU performance and efficiency among Windows 11 PC platforms.
Ecosystem Shift: How On-Device AI Will Spread to All Platforms (Not Just Snapdragon)
Qualcomm also unveiled a new platform for Snapdragon Sound (S7) that uses AI for advanced noise cancellation; and Snapdragon Seamless, which enables Snapdragon devices to work seamlessly together across manufacturers and operating systems. CEO Cristiano Amon also showed how AI will have a profound impact on device UX moving forward, and how Snapdragon’s on-device AI capabilities across a broad range of consumer electronics categories is yet another key differentiator for Qualcomm.
“We are entering the era of AI, and on-device generative AI will play a critical role in delivering powerful, fast, personal, efficient, secure, and highly optimized experiences,” said Amon. “Snapdragon is uniquely positioned to help shape and capitalize on the on-device AI opportunity, and you will see generative AI going virtually everywhere that Snapdragon goes. Our years of AI development, including the most powerful combination of CPU, NPU, and GPU available in devices, and our support of all leading models running natively, means we can bring the benefits of generative AI to users worldwide and across multiple device categories. The partner support we have at Snapdragon Summit is a testament to our standing in the industry as an on-device AI leader.” Partners including Microsoft, Meta, Google, and multiple others were on hand to show their support of on-device AI at the Summit. Devices powered by these platforms are expected to start hitting the shelves in late 2023 and early 2024.
Disruption Diaries: On-Device AI’s Impact on Mobile and PC Markets Puts New Pressure on Apple Silicon and UX
Although the most advanced and high-performing on-device AI tasks will initially be featured in premium-tier devices, it is not difficult to see how on-device AI capabilities are already creating a UX course correction for device chipmakers and their OEMs. As with every other wave of disruption, today’s premium features will become tomorrow’s basic features, and so we can expect many of those on-device AI features to trickle down to lower-priced tiers in the coming years. What this means for the industry as a whole, though, is that on-device AI is about to become an expected feature set for PCs, mobile phones, audio components, extended reality (XR) headsets, and more. And because on-device AI (and especially the generative kind) is going to quickly become a key market differentiator for OEMs looking to maintain – let alone gain – market share in an increasingly competitive landscape, it is going to become especially critical for device OEMs to choose their semiconductor partners well or risk being late to the on-device generative AI party and losing relevance.
On the one hand, this opportunity is significant for Qualcomm to stand out, primarily in the mobile market, where competition from Apple’s own silicon (and to a lesser extent from legacy Android competitors) made Snapdragon’s “traditional” performance advantages less relevant to consumers than they ought to be. For premium handset OEMs that might have been seduced by the lure of cutting costs at the expense of tiers of performance that significant slices of mobile users might not ultimately care that much about, Qualcomm has a compelling counter: CPU and GPU performance is not just about cameras, media, productivity, and connectivity anymore. It is also about enabling enterprise-class generative AI use cases directly on devices – not only that but doing it better and with a much better thermal envelope than everyone else. I feel that this new layer of on-device AI capabilities protects Qualcomm’s competitive flanks while simultaneously giving its Snapdragon tech stack juice to push on ahead into 2024 with a real boost in UX value for end users.
On the other hand, Qualcomm’s advances in on-device AI capabilities seem to present a significant challenge for Apple, whose own AI UX continues to feel dated. Whether Apple either failed to anticipate the scale and speed of the market shift that on-device AI might drive or deliberately chose to dismiss it for now, perception is reality: The market’s questions around the possibility that Apple might have fallen behind in AI are growing louder by the day, and Apple does not seem to have a counter yet. Even if we put aside Siri’s limited performance compared with competing assistants (a nagging UX black eye that should be easy for Apple to fix), I still do not see enough of a push at Apple to prioritize on-device AI. Perhaps that does not matter. Perhaps Apple is on its own product development timeline. Perhaps Apple’s next generation of chips will address this market vulnerability. Apple’s acquisitions and spend focused on AI certainly indicate that it takes AI very seriously. The two-part question I find most interesting is: Is Apple playing catchup? And if yes, how fast can Apple catch up to Qualcomm?
In the PC space, where Qualcomm is still a relatively new entrant, the competitive dynamics look different, but the fundamentals remain the same: on-device AI signals a major sea change for PCs. This reality gives Qualcomm an opportunity to push into the PC market with a pretty solid differentiation story against incumbent chipmakers such as Intel and AMD (and more recently Apple).
While Apple’s silicon continues to both impress and provide more velocity and agility to the company’s product development trajectory, Snapdragon’s on-device AI specs feel like a bit of a shot across Apple’s bow. Nothing for Apple to worry about yet, as Snapdragon’s PC market share is still relatively small, but Apple’s competitive vulnerability when it comes to AI feels all the more exposed now that Qualcomm has shown what it is bringing to the PC market.
While I feel that the real fight will be between x86 and Arm-based PC platforms, it will be interesting to see how quickly Apple reacts to this AI-forward shift in the PC space. What strikes me about this equation is that Apple, which was on the offensive when it disrupted the PC market with its own silicon, might be on the defensive now as Qualcomm disrupts the PC market with its impressive on-device AI play. I cannot help but suspect that these exciting new capabilities will help bring some much-needed lift to the PC market in 2024.
Overall, Qualcomm’s early and consistent investments in thorough AI integration across its entire Snapdragon ecosystem appear to be paying off at exactly the right time. If nothing else, the era of on-device AI certainly just got a lot more interesting.
Disclosure: The Futurum Group is a research and advisory firm that engages or has engaged in research, analysis, and advisory services with many technology companies, including those mentioned in this article. The author does not hold any equity positions with any company mentioned in this article.
Analysis and opinions expressed herein are specific to the analyst individually and data and other information that might have been provided for validation, not those of The Futurum Group as a whole.
Other insights from The Futurum Group:
Qualcomm Revenue in Q3 Dips to $8.44 Billion as Mobile, IoT Soften
Qualcomm and Microsoft: On-Device AI Will Make the PC Experience Shine
Qualcomm Works with Meta on On-Device AI: On-Device AI Without a Cloud in the Sky
Image Credit: Qualcomm
Author Information
Olivier Blanchard has extensive experience managing product innovation, technology adoption, digital integration, and change management for industry leaders in the B2B, B2C, B2G sectors, and the IT channel. His passion is helping decision-makers and their organizations understand the many risks and opportunities of technology-driven disruption, and leverage innovation to build stronger, better, more competitive companies.
