The News: At the Snapdragon Summit, Qualcomm unveiled SoCs for PCs and smartphones that could unleash on-device AI:
- Snapdragon X Elite
- Features the custom integrated Qualcomm Oryon CPU that Qualcomm says delivers up to 2x faster CPU performance versus the competition, matching competitor peak performance with one-third of the power.
- Snapdragon X Elite can run generative AI models with more than 13 billion parameters on-device and 4.5x faster AI processing power than competitors, 30 tokens per second.
- Snapdragon X Elite is expected from leading OEMs starting mid-2024.
- Snapdragon 8 Gen 3
- Qualcomm says Snapdragon 8 Gen 3 is its first mobile platform “meticulously designed with generative AI in mind.”
- Delivers industry-leading AI with the ability to run 10 billion parameter AI models on-device at 20 tokens per second.
- Flagship Android devices powered by Snapdragon 8 Gen 3 are expected to be available in the coming weeks.
Read the full Qualcomm on-device AI press release from Snapdragon Summit here.
Read the full Qualcomm Snapdragon 8 Gen 3 press release from Snapdragon Summit here.
Read the full Qualcomm Snapdragon X Elite press release from Snapdragon Summit here.
A few more facts discussed at the Summit are relevant to the news and our analyst take:
- Qualcomm revealed the installed base of devices with Snapdragon SoCs is currently 3 billion.
- Qualcomm has a comprehensive range of partners for realizing on-device AI, including Microsoft, Meta, Google, Baidu, OpenAI, Bloom, Stability AI, Zhipu-AI, EleutherAI, Baichaun, Yoodoo, and RWKV.
- Qualcomm is advancing large language models (LLMs) on-device by leaning toward smaller models, anywhere between 6 billion (6B) parameters and, at this time, 30 billion parameter LLMs. Snapdragon partner and smartphone OEM Honor said 6B parameter model is the minimum size model they have found for on-device AI with no cloud connection. Smaller than that “it’s just a toy,” said Honor CEO George Zhao. Zhao went on to say the limitation for bigger models is the battery. “You can’t run larger models because of the power consumption,” said Zhao. Honor has built its own LLM at 6B parameters.
- Qualcomm SVP Ziad Asghar spoke briefly about an intriguing idea – “multimodal AI.” For on-device AI, particularly smartphones, this marries LLMs with diffusion models, creating a fusion of language-based outputs with image and video-based outputs. This makes sense in thinking about smartphone onboard cameras.
With Snapdragon, Qualcomm Sets the Pace for On-Device AI
Analyst Take: Qualcomm’s latest Snapdragon SoCs seem poised to introduce legitimate on-device AI, a concept that might seem to defy the current convention of compute-munching AI workloads. Here are some thoughts about this opportunity, competitive advantages, and challenges
Opportunity
Installed base of devices with Snapdragon today: 3 billion. That in itself is compelling. Why? Because enterprises are always interested in the size of the market opportunity and the speed in which they might be able to penetrate it. Think about how the value proposition of mobile apps became so compelling of a business opportunity. Enterprises are interested in AI use cases, whether they become mobile apps or PC software or infused capabilities, in a 3 billion installed base market opportunity.
Competitive Advantages
Not many players make GPUs. Even less make GPUs for “mobile” devices, primarily PCs and smartphones. Four to think of are Qualcomm, Intel, AMD, and Apple. Of those, which have been thinking about on-device AI the most and for the longest? Probably Qualcomm and Apple. So first competitive advantage is GPU experience and capability.
The second competitive advantage Qualcomm has is experience with AI. Executives at the Snapdragon Summit said on at least two occasions that Qualcomm has been working with AI for more than 10 years. All that time, the company has been thinking about on-device AI. The only difference today is an even bigger opportunity with generative AI.
The third competitive advantage Qualcomm has right now is it feels like the company is further along in terms of the partnerships and ecosystems to support on-device AI than some of the others – this includes its AI stack and maybe a bigger developer ecosystem than some of its competitors (particularly around mobile). It was evident at Snapdragon Summit that Qualcomm has also thought pretty deeply about pragmatic and logical use cases for on-device AI – of course, there is the ubiquitous personal assistant, but they also showed some nifty camera-based use cases and talked more about video conferencing. This was a breath of fresh air over some of the more nebulous AI use cases of the past – augmented reality, mixed reality, etc.
Challenges
It is early days for generative AI use cases. Today, there are limitations (battery) to how much AI compute smartphones and PCs can process locally. Many use cases will require a hybrid approach, which Qualcomm already promotes, to deliver of many AI use cases.
Conclusions
We have gone from an era where the best of PCs were made smaller more and more efficient to create a powerful mobile device. In the AI PC era, we are moving to PCs that emulate mobile devices. High-powered, super-efficient, touch-enabled, and very lightweight devices. Qualcomm has the pedigree and with Arm + the Apple success and recent NVIDIA news (moving into Arm), this seems very well timed.
It is early days for generative AI, regardless of whether it is cloud based or on-device. Qualcomm has an enormous opportunity and the company appears to be all in on pursuing it. The next 2 years will reveal a great deal about how and where we can use AI, and the bet here is that Qualcomm will have something to say about how that becomes reality.
Disclosure: The Futurum Group is a research and advisory firm that engages or has engaged in research, analysis, and advisory services with many technology companies, including those mentioned in this article. The author does not hold any equity positions with any company mentioned in this article.
Analysis and opinions expressed herein are specific to the analyst individually and data and other information that might have been provided for validation, not those of The Futurum Group as a whole.
Other insights from The Futurum Group:
Qualcomm Raises Bar for On-Device Generative AI at Snapdragon Summit
5G Factor VRN: Qualcomm Makes Waves with Snapdragon G Series Launch
Qualcomm-Meta Llama 2 Could Unleash LLM Apps at the Edge
Author Information
Based in Tampa, Florida, Mark is a veteran market research analyst with 25 years of experience interpreting technology business and holds a Bachelor of Science from the University of Florida.
Daniel is the CEO of The Futurum Group. Living his life at the intersection of people and technology, Daniel works with the world’s largest technology brands exploring Digital Transformation and how it is influencing the enterprise.
From the leading edge of AI to global technology policy, Daniel makes the connections between business, people and tech that are required for companies to benefit most from their technology investments. Daniel is a top 5 globally ranked industry analyst and his ideas are regularly cited or shared in television appearances by CNBC, Bloomberg, Wall Street Journal and hundreds of other sites around the world.
A 7x Best-Selling Author including his most recent book “Human/Machine.” Daniel is also a Forbes and MarketWatch (Dow Jones) contributor.
An MBA and Former Graduate Adjunct Faculty, Daniel is an Austin Texas transplant after 40 years in Chicago. His speaking takes him around the world each year as he shares his vision of the role technology will play in our future.
