The News: MediaTek has announced that it is working closely with Meta’s Llama 2, the company’s next-generation open-source large language model (LLM) to build a complete edge computing ecosystem designed to accelerate AI application development on smartphones, IoT, vehicles, smart home, and other edge devices. Meta’s LLM will work alongside MediaTek’s latest APUs and NeuroPilot AI Platform. You can read the full press release on the MediaTek website.
MediaTek Announces On-Device Generative AI Powered by Meta’s Llama 2
Analyst Take: As with most transformative technologies these days, mobile is where innovation first learns to walk and run. MediaTek’s partnership announcement with Meta comes as demand for on-device/edge generative AI capabilities is entering a bit of a fever pitch. By folding Llama 2, Meta’s next-generation open-source Large Language Model (LLM) into its latest APUs and NeuroPilot AI platform, MediaTek is looking to build an edge computing ecosystem that will enable it to accelerate AI application development in smartphones, IoT, vehicles, smart home, and other edge devices powered by its SoCs.
MediaTek, the fabless semiconductor company powering more than 2 billion connected edge devices every year, needs a strong on-device/edge AI story to take to the market at a time when device SoC rivals like Qualcomm and Apple have already begun to prioritize on-device generative AI capable features in their premium mobile handset tiers as high-value market differentiators.
To better understand why on-device generative AI capabilities matters so much to the industry, consider that for now, most generative AI processing is still performed in the cloud. This is not ideal for a plethora of reasons, ranging from cost (a lot of generative AI processing can be handled more cheaply on a handheld device than in a data center) and connectivity challenges (losing connection to cloud computing services means lag or no generative AI processing for your device at all, which is both a productivity killer and a UX nightmare), to privacy (to keep prompts and generative AI work products better protected from snooping and third-party interference). Enabling generative AI applications to run directly on devices will lower latency, reduce operational costs, deliver more seamless performance, provide users with the ability to work despite having limited to no connectivity, and enhance privacy and security.
Device OEMs therefore need leading-edge SoCs and platforms capable of delivering on these capabilities, and this is a challenge for MediaTek, whose success in mobile can be defined by a single, enviable characteristic: consistent ability to put out attractively priced, high-performance products that fit well within the boundaries of budget-minded handset tiers. That strategy has served the company well, but it has struggled to penetrate premium tier SoC markets currently led by rivals Qualcomm and Apple. While MediaTek still occupies a strong 31% global market share of smartphone application processors (against Qualcomm’s 28% and Apple’s 26% market shares for Q1-2023), the Taiwan-based company has yet to gain traction in premium tiers, particularly in the coveted North American market.
To remedy this problem, MediaTek needs more than just capable CPUs, GPUs, and connectivity solutions now. It also needs a strong, credible, competitive on-device/edge generative AI story to take to handset OEMs. MediaTek’s Llama 2 announcement is the kernel of that story, and adding a credible partner like Meta is a check in the plus column. According to JC Hsu, Corporate Senior Vice President and General Manager of Wireless Communications Business Unit at MediaTek, “the increasing popularity of generative AI is a significant trend in digital transformation, and our vision is to provide the exciting community of Llama 2 developers and users with the tools needed to fully innovate in the AI space.”
The timing of the announcement is also interesting, as MediaTek’s next-generation flagship chipset, which will be introduced in the fall, is expected to feature a software stack optimized to run Llama 2, as well as an upgraded APU with Transformer backbone acceleration, reduced footprint access, and use of DRAM bandwidth to enhance LLM and AIGC performance. This helps highlight the importance of this otherwise easy-to-miss announcement. MediaTek expects Llama 2-based AI applications to become available for smartphones powered by the next-generation flagship SoC, scheduled to hit the market by the end of the year.
In the Android space, will MediaTek finally give Qualcomm a run for its money in the premium and flagship tiers? That remains to be seen. Snapdragon enjoys a significant head start in those tiers, to say nothing of its custom silicon advantage, but it will be interesting to see how MediaTek leverages this new opportunity to disrupt the status quo. Mobile aside, what I will be especially interested in over the coming 18 to 24 months is how MediaTek will expand these on-device generative AI capabilities to its ecosystem of mobile-adjacent SoCs and use cases. Mobile is only part of this new wave of AI-capable digital transformation.digita
Disclosure: The Futurum Group is a research and advisory firm that engages or has engaged in research, analysis, and advisory services with many technology companies, including those mentioned in this article. The author does not hold any equity positions with any company mentioned in this article.
Analysis and opinions expressed herein are specific to the analyst individually and data and other information that might have been provided for validation, not those of The Futurum Group as a whole.
Other insights from The Futurum Group:
The Future of AI is Hybrid: Look No Further than Your Devices to Scale Generative AI
Snapdragon Chip Outperforms MediaTek in Xiaomi Phone Cameras
Author Information
Olivier Blanchard is Research Director, Intelligent Devices. He covers edge semiconductors and intelligent AI-capable devices for Futurum. In addition to having co-authored several books about digital transformation and AI with Futurum Group CEO Daniel Newman, Blanchard brings considerable experience demystifying new and emerging technologies, advising clients on how best to future-proof their organizations, and helping maximize the positive impacts of technology disruption while mitigating their potentially negative effects. Follow his extended analysis on X and LinkedIn.
