The News: Generative AI is widely considered the fastest-growing technology innovation in history. It has captured the imagination of consumers and enterprises around the globe, spawning incredible innovation and along with it a fast-mutating market ecosystem. Generative AI has also caused a copious amount of news and hype. To avoid AI “fear of missing out” (FOMO) and find the right path, the wise will pay attention to underlying trends and not be distracted by every announcement and news byte.
Year-End AI Model Trends: Anthropic, Microsoft, Google, ETH Zurich
Analyst Take: Here are the top AI model trends as of December 2023.
LLM Competition and the Rise of Open Source
Pricing for large language models (LLMs) could be dropping across the board. The main reason is more open source options (e.g., see Anthropic lowers pricing). In November, Anthropic dropped the per-token pricing of Claude 2.1.
The AI market is showing the classic (if accelerated) pattern of tech adoption, namely that moving into commercialization requires scalable solutions at competitive prices. Open source LLMs are putting some pressure on private plays from industry darlings such as OpenAI Cohere Aleph Alpha, but be forewarned: Open source tools have pros and cons. The cons include the following:
- Susceptible to vulnerabilities. With so many contributors, security is not easy.
- No vendor support. Enterprises that go open source are on their own. Literally, there is no vendor to squeeze for a fix.
- Requires customization. This is both a pro and a con, but open source code must be customized and enterprises have to carry that load.
“Small” Language Models Proliferating
Smaller language models trained on small data sets continue to proliferate, outpacing general-purpose LLMs trained on massive data sets. Some examples:
- Microsoft Orca 2: The Biggest Generative AI Breakthrough Since ChatGPT. On November 20, Microsoft Research announced the debut of Orca 2, the company’s latest step in exploring the capabilities of smaller language models (LMs). Orca 2 comes in two sizes, 7 billion parameters and 13 billion parameters.
- Gemini Nano is smallest language model yet. On December 6, Google announced its Gemini suite of language models, including Nano. Nano 1 is built on 1.8 billion parameters and Nano 2 is built on 3.25 billion parameters.
AI Compute Workloads Will Get Smaller
The massive compute workloads required for training LLMs has worried the AI ecosystem since ChatGPT launched. Can enterprises afford the compute required to train and run (inference) AI models? Will the economics work against commercial AI solutions? High-performing smaller models will cost less to support than larger LLMs and ensure better paths to commercialized AI.
Smaller LMs Now on Par, Better Performers than LLMs
It was believed that LLMs would outperform smaller LMs. There are now a long string of smaller models performing as well as or better than the largest LMs. This trend started with Meta’s Llama 2 13B but now Orca 2’s 13B and 7B outperform both Llama 2 13B and Mistral AI’s 7B . These improvements come from changing how models learn, as evidenced by Orca’s approach. Better performance for less cost? That would seem to be a way for most enterprises to go.
Unleashing On-Device AI
As smaller LMs get both smaller and better, powerful use cases for on-device AI for smartphones and PCs become more likely. Look for on-device AI champions such as Qualcomm, Dell Technologies, and others to parlay smaller LMs into on-device AI.
LLMs/Foundation Models Mutate for Better Modeling Techniques
New players continue to enter the market and model vendors and other players are forming alliances. Some details:
- New technique can accelerate language models by 300x. “Researchers [at] ETH Zürich have developed a new technique that can significantly boost the speed of neural networks. They’ve demonstrated that altering the inference process can drastically cut down the computational requirements of these networks …. The researchers believe that with better hardware and low-level implementation of the algorithm, there could be potential for more than a 300x improvement in the speed of inference. This could significantly address one of the major challenges of language models—the number of tokens they generate per second.”
- Skeleton-of-Thought: Parallel decoding speeds up and improves LLM output. A recent paper published by Microsoft Research outlines a method to speed up LLM answers and possibly improve them. Initial results are twice as fast when applied to GPT-3.5 and GPT-4, with Skeleton-of-Thought as the basis because LLMs answer questions based on sequential analysis, while humans do not necessarily. As quoted from the paper, “… humans may not always think about questions and write answers sequentially. In many cases, humans first derive the skeleton of the answer and then add details to explain each point.”
Conclusion
I have said that AI compute workloads have to come down; they are unsustainable and kill use of AI economics. Lots of progress in both hardware and software is driving cost down via more efficient chips, hacks, etc. But these newer developments might be the kind of scientific breakthroughs that move the market.
Disclosure: The Futurum Group is a research and advisory firm that engages or has engaged in research, analysis, and advisory services with many technology companies, including those mentioned in this article. The author does not hold any equity positions with any company mentioned in this article.
Analysis and opinions expressed herein are specific to the analyst individually and data and other information that might have been provided for validation, not those of The Futurum Group as a whole.
Other insights from The Futurum Group:
Key Trends in Generative AI – The AI Moment, Episode 1
Top Trends in AI This Week: October 9, 2023
Top Trends in AI This Week: August 25, 2023
Author Information
Based in Tampa, Florida, Mark is a veteran market research analyst with 25 years of experience interpreting technology business and holds a Bachelor of Science from the University of Florida.
