The News: On November 15, as part of Ignite 2023, Microsoft announced the launch of Windows AI Studio, a new AI experience to help enterprises and developers jumpstart local AI development and deployment on Windows. Here are the key details:
- Tools and models from Azure AI Studio. For full analysis, read my earlier post “With Azure AI Studio, Microsoft Contends for Top Dev Platform.”
- Hybrid cloud and on device. As stated in the press release “Windows AI Studio brings us closer to supporting Hybrid Loop development patterns and enabling hybrid AI scenarios across Azure and client devices. This gives developers greater choice either to run their models on the cloud on Azure or on the edge locally on Windows (or across the two) to meet their needs. Prompt Flow enables developers to implement this hybrid pattern by switching between local SLMs and cloud LLMs.”
- Models optimized for Windows graphics processing units (GPUs) and neural processing units (NPUs), starting with Llama 2 7B, Mistral 7B, Falcon 7B, and Stable Diffusion XL. ONNX Runtime is the gateway to Windows AI, and DirectML is the native Windows machine learning (ML) application programming interface (API), and together they give developers access to a simplified yet highly performant AI development experience. Earlier this year at Inspire, Microsoft shared details on how developers will be able to run Llama 2 with DirectML and the ONNX Runtime. Microsoft has a sample showing the company’s progress with Llama 2 7B; Microsoft can now run it locally and performantly on varied Windows hardware.
Read the blog post on Microsoft Windows AI Studio here.
Read the blog post on the preview support for Llama 2 7B in DirectML here.
Windows AI Studio: Jumpstart For On-Device AI?
Analyst Take: On-device AI is both a massive opportunity and a significant challenge. Just the installed base of Windows 10 users is estimated at 1.3 billion, to start thinking about a portion of the addressable market. But running AI locally without powerful datacenter computers means on-device use cases likely should not require real-time inferencing and are use cases that run on low-power compute. Microsoft’s investment in on-device AI through the Windows AI Studio will be a key driver of on-device AI. Here’s why.
Leveraging the Microsoft AI Toolkit
The design of Azure AI Studio and the Windows AI Studio reflects real-word learnings that Microsoft has culled while developing other elements of its AI offerings, namely, the collaboration with OpenAI, and the development of Copilot and integration of generative AI into Microsoft’s signature Office 365 apps. The design reflects mature AI thinking on many levels – for example, the highlight of the focus on RAG, fine-tuning on proprietary enterprise data – is a reflection of the rapid evolution of generative AI to a place where the general large language model (LLM) is not so useful for enterprises than more refined model use. The integrations are mature thinking – to Microsoft Fabric, third-party code platforms such as VS Code, GitHub Codespaces, and LangChain, model partners such as Hugging Face, Nvidia and Meta.
Perhaps most important, the real-world learnings Microsoft leveraged here have to do with security and AI safety. Microsoft has meticulously built systems to manage AI risk for the deployment of AI through its Copilot and AI-infused applications that the company is able to bring to bear in Azure AI Studio. These AI safety policies and guardrails could be a differentiator for Microsoft in the quest to capture enterprise AI business.
Device Compute Experience
On-device applications are nothing new to Microsoft and Windows and it is clear in the blog post that the team is leveraging plenty of learnings about maximizing power efficiency to run local apps. The success of the Llama 2 7B testing is testament to that knowledge, and it is important to note that while other development teams/platforms have been working to deliver on-device AI, it is hard to point to another platform showing this kind of progress, or perhaps rather, the maturity of the thinking. Finding models that are small enough but smart enough to run locally on-device will be a key element to unleashing potent on-device AI use cases.
Conclusion
Microsoft and many of its partners – device chip makers such as Qualcomm, Intel, and AMD as well as OEMs such as Dell, Lenovo, and a host of others – are working to break through with real-life, useful on-device AI use cases. The Windows development community will be a critical component, the creative power, to drive that forward. (Note that the on-device AI market opportunity is PCs, but perhaps more critically, it is smartphones, so Windows is an important element and driver, but not the only component of on-device AI). Look for Microsoft to continue to invest heavily into the Windows AI Studio, and for key partners, particularly Qualcomm, Intel, AMD, and Dell to join or align in promoting Windows AI development.
Disclosure: The Futurum Group is a research and advisory firm that engages or has engaged in research, analysis, and advisory services with many technology companies, including those mentioned in this article. The author does not hold any equity positions with any company mentioned in this article.
Analysis and opinions expressed herein are specific to the analyst individually and data and other information that might have been provided for validation, not those of The Futurum Group as a whole.
Other insights from The Futurum Group:
Microsoft Ignite Showcases AI Advancements with Copilot in Teams
Microsoft’s AI Safety Policies: Best Practice
Under The Hood: How Microsoft Copilot Tames LLM Issues
Author Information
Mark comes to The Futurum Group from Omdia’s Artificial Intelligence practice, where his focus was on natural language and AI use cases.
Previously, Mark worked as a consultant and analyst providing custom and syndicated qualitative market analysis with an emphasis on mobile technology and identifying trends and opportunities for companies like Syniverse and ABI Research. He has been cited by international media outlets including CNBC, The Wall Street Journal, Bloomberg Businessweek, and CNET. Based in Tampa, Florida, Mark is a veteran market research analyst with 25 years of experience interpreting technology business and holds a Bachelor of Science from the University of Florida.
