Maia 300 Pairs Software-Defined Dataflow With 288 GB of HBM4 and a 12,288-Accelerator Scale-Up Fabric as Microsoft Makes Maia Its Primary Inference System From 2027. Microsoft introduced Maia 300, its third-generation inference accelerator, and committed to multi-gigawatt Maia deployments starting in 2027 with Maia as the primary inference system for its products. Futurum examines how a decode-first design built on Maia 200’s production record shifts the hyperscaler silicon mix toward post-training and agentic inference.
What Is Covered in This Article:
- Maia 300 launch on TSMC N2P with 288 GB of HBM4 at 18 TB/s per accelerator
- Multi-gigawatt Maia deployments from 2027 as Microsoft’s primary inference system
- Maia 200 production results and MAI-Thinking-1 silicon co-design
- One Ethernet-based scale-up fabric spanning up to 12,288 accelerators
- Positioning against Google TPU 8i, AWS Trainium, and NVIDIA GPUs
The News: Microsoft introduced Maia 300, the third generation of its Azure Maia accelerator, built for long-context, reasoning-intensive, and decode-heavy agentic inference. The multi-chiplet design places compute on TSMC N2P and moves memory-controller and I/O functions onto separate dies, with eight HBM4 stacks delivering 288 GB of capacity and 18 TB/s of bandwidth per accelerator. Against Maia 200, Microsoft cites 36% more compute FLOPs, 2.5x the HBM bandwidth, 33% more capacity, roughly 2x the HBM bandwidth per watt, and 14% more scale-up bandwidth. New enhanced MXFP4 and sub-3-bit storage formats cut the bytes moved per token, and memory shifts to a NUMA organization across chiplets so software can place data near the compute that uses it. A 167 kW rack houses 96 accelerators across 24 four-accelerator trays, and a two-tier Ethernet-based scale-up network joins up to 128 racks into a superpod of 12,288 accelerators with no separate scale-out network. Microsoft says the full 96-accelerator configuration delivers up to 7x the rack-scale token throughput of Maia 200 and more than 2.5x the performance per watt, and projects a 40% improvement in tokens per megawatt over leading GPU systems.
Microsoft also set the deployment plan. Scott Guthrie, Executive Vice President of Cloud + AI at Microsoft, said Maia already runs production workloads for Microsoft Copilot, GitHub Copilot, Microsoft Foundry, and Microsoft’s own AI models, and that Microsoft will scale Maia to multi-gigawatt deployments starting in 2027, making it the primary inference system for Microsoft products and services.
Microsoft Maia 300 Bets Gigawatts on Dataflow Inference Silicon
Analyst Take: Microsoft Maia 300 turns the post-training silicon thesis into a gigawatt procurement plan. Futurum argued in January that Maia 200 signaled an XPU shift toward reinforcement learning, and in July Microsoft justified new silicon by pointing to RL for the second time in six months, prompting Futurum to ask whether reinforcement learning had become the hardware bottleneck. Maia 300 is the hardware answer. Its compute grows by 36% while its memory bandwidth grows by 2.5x, a ratio that only makes sense for memory-bound workloads, and the decode-heavy generation behind agentic inference and RL rollouts fits that profile.
The scale is significant. Futurum’s semiconductor market model projects deployed AI data center critical IT power rising from 26.2 GW in 2026 to 47.1 GW in 2027 in the base case, at about $15.4 billion of accelerator spending per gigawatt. Every gigawatt Microsoft fills with Maia instead of merchant GPUs moves a slice of that spend to first-party silicon. The bear case is that every efficiency figure behind the plan comes from Microsoft, Maia runs only inside Microsoft, and no independent benchmark has yet tested Maia against the GPUs it displaces.
Maia 200’s Production Record Earns Maia the Primary Inference Role
Microsoft is scaling Maia because Maia 200 delivered in production. Guthrie now ranks Maia 200 as the best-performing AI infrastructure in Microsoft’s fleet on the real-world metrics its AI services track. At its January launch, Microsoft said Maia 200 offered 30% better performance per dollar than the latest hardware in its fleet, and it now reports up to 1.7x more tokens per watt than leading GPU infrastructure for certain Teams, Copilot, and Cowork experiences. At Hot Chips 2026, where Futurum attended, the Maia team claimed roughly 40% better performance per dollar than other leading accelerators in Azure. The Microsoft 365 workloads behind those numbers are agentic. Jim Kleewein, Technical Fellow for Microsoft 365, described Copilot and Cowork running on Maia 200 to summarize information, compose content, and complete multi-step tasks, with usage data on responsiveness and reliability flowing back to the infrastructure team to shape Maia 300.
Microsoft’s own frontier models make the same case. Microsoft co-designed MAI-Thinking-1, a 35B-active Mixture of Experts reasoning model that scores 97% on AIME 2025 and 53% on SWE-Bench Pro, with Maia 200, and Microsoft AI reported 30% better performance per dollar and 1.4x better performance per watt than GB200 when running MAI models end to end. Mustafa Suleyman, CEO of Microsoft AI, framed chip-model co-design as a requirement for frontier model development and said lessons from running MAI models on Maia 200 fed directly into the Maia 300 design. The Microsoft Superintelligence team has used Maia 200 for synthetic data generation and reinforcement learning since launch, while NVIDIA and AMD continue to support Microsoft’s model training. The implication is that Maia already runs both sides of Microsoft’s AI business: first-party products that serve customers and the post-training loop that improves the models behind them, with each side informing the next chip.
Maia 300 Trades FLOPs for Memory Bandwidth to Serve Decode
Maia 300 spends its silicon budget on memory. Rani Borkar, President of Azure Hardware Systems and Infrastructure, set out the design premise: AI economics now turn on how efficiently a whole system generates tokens at scale, and peak compute matters less. RL rollouts and agentic reasoning share a workload profile in which a model generates long responses from comparatively short inputs, so performance depends on how fast weights and KV cache stream from HBM. Futurum’s analysis is that this memory-bound decode, rather than pretraining arithmetic, now sets the shape of hyperscaler inference demand.
Maia 300 meets it on four fronts. HBM4 delivers 18 TB/s per accelerator. Roughly 2x the HBM bandwidth per watt stretches a fixed power envelope further. The new MXFP4 and sub-3-bit formats apply to KV storage as well as computation, according to Saurabh Dighe, CVP of Azure Hardware Systems Planning and Architecture, which shrinks the cache that grows with every turn of a long-context session. NUMA locality keeps data next to the compute region that consumes it, which shortens on-die travel and cuts data movement energy. Microsoft’s own description lists multi-step execution, long-context decoding, and tool use as the workloads changing the arithmetic intensity of inference. The trade-off is flexibility. A chip balanced this far toward bandwidth cedes training to GPUs by design, and sub-3-bit formats will need production accuracy evidence across model families before customers outside Microsoft would accept them.
Software-Defined Dataflow Moves Maia Closer to Startup Architectures
Maia’s Software Defined Locally Accessed Dataflow Architecture (SDLA) gives software explicit control over where data lives and when it moves. At Hot Chips, the Maia team described a design with no implicit caching: control processors queue work for separate DMA and compute engines, synchronization runs through semaphores, and data-oblivious control flow yields deterministic schedules that help a service meet tight latency targets. SDLA does not raise peak HBM bandwidth, which comes from the memory stacks themselves. It raises effective bandwidth by overlapping transfers with compute and reducing the bytes moved, and Microsoft’s Hot Chips data showed GEMM and collective kernels tracking close to the roofline across a wide range of sizes.
This approach resembles dataflow chips from Groq, SambaNova, Cerebras, and Tenstorrent more than it resembles TPU or Trainium, which apply explicit control mainly within large cores. The difference from the startups is memory, since Maia pairs dataflow with hyperscaler-scale HBM instead of SRAM-only or GDDR designs. Programmability has stalled dataflow startups because every customer must port its models. Microsoft sidesteps that problem as its own customer, and while the preview Maia SDK offers PyTorch integration, a Triton compiler, and a low-level programming language, outside developers have no route to rent Maia capacity. The risk is that a shift in model architecture reopens the porting cost that a captive workload currently absorbs.
One Scale-Up Fabric Replaces the Scale-Out Network
Maia 300 extends a single communication model from the tray to the superpod. Direct links pair accelerators within a tray, a switched electrical fabric joins all 96 accelerators in a rack, and optical second-tier switches connect up to 128 racks, giving three latency domains of zero, one, and three switch hops. Each accelerator’s split between in-rack and cross-rack bandwidth is set by job packing at run time, with up to 750 GB/s to any peer in the superpod when only a subset of accelerators communicates across racks. Prefill and decode workers can be sized independently, with KV state moving directly to decode workers over the same fabric, and schedulers can draw on hot-spare trays in other racks. Google’s TPU 8i scales to 1,152 chips per pod, AWS pairs 144-chip Trainium3 UltraServers with EFA scale-out networking, and NVIDIA pairs NVLink racks with Spectrum-X Ethernet for scale-out. OpenAI’s Jalapeño, presented at Hot Chips, is the closest parallel, with scale-up Ethernet reaching 2,048 chips. Maia 300’s 12,288-accelerator domain is the largest scale-up fabric any hyperscaler has disclosed for inference. The open question is whether optical second-tier links stay reliable and cost-effective at that size, since a fault domain that broad depends on the hot-spare and telemetry systems Microsoft describes performing in the field.
Maia 300 Leads Hyperscaler ASICs on Bandwidth as the Silicon Mix Sorts by Workload
On paper, Maia 300 out-specs every announced hyperscaler ASIC on memory bandwidth. Google’s inference-tuned TPU 8i pairs 288 GB of HBM3e at 8.6 TB/s with 384 MB of on-chip SRAM, and AWS Trainium3 ships with 144 GB at 4.9 TB/s. Much of that lead comes from reaching HBM4 first, and AWS has said Trainium4 will bring 4x Trainium3’s memory bandwidth, which could close the gap in the same 2027 window. Go-to-market separates the three more than specs do. Google and AWS rent TPU and Trainium to external labs, with Anthropic training and serving Claude on both, while Maia’s validation comes entirely from Microsoft’s own products, MAI models, and OpenAI models served through Microsoft. Maia also fills a role AWS has filled from outside, since AWS pairs Trainium prefill with Cerebras CS-3 systems for decode. Microsoft built its own decode engine. Hyperscaler silicon is now splitting by workload, and vendors are sorting along those lines: GPUs keep pretraining, while purpose-built silicon takes decode, RL rollouts, and long-context agents. Futurum expects the share of post-training and inference spend that moves off general-purpose GPUs at Microsoft to exceed market expectations through 2027.
What to Watch:
- Whether Maia 300 reaches volume deployment across multiple Azure regions in 2027
- Whether Microsoft submits Maia 200 or Maia 300 results to MLPerf Inference or Signal65 PINNACLE
- Whether Microsoft opens Maia capacity to third-party labs or Azure customers
- Whether new MAI models after MAI-Thinking-1 ship co-designed for Maia 300
- Whether Trainium4 and HBM4-based TPUs close Maia 300’s bandwidth lead in 2027
Read the full Maia 300 announcement and the Maia 300 architecture deep dive on the Microsoft Azure blog.
Sources
- Deep dive into Azure Maia 300 architecture, Microsoft, October 2026
Declaration of generative AI and AI-assisted technologies in the writing process: This content has been generated with the support of artificial intelligence technologies. Due to the fast pace of content creation and the continuous evolution of data and information, The Futurum Group and its analysts strive to ensure the accuracy and factual integrity of the information presented. However, the opinions and interpretations expressed in this content reflect those of the individual author/analyst. The Futurum Group makes no guarantees regarding the completeness, accuracy, or reliability of any information contained herein. Readers are encouraged to verify facts independently and consult relevant sources for further clarification.
Disclosure: Futurum is a research and advisory firm that engages or has engaged in research, analysis, and advisory services with many technology companies, including those mentioned in this article. The author does not hold any equity positions with any company mentioned in this article.
Analysis and opinions expressed herein are specific to the analyst individually and data and other information that might have been provided for validation, not those of Futurum as a whole.
Read the full Futurum Group Disclosure.
Other Insights From Futurum:
Microsoft Splits the Copilot Pricing Model Into Everyday and Advanced AI
Microsoft Copilot Becomes an Agentic Work Platform
Autonomy Over Analytics: The Read-Write Decree Rewiring Enterprise Data Platforms
Author Information
Brendan is Research Director, Semiconductors, Supply Chain, and Emerging Tech. He advises clients on strategic initiatives and leads the Futurum Semiconductors Practice. He is an experienced tech industry analyst who has guided tech leaders in identifying market opportunities spanning edge processors, generative AI applications, and hyperscale data centers.
Before joining Futurum, Brendan consulted with global AI leaders and served as a Senior Analyst in Emerging Technology Research at PitchBook. At PitchBook, he developed market intelligence tools for AI, highlighted by one of the industry’s most comprehensive AI semiconductor market landscapes encompassing both public and private companies. He has advised Fortune 100 tech giants, growth-stage innovators, global investors, and leading market research firms. Before PitchBook, he led research teams in tech investment banking and market research.
Brendan is based in Seattle, Washington. He has a Bachelor of Arts Degree from Amherst College.
