Analyst(s): Nick Patience
Publication Date: August 11, 2026
NVIDIA has announced Nemotron 3.5 Lightning, a 30-billion-parameter open mixture-of-experts model built for the high volume of specialized calls that always-on agents generate, alongside NeMo Switchyard, an open source model routing library. Together, they extend NVIDIA’s open model strategy into the layer that decides which model handles each step of an agentic workflow. Of the two, the routing library carries the greater strategic weight.
What Is Covered in This Article:
- Nemotron 3.5 Lightning is a 30-billion-parameter mixture-of-experts model with roughly 3 billion active parameters, distilled from Nemotron 3 Ultra. NVIDIA claims four times the throughput of comparable models in its class and up to 30% faster completion of agentic benchmark workloads.
- The model ships with open weights, open post-training datasets, and published training recipes, and runs locally on Jetson, GeForce RTX, DGX Spark, and DGX Station.
- NeMo Switchyard is an open source routing library that selects a model for each step of an agentic workflow, with integrations already in place at OpenRouter, LiteLLM, and Kong. NVIDIA’s benchmark data shows a routed pairing cutting task completion cost to roughly one-third, with completion rates broadly unchanged.
- Early post-training results from CrowdStrike, CodeRabbit, Harvey with Trajectory, and Lila position specialization, rather than general intelligence, as the commercial argument for the model.
The News: NVIDIA has announced NVIDIA Nemotron 3.5 Lightning, a 30-billion-parameter open mixture-of-experts (MoE) model with approximately 3 billion active parameters, and NVIDIA NeMo Switchyard, an open source model routing library for AI agents. Lightning is positioned for always-on agents that issue a constant stream of specialized calls, and NVIDIA states the model delivers four times the throughput of comparable models in its class at equivalent intelligence, completing agentic benchmark workloads up to 30% faster than open peers of similar capability. The model is distilled from Nemotron 3 Ultra and was built with contributions from the Nemotron Coalition. It combines a hybrid Mamba-Transformer architecture with multi-token prediction, a latent mixture-of-experts design that doubles the number of experts available at inference, and speculative decoding. NVIDIA is releasing open weights alongside post-training datasets and training recipes, and the model runs locally on Jetson, GeForce RTX, DGX Spark, and DGX Station. NeMo Switchyard routes requests across a developer-defined pool of models using customizable algorithms and policies, and NVIDIA has worked with OpenRouter, LiteLLM, and Kong to embed it inside gateways developers already run.
Who Decides Which Model Runs? NVIDIA Would Like a Say
Analyst Take: NVIDIA announced Nemotron 3.5 Lightning and NeMo Switchyard together. Lightning is a 30B open MoE model tuned for throughput on the repetitive calls that fill most of an agent’s working life, while Switchyard decides when to call it. On the Artificial Analysis composite intelligence index NVIDIA used cited, Lightning scores 24, which is near the bottom of a chart topped by Claude Opus 5 at 63, and marginally below NVIDIA’s own Nemotron 3 Super at 26. NVIDIA’s argument is that most agentic work does not need frontier reasoning, and that the industry has spent two years paying frontier prices for execution-grade tasks.
The Efficiency Frontier Is the Argument
The technical story centers on tokens per second per dollar. A hybrid Mamba-Transformer backbone carries multi-token prediction, a latent MoE design that doubles the effective expert count at inference, and speculative decoding. On NVIDIA’s agentic benchmark, Lightning matches Qwen3.6-35B on accuracy while completing 10,000 tasks in roughly 17 H100 GPU hours against approximately 24 for Qwen and 25 for Gemma 4 26B, which also trails on accuracy. NVIDIA distilled Ultra into a Nano-scale footprint in about six weeks, including evaluation. With 3B active parameters, the model fits on a workstation, and NVIDIA is explicit that it should run on Jetson, GeForce RTX, DGX Spark, and DGX Station rather than through a cloud API call for every task.
Post-Training Is Where the Claim Will Be Tested
The early access customization data is NVIDIA’s strongest evidence. CrowdStrike lifted the benign recall by 45 points to approach a customized Nemotron 3 Super at roughly a fifth of the cost. CodeRabbit improved its coding router by 34 points and built the agent for $85 in two hours on Baseten using a standard NeMo AutoModel recipe. Harvey, with post-training partner Trajectory, gained 8.3 points on legal task completion. Lila added 36 points on the energy simulation, having already beaten Opus 4.8 out of the box. The caveat is that all four are hand-selected early-access partners with deep machine-learning benches. NVIDIA has not shown what customization looks like for an organization with a data team and no research function, which is precisely the gap that keeps agentic pilots from reaching production.
Switchyard Is a Claim on the Control Point
We feel that the router is the more consequential release of the two. Whoever controls model selection controls which silicon serves which token, and NVIDIA is giving that capability away. Switchyard is a Rust proxy and library that translates between OpenAI Chat, Anthropic Messages, and OpenAI Responses formats, so an agent keeps speaking its native API while requests are served by vLLM, NVIDIA NIM, Ollama, or any OpenAI-compatible endpoint. Routing algorithms run from random selection through classifier-based routing to signal-driven stage routing, and developers can write their own. The protocol translation is the quiet part and the important part: it makes model substitution invisible to the application above it, and substitution economics require substitutability.
NVIDIA has chosen distribution over confrontation, having OpenRouter, LiteLLM, and Kong integrate its routing rather than competing with them. That buys reach on day one and thins gateway differentiation over time toward observability, billing, and access control. The commercial logic underneath is straightforward: routing execution-tier work to small models running on Jetson, RTX, DGX Spark, and DGX Station keeps that inference on NVIDIA hardware rather than on cloud APIs served by TPUs, Trainium, or other accelerators.
Read the Cost Chart Carefully
NVIDIA’s headline claim is frontier-level accuracy at roughly one-third of the cost, benchmarked against running Opus 4.8 for every step, which no cost-sensitive team does. But there are other, more useful findings to be found. Adding Lightning alongside Opus took benchmark cost from approximately $180 to around $95, with task completion essentially flat; adding two further open models reached roughly $72 for less than a point of completion improvement. Almost all of the benefit came from the first substitution. In NVIDIA’s own data, the marginal return on an elaborate model pool is small, which argues for a narrow and well-understood set rather than the sprawling system of models the narrative implies. But there are also operational costs to be considered: every routed request needs per-step attribution for debugging, evaluation, and audit, and in regulated sectors which model produced a given output is a compliance question rather than a curiosity.
The Sovereignty Angle
Open weights, published datasets and recipes, and a 30B footprint that runs on a single workstation make a credible substrate for jurisdictionally constrained deployment. For governments and regulated industries, the binding constraint has usually been capability at a deployable scale, and a router that keeps sensitive steps local while sending only benign ones outward is a more practical sovereign architecture than either a frontier API or a weak local model. The distinction that matters is jurisdictional reach: open weights on domestically operated hardware remove a foreign provider dependency at the inference layer, while the silicon supply chain dependency remains intact. Expect that gap between operational sovereignty and sovereign branding to be blurred further.
What This Means for Buyers
Enterprises running agentic workloads should measure their own token distribution before adopting anything. What proportion of calls genuinely requires frontier reasoning? If the answer is 10% to 20%, which is typical for well-instrumented agents, a routed architecture pays for the engineering effort. If a team cannot answer that question with data, routing will add complexity without capturing the savings. Start with a two-model pool, instrument per-step attribution before expanding it, and treat NVIDIA’s customization proof points as an upper bound rather than a forecast.
What to Watch:
- Whether Switchyard’s neutrality holds in practice. The algorithms are open and the model pool is developer-defined, but default configurations, reference recipes, and documentation are where preference gets expressed. Watch what the out-of-the-box defaults actually do.
- How OpenRouter, LiteLLM, and Kong respond over the next two quarters. Integrating NVIDIA’s routing buys reach today and erodes differentiation tomorrow, and competing routers from the hyperscalers are already embedded in their own model catalogs.
- Whether the customization story generalizes beyond the early access cohort. The market test is a mainstream enterprise with a data team and no research organization producing a comparable accuracy lift on its own workload.
- Whether enterprises build the observability required to operate routed systems safely. Per-step model attribution, evaluation across a heterogeneous pool, and durable audit trails are prerequisites for regulated deployment, and none of them ship with the router.
See the full announcement details on the NVIDIA website and the NeMo Switchyard open source repository for routing algorithms, protocol translation details, and configuration documentation on GitHub.
Disclosure: Futurum is a research and advisory firm that engages or has engaged in research, analysis, and advisory services with many technology companies, including those mentioned in this article. The author does not hold any equity positions with any company mentioned in this article.
Analysis and opinions expressed herein are specific to the analyst individually and data and other information that might have been provided for validation, not those of Futurum as a whole.
Other Insights From Futurum:
Meta Reopens Its Models. Is This a PC Play or a Policy Play?
NVIDIA’s Open Secure AI Alliance Bets Open Models Beat Closed Ones on Defense
At GTC 2026, NVIDIA Stakes Its Claim on Autonomous Agent Infrastructure
NVIDIA Bolsters AI/HPC Ecosystem with Nemotron 3 Models and SchedMD Buy
OpenShell Redraws the Agent Control Plane – Open Standard or Product Launch?
Author Information
Nick Patience is VP and Practice Lead for AI Platforms at The Futurum Group. Nick is a thought leader on AI development, deployment, and adoption - an area he has researched for 25 years. Before Futurum, Nick was a Managing Analyst with S&P Global Market Intelligence, responsible for 451 Research’s coverage of Data, AI, Analytics, Information Security, and Risk. Nick became part of S&P Global through its 2019 acquisition of 451 Research, a pioneering analyst firm that Nick co-founded in 1999. He is a sought-after speaker and advisor, known for his expertise in the drivers of AI adoption, industry use cases, and the infrastructure behind its development and deployment. Nick also spent three years as a product marketing lead at Recommind (now part of OpenText), a machine learning-driven eDiscovery software company. Nick is based in London.
