NVIDIA is putting more weight behind a version of AI that doesn’t have to live entirely in somebody else’s cloud.
The company has expanded its local AI ecosystem with Nemotron 3.5 Lightning, a customizable open-weight model built for always-on AI agents, alongside new tools designed to help developers run increasingly capable models on PCs, workstations and edge systems.
It’s part of a broader push happening across open-source AI right now. Bigger models are becoming easier to compress. Consumer and workstation GPUs are getting powerful enough to run workloads that once demanded a data center. And AI agents are moving from simple chat windows toward software that can actually work through files, tools and multi-step jobs.
NVIDIA clearly wants its hardware sitting underneath a lot of that activity.
Nemotron 3.5 Lightning Is Built for AI Agents That Keep Working
Nemotron 3.5 Lightning is a 30-billion-parameter mixture-of-experts model aimed at high-volume agentic workloads.
NVIDIA says it can generate output up to four times faster than comparable open models in its class while cutting agent task completion time by roughly 30%. Those numbers come from NVIDIA’s own testing, so they should be treated as vendor benchmarks rather than universal performance guarantees.
The more interesting part may be what NVIDIA expects people to do with it.
Because Nemotron 3.5 Lightning is customizable, developers can post-train it around their own data, terminology and workflows. A company could tune it for code review. Another might build an internal agent that knows its support procedures. A creator could potentially adapt it around a particular writing or production workflow.
This isn’t really about having another chatbot available on a PC.
It’s about giving smaller, specialized models a job inside larger AI systems.
NVIDIA Wants AI Agents to Use More Than One Model
There’s an assumption buried inside many AI products today: send the request to the smartest model available and hope the economics work out.
NVIDIA is betting that agents will become more selective.
A difficult reasoning step might need a frontier model. Classifying a document probably doesn’t. Routine tool calls could go somewhere cheaper and faster. Code review may work better with a model trained specifically for code.
That is where NeMo Switchyard enters the picture.
The open-source routing library is designed to examine individual parts of an agent workflow and send them to different models depending on factors such as capability, latency and cost. Developers can use open models, proprietary models or NVIDIA models rather than building the entire application around one provider.
Think of it less like choosing an AI model and more like dispatching work.
The agent asks for something. The router decides who should handle it.
The Cost Argument Could Be Just as Important as Speed
Running every step of an autonomous agent through an expensive frontier model can get expensive surprisingly quickly.
Agents don’t necessarily make one request and stop. They search, reason, call tools, inspect results, recover from errors and try again. A relatively simple user request can become many model calls behind the scenes.
NVIDIA says its internal testing of NeMo Switchyard maintained frontier-level task performance while reducing completion cost to around one-third of using Opus 4.8 alone.
Other tests reveal the tradeoffs more clearly.
LangChain tested routing between Nemotron 3.5 Lightning and Claude Opus 4.8 across 145 multi-turn agent tasks. NVIDIA reports that the setup reduced costs by 74%, with only 7% of calls sent to the frontier model, although accuracy dropped by roughly six percentage points.
That last number matters.
Routing isn’t magic. Saving money can mean accepting a performance tradeoff. What changes is that developers get another dial to turn instead of treating every model request exactly the same.
Local AI Is Getting Less Local-Looking
The phrase “local AI” used to bring to mind tiny language models running slowly on a laptop.
That picture is becoming outdated.
NVIDIA’s current ecosystem includes Nemotron 3.5 Lightning running across RTX PCs, DGX Spark, DGX Station, Jetson systems, RTX PRO workstations, data centers and cloud environments. The model is also being supported through tools including vLLM, Ollama, llama.cpp, LM Studio and Unsloth.
And Nemotron isn’t arriving alone.
NVIDIA’s August local AI push highlights a rapidly growing collection of open models, including Meta’s Muse Glimmer for local agentic workloads, DeepSeek V4 Flash, Poolside AI’s Laguna S 2.1 coding model and multimodal systems capable of generating video and audio locally.
Some of these models are enormous.
Yet quantization, mixture-of-experts architectures and newer GPU hardware are making it possible to run models locally that would have sounded absurdly large for a desktop machine only a short time ago.
Privacy Is Quietly Becoming Part of the Local AI Pitch
There’s another reason local agents are interesting: they can work with information people may not want continually uploaded to external servers.
NVIDIA specifically describes local agent use cases involving documents, messages, files, credentials and application data. Models running on-device can process that information without requiring every inference request to leave the machine.
That won’t eliminate cloud AI.
It probably won’t even reduce cloud usage everywhere.
But a hybrid setup starts to look much more practical: keep routine or sensitive work local, send difficult problems to more powerful remote models when necessary, and put a routing layer in the middle deciding which is which.
That’s a very different architecture from simply opening a chatbot in a browser.
NVIDIA Is Betting on a System of Models
Nemotron 3.5 Lightning is another model launch, but the larger story is NVIDIA’s idea of what AI software becomes next.
Not one giant model doing everything.
A collection of models, agents and tools passing work between one another, with some running locally and others in the cloud.
If that model of AI wins, inference efficiency starts mattering almost as much as raw intelligence. The question stops being Which model is the best?
It becomes: Which model should handle this particular step?
NVIDIA is building hardware for that world.
Now it’s building more of the software around it too.
