Nick Ustaran-Anderegg
&
August 11, 2026
Today, we are proud to announce a partnership between AgileRL and NVIDIA that makes the Nemotron family of models available on Arena. NVIDIA created Nemotron to be an open, capable reasoning foundation. Companies can now take these models directly into Arena and train them into superintelligent agents specialized for their task, with the post-training and deployment handled by the platform. The result for companies is a single path from a NVIDIA model to a superintelligent agent running in production.
NVIDIA gave us early access to Nemotron Lightning 3.5 ahead of release, and we trained it on the two use cases below to show how Arena can be used to create specialized models that beat frontier APIs at a variety of tasks. Nemotron Lightning 3.5 is a hybrid Latent Mixture-of-Experts model, and has 3B active parameters and 30B parameters in total.
The first use case is Sudoku, on hard boards, which is a good proxy for a long-running agentic workflow. Solving it requires dozens of consecutive decisions, long contexts, and a scoring rule that cannot be gamed. The base model does not finish a single board. Claude Sonnet 5, run on the same environment, solves 10%. Trained on Arena, Nemotron Lightning 3.5 solves 88% of them outright.
The second use case takes an off-the-shelf customer support dataset, and trains an agent that both files tickets and writes replies for the customer. Base Nemotron Lightning 3.5 trails Claude Sonnet 5 on performance. After Arena training, it overtakes Sonnet on intent accuracy (99% vs 90%) and reply quality, according to our rubrics.
Arena is the platform for creating AI agents specialized at your task. Most teams arrive with a dataset or an RL environment, and Arena validates what you have before you spend anything on compute. Starting with neither is also fine; our team is on hand to build both with you from a plain description of the job the agent has to do. Training runs on an async-RL engine with evolutionary hyperparameter tuning, which is where our 7x speedup against TRL and ART comes from. Workloads run inside your own environment and the weights never leave it. When a run finishes, deploy it in one click onto your own infrastructure and keep training it on live feedback.
A hard Sudoku board opens with about 47 empty cells. The model fills one cell per turn, up to 50 turns, and one malformed answer ends the episode on the spot. Achieving a full solution requires careful reasoning at each step, including forecasting the consequences of the agent's actions, with a large context of 50,000+ tokens.
Untrained, Nemotron Lightning 3.5 is a capable model. Hand it a hard Sudoku board, and you can watch it reason about the position carefully - but not achieve success. After all, the model has never been trained to play Sudoku. Train it on Arena and the mean score comes out at 0.94, where a completely filled board scores 1.0. Most of that comes from boards it fills outright and the rest from boards that stop a few cells short. The unfinished boards display a common error: the agent makes one wrong placement, burns the remaining spare turns correcting or compounding it, and digs into a hole it can't climb out of before the tight 50-turn limit. The model's 88% full completion rate shows that, with Arena, we are able to take a naive model and tune it into an expert at a challenging, multi-turn environment.
We ran the same boards against Claude Sonnet 5 with the same 50-turn limit and conditions that training uses. Across 100 hard boards, Sonnet solved 10, for a 10% solve rate and a mean score of 0.11. 85 of the 100 runs died on a format error: the move has to arrive as \boxed{row col value}, and a single malformed answer ends the episode immediately. On 60 of those boards, the failure landed in the first two turns. Often, Sonnet's reasoning spends the entire token budget and leaves an empty reply, so the environment never sees a legal move. One near-miss filled 45 cells and then ran out of turns at a score of 0.90. Sonnet can play Sudoku; just not as well as a fine-tuned Nemotron Lightning 3.5.
We used CISPO on Arena's async pipeline, with rollout generation and learning running on separate GPUs and LoRA adapters synced between them, across 4 H100 80GB GPUs. The training run took 6 hours, and Arena comfortably handled context lengths of over 50,000 tokens while maintaining high token throughput. Our training optimizations and async-RL engine allow compute infrastructure to be fully utilized, resulting in fast and cost-effective learning.
On an off-the-shelf Bitext support dataset, base Nemotron Lightning 3.5 files 71.1% of 2,728 held-out messages under the right one of 27 intents. The errors also cluster, with the model repeatedly confusing similar requests. The model's pretraining is generally strong, but it has not been trained on this specific task, and so it also struggles with responding to customers in the prose and tone that we would like.
During training on Arena, the agent does two jobs on every message and the reward scores both: file the request under the right intent, and write the reply the customer reads. Every point comes from a comparison against data the business already owns, with labelled intents and gold-standard replies supplied via the dataset.
We scored Claude Sonnet 5 the same way we scored Sudoku: same prompt, same format contract, same reward the training run uses. On a 600-message slice of the held-out set, Sonnet with thinking on reaches 90% intent accuracy and perfect format validity. Base Nemotron sits behind it at 71.1%. After Arena training, Nemotron Lightning 3.5 beats Sonnet on all metrics:
Reply fidelity is a measurement of the difference between the model's reply and the desired reply in the dataset. Slot F1 measures whether a reply correctly references customer data, such as an order number or an account type, when needed. Targeting these ensures that reply quality is high, and matches the behaviour expected for the agent's real-world setting.
The base model's intent classification errors were mostly clustered on similar cases, such as confusing a user asking to be sent an invoice with asking about one they already have. In practice, getting these intents wrong leads to misfiled tickets that sit in the wrong queue until somebody notices.
Getting the format right counts for as much as getting the label right, because a label is only worth anything if the system downstream can read it. 7.6% of the base model's answers came back in a shape no parser would accept, which in production is a ticket that quietly fails to route.
Here is an example of a customer message, and the agent's replies:
Base model: ✗ Incorrect intent and response
Fine-tuned agent: ✓ Correct intent and acceptable response
After training, the same message is correctly filed as get_refund and answered with a request for the amount and the reason, closely resembling the gold example. This is the difference between a customer who gets their money back and one who gets a second reply about policy.
This customer support example demonstrates fine-tuning with Arena against a table of historical decisions, which is the form most companies already have their expertise sitting in. Like in the Sudoku case, a 3B-active open model, post-trained on Arena, overtakes Claude Sonnet 5. Nemotron Lightning 3.5 brings the language ability and Arena lets you turn that static dataset into a specialized, useful agent.
Teams in industries including banking, aerospace and government are moving off frontier model APIs, and control is usually what starts the conversation. Depending on a provider means trusting that the model stays available and that its behavior does not shift between releases. Where data sovereignty is a requirement, that dependency is not something a contract fixes. A model you trained runs where your data already lives.
Another important consideration is cost. Per-token pricing compounds faster than most teams budget for, which is why large enterprises are already capping token spend. A model you host is a fixed cost on hardware you own, and it does not grow with the number of tickets arriving at your support desk.
Performance on your actual task matters more than general capability. Your intent taxonomy, your escalation policy, your product names and your house tone are internal conventions that appear in no model's pretraining set, and a model that has not been trained on them will not infer them. A prompt can describe a convention, but training is what makes an AI agent expect it. The Sudoku and customer support case numbers make that concrete. On hard Sudoku, Arena-trained Nemotron Lightning 3.5 solves 88% of boards against Sonnet's 10%. On customer support, it reaches 99% intent accuracy against Sonnet's 90%, from a base that started at 71%. Specialization beats raw scale when the job is scored against your data. Both use cases above took a model that could not do the task and produced one that could.
We are partnering with NVIDIA on post-training across their open models. Bring a task: a dataset, or a description of what your agent has to get right. We will train and deploy an agent on it with you in a live session, on your infrastructure, with the weights yours to keep.
