Karpo Decision goes open-source; 31B model ties first on a specified Decision Index version

MachinePulse releases 0.8B–31B decision models that score options in one forward pass. The 31B model ties first at 73.9 in the November 12 Decision Index 0.4.1 snapshot. Latency and costs depend on test configuration; single decisions differ from full workflows.
Editorial correction (November 18, 2026): The tied ranking, 32,768-token context limit and cost conversion have been corrected against the model card. Single-step latency is distinguished from complete workflows.
A new open-source contender has just emerged in Silicon Valley's fiercely competitive new field!
Recently, Anthropic, approaching its stock-market debut, revealed an enormous compute shortfall in its IPO materials, throwing cold water on an AI industry racing to increase parameter counts.
General-purpose large models excel at programming and mathematics, but when they actually do work for people, the compute bill becomes a bottomless pit.
Long reasoning chains consume more tokens than text generation, while browser operation is more expensive still, with an average operation costing 4.8 times as much as complex reasoning!

Amid intense anxiety, Silicon Valley's decision-model battle has begun in earnest.
Early benchmark Jev pioneered fast thinking that judges without writing. Giants including OpenAI, Microsoft, Amazon and Cloudflare followed.
A couple of days ago, Google CEO Sundar Pichai also joined in, prominently launching the closed-source decision model Gemini Decision 1.

Amid the battle of giants, open-source dark horse Karpo Decision 31B has burst onto the scene!
The model card's November 12, 2026 snapshot of Decision Index 0.4.1 contains 300 entries. Karpo scored 73.9, tied first with GLiDE-27B under the leaderboard's tiering rules. This is a particular evaluation version, not global first place across every decision task.

https://huggingface.co/spaces/multimodalart/jev-decision-index
The full Karpo Decision family, from 0.8B to 31B, is now open-source!
The winning 31B flagship reports single-GPU median latency as low as 94 milliseconds, calibration error down to 0.023, and simultaneous evaluation of up to 255 options per question.

Taking on Google's closed-source flagship
An open-source dark horse wins three rounds
Unlike traditional models that generate text token by token, Karpo Decision is built for agents' frequent decisions.
Given the current state and a question, one forward pass directly calculates probabilities for every option without generating any tokens.
With this fast judgment mechanism, Karpo reached the top of the overall leaderboard upon release and beat Google's closed-source Gemini Decision 1 in several evaluations!
More dramatically, the winning open-source contender was itself produced by post-training Google's open Gemma 4 31B foundation model.

This article cites different results from Decision Index and JevBench. Their tasks, metrics and configurations cannot be mixed directly. A tied Decision Index ranking also does not establish that Karpo surpasses every open- and closed-source model across the board.
Decision Index spans thirty-seven public benchmarks and undisclosed private questions, attracting 300 models from around the world.
JevBench, meanwhile, is run by an independent organization on its own GPUs, blind and fully offline, with 1,200 of its 1,500 questions permanently held back.
On the overall Decision Index leaderboard, Karpo took the top position with 73.9 points, well above early benchmark Jev 1.14's 60.2.
Alongside its higher score, median latency was only 94 milliseconds, versus Jev 1.14's 196 milliseconds: roughly twice as fast!
The greater distinction came in private blind tests intended to resist leaderboard gaming.
When the public leaderboard's top ten entered private blind tests of the same skills, scores fell an average 6.8 points.
Karpo dropped only 4.3 points, the smallest decline among the top ten, and scored 78.6 on private questions from new domains!

Against Google's Gemini Decision 1, Karpo also produced a series of wins.
Although both tied at 92.2% on JevBench's public set, differences emerged in agent tasks.
The first round tested agents' core action decisions.
The established When2Call benchmark, for example, tests whether an agent should call a tool, ask a follow-up question or stop during a conversation.
Karpo scored 90.4%, exceeding Google's 79.9% by 10.5 percentage points.
On the multimodal Mind2Web test of clicking buttons from webpage screenshots, Karpo likewise scored 90.0%, ahead of Google's 86.2%.
The second round tested implied meaning and fine-grained classification.
On the New Yorker cartoon-caption test for understanding implicit tone, Karpo reached 90.5%, beating Google's 79.7% by 10.8 percentage points!
For Amazon ESCI e-commerce classification, with its highly granular product intents, Karpo scored 64.7%, ahead of Google's 58.7%.

The third round examined all thirty-seven public benchmarks.
Across Decision Index's thirty-seven public benchmarks, Karpo scored 75.0, exceeding Google's 60.9 by 14.1 points!
It also won across unfamiliar domains (72.3%), knowledge accuracy (56.0%) and safety judgments (93.4%).
It also moved ahead of open-source rivals including KnowLine, GLiDE and Sieve, previously the leaderboard's top three.

In JevBench's newly released official 1,500-question offline blind test, Karpo also delivered striking results.
In this rigorous test independently run by the leaderboard maintainer, Karpo achieved a current capability score of 89.53 points.
More notably, it scored 85.67 on the 1,200 sealed private questions, above its 80.45 on public questions, while its calibration score reached 96.00!

Converted to the leaderboard's common scale, Karpo entered the global top three among 261 open-source models with an adjusted capability score of 87.2 points.
Against Google in the same evaluation, it led in all three core metrics: overall score (77.4), decision intelligence (80.2) and calibration (94.1).
Meanwhile, Karpo's cost per thousand decisions was just $0.025, equivalent to 40,000 judgments for one dollar.

Alongside the 31B flagship's top ranking, MachinePulse applied the same training recipe to five smaller sizes from 0.8B to 12B parameters.
From lightweight on-device models to single-GPU workhorses, every parameter scale entered the leading tier of its size class.

Screening thirty-six events in a second
Choosing dishes for ten diners in a blink
Beyond benchmark wins, how fast is Karpo in real scenarios?
Traditional large models respond to problems by generating lengthy analysis token by token.
Karpo writes nothing. One forward pass calculates probabilities for all options in milliseconds.
Karpo's website has added two live interactive challenges to illustrate the speed difference of answering dozens of questions in one forward pass.
In the City Scanner challenge, AI helps choose a weekend outing from thirty-six events around the city.
Suppose you are meeting a friend on Friday evening, with these conditions: tonight, two people, under $50 each, within five kilometers and suitable for conversation.
The model must assess every event as a match, a rejection or insufficient information against those requirements.
The system sends the same request simultaneously to Karpo and the general-purpose Gemini 3.5 Flash, timing both.
After clicking start, Karpo answers all thirty-six questions in just 1.12 seconds!
It also selects ten matches, twenty-three rejections and three insufficient-information cases, agreeing with the rules on all thirty-six questions.
The general-purpose model on the right, still emitting text token by token, leaves the user waiting as the timer passes five seconds and continues spinning.

The other challenge, Ten Appetites, One Menu, pushes multi-constraint tradeoffs to the limit.
A restaurant has ten diners, each with calorie limits, a budget, allergens and taste preferences.
We tried choosing dishes manually on the left, watching restrictions and budgets while selecting ten dishes. After five seconds of work, we had still accidentally violated two dietary restrictions.
On the right, Karpo handles the whole menu and ten diners in one request, taking only 916 milliseconds to serve up all ten dishes, earning full marks for hard constraints and soft preferences.
Even if a dish suddenly sells out, it can reassign dishes for only the affected diners within a few hundred milliseconds.

Beyond the interactive challenges, we also tested automatic browser-agent restaurant booking on the website's playground.
The agent's goal was to book a table for two at Lumen restaurant on Friday at 7:30 pm.
On a real booking webpage, Karpo immediately supplied the highest-probability action at each step, selecting "7:30 pm," then "Find a table," and confirming the reservation.
The approximately 130 milliseconds cited describes single-step model-decision latency. Complete booking also involves tool execution, network round trips and external services. The time for several model calls cannot simply be treated as end-to-end reservation time.

Removing token-by-token generation
Calculating the answer in one forward pass
Karpo can answer dozens of questions in a few hundred milliseconds because it changes the underlying token-by-token generation mechanism.

The model card specifies a 32,768-token limit for one decision sequence. Longer material can be split into independent questions, but this does not mean a single sequence supports 256K context. Its focus is constrained decision outputs.
It removes token-by-token decoding and instead adds a lightweight linear output layer to the end of the sixty-layer backbone (Readout head).
When a question enters the model, candidate answers are condensed into individual single-token labels. The head scores up to 255 options simultaneously and converts them directly into calibrated probabilities.
For multiple-choice, yes/no and scoring interfaces, the model returns constrained decisions, reducing free-text formatting errors. This does not guarantee semantic correctness, infallible interfaces or automatic abstention on unfamiliar questions.

After accelerating individual questions, Karpo also introduces a branching-attention mechanism to minimize computation when asking dozens of questions about a long webpage (Branch Attention).
The system places the entire question list first, having the model read the subsequent long webpage with those questions in mind, combining both into a shared block whose KV cache is computed only once.
Each question then receives a parallel answer branch after the shared block, with position encodings restarting at the same location for every branch, keeping dozens of questions isolated without cross-contamination.
For complex webpages thousands of tokens long, median latency is 371 milliseconds for one question, while asking sixteen to sixty-three questions together takes only 653 milliseconds!

Self-improvement through RSI
The hardest problem in training decision models is preventing a leaderboard-gaming machine that merely memorizes formats.
The Karpo team found that full-parameter fine-tuning alone could raise the public score to 75.09, while unfamiliar-domain accuracy actually fell 2.3 percentage points.
More ironically, leaderboard tricks such as reading options in both orders and averaging did nothing for real generalization, while doubling latency to 162 milliseconds.
This shows that a higher score on a public dataset does not mean a model can actually hold its own in real business use. The team therefore decisively cut out these leaderboard-chasing tricks that slowed the model down.
The team describes an RSI loop for ongoing training improvement, including evaluation and selection of candidate data and models. This does not mean deployed models automatically update after every use or necessarily become smarter with usage.
The loop uses public datasets and targeted synthetic data to fill capability gaps. Each candidate is evaluated with a 60% public-set and 40% unfamiliar-domain weighting and advances only if it genuinely beats its predecessor.

For combining models, the team likewise rejected elaborate tricks, finding complex search algorithms prone to overfitting. Classical Model Fusion, uniformly averaging weights, was the most stable approach.
The final winning Karpo therefore uniformly averages three checkpoints from early, middle and late full-parameter fine-tuning with two checkpoints from a separate LoRA training run.
The logic is complementary: early weights generalize better, later ones specialize more, while LoRA learns less and forgets less.
After merging five weight sets into one, inference cost remains the same as a single model, while unfamiliar-domain accuracy exceeds the final single-model checkpoint by 1.6 percentage points!

With this combination against overfitting, Karpo beat Jev 1.14 across all ten major categories and 283 subcategories of unfamiliar-domain evaluation, including clinical medicine and court judgments, for a 10–0 sweep!

Beyond selecting correctly, decision models need awareness of their uncertainty. Karpo introduces reinforcement learning for calibrated decisions (RLCD), encouraging truthful assessment of confidence.
After calibration reinforcement, Karpo's expected calibration error, ECE, on JevBench falls to 0.023, substantially better than Google and Jev 1.14.
Calibration error measures how confidence corresponds to accuracy across many samples. It does not guarantee exactly eight correct judgments in any ten labeled 80% confident, and calibration figures from different benchmarks should not be directly compared.
In blind yes/no tests with missing information, Karpo also decided on 92.0% of questions and abstained on the remaining 8.0% when uncertain, handing those back to humans for safety.

A powerful decision model on one GPU
To maximize single-GPU speed after training the 31B model, engineers used Triton to handwrite dedicated kernels for Gemma 4's global-attention layers, accelerating single-GPU inference by nearly 2x.
Combined with padding-free packing, FP8 quantization and CUDA graph replay, memory use on one RTX PRO 6000 was held at around 60 GB, without losing evaluation score.
Across two thousand requests sampled from the benchmark, single-GPU median latency ultimately reached 73 milliseconds, with throughput of 7,100 tokens per second. Even the most compute-intensive long-webpage inference time was halved to 3.6 seconds!

Real scenarios as the training ground
The personal-agent opportunity arrives
At this point, many may wonder: how can a startup team beat Google head-on?
The answer lies in its focus on personal agents as its field of operation.
Overseas, MachinePulse has long been refining its local-services agent Karpo.
Searching citywide for restaurants and weekend events, opening pages to compare room types, selecting time slots and booking: every step is fundamentally a frequent judgment.
Such small decisions are everywhere in daily life.

In a supermarket, choosing a carton of milk from a full shelf to suit your tastes and physical needs.
At night, entering a bar you like, chatting and winding down after an exhausting day.
These seemingly effortless moments become repeated millisecond-level tradeoffs and decisions for an agent.

But general models' pauses of several seconds, subtle judgment errors and extra tokens translate into a severe experience gap for real users.
AI labs can focus on benchmarks; frontline teams face actual interaction pain points every day.
In practice, vertical agent products themselves are excellent model-training grounds.
Multi-constraint tradeoffs, sequential webpage operations and exception recovery in real business force teams beyond paper scores to solve latency, cost and calibration problems unseen in laboratories.
The cited $0.025 per thousand decisions converts to approximately $0.000025 per decision, not several cents each. Actual costs vary with hardware, load and service setup. Low-latency constrained decisions offer another option for some agent tasks.
MachinePulse has open-sourced this entire family of 0.8B–31B decision models with native webpage-action judgment, supplying practical tools for personal-agent creators.

Let general-purpose large models focus on deep thinking, and give millisecond-level, frequent judgments to Karpo decision models.
The model offers an open-source solution for constrained decisions on one GPU. Suitability still requires task-specific validation of accuracy, calibration, latency and tool safety; one leaderboard alone is insufficient for model selection.
This article was originally published by AI Era. Authors: Moses and David.
Related topics
After reading
Join the AINRK community
Read how to join the community and connect with people working in AI.




