A 2 GB math model claimed 94% on AIME 2026. We graded it

Checking a benchmark claim is now cheap. What a one-try test can tell you, what it cannot, and why this model is not a chatbot.

Just now

Every model launch arrives with a table of scores. Most teams read it and move on. When the model runs on one rented card, you can check it.

Paradigma says its first model, Limite 1B Violetto, a single 2.1 GB file, scores 94% on AIME 2026, an American invitational exam for top high school students. So we sat it down for the exam on one RTX 4090 on our cloud. It solved 27 of the 30 problems, with one attempt at each.

Summary of our Limite 1B Violetto test: 27 of 30 AIME 2026 problems right on one try, 211 tokens per second on an RTX 4090 (85 on an RTX 2000 Ada 16 GB), and 5.2 GB of card memory under a 4 GB cap

The lesson is a habit: grade a vendor’s number before you plan around it.

Who did the work

Paradigma’s launch post names Prignano, Cirillo, Morosini, Cerovaz, Bartolocci, Rodolà, Starace and Pappone as authors. They trained it from scratch in about six weeks and released it under Apache 2.0, which allows commercial use, with their evaluation code. They also wrote the plugin that lets vLLM, the open-source program that serves models, load it.

The exam itself comes from MathArena, who publish the problems as a dataset for testing models. We first heard of Limite in a news video from the AI Search channel. Our part: packaging Paradigma’s plugin to deploy, running it on two cards and checking every answer.

Same rules, one deliberate change

A fair check reruns the vendor’s test. We used the same 30 problems at the dataset version Paradigma’s evaluator pins, with the same instruction, randomness settings and length limit. A script compared the answer the model boxed at the end with the official one.

We changed one thing. Paradigma’s 94.01% averages 32 attempts per problem. We allowed one attempt, and one attempt swings more than an average of 32. So 27 of 30 tells you the model belongs in the range Paradigma reports, not that its exact decimal is right.

Get InferenceKey’s stories in your inbox

Join StartsNest for free to get updates from this writer.

Remember me for faster sign in

Two misses were wrong answers. The third kept writing for 14 minutes, roughly 111,000 tokens (a token is a small chunk of text, roughly a word), and then stopped mid-sentence. We could not pin down why.

The limits of a check like this

The AIME 2026 problems went public in February, months before Limite was trained. We did not test whether they leaked into its training data. On ArXivMath May 2026, a newer set, Paradigma’s own table shows Limite at 25% against about 57% for GLM 5.2. Our result agrees with Paradigma on this exam, not on problems the model could not have seen.

We graded only the final number, not the reasoning. We skipped BeyondAIME, the set where Paradigma puts Limite next to a model thirty times larger. That comparison stays their claim. The full write-up of the Limite 1B test lists each limit, every problem’s result and Paradigma’s full score table.

A solver for a pile of problems, not a chat partner

Before answering, it thinks out loud: it drafts an idea, tests it, drops it and starts over, and you wait for all of it. Its shortest correct answer used 3,203 tokens. Its longest ran to about 101,000. Half the problems took fewer than about 21,000.

Alone, one answer streams at about 211 tokens per second on the 4090: a typical problem takes under two minutes. Hand over all 30 at once and the card works on them side by side, at its peak producing about 17 times the output of a single answer. But the job ends only when its slowest problem does. Ours ran for 16 minutes.

That shared run is where the economics live. At about 0.80 euros an hour for a 4090, the 30 problems used about 22 euro cents of working time, and 25 of those answers were correct. That puts a solved problem under one euro cent.

Paradigma says plainly that this is a math model. Ask it anything else and it bends the question into math. In their own examples, “What is photosynthesis?” became a problem about dividing cells. It is not a chat assistant or a tool for an agent.

A modest card will do

A real RTX 2000 Ada, a 16 GB workstation card, solved both problems we gave it, 2.5 times slower than the 4090. We also limited the server to roughly 4 GB of the 4090’s memory, which copies a smaller card’s memory but not its weaker chip. Two short problems still came back right, just as fast. Less memory means fewer long answers at once, not slower ones.

The hands-on report on inferencekey.com covers each memory step, how the model came up in 31 seconds on a warm machine, and one rough edge in our published SDK that stays open until the next release. Our SDK example deploys the model and grades it for you.

Grade the table yourself

When a small open model posts a score, reuse the vendor’s setup, run it once on a card rented by the hour, and note what your run cannot show. A score you checked, with its limits beside it, beats one you only read.

The commands, every result and the cost per problem are in the complete Limite 1B Violetto test on inferencekey.com.