Gemini 4 Carbon tops the MathArena ranking for mathematical reasoning with an estimated performance of 85.4% and a cost of $15.70 per problem. GPT-6.1 Sol ranks third, at 80.6% and $1.07: the estimated improvement is 4.8 percentage points, but Carbon costs about 14.7 times more per attempt.
The ranking, updated by MathArena with results for Gemini 4 Carbon and Claude-Fable-5.2, also shows that the top position does not automatically make a model the best choice for every use case. Carbon outperforms Sol in the overall estimate, but that margin comes with a much higher cost and results that vary considerably by test.
A measurable advantage, with uncertainty
MathArena places Carbon first with a range of 85.4% ± 3.0 points. Claude-Fable-5.2 ranks second at 82.8% ± 5.3, and GPT-6.1 Sol third at 80.6% ± 2.7. The ranges for Carbon and Sol overlap; therefore, the difference between their estimates does not indicate an identical win across all tasks or eliminate statistical uncertainty.
The composition of the ranking itself matters. MathArena aggregates results from available tests and, when a model has not attempted a particular question, estimates its result based on its observed performance and the characteristics of that question. The models do not have results recorded for the same sets. The overall percentage therefore combines measurements and estimates; it is not the result of a single shared test conducted under identical conditions.
Cost is also an aggregate estimate. MathArena weights tests by the number of questions, excludes retired tests and Project Euler, and estimates costs when data is missing. The $15.70 figure represents one attempt across the mix of tests considered, not a universal rate for any mathematical problem.
The table doesn't tell the whole story
A test-by-test comparison shows why a single ranking can hide different profiles. In October's BrokenArXiv, Carbon leads Sol by 2.47 points, while in the September edition Sol is slightly ahead. In ArXivLean, the performance gap is much larger, but Carbon also incurs a high cost per problem.
| Model and test | Performance | Cost per problem |
|---|---|---|
| Carbon, October BrokenArXiv | 82.72% | $18.06 |
| GPT-6.1 Sol, October BrokenArXiv | 80.25% | $1.30 |
| Claude-Fable-5.2, October BrokenArXiv | 73.46% | $39.78 |
| Carbon, September BrokenArXiv | 82.39% | $18.60 |
| GPT-6.1 Sol, September BrokenArXiv | 82.70% | — |
| Carbon, September ArXivLean | 62.26% | $45.11 |
| GPT-6.1 Sol, September ArXivLean | 35.85% | $1.16 |
BrokenArXiv presents plausible claims drawn from conjectures alongside other claims that recent papers have refuted; the model must assess a proposition without being told in advance that it is false. The score rewards identifying it as false and gives zero if the model claims to have proved a false proposition. In this context, the October result measures not only the ability to produce a proof, but also whether the model detects an incorrect premise.
ArXivLean evaluates formal proofs. In the September round, Carbon scored 62.26% compared with Sol's 35.85%. This gap, much larger than the one in October's BrokenArXiv, may make Carbon more appealing for this kind of work, although its recorded cost was $45.11 per problem compared with $1.16 for Sol.
More answers, more spending
Cost is not explained solely by the base output price. In the configuration recorded for Carbon and Sol, both models have the same output price: $10 per million tokens. However, in October's BrokenArXiv, Carbon generated an average of 563,757 output tokens per attempt, compared with Sol's 89,623. The difference in volume helps explain why spending per problem is so different.
Carbon's test data also points to heavy output usage. For September's ArXivLean, MathArena records 1,064,444 generated tokens and a cost of $45.11 per problem. In BrokenArXiv, October attempts averaged 563,757 tokens and $18.06. These costs are not interchangeable: they depend on the evaluated set and the work done on each attempt.
This changes the practical calculus for teams evaluating models. If the goal is to maximize the success rate on a demanding mathematical task, paying more for Carbon may make sense; if query volume is high and Sol achieves a sufficient result, its lower cost per attempt matters a great deal. The relevant comparison is not just “which model ranks first,” but how much it costs to achieve the performance level a particular application requires.
A leader that doesn't always finish on time
Higher performance comes with an operational limitation. The Decoder notes that Carbon's most common failure mode on MathArena is exceeding the available time. It also says that it almost never gives up on a task: that persistence can help when a problem requires exploration, but it can consume resources without producing a solution on time.
In one case, Carbon started a script that kept running for more than eleven hours without checking its progress. The example does not allow us to calculate how often this behavior occurs, but it illustrates the kind of cost that an average score does not capture: in addition to token charges, a prolonged run can tie up capacity and leave a task unresolved within the expected time frame.
For developers, the implication is that a reasoning benchmark and a product evaluation answer different questions. MathArena measures performance on mathematical problems under its protocol; the experience of an application also depends on whether the model finishes within the allowed time and whether the cost per attempt is affordable.
Choose by task, not by medal
The comparison among the top three illustrates the trade-off. Carbon achieves the best overall result and outperforms Sol in October's BrokenArXiv, but Sol comes out ahead in September. Fable 5.2 ranks second in the aggregate ranking, although its estimated $40.20 per problem makes it even more expensive than Carbon; in October's BrokenArXiv, it scores 73.46% at a cost of $39.78.
MathArena also publishes results by round and allows users to inspect response traces. For a company considering these models, that level of detail is more useful than simply transferring the ranking to production: a task resembling ArXivLean supports a different conclusion from an evaluation focused on BrokenArXiv, and the overall estimate is no substitute for measuring costs on the set of problems that will actually be used.
The news, then, is not simply that Google has displaced OpenAI from the top spot. It is that Carbon has a specific estimated advantage in mathematics in exchange for more tokens, higher costs, and a risk of exceeding the execution time. The question for each team is how much that improvement is worth for its actual workload and whether it can tolerate the cost of attempts that do not finish on time.
Sources: The Decoder, MathArena, MathArena, AI Ping
