Ox Alpha Benchmarks: Does It Really Beat GPT-5.6 and Claude 5?

Ox Alpha became one of the most discussed AI models of 2026 after an early coding test appeared to show the anonymous model scoring 80% on DeepSWE—above results associated with GPT-5.6 Sol, Claude Fable 5, GLM-5.3 and Grok 4.6.

The headline travelled faster than the methodology.

The 80% result came from an initial sample of only 10 DeepSWE tasks. A subsequent evaluation performed on a larger subset produced a score of approximately 63%. A separate community evaluator later reported 62.8% from a complete run across the benchmark, although two agents reached the 90-minute time limit.

Ox Alpha still has no official entry on the DeepSWE leaderboard.

The available evidence suggests that Ox Alpha is a highly capable coding model. It does not prove that Ox Alpha is better than GPT-5.6 or Claude 5.

TL;DR: Ox Alpha scored 80% on an initial 10-task DeepSWE sample. A larger subset produced approximately 63%, while a subsequent community full run reported 62.8%, with two tasks reaching the time limit. These results suggest competitive coding performance, but Ox Alpha does not yet have a verified entry on the official DeepSWE leaderboard. Claims that it definitively beats GPT-5.6 or Claude 5 remain unsupported.

Ox Alpha Benchmark Results at a Glance

Test	Reported Ox Alpha result	What the result means
Initial DeepSWE subset	Approximately 80%	Eight successful tasks from a sample of ten; highly sensitive to individual results
Larger DeepSWE subset	Approximately 63%	More representative than the initial test, but the subset size was not disclosed
Community full DeepSWE run	62.8%	Complete single run; two agents reached the 90-minute time limit
Terminal-Bench 3.0 community run	26.6%	Single pass rather than the five-run average used by the official leaderboard
EVMBench smart-contract audit test	37.2% micro recall	Competitive with GPT-5.6 Sol and Grok 4.6 in a small 12-audit community evaluation
Official DeepSWE leaderboard entry	Not available	Ox Alpha has not been added to the official leaderboard

The correct interpretation is not that Ox Alpha failed to live up to the initial excitement. Its results consistently fall near the leading group of coding models.

The problem was presenting an eight-out-of-ten result as proof that an anonymous model had defeated complete evaluations of frontier systems.

For the model’s specifications, availability and unknown origin, read our complete Ox Alpha AI guide.

What Was the Original Ox Alpha Benchmark Claim?

The first widely circulated Ox Alpha coding result came from developer Ben Davis, who tested the model on ten tasks selected from DeepSWE.

Ox Alpha reportedly completed eight successfully, producing an 80% result.

The comparison accompanying the test included the following figures:

Model	Reported DeepSWE result
Ox Alpha	80%
Claude Fable 5	65%
GLM-5.3	62%
Grok 4.6	62%
GPT-5.6 Sol	52%

This table created the impression that an anonymous, temporarily free model had immediately defeated the most advanced coding systems from OpenAI, Anthropic, xAI and Z.ai.

That is not what the test proved.

Ox Alpha’s 80% came from a ten-task sample. The other figures were not all generated through a new, controlled head-to-head evaluation using those same ten tasks under identical conditions.

The table combined a small Ox Alpha experiment with results associated with broader model evaluations. It was an interesting early signal, not a new coding leaderboard.

Why Ten Tasks Are Not Enough

DeepSWE contains 113 long-horizon software-engineering tasks.

When a model is evaluated on only ten tasks, every individual success or failure changes its result by ten percentage points:

• eight completed tasks produce 80%;
• seven completed tasks produce 70%;
• six completed tasks produce 60%;
• five completed tasks produce 50%.

A difference of one task can therefore transform the headline from “Ox Alpha beats every frontier model” into “Ox Alpha performs near the middle of the leading group.”

The result also depends on which tasks were selected. A model may perform exceptionally well on one programming language, repository structure or type of engineering problem while struggling with another.

A small sample cannot establish whether the model performs consistently across the full benchmark.

The Larger DeepSWE Subset Produced Approximately 63%

A later Ox Alpha evaluation conducted by Wenqi Huang, a DataCurve researcher involved in creating DeepSWE, produced about 63% on a larger subset.

The run reportedly used an average of approximately 47,000 output tokens. Huang described the result as competitive with leading open models and just behind Grok 4.6 among closed systems.

The original test’s author subsequently clarified that the 63% evaluation was larger and more useful than his initial ten-task sample.

The exact number of tasks in the subset was not disclosed. It should therefore not be inserted directly into the official leaderboard as though all evaluation conditions were identical.

The original larger-subset result is still valuable because it moved the discussion away from the unstable 80% figure and towards a more plausible performance range.

A Complete Community Run Reported 62.8%

After the larger-subset result appeared, another evaluator reported completing a full Ox Alpha run across DeepSWE.

The reported result was 62.8%.

Two agents reached the 90-minute time limit, and the evaluator indicated that additional runs would be necessary to establish a reliable average.

This is currently the most complete public Ox Alpha result, but it is not equivalent to an official leaderboard entry.

The distinction matters:

• it represents one complete community run;
• it has not been published by DataCurve as an official configuration;
• it does not include an official confidence interval;
• repeated runs have not yet established the model’s variance;
• two tasks were affected by execution time limits;
• the exact serving conditions may differ from official leaderboard configurations.

The 62.8% result closely matches the approximately 63% larger-subset evaluation. That consistency makes the model’s likely performance range more credible.

It does not transform the result into an official ranking.

The correction and subsequent benchmark discussion can be followed in the public evaluation thread.

What Is DeepSWE?

DeepSWE is a benchmark designed to evaluate AI coding agents on original, long-horizon software-engineering assignments.

Unlike a basic coding test that asks a model to generate a function or solve an isolated programming problem, DeepSWE requires agents to work inside real repositories.

A model may need to:

1. understand the requested change;
2. inspect an unfamiliar codebase;
3. identify the relevant files;
4. modify several connected components;
5. run tests;
6. interpret failures;
7. revise the implementation;
8. produce a working solution.

The benchmark contains:

• 113 original tasks;
• 91 active open-source repositories;
• TypeScript, Go, Python, JavaScript and Rust projects;
• isolated execution environments;
• programme-based verification;
• long, multi-step engineering assignments.

The tasks were written from scratch instead of being extracted directly from existing public pull requests. This is intended to reduce the risk that a model encountered the solution during training.

DeepSWE is therefore more relevant to autonomous software engineering than short-form coding benchmarks. It tests whether an AI agent can complete work, not merely generate plausible-looking code.

The benchmark’s methodology and task structure are available in its official GitHub repository.

Where Ox Alpha Would Sit on the Official DeepSWE Leaderboard

As of August 24, 2026, the official DeepSWE leaderboard lists the following leading configurations:

Model configuration	DeepSWE result
Claude Opus 5, max reasoning	74% ±4%
GPT-5.6 Sol, max reasoning	73% ±3%
Claude Fable 5, max reasoning	70% ±4%
GLM-5.3, max reasoning	69% ±3%
Kimi K3, max reasoning	69% ±5%
GPT-5.6 Luna, max reasoning	67% ±4%
GPT-5.5, xhigh reasoning	67% ±6%
Grok 4.6, xhigh reasoning	67% ±2%
Gemini 3.7 Flash, high reasoning	65% ±2%
DeepSeek V4 Pro, max reasoning	63% ±6%

A 62.8% complete Ox Alpha run would place the model around DeepSeek V4 Pro and below the leading Claude, GPT, GLM, Kimi, Grok and Gemini configurations.

It should not be added as another row in the official table because the evaluation conditions and number of repetitions are not identical.

If a future official evaluation confirms a result near 63%, Ox Alpha would qualify as a competitive coding model. It would not lead the benchmark.

That would still be impressive for a free preview model with an undisclosed developer. It would not justify the claim that Ox Alpha had defeated GPT-5.6 Sol or Claude Opus 5.

Our comparison of the best AI models in 2026 explains why model selection must consider reliability, cost, speed and complete workflow execution—not only a single benchmark percentage.

Ox Alpha on Terminal-Bench 3.0

A separate community evaluation reported that Ox Alpha achieved 26.6% on Terminal-Bench 3.0.

Terminal-Bench evaluates whether an AI agent can complete practical tasks inside a terminal environment. This can involve configuring software, navigating files, using command-line tools and recovering from operational errors.

The reported result placed Ox Alpha above Claude Opus 4.8 but below Claude Fable 5, between GLM-5.3 and Grok 4.6.

Several limitations remain:

• the Ox Alpha result came from one pass;
• official comparisons commonly average five trials;
• two tasks were reportedly excluded because the evaluator lacked the required H100 hardware;
• the run has not been adopted as an official leaderboard result;
• a single pass cannot measure consistency across repeated attempts.

The test provides additional evidence that Ox Alpha is capable of agentic technical execution. It does not establish a definitive ranking.

Ox Alpha on EVMBench

Ox Alpha has also been tested on a small set of smart-contract security audits using EVMBench.

The community evaluation used 12 frozen audits and reported the following micro-recall results:

Model	Micro recall
GPT-5.6 Sol	40.3%
Ox Alpha	37.2%
Grok 4.6 Build	36.4%
GLM-5.3	11.6%

Ox Alpha identified 16 of 43 relevant issues and reportedly achieved 80% precision. It was also reported as the fastest tested model, with a median time of 52 seconds per audit.

These results place Ox Alpha close to GPT-5.6 Sol and Grok 4.6 Build in this particular experiment.

The limitations are substantial:

• only 12 audits were included;
• Ox Alpha received one run per audit;
• evaluation involved a single reviewer;
• the test was self-scored;
• the comparison was not presented as an official ranking;
• the frontier-model results were obtained under partially different repetition conditions.

EVMBench therefore reinforces the broader pattern: Ox Alpha appears competitive, but the available evidence remains too limited for absolute conclusions.

Does Ox Alpha Beat GPT-5.6?

No verified benchmark currently proves that Ox Alpha is better than GPT-5.6.

The original 80% DeepSWE result was based on ten tasks. The complete community run produced 62.8%, while GPT-5.6 Sol’s official DeepSWE configuration is listed at approximately 73%.

These results were not generated under fully identical evaluation conditions, so the percentages should not be treated as a direct head-to-head ranking. They provide no evidence that Ox Alpha consistently outperforms GPT-5.6 Sol.

The EVMBench community test placed Ox Alpha close to GPT-5.6 Sol, with micro recall of 37.2% against 40.3%.

The evidence suggests that Ox Alpha can compete with GPT-5.6 on some coding and agentic tasks. It does not demonstrate that Ox Alpha is the stronger model overall.

Does Ox Alpha Beat Claude 5?

There is also no verified evidence that Ox Alpha consistently beats Claude 5.

Claude Opus 5 currently leads the official DeepSWE table at approximately 74%, while Claude Fable 5 is listed at approximately 70%. The complete Ox Alpha community run reported 62.8%.

Different repositories, reasoning settings, serving conditions and agent configurations can produce different results. Ox Alpha may outperform Claude on individual tasks, but the existing benchmark data does not support a general victory claim.

What the Benchmarks Reveal About Ox Alpha

The available tests support five conclusions.

1. Ox Alpha Is a Serious Coding Model

The larger DeepSWE subset, complete community run, Terminal-Bench test and EVMBench evaluation all indicate meaningful software-engineering capability.

Ox Alpha should not be dismissed as a novelty created around an anonymous launch.

2. The Initial 80% Result Was Overinterpreted

The result itself was not necessarily false. Eight of ten tasks can legitimately produce an 80% score.

The problem was treating that small sample as proof that Ox Alpha had surpassed complete evaluations of frontier models.

3. Ox Alpha Appears Better at Long, Complex Work Than Simple Responses Suggest

Developer reports describe the model as capable of handling subagents, extended assignments and complex instructions. This aligns with its positioning as a reasoning model for sustained agentic work.

These are precisely the capabilities required by enterprise AI agents that must perform complete workflows rather than answer isolated questions.

4. Speed and Efficiency Remain Potential Weaknesses

Developer reports indicate that Ox Alpha can produce good code and handle long assignments effectively, but may feel slow at higher reasoning settings. It has also been observed leaving dead code behind.

The complete community run included two agents that reached the 90-minute time limit.

A model can achieve a strong benchmark result while still consuming excessive time, tokens or supervision.

Businesses do not pay benchmark scores. They pay for completed work.

5. Ox Alpha Still Needs an Official Repeated Evaluation

A credible final assessment requires:

• all 113 DeepSWE tasks;
• the same agent harness used for comparison models;
• disclosed reasoning settings;
• repeated trials;
• complete task-level results;
• token and latency measurements;
• failure analysis;
• an official leaderboard submission;
• independent reproduction.

Until that evaluation exists, Ox Alpha should be described as competitive—not dominant.

Why Coding Benchmarks Matter for Replacing Developers

Software engineering is not replaced when a model generates a convincing code sample.

It is replaced when an agent can reliably:

• understand a requirement;
• navigate an existing repository;
• implement a correct change;
• test its own work;
• recover from failures;
• avoid regressions;
• document the result;
• complete the assignment without continuous human intervention.

DeepSWE and Terminal-Bench are useful because they move evaluation closer to this standard.

A completion rate near 63% would still leave a substantial percentage of assignments unfinished or incorrectly implemented. Human review, validation and recovery would remain necessary.

However, replacement does not require a model to complete 100% of all possible software tasks. It requires the system to absorb enough recurring work that a company needs fewer developers to produce the same output.

This is the same economic mechanism examined in our ranking of the jobs AI will replace.

The Verdict

Ox Alpha has not been proven to beat GPT-5.6 or Claude 5.

The viral 80% DeepSWE result came from a sample of only ten tasks. A larger subset produced approximately 63%, while a subsequent complete community run reported 62.8%.

The agreement between those two broader evaluations makes Ox Alpha’s likely coding performance more credible. It places the model near the frontier group without establishing it as the leader.

Additional Terminal-Bench and EVMBench tests support the conclusion that Ox Alpha is a capable coding and agentic model. They remain community-run evaluations rather than definitive benchmark results.

The honest verdict is stronger than the hype:

Ox Alpha appears capable enough to compete with leading coding models on real software-engineering tasks, despite being anonymous and temporarily free.

That is already significant.

It does not need a misleading 80% headline to matter.

Frequently Asked Questions

What Is the Ox Alpha Benchmark Score?

Ox Alpha initially scored approximately 80% on a ten-task DeepSWE sample. A larger subset produced approximately 63%, while a subsequent complete community run reported 62.8%. Ox Alpha does not currently have an official DeepSWE leaderboard result.

Did Ox Alpha Score 80% on DeepSWE?

Yes, but the result came from only ten tasks. Completing eight produced the reported 80% score. It should not be treated as a complete DeepSWE benchmark result.

What Did Ox Alpha Score on the Complete DeepSWE Benchmark?

A community evaluator reported 62.8% from a complete run. Two agents reached the 90-minute time limit. The result represents one community evaluation and has not been added to the official leaderboard.

Did Ox Alpha Beat GPT-5.6?

Not in a verified complete comparison. Ox Alpha’s community full run produced 62.8%, while GPT-5.6 Sol is listed at approximately 73% on the official DeepSWE leaderboard. The runs were not conducted under fully identical conditions.

Is Ox Alpha Better Than Claude 5?

There is not enough evidence to conclude that Ox Alpha is better than Claude 5. Claude Opus 5 and Claude Fable 5 currently have higher official DeepSWE results than the reported Ox Alpha community run.

Is Ox Alpha on the Official DeepSWE Leaderboard?

No. As of August 24, 2026, Ox Alpha does not have an official entry on the DeepSWE leaderboard.

Is Ox Alpha Good for Coding?

Available tests indicate that Ox Alpha is a capable coding model, particularly for long-running agentic assignments. Its consistency, speed and official performance across repeated complete evaluations still require verification.

What Is the Most Reliable Ox Alpha Benchmark Result?

The complete community run reporting 62.8% is more informative than the original ten-task result. The approximately 63% larger-subset evaluation supports a similar performance range. Neither is currently an official DeepSWE leaderboard result.