DeepSeek V4 review 2026: benchmarks, pricing, my test run

I almost skipped this release. The April preview was fine in a forgettable way, and I assumed V4 would be another quiet update I could ignore. Then Flash-0731's model card showed a 20-point jump on Terminal Bench, the kind of number that makes you actually run the thing.

I spent a few hours with DeepSeek V4 this week, and here's the honest version of the story: the value math is real, the coding claims mostly hold up, and the gap between DeepSeek's own benchmark table and an independent government evaluation is the most interesting part of the release. This review covers both models, the official numbers, the NIST CAISI cross-check, and my own API test run from August 3, 2026, three days after the Flash-0731 release. If you only want the scores, the short version is next.

The short version

V4-Flash-0731 at $0.14/$0.28 per million tokens is the best value in agentic coding right now.

V4-Pro is not the frontier model DeepSeek's charts suggest. CAISI puts it about eight months behind the US frontier, and it's priced accordingly. Buy Flash for cost, rent Pro for hard reasoning. And never trust a vendor's benchmark table without a cross-check.

• Flash-0731 shipped July 31, 2026 as the first stable V4 release, same price as the April preview with a big capability jump.
• Both models are MIT-licensed open weights with a native 1M token context. V4-Pro is still a preview.
• The legacy deepseek-chat and deepseek-reasoner aliases died July 24, 2026. The swap is one line of config.

What DeepSeek V4 is

DeepSeek shipped V4 on April 24, 2026 as two open-weight models, both with a 1M token context window and an MIT license.

The architecture is the interesting part. Compressed sparse attention (CSA) plus heavily compressed attention (HCA) makes long context cheap. Manifold-constrained hyper-connections and the Muon optimizer replace the old residual wiring and AdamW, and expert weights are stored in FP4, which halves memory.

V4-Pro runs 1.6 trillion total parameters with 49B active per token. Flash runs 284B total, 13B active. Both carry a 1M token context window and a 384K token output cap, and both are MIT-licensed. The API model IDs are deepseek-v4-pro and deepseek-v4-flash. Pro is still a preview build; Flash-0731 went stable on July 31.

The Flash-0731 release is the sleeper story. Same architecture, same price, but DeepSeek's model card shows Terminal Bench 2.1 jumping from 61.8 to 82.7, past the April preview's V4-Pro score of 72.1.

The cheap model became the strong model overnight.

I tested the deepseek-v4-flash endpoint, which the API docs confirm now serves Flash-0731, so my hands-on numbers below are for the stable build, exactly what a new user gets.

Moving to V4

If you were on the old aliases, you're already past the deadline. deepseek-chat and deepseek-reasoner stopped working on July 24, 2026. The migration itself is one line: swap the model ID, keep everything else. The API is OpenAI-compatible, and there's an Anthropic-compatible endpoint too. Claude Code, GitHub Copilot, and OpenCode can all use DeepSeek as a backend with a config change and no code changes. A lot of teams are doing exactly that for cost reasons.

Two gotchas from my test:

• deepseek-v4-pro defaults to thinking mode on. Pass thinking: {"type": "disabled"} for direct output.
• Set your own max_tokens. I clipped two responses in my first run because I left it at 64.

DeepSeek vs GPT: the official numbers

Every number in this section is vendor-reported: DeepSeek's technical report (arXiv:2606.19348) for V4, and each competitor's own release materials for theirs. "n/r" means not reported in the materials I used. That caveat matters, and the independent check below is why.

Read honestly: V4-Pro leads on LiveCodeBench and Codeforces, ties everyone on SWE-bench Verified within noise, and trails clearly on knowledge. Humanity's Last Exam and SimpleQA are the weak spots. The SimpleQA gap (57.9 vs Gemini's 75.6) matches what people report in daily use: ask V4 about facts and it drifts.

What CAISI found

NIST's Center for AI Standards and Innovation is the US government's evaluation shop for commercial AI, running unclassified evals of frontier models across five domains: cyber, software engineering, natural sciences, abstract reasoning, math. Most of its benchmarks are closed. That's the whole point: the model can't have trained on them. It's independent of the vendors it tests; how much its government framing colors the results is your call.

CAISI evaluated V4-Pro in April 2026 and published on May 1. Three findings stood out:

1. V4 lags the US frontier by roughly 8 months. That's CAISI's aggregate measure, an IRT-style score over 16 benchmarks and 35 models. DeepSeek's own table makes V4 look like a peer of GPT-5.4 and Opus 4.6, released about two months earlier. CAISI's closed suite puts V4 at roughly GPT-5's level, released about eight months earlier. That's the contamination gap, quantified.

2. V4 is still the most capable Chinese model CAISI has evaluated. Its strongest areas are math, software engineering, and natural sciences. V3 trailed by 12-14 months. V4 trails by 8. The trend is closing.

3. It wins on capability per dollar. V4 was cheaper than GPT-5.4 mini on 5 of 7 benchmark tasks, ranging from 53% less expensive to 41% more expensive depending on the task.

The weaknesses CAISI found are exactly the ones DeepSeek's report doesn't feature: ARC-AGI-2 semi-private, PortBench (a held-out software engineering eval CAISI built), CTF-Archive-Diamond (cyber). Reasoning under novel conditions, not canned coding.

One detail I respect: CAISI reproduced DeepSeek's self-reported GPQA-Diamond results before running its own suite, which rules out serving or configuration errors. I wish every benchmark article did that.

My hands-on test

Small smoke test, August 3, 2026: 11 tasks, two models, rubrics I wrote in advance, answers that could be checked against ground truth.

I executed and tested every code answer, and parsed the JSON responses. Exact answers were checked against the rubrics. Thinking mode was off on both models so the outputs are comparable. Full prompts and raw logs are in the test notes at the bottom.

Total cost for all 22 calls: $0.0013. Flash averaged 1.1s per response, Pro 2.0s.

I ran the battery twice to check the one failure, and Flash answered "341" both times.

I don't know why 341. I just know a model priced at $0.28 per million output tokens can't reliably multiply 23 by 25 with thinking off.

Admittedly, 11 tasks can't prove the 79% SWE-bench number isn't contaminated. But they do prove that even if that leaderboard is cooked, the live API still writes passing Python at $0.28 per million output tokens.

A few things surprised me:

• Both models wrote clean Python that passed its tests on the first try. The bug-fix answers were genuinely elegant: Flash returned sorted(set(xs)), which is exactly the fix I'd write.

• Neither model respected "answer with just the number" on the first math task. They explained, then buried the answer. Pro did this on both math tasks. With thinking off, both models overexplain.

• Pro's IPv4 code rejected leading zeros ("1.2.3.04" as False). Flash's didn't. The expensive model was stricter.

• Flash's product copy used the word "robust". Which is funny, because that word sits on every AI writing tell list I've read. Even the model itself can't shake the tell.

• Pro wrapped its JSON summary in a code fence. Flash returned bare JSON. Both parsed fine.

Pricing: the actual story

Pro's price was cut 75% in July, from $1.74/$3.48 at launch to $0.435/$0.87. Reuters reported the cut.

On the Artificial Analysis Intelligence Index, Flash-0731 scores 50, third among open weights, at about $0.03 per index task, the cheapest on the whole board. Gemini 3.6 Flash scores the same 50 and costs $0.56 per task.

The long context pricing is where the gap gets silly. GPT-5.4 charges $2.50 per million input tokens up to 272K, then doubles it beyond that. DeepSeek charges a flat $0.14 per million. Feed a 300K token document to both and you pay about 18× more on the GPT side before a single output token.

One thing I couldn't verify: several third party posts mention peak hour pricing, roughly 9:00-12:00 and 14:00-18:00 Beijing time billed at 2×. It's not in the official pricing docs I could reach, so treat it as unconfirmed. If it's real, it changes the cost math for Asian timezone workloads.

Bottom line

Use it if you pay for coding agents out of pocket, run batch jobs over long documents, or want open weights you can self-host and audit. The 1M token context at these prices makes whole repo passes practical in a way GPT-5.4's $2.50/$15 simply isn't.

Skip it if you need frontier knowledge or novel reasoning. SimpleQA and CAISI's closed tasks both say no. Skip it for multimodal input too, since V4 is text only.

Enterprise data residency with western SLAs is a different conversation. And if you can't tolerate a wrong answer on a simple multiplication without noticing, this isn't the model for you, at least not with thinking off.

Cheap and correct are different products.

DeepSeek V4-Flash-0731 gives you the most capability per dollar of any LLM right now, and it's not close. V4-Pro is a good model wearing a price tag sized for a frontier model: competent, but CAISI's 8-month lag is the number I'd quote in a procurement meeting, not the SWE-bench table.

The thing I want to test next is Pro with thinking mode on a real 100K token codebase, where the extra latency and cost might actually buy something. I'll report back when I've done it.

Test notes & sources

My own data

11 tasks per model, run once per battery on 2026-08-03 via the official DeepSeek API (api.deepseek.com, OpenAI-compatible endpoint). No temperature override, max_tokens 64-512 per task, thinking disabled on both models for comparability. I executed and tested code answers, parsed JSON answers, and checked exact answers against values set in advance. The full battery ran twice; the single failure reproduced. N=1 per task per model. These are smoke-test signals, not a statistical eval.

Vendor-reported numbers

All DeepSeek benchmark numbers come from the technical report (arXiv:2606.19348, submitted 2026-04-26) and the Hugging Face model cards. Competitor numbers come from GPT-5.4, Claude Opus 4.6, and Gemini 3.1 Pro release materials as compiled in April 2026 coverage. Pricing comes from official pricing pages and the Reuters report on the July cut, retrieved 2026-08-03.

The independent check

NIST CAISI's "Evaluation of DeepSeek V4 Pro" (2026-05-01, updated 05-02) used non-public benchmarks across five domains. Also CAISI's "Evaluation of DeepSeek AI Models" (2025-09) and the NIST AI 800-2 draft (2026-01) for methodology context.

What I didn't test

The 1M token context at full length (my needle test was 426 tokens), and self-hosting cost and latency (I used the API). There's also no "stable" V4-Pro release to test, because none is announced. I also didn't test Pro's long-context or multi-step reasoning stability, so if that's what you need, treat the Pro verdict as provisional.

Sources

• DeepSeek V4 technical report, arxiv.org/abs/2606.19348 (2026-04-26)
• DeepSeek API docs: model IDs, Flash-0731 note, pricing, api-docs.deepseek.com
• NIST CAISI, Evaluation of DeepSeek V4 Pro, nist.gov/news-events/news/2026/05/caisi-evaluation-deepseek-v4-pro (2026-05-01)
• NIST CAISI, Evaluation of DeepSeek AI Models (2025-09-30)
• NIST AI 800-2 ipd, Practices for Automated Benchmark Evaluations (2026-01)
• OpenAI API pricing: GPT-5.4 / 5.4 mini, openai.com/api/pricing (July 2026)
• Anthropic and Google pricing pages: Claude Opus 4.6, Gemini 3.1 Pro (retrieved 2026-08-03)
• Reuters: DeepSeek V4-Pro price cut (July 2026)
• felloai.com DeepSeek V4 page: Flash-0731 details, Artificial Analysis figures (retrieved 2026-08-03)
• My API test logs: 22 raw responses, 2026-08-03 + 2026-08-07 rerun, published

Disclosure: no sponsors, no affiliate links. I paid for the API calls out of pocket, $0.0013 total, which is itself a data point.

P.S. August 4, 2026: I re-ran the 23×25 task with Flash's thinking mode on, and it answered 575 three times in a row. Pro, which defaults to thinking on, was correct too. So the multiplication miss was a thinking-off artifact: same prompt, two different answers, and the only thing I changed was one flag.

Update, August 7, 2026: I re-ran the full 11-task battery on both models with thinking off (same sampling parameters, prompts pre-registered from the published task set) and archived every request and every raw response — 22 calls, full logs published. Scores matched the original run (Pro 11/11, Flash 10/11). One honest wrinkle: Flash's miss moved. On August 3 it failed 23×25 with thinking off (answered 341 twice); on August 7 it solved that one but answered a quadratic "9, -4" (the roots are 4 and -3). Same score, different failure — which is exactly the kind of wobble you should expect from N=1 smoke tests with thinking off, and exactly why we publish the raw logs rather than a summary table.