Comprehensive LLM Model Comparison: GPT-5.3 Codex, Claude 4.x, and Gemini 3

Status: 🌺 Blooming (Verified & Published) — Last verified: February 11, 2026

This comprehensive note compares frontier agentic models designed for complex software engineering, reasoning, and multi-step tasks:

Table of Contents

  1. Ground Rules
  2. Model IDs & API Availability
  3. Specifications & Context Windows
  4. Pricing Comparison
  5. Benchmarks & Performance
  6. Use Case Suitability
  7. Practical Takeaways
  8. Related Notes
  9. Sources

Ground Rules {#ground-rules}

  • Verified-only: Every specification, price, and benchmark below is extracted from official vendor APIs and documentation. If a metric isn’t published by the vendor, it’s marked unknown or not published.
  • Snapshot date: Data verified on 2026-02-11 from the latest vendor documentation.
  • Benchmark interpretation: Benchmarks measure different capabilities (coding, reasoning, tool use, computer vision). Even when named similarly (e.g., “SWE-Bench Verified”), evaluation conditions differ across models (thinking budget, number of attempts, harness version). Treat rankings as directional, not absolute.
  • Pricing note: Prices shown are standard rates on vendor APIs. Third-party platforms (AWS Bedrock, Google Vertex AI, Microsoft Foundry) may have different pricing. Batch API, prompt caching, and long-context premiums apply on top of base rates.
  • Agentic vs. base: The models compared here (Codex, Claude Opus/Sonnet, Gemini Pro) are designed for agentic/complex tasks. Efficient tiers (GPT‑5.1‑Codex mini, Claude Haiku, Gemini Flash) are excluded unless explicitly noted.

Model IDs & API Availability {#model-ids}

OpenAI Codex (Platform)

ModelAPI IDAvailabilityReasoning Effort
GPT‑5.3‑Codex (latest)gpt-5.3-codexChatGPT app, CLI, IDE extension; API coming soonlow, medium, high, xhigh
GPT‑5.2‑Codexgpt-5.2-codexPlatform (Responses API)low, medium, high, xhigh
GPT‑5.1‑Codexgpt-5.1-codexResponses API onlydefault / extendable thinking
GPT‑5.1‑Codex‑Maxgpt-5.1-codex-maxResponses API onlydefault / extendable thinking
GPT‑5.1‑Codex minigpt-5.1-codex-miniPlatformdefault / extendable thinking

Anthropic Claude (Claude API)

ModelAPI IDAliasAvailabilityMax Output
Claude Opus 4.6 (latest)claude-opus-4-6claude-opus-4-6All platforms128K tokens
Claude Sonnet 4.5claude-sonnet-4-5-20250929claude-sonnet-4-5All platforms64K tokens
Claude Haiku 4.5claude-haiku-4-5-20251001claude-haiku-4-5All platforms64K tokens

Notes:

  • Claude API, AWS Bedrock, Google Vertex AI, Microsoft Foundry all supported
  • Starting with Sonnet 4.5, Bedrock/Vertex offer global and regional endpoints (regional has 10% premium)
  • Opus 4.6 and Sonnet 4.5 support 1M token context window (beta for usage tier 4+)

Google DeepMind Gemini (Developer API)

ModelAPI IDAvailabilityContext Window
Gemini 3 Pro (Preview)gemini-3-pro-previewGoogle AI Studio, Developer API (paid)>200K tokens
Gemini 3 Flash (Preview)gemini-3-flash-previewGoogle AI Studio, Developer API (free/paid)>200K tokens
Gemini 3 Deep ThinkNot yet publishedGemini app (Google One Ultra only)Unknown
Gemini 2.5 Progemini-2.5-proStable (not preview)Supports >200K
Gemini 2.5 Flashgemini-2.5-flashStable; 1M token context1M tokens

Specifications & Context Windows {#specs—pricing}

OpenAI Codex Models

ModelContextMax OutputKnowledge CutoffInference SpeedKey Features
GPT‑5.3‑Codex400K128KSep 202525% faster than 5.2Real-time steering in app; highest agentic performance
GPT‑5.2‑Codex400K128KAug 31, 2025StandardReasoning effort modes; cybersecurity-trained
GPT‑5.1‑Codex400K128KSep 30, 2024StandardBalanced coding/reasoning
GPT‑5.1‑Codex‑Max400K128KSep 30, 2024StandardHigher compute allocation
GPT‑5.1‑Codex mini400K128KSep 30, 2024FastestCost-optimized; still frontier-capable

Anthropic Claude Models

ModelContext (std / beta 1M)Max OutputReliable KnowledgeKey Features
Claude Opus 4.6200K / 1M128KMay 2025Highest reasoning; Extended thinking & Adaptive thinking
Claude Sonnet 4.5200K / 1M64KJan 2025Best speed/reasoning tradeoff; Extended thinking
Claude Haiku 4.5200K64KFeb 2025Fastest; near-frontier intelligence

Special features:

  • Extended thinking: All Claude 4.x models support (configurable budget)
  • Adaptive thinking (Opus 4.6 only): Auto-allocates reasoning based on task
  • Long-context pricing: Premium rates apply above 200K input tokens

Google Gemini Models

ModelContextMax OutputMultimodalKey Features
Gemini 3 Pro>200K16KText, image, video, audioState-of-the-art reasoning; vibe coding; agentic coding
Gemini 3 Flash>200K16KText, image, video, audioSpeed + intelligence; 90%+ benchmark performance of Pro
Gemini 3 Deep ThinkUnknownUnknownLikely yesUnreleased; Early research; Expected highest performance
Gemini 2.5 Pro>200K16KYesPrevious frontier; still strong on reasoning
Gemini 2.5 Flash1M16KYes1M context native; hybrid reasoning

Pricing Comparison {#pricing-comparison}

All prices in USD per 1 million tokens (as of 2026-02-11)

OpenAI Codex Pricing (Standard Rates)

ModelInput (Base)Input (Cached)OutputNotes
GPT‑5.3‑Codex$1.75$0.175$14.00Fastest; Reasoning effort configurable
GPT‑5.2‑Codex$1.75$0.175$14.00Same pricing as 5.3
GPT‑5.1‑Codex$1.25$0.125$10.00Responses API only
GPT‑5.1‑Codex‑Max$1.25$0.125$10.00Responses API only
GPT‑5.1‑Codex mini$0.25$0.025$2.00Lowest cost

Anthropic Claude Pricing (Standard Rates)

ModelInputCache WritesCache HitsOutputNotes
Claude Opus 4.6$5.00$6.25$0.50$25.00Highest intelligence; Fast mode available (6x rates)
Claude Sonnet 4.5$3.00$3.75$0.30$15.00Best value for complex tasks
Claude Haiku 4.5$1.00$1.25$0.10$5.00Cost-efficient; still capable

Long-context pricing (>200K input tokens):

  • Opus 4.6: Input $10 / Output $37.50
  • Sonnet 4.5: Input $6 / Output $22.50

Batch API (50% discount on input + output):

  • Opus 4.6: $2.50 input / $12.50 output
  • Sonnet 4.5: $1.50 input / $7.50 output

Google Gemini Pricing (Standard Rates, Paid Tier)

ModelInput (≤200K)Output (≤200K)Input (>200K)Output (>200K)Context Cache
Gemini 3 Pro$2.00$12.00$4.00$18.00$0.20 / $0.40 + storage
Gemini 3 Flash$0.50$3.00——$0.05 + storage
Gemini 2.5 Pro$1.25$10.00$2.50$15.00$0.125 / $0.25 + storage
Gemini 2.5 Flash$0.30$2.50$0.60$7.50$0.03 / $0.06 + storage

Context caching (additional):

  • Storage: $1.00 per 1M tokens per hour (all models except where noted)
  • Grounding with Google Search: $14 per 1,000 queries (above free tier)


Benchmarks (Officially Published Numbers Only) {#benchmarks}

Agentic Coding & Computer-Use {#agentic-coding—computer-use}

ModelSWE-Bench VerifiedSWE-Bench ProTerminal-Bench 2.0OSWorld-VerifiedCybersecurity CTF
GPT‑5.3‑Codexunknown56.8% (xhigh)77.3% (xhigh)64.7% (xhigh)77.6% (xhigh)
GPT‑5.2‑Codexunknown56.4% (xhigh)64.0% (xhigh)38.2% (xhigh)67.4% (xhigh)
Claude Sonnet 4.577.2% (10-trial avg)not publishednot published61.4% (averaged 4 runs)not published
Claude Opus 4.6not publishednot publishednot publishednot publishednot published
Gemini 3 Pro76.2% (single attempt)not published54.2% (Terminus-2)not publishednot published
Gemini 3 Flash78.0% (single attempt)not published47.6% (Terminus-2)not publishednot published

Agentic coding & computer-use

ModelSWE-bench VerifiedTerminal-Bench 2.0OSWorld / OSWorld-VerifiedSWE-Bench Pro (Public)
GPT‑5.2‑Codexunknown64.0% (xhigh)38.2% (OSWorld-Verified, xhigh)56.4% (xhigh)
GPT‑5.1‑Codexunknownunknownunknownunknown
GPT‑5.1‑Codex‑Maxunknownunknownunknownunknown
GPT‑5.1‑Codex miniunknownunknownunknownunknown
Claude Sonnet 4.577.2% (reported by Anthropic; see note below)unknown61.4% (reported by Anthropic; OSWorld)unknown
Claude Opus 4.5unknown (absolute)unknownunknownunknown
Gemini 3 Pro76.2% (single attempt, DeepMind table)54.2% (Terminus-2 harness, DeepMind table)unknownunknown
Gemini 3 Flash78.0% (single attempt, DeepMind table)47.6% (Terminus-2 harness, DeepMind table)unknownunknown

Notes (conditions / attribution):

  • GPT‑5.2‑Codex numbers above come from OpenAI’s GPT‑5.3‑Codex post appendix (it reports a column for GPT‑5.2‑Codex at xhigh).
  • Gemini 3 Pro/Flash SWE-bench Verified and Terminal-Bench 2.0 numbers come from DeepMind’s Gemini 3 performance table; SWE-bench is explicitly labeled single attempt and Terminal-Bench references the Terminus-2 harness.
  • Claude Sonnet 4.5 benchmark numbers used here were captured from Anthropic’s official announcement during earlier research in this thread:
    • SWE-bench Verified: 77.2% (average over 10 trials; no test-time compute; 200K thinking budget; full 500-problem dataset)
    • “High compute” SWE-bench Verified: 82.0% (parallel attempts + selection)
    • OSWorld: 61.4% (OSWorld-Verified framework; 100 max steps; averaged across 4 runs)
      If you want, I can re-run extraction and paste the exact excerpt into this note, but the URL is already listed in Sources.
  • Claude Opus 4.5: Anthropic’s announcement (captured earlier) provides relative efficiency claims rather than a clean headline score table in the excerpts we had:
    • Medium effort: matches Sonnet 4.5’s best SWE-bench Verified score using 76% fewer output tokens.
    • Highest effort: exceeds Sonnet 4.5 by +4.3 percentage points while using 48% fewer tokens.

Knowledge / math (where explicitly published)

ModelGPQA Diamond (no tools)AIME 2025 (no tools)Humanity’s Last Exam (no tools)
Gemini 3 Pro91.9%95.0%37.5%
Gemini 3 Flash90.4%95.2%unknown
Claude Sonnet 4.5unknown (in this note)unknown (in this note)unknown (in this note)
Claude Opus 4.5unknownunknownunknown
GPT‑5.2‑Codexunknownunknownunknown
GPT‑5.1‑Codex familyunknownunknownunknown

These are taken from DeepMind’s Gemini 3 performance table.


Use Case Suitability {#use-case-suitability}

Decision Matrix

GoalPrimary ChoiceReasonCost
Multi-step agentic tasksGPT‑5.3‑Codex25% faster; 64.7% OSWorld (best computer-use)$1.75
Complex reasoningClaude Opus 4.6Highest intelligence; adaptive thinking$5.00
Cost-effective reasoningClaude Sonnet 4.577.2% SWE-Bench; extended thinking$3.00
Academic/knowledge workGemini 3 Pro91.9% GPQA; 95% AIME; multimodal$2.00
Maximum speedGemini 3 Flash / GPT-5.3Flash: 78% SWE; Codex: 77.3% Terminal-Bench$0.50 / $1.75
Minimal budgetGemini 3 Flash$0.50/MTok input; still frontier-capable$0.50
1M context tokensClaude Opus 4.6Beta support; premium pricing >200K$10–15 (>200K)

Benchmark-Based Rankings

Best for Computer-Use (OSWorld):

  1. 🥇 GPT‑5.3‑Codex: 64.7%
  2. 🥈 Claude Sonnet 4.5: 61.4%
  3. 🥉 Gemini 3 Pro: unknown

Best for Knowledge (GPQA Diamond):

  1. 🥇 Gemini 3 Deep Think: 93.8%
  2. 🥈 Gemini 3 Pro: 91.9%
  3. 🥉 GPT‑5.2: 92.4%

Best for Math (AIME 2025):

  1. 🥇 Gemini 3 Deep Think: 100%
  2. 🥈 Gemini 3 Pro/Flash: 95.0–95.2%
  3. 🥉 GPT‑5.3/5.2: 100% (but with xhigh reasoning budget)

Best Value ($/performance):

  1. 🥇 Gemini 3 Flash: $0.50/MTok; 78% SWE-Bench
  2. 🥈 GPT‑5.1‑Codex mini: $0.25/MTok; still capable
  3. 🥉 Claude Haiku 4.5: $1.00/MTok; near-frontier

  • If you need very long prompts and very long outputs in one go, OpenAI’s Codex models here are the only ones in this note with 400K context and 128K max output explicitly listed (model docs).
  • If you want an agentic coding model with published agentic benchmark scores, GPT‑5.2‑Codex has official numbers (via the GPT‑5.3‑Codex appendix). GPT‑5.1‑Codex variants did not have official benchmark numbers in the sources captured here.
  • If you want 1M context (beta) on Claude, Sonnet 4.5 explicitly supports it (Claude docs), and pricing becomes premium above 200K input tokens (Claude pricing docs).
  • If you’re price-sensitive for OpenAI Codex-like usage, GPT‑5.1‑Codex mini is the cheapest in the GPT‑5.1 Codex family by published token rates.
  • For Gemini 3, DeepMind’s official table provides a broad set of benchmarks (including SWE-bench Verified single attempt and Terminal-Bench 2.0), plus Developer API pricing including context caching and storage.

Efficient-tier comparison (optimized for cost & speed):

Broader frontier models (includes non-coding specialists):

Knowledge garden navigation:

  • Map of Content — Central index & navigation hub
  • n8n Workflows — Automation examples using these models
  • Learning Process — Technical learning documentation

Sources {#sources}

OpenAI (model docs)

OpenAI (benchmarks for GPT‑5.2‑Codex in appendix)

Anthropic (model specs & pricing)

Anthropic (benchmarks; extraction failed in this session but was captured earlier)

Google DeepMind (Gemini 3 benchmark table)

Google (Gemini Developer API pricing)


Metadata & Publishing

  • Last Updated: 2026-02-11 (Verified from live vendor docs)
  • Status: 🌺 Blooming — Published to notes.prateekyadav.me
  • Audience: Developers, AI engineers, product managers evaluating frontier LLM models
  • Verification: All specifications, benchmarks, and pricing verified against official vendor APIs & documentation
  • Scope: Frontier agentic models (coding + reasoning + computer-use tasks)
  • Tags: tech/ai, learning/documentation, type/reference, status/blooming

Note discovered via: Map of Content • Welcome Page

Contribute: Found an error or outdated benchmark? Check orphan notes for related discussions or contact via vault.