The right model is not always the newest model, the largest model, or the one at the top of a public benchmark. It is the model—and the surrounding system—that reliably completes your business task within your quality, risk, latency, and cost constraints.
Free 2026 buyer's guide
LLM Selection Cheat Sheet
A concise four-page guide to shortlisting models, running a business-specific evaluation, and comparing quality, reliability, safety, latency, cost, and deployment fit.
No email required · Updated September 2026
Start with the job, not the leaderboard
Model selection becomes much clearer when the team first defines the user, workflow, source data, expected output, acceptable latency, and cost of a wrong answer. A single organization may use a fast, economical model for routine extraction and a frontier reasoning model only for the cases that need it.
Task quality
Does it produce the correct, complete output on representative work from your business?
Reliability
Does it stay dependable across repeated runs, hard cases, ambiguous inputs, and tool failures?
Risk and security
Can you control data handling, permissions, citations, refusals, human review, and audit evidence?
Operating fit
Does the complete system meet your latency, throughput, integration, deployment, and support needs?
Total cost
Measure cost per successful task, including retrieval, tools, retries, review, and ongoing operations.
Portability
Can you version the model, monitor changes, and move workloads when performance or economics change?
Current model and evaluation snapshot
This is a dated shortlist—not a universal ranking. The public results below were reported in September 2026 and can help identify candidates for your own evaluation. Model settings, tools, reasoning effort, and the evaluation harness materially affect results.
| Model | AA Index v4.1.1 | Selected published signal | Published API price / 1M tokens |
|---|---|---|---|
| OpenAI GPT-6 Astra ↗ | 61.2 | AutomationBench 41.4%; BrowseComp 91.5%; OSWorld 2.0 partial 72.6% | $10 input / $50 output |
| Claude Fable 5.1 ↗ | 65.7 | AutomationBench 31.4% in OpenAI's comparison; positioned by Anthropic for coding and knowledge work | $10 input / $50 output |
| Claude Opus 5 ↗ | 63.1 | AutomationBench 26.9%; BrowseComp 90.8%; OSWorld 2.0 partial 70.2% | Check current pricing ↗ |
| Gemini 3.8 Flash ↗ | 58.7 | Google positions it for long-horizon engineering, autonomous agents, multimodal inputs, and enterprise workflows | $0.75 input / $3.75 output introductory pricing through Dec. 31, 2026 |
AA Index and selected comparison results: OpenAI, September 3, 2026. Pricing and availability can change; verify the linked provider page before making a decision. A missing score is not evidence of poor performance.
What public benchmarks cannot decide for you
A benchmark cannot tell you whether the model understands your terminology, uses your retrieval system well, returns valid structured data, respects your permissions, or produces an answer that a real user can act on. It also cannot measure the complete cost of retries, tool calls, review, and failure.
The best model is the least expensive system that reliably meets your acceptance criteria—not necessarily the model with the highest public score.
Run an evaluation that reflects your business
- Define acceptance criteria. Write down what a correct answer must contain, what it must avoid, and when a person must approve the result.
- Assemble representative cases. Include normal work, difficult cases, incomplete inputs, safety tests, and examples that should be refused or escalated.
- Freeze the test setup. Use the same prompts, sources, tools, permissions, output format, and reasoning settings for every candidate.
- Run repeated trials. Measure average performance and worst-case failures, because generative systems can vary from one run to the next.
- Review errors, not only averages. A high score may still be unacceptable if the remaining failures include fabricated citations, privacy violations, or unauthorized actions.
- Pilot with real users. Measure time saved, task completion, review burden, adoption, and business impact before expanding.
Useful starting points include OpenAI's evaluation guidance, Amazon Bedrock model evaluation, and the NIST AI Risk Management Framework.
Need a defensible decision?
We can turn your workflow into an evaluation and production plan.
Ai Applied helps teams define the use case, shortlist models, build representative tests, assess security and operating fit, and select an architecture that can evolve as models change.