AI Reality CheckClear answers. Real evidence.The Free Tier Reality Check · September 2026
Each product received the same 12 practical tasks. We preserved 108 responses, evaluated capability independently, audited instruction-following and recorded what free access was actually like.
The central finding
Grok and Claude led the recorded capability means. ChatGPT and Claude both passed all 12 instruction-compliance checks. Copilot’s recoverable answers were undermined by severe display problems throughout the review. Which result matters most depends on what you need from the product.
Capability
Scored from 1–5 under the frozen evaluation rubric. Ordering shows the recorded means, not a statistically proven league table.
Instruction audit
A separate audit checked whether each answer actually followed the controlled prompt. These categories were not folded into capability scores.
What the averages cannot show
All nine accepted the controlled prompts. The experience of getting a usable answer was not equal.
Severe display and retrieval failures affected all 12 tests. Answers were often blank, offscreen or partly visible, although the text was usually recoverable through Copy.
Anonymous access stopped after two prompts. The anonymous attempt was excluded and the controlled run was completed after signing into a confirmed Free account.
One task returned an app-access request instead of the requested email, leaving 11 of 12 usable answers recovered.
The 12-task candidate run completed. A later, separate evaluator workload exhausted the free allowance and required waiting for a reset.
Read this before quoting the results