Why FixBugs Switched Away From OpenRouter
OpenRouter looked like the simplest way to give FixBugs users every model. Five reasons we moved the default to Claude and Codex subscriptions and direct provider keys.
FixBugs v0.0.1 had one LLM setting: an OpenRouter API key. It gave users hundreds of models behind one key, and we thought that was both simple and sufficient.
We have since made Claude and ChatGPT (Codex) subscriptions the default, with direct keys for several providers as the alternative.
Five reasons:
- Model performance depends on the provider - Deepseek v4 Flash's perf on the TAU-Bench was found to vary between 58% and 81% based on the provider.
- Zero data retention (ZDR) depends on each provider's own policy, and OpenRouter doesn't give any guarantees.
- Bills are hard to predict. - Reliable providers like Alibaba can charge exhorbitantly.
- Most developers already pay for Claude Code or Codex.
1. No guarantees on Model Performance
OpenRouter sends each model ID to one of many providers. Each inference provider picks its own quantization level, KV-cache precision and other model deployment parameters.
Mo Moustafa runs an assistant that has handled over 18 million messages, about a third of them on open models through OpenRouter. In So you want to use OpenRouter? (HN discussion), he pulled OpenRouter's own provider grouped benchmark result for one model:
- In July, Fireworks (OpenRouter alternative) scored 46% on the same benchmark, 30 points under first-party.
- On GPQA (knowledge), four providers fell from about 90 to between 70 and 76.
Some failures come back as 200 OK:
| Failure | What he saw |
|---|---|
| Blind vision | DeepInfra's Qwen3.5 122B read "K" as "R" and red as blue. Venice and Together replied "no image provided" for MiniMax M3. |
| Empty completions | StreamLake carried ~20% of his DeepSeek traffic and 92% of the empty responses. |
| Ignored parameters | reasoning.effort was accepted everywhere and had little effect based on evaluations. |
| Strict JSON | "using strict json output also does not work on all the endpoints of the models" (alexcz on HN). |
FixBugs passes each structured result to the next step of a debugging workflow, If the host serves a weaker model, the next step builds on a wrong hypothesis. We ran into a lot of these problems ourself.
Pinning providers fixes quality but removes fallback. Moustafa pinned three providers with fallbacks off. Two weeks later one host had dropped the model and the other two were returning 429 (Too many requests). His app went down.
2. Zero data retention you can't check
Paid FixBugs plans enforce ZDR. OpenRouter offers ZDR too. Its docs describe what that means:
- It filters routing to endpoints whose provider policy says ZDR.
- For private endpoints, retention status is self-declared, which OpenRouter does not independently verify.
- From October 27, 2026, all inference, including ZDR traffic, is screened for flagged content. Such prompts "may be retained for human review, reporting, and preservation."
- In-memory prompt caching doesn't count as retention.
- Plugins and tools, such as web search, have their own retention policies.
In Section 16 of its Terms of Service, OpenRouter explicitly states:
"OPENROUTER MAKES NO REPRESENTATION OR WARRANTY REGARDING ANY MODEL PROVIDER'S DATA HANDLING, RETENTION, TRAINING, SECURITY, AVAILABILITY, OR INTELLECTUAL PROPERTY PRACTICES."
No one can prove whether a provider followed zero data retention. With a direct key you at least know which company holds it, and the retention terms are the ones you agreed with them. Our BYOK post lists them per provider.
3. Most developers already pay for Claude Code or Codex
JetBrains' 2026 survey of 15,000+ developers (April and May 2026) asked about use at work:
| Tool | May '26 | January 2026 |
|---|---|---|
| Claude Code | 39% (47% in the US) | 3% |
| GitHub Copilot | 21% | 29% |
| Codex | 16% | 0% |
| Cursor | 12% | 12% |
90% of those developers use an AI coding agent at least weekly. 68% use one daily.
Claude Code and Codex come with a subscription that the developer or their employer already pays for.
An OpenRouter key means a second bill, paid upfront. We wanted to give users the choice of which inference provider to use.
4. The alternatives don't serve frontier models
FixBugs needs four models from each provider: a premium model, a fast worker, a multimodal model and a fallback. It also needs strict JSON together with tool calls. None of these platforms gave us that:
| Platform | What we found |
|---|---|
| Groq | 6 production models: Llama 3.1 8B, Llama 3.3 70B, GPT OSS 120B and 20B, and two Whisper models. Strict structured outputs can't be combined with streaming or tool use. |
| Cerebras | 3 models in our inventory. No multimodal fallback. |
| Fireworks | Fast open-weight inference, no Claude, GPT or Gemini. Tools and JSON schemas do not work concurrently. Prompt caching is not guaranteed. |
| Together | Two models behind the same endpoint handled schemas and tools differently. Kimi K2.6 and K2.7 sometimes returned fenced JSON instead of the requested schema. |
5. Fees went up, and bills are hard to predict
OpenRouter's current fees, per its FAQ and TrueFoundry's tracker:
| Charge | Now |
|---|---|
| Buying credits, Standard | 5.5%, $0.80 minimum |
| Buying credits, Business (since Sept 2026) | 8% |
| Using your own provider key through OpenRouter (BYOK) | Free up to $25,000/month at list price, then 5%. Before July 14, 2026 the allowance was 1M requests/month. |
| Unused credits | Expire after a year of account inactivity |
| Tax | Prices exclude tax. Stripe adds VAT or GST at purchase where required. |
Example: a team buying $2,000 of credits a month pays $110 in fees on Standard, or $160 on Business. OpenRouter has also said it will replace the BYOK fee with a fixed monthly subscription, at a price not yet announced. Stripe agreed to acquire OpenRouter in August 2026.
Model prices are harder to predict:
- Each host sets its own price for the same model.
- Moving between providers throws away cached input. One HN commenter estimated that unpinned agentic coding costs about 2x because of this.
What we run now
| v0.0.1 | Now | |
|---|---|---|
| Default sign-in | One OpenRouter key | Claude or ChatGPT subscription, with options for OpenRouter and a dozen alternative providers |
| Who serves the model | Whichever host OpenRouter picks | The lab that trained it, or the provider you chose |
OpenRouter is still one of the many inference providers you can use. DeepSeek's and Z.ai's own APIs lack JSON-schema output, so OpenRouter is how FixBugs users can reach those models.
If you use OpenRouter, check the per-provider benchmark board, then pin a short list of providers from the FixBugs Settings page.
Sources
- Mo Moustafa, So you want to use OpenRouter?, September 2026, and the Hacker News discussion
- Stack Overflow Developer Survey 2026: AI code agents
- JetBrains Research, AI coding agent adoption in 2026
- OpenRouter, Zero Data Retention, FAQ and Tax ID
- r/openrouter, Is this a billing bug or a real change?
- TrueFoundry, OpenRouter BYOK explained
- Groq, Models and Structured outputs