New: FixBugs for VS Code is now available. Install from Marketplace 

OpenRouter

Why FixBugs Switched Away From OpenRouter

OpenRouter looked like the simplest way to give FixBugs users every model. Five reasons we moved the default to Claude and Codex subscriptions and direct provider keys.

Kirti Rathore··6 min read

FixBugs v0.0.1 had one LLM setting: an OpenRouter API key. It gave users hundreds of models behind one key, and we thought that was both simple and sufficient.

We have since made Claude and ChatGPT (Codex) subscriptions the default, with direct keys for several providers as the alternative.

Five reasons:

  1. Model performance depends on the provider - Deepseek v4 Flash's perf on the TAU-Bench was found to vary between 58% and 81% based on the provider.
  2. Zero data retention (ZDR) depends on each provider's own policy, and OpenRouter doesn't give any guarantees.
  3. Bills are hard to predict. - Reliable providers like Alibaba can charge exhorbitantly.
  4. Most developers already pay for Claude Code or Codex.

1. No guarantees on Model Performance

OpenRouter sends each model ID to one of many providers. Each inference provider picks its own quantization level, KV-cache precision and other model deployment parameters.

Mo Moustafa runs an assistant that has handled over 18 million messages, about a third of them on open models through OpenRouter. In So you want to use OpenRouter? (HN discussion), he pulled OpenRouter's own provider grouped benchmark result for one model:

Bar chart of TAU-Bench Airline scores for DeepSeek V4 Flash 0731 by OpenRouter host: DeepSeek first-party 81.3, Alibaba Cloud 76.8, NextBit 76.6, Together 75.0, Cloudflare 75.0, DeepInfra 73.9, OpenInference 70.8, Nebius 65.3, DigitalOcean 58.4
Same model ID, 23-percent spread on tool calling.
  • In July, Fireworks (OpenRouter alternative) scored 46% on the same benchmark, 30 points under first-party.
  • On GPQA (knowledge), four providers fell from about 90 to between 70 and 76.

Some failures come back as 200 OK:

FailureWhat he saw
Blind visionDeepInfra's Qwen3.5 122B read "K" as "R" and red as blue. Venice and Together replied "no image provided" for MiniMax M3.
Empty completionsStreamLake carried ~20% of his DeepSeek traffic and 92% of the empty responses.
Ignored parametersreasoning.effort was accepted everywhere and had little effect based on evaluations.
Strict JSON"using strict json output also does not work on all the endpoints of the models" (alexcz on HN).

FixBugs passes each structured result to the next step of a debugging workflow, If the host serves a weaker model, the next step builds on a wrong hypothesis. We ran into a lot of these problems ourself.

Pinning providers fixes quality but removes fallback. Moustafa pinned three providers with fallbacks off. Two weeks later one host had dropped the model and the other two were returning 429 (Too many requests). His app went down.

2. Zero data retention you can't check

Paid FixBugs plans enforce ZDR. OpenRouter offers ZDR too. Its docs describe what that means:

  • It filters routing to endpoints whose provider policy says ZDR.
  • For private endpoints, retention status is self-declared, which OpenRouter does not independently verify.
  • From October 27, 2026, all inference, including ZDR traffic, is screened for flagged content. Such prompts "may be retained for human review, reporting, and preservation."
  • In-memory prompt caching doesn't count as retention.
  • Plugins and tools, such as web search, have their own retention policies.

In Section 16 of its Terms of Service, OpenRouter explicitly states:

"OPENROUTER MAKES NO REPRESENTATION OR WARRANTY REGARDING ANY MODEL PROVIDER'S DATA HANDLING, RETENTION, TRAINING, SECURITY, AVAILABILITY, OR INTELLECTUAL PROPERTY PRACTICES."

No one can prove whether a provider followed zero data retention. With a direct key you at least know which company holds it, and the retention terms are the ones you agreed with them. Our BYOK post lists them per provider.

3. Most developers already pay for Claude Code or Codex

Bar chart from the Stack Overflow Developer Survey 2026: Claude Code 65.5%, GitHub Copilot 58.7%, OpenAI Codex 29.5%, Cursor 25.5%, Google Antigravity 16.0%, Gemini Code Assist 14.6%, OpenCode 13.6%, JetBrains AI 12.8%
Two in three respondents used Claude Code. Data: .

JetBrains' 2026 survey of 15,000+ developers (April and May 2026) asked about use at work:

ToolMay '26January 2026
Claude Code39% (47% in the US)3%
GitHub Copilot21%29%
Codex16%0%
Cursor12%12%

90% of those developers use an AI coding agent at least weekly. 68% use one daily.

Claude Code and Codex come with a subscription that the developer or their employer already pays for.

An OpenRouter key means a second bill, paid upfront. We wanted to give users the choice of which inference provider to use.

4. The alternatives don't serve frontier models

FixBugs needs four models from each provider: a premium model, a fast worker, a multimodal model and a fallback. It also needs strict JSON together with tool calls. None of these platforms gave us that:

PlatformWhat we found
Groq6 production models: Llama 3.1 8B, Llama 3.3 70B, GPT OSS 120B and 20B, and two Whisper models. Strict structured outputs can't be combined with streaming or tool use.
Cerebras3 models in our inventory. No multimodal fallback.
FireworksFast open-weight inference, no Claude, GPT or Gemini. Tools and JSON schemas do not work concurrently. Prompt caching is not guaranteed.
TogetherTwo models behind the same endpoint handled schemas and tools differently. Kimi K2.6 and K2.7 sometimes returned fenced JSON instead of the requested schema.

5. Fees went up, and bills are hard to predict

OpenRouter's current fees, per its FAQ and TrueFoundry's tracker:

ChargeNow
Buying credits, Standard5.5%, $0.80 minimum
Buying credits, Business (since Sept 2026)8%
Using your own provider key through OpenRouter (BYOK)Free up to $25,000/month at list price, then 5%. Before July 14, 2026 the allowance was 1M requests/month.
Unused creditsExpire after a year of account inactivity
TaxPrices exclude tax. Stripe adds VAT or GST at purchase where required.

Example: a team buying $2,000 of credits a month pays $110 in fees on Standard, or $160 on Business. OpenRouter has also said it will replace the BYOK fee with a fixed monthly subscription, at a price not yet announced. Stripe agreed to acquire OpenRouter in August 2026.

Model prices are harder to predict:

  • Each host sets its own price for the same model.
  • Moving between providers throws away cached input. One HN commenter estimated that unpinned agentic coding costs about 2x because of this.

What we run now

v0.0.1Now
Default sign-inOne OpenRouter keyClaude or ChatGPT subscription, with options for OpenRouter and a dozen alternative providers
Who serves the modelWhichever host OpenRouter picksThe lab that trained it, or the provider you chose

OpenRouter is still one of the many inference providers you can use. DeepSeek's and Z.ai's own APIs lack JSON-schema output, so OpenRouter is how FixBugs users can reach those models.

If you use OpenRouter, check the per-provider benchmark board, then pin a short list of providers from the FixBugs Settings page.

Sources