FixBugs is now Free Forever
How implemented the free BYOK tier in FixBugs, and learned while qualifying providers from frontier AI labs.

Recently I realized that having a free forever bring-your-own-key plan just fits perfectly with our self-hosted zero data retention philosophy.
So, without wasting too much time business-thinking and thumb-twiddling, we added it.
Doing so was a learning experience in how LLM APIs have evolved over the last year.
Why an LLM API adapter layer was necessary
FixBugs was originally built and tested over the DeepSeek model. Why? it was cheap and quite good.
DeepSeek, and almost all other inference providers at the time had an OpenAI-Compatible API, which gave developers a consistent request/response contract to work with.
FixBugs VSCode Extension v0.0.1 gave users access to hundreds of models through one OpenRouter API key. Initially we thought this was good enough.
It wasn't. There were so many reasons that this deserves its own post: Why FixBugs switched away from OpenRouter.
Extending our OpenRouter based implementation to our target set of 20+ providers in 2026 turned into four different wire protocols, strategies for efficient prompt caching, and other surprises.
A successful "Hi" to an LLM API is not enough
FixBugs pushes LLM APIs to its limits. When you are implementing complex workflows at production level reliability, you need a clear and reliable API contract. Most providers do not provide this.
This meant that before adding a LLM provider, we had to qualify it first.
We wrote a script that automated this qualification process, sending requests to providers and checking:
- structured JSON schema output, strict tool contracts, and the combination of both;
- streaming text, reasoning, prompt caching, terminal state, and consistent error statuses;
- the complete premium, worker, multimodal, reasoning, and fallback profile for each tier.
- usage reporting, output limits, and model-specific fields;
The most reliable validation however, were our E2E tests, which revealed a lot of model and provider specific insufficiencies.
Nine Providers made the cut for FixBugs
| Provider | Why we wanted it | Lessons learned |
|---|---|---|
| OpenRouter | One key provides a broad model market plus routing and per-request privacy controls. | Llama Maverick and Seed models passed basic calls but sometimes ignored a required tool. ZDR eligibility, cache affinity, price, and tool support are provider specific. |
| OpenAI | It is still the clearest baseline for strict schemas and tools. | The API surface is consistent; the models are not. Optional fields and tool-reasoning levels vary per model. |
| Anthropic | Haiku is a useful fast worker, while Fable, Opus and Sonnet supply a stronger fallback and multimodal path. | Its Messages API is not OpenAI-compatible. You need a separate adapter layer for their API. |
| Google Gemini | Multimodality is native, and Flash/Flash-Lite provide useful role separation. | The Models API listed Gemini 2.5 models that returned 404 for inference on our new account. |
| xAI | Grok supplied a complete coding-oriented premium, worker, and multimodal family. | Again, they have their own Responses API which is not OpenAI-compatible. |
| Mistral AI | Open weights, EU-hosted APIs, multimodal Large/Medium models, and small edge-capable Ministral workers. | Raw generation was often fast; Response quality even for premium models is variable. |
| Together AI | It is a deployment platform as much as an inference vendor: serverless models, fine-tuning, dedicated endpoints. | Two models behind the same OpenAI-compatible endpoint can implement schemas and tools differently. |
| Fireworks AI | Its product is optimized inference for open models, making it attractive for latency- and cost-sensitive agents. | Prompt cache entries are replica-local. Session affinity and provider-specific metadata matter. |
| Moonshot AI / Kimi | Kimi targets long-horizon coding, reasoning, and large-context agent work. | Moonshot's docs advertised direct K3 while our funded account still rejected a minimal K3 call. They should focus more on building out their model serving capacity. |
One contract, several dialects
Within FixBugs, we use two types of LLM API adapters: LLMDriver (optimized for throughput) and StreamingLLMDriver (optimized for interactivity).
These adapters have been made robust with bounded retries, schema validation, attempt resets, usage accounting, and several user-centric failure mode handling.
Replacing those drivers for BYOK would have thrown away the difficult lessons.
Instead, we chose to add a provider translation layer that sits below them.
flowchart LR
D[LLMDriver and StreamingLLMDriver] -->|Request| E[BYOK execution boundary]
E --> T{Provider transport}
T --> O[OpenAI-compatible]
T --> X[xAI Responses]
T --> H[Anthropic Messages]
T --> G[Gemini GenerateContent]
O -->|wire response or SSE| N[Normalizer]
X -->|Responses items or events| N
H -->|content blocks or events| N
G -->|parts or events| N
N -->|text, tools, usage, errors| DThe drivers still ask for text, tools, or a typed result. The lower layer decides what that means on a particular wire.
Why Groq, Cerebras, Deepseek and Z.AI are missing
Groq did not give us complete set of premium, worker, multimodal, and fallback models we needed.
Its strict structured-output mode also could not be combined with streaming or tool use, a limitation still stated in Groq's documentation.
Cerebras had even fewer models: our inventory returned three models, and we had no qualified multimodal fallback. Very fast text inference is not enough when you try to make it a part of a complex workflow.
Deepseek had the requisite model set but did not support JSON schema compliance on its API. Same with Z.ai. Both model families remain available through OpenRouter.
OpenAI: few surprises to note
gpt-5-minirejectedtemperature, so the adapter omits it.gpt-5-nanoat high reasoning did not adhere to prompts requiring it to call context-gathering tools.
While the API contract says the provider supports reasoning and tools this is not guaranteed as you try a combination of the model/reasoning level/tool choice parameters.
This is why a provider-level capability flag is not sufficient. For reliable behavior the harness has to be knowledgeable about model limitations.
Gemini: OpenAI compatibility is partial
Gemini is not fully OpenAI API compatible. So we switched to use their own API for reliability.
Implementation note - Gemini API's tool calls are not stateless. Each tool call has to carry an ID and a thought signature that must be provided again on the next turn:
Request:
{
"functionCall": {
"id": "call-7",
"name": "inspect_trace",
"args": "..."
},
"thoughtSignature": "opaque-provider-state"
}
Response:
{
"functionResponse": {
"id": "call-7",
"name": "inspect_trace",
"response": {"result": "..."}
}
}
Dropping the thought signature can break the next model turn and dropping the function ID can mispair parallel calls.
xAI: Another Provider Specific API
xAI's provides an OpenAI-compatible API, but we found it unreliable. We switched to using their recommended Responses API and abstracted away the transport part for both Gemini and xAI.
{
"model": "grok-4.5",
"input": [{"role": "user", "content": [{"type": "input_text", "text": "..."}]}],
"store": false,
"prompt_cache_key": "opaque"
}
Interestingly -
- They give you API level switches for specifying whether the prompt/response may be stored. This defaults to "true".
- An unique prompt cache key must be provided by the user to enable prompt caching.
Kimi: one model family, three different contracts
Together
Together's Kimi K2.6 and K2.7 sometimes ignored JSON response schemas or returned fenced JSON.
Since fenced JSON was actually reliable, we translated into structured JSON internally.
Fireworks
Fireworks had the inverse problem: schema-only and tool-only calls worked, but they did not work together. We had to write an adapter for this.
Moonshot
Direct Moonshot was different again. During qualification, Moonshot's K2.6 and K2.7 gave us JSON-object mode rather than strict JSON. We add a translation layer here as well.
| Feature | JSON Object Mode (json_object) | Strict JSON Schema Mode (json_schema, strict: true) |
|---|---|---|
| Purpose | Ensures the output is valid JSON | Ensures the output follows a predefined JSON schema |
| Valid JSON | ✅ Guaranteed under normal completion | ✅ Guaranteed under normal completion |
| Schema Enforcement | ❌ No schema enforcement | ✅ Enforces supported JSON Schema constraints |
| Required Fields | ❌ Not guaranteed | ✅ Enforced |
| Data Types | ❌ Not guaranteed | ✅ Enforced (string, number, boolean, array, etc.) |
| Field Names | ❌ Model may generate different field names | ✅ Must match the schema |
| Nested Objects & Arrays | ⚠️ Structure depends on model instructions | ✅ Structure validated against the schema |
| Provider Compatibility | ✅ Generally widely supported | ⚠️ Varies by provider, model, and supported schema subset |
Mistral: Fast and high quality
| Model | Output Speed (tokens/s) | Median Time to First Token (TTFT) |
|---|---|---|
| Mistral Medium 2604 | 165 | 0.51s |
| Ministral 8B | 134 | ~0.5s |
| Ministral 14B | 120 | ~0.5s |
| Mistral Large 2512 | 53 | 6.6s |
Benchmark methodology: Measurements were collected via direct API calls after a warm-up, using a single fixed short prompt and a 256-token output limit.
Mistral Large 3 passed our complete FixBugs matrix, which was rare for a frontier open weights model during our evaluations.
Beyond keys: subscription inference
After qualifying the nine API providers, we also added Codex and Claude subscription inference support to FixBugs. This lets developers use an existing subscription instead of configuring a separate API key, where the provider allows it.
Point 3 in our OpenRouter post explains the motivation. If you already pay for ChatGPT or Claude, creating another account, buying API credits, and managing another key adds extra setup and cost just to use a debugging tool.
There is an important difference between the two providers:
- Codex (OpenAI): FixBugs supports Codex subscription inference through OpenAI's official OAuth-based integration. Eligible users can authenticate with their ChatGPT account and use their existing subscription for supported inference requests, subject to plan limits and eligibility requirements. No separate OpenAI API key or API credits are required for this path.
- Claude (Anthropic): We added Claude subscription integration, but Anthropic has since restricted third-party use of subscription credentials. Under its current policy, third-party applications cannot directly reuse Claude subscription credentials for API inference. As a result, Claude subscription inference is not currently a supported option in FixBugs. Developers who want to use Claude through FixBugs should use an Anthropic API key instead.
In short, FixBugs supports Codex subscription inference today. Claude is available through API-key-based access, but not through direct subscription reuse under Anthropic's current rules.
Separation of Responsibilities
We kept the sign-in flow in a layer above the code that translates model requests. FixBugs still owns prompts, conversation state, tool execution, and result validation. The provider-specific adapters are the final layer before the model call. We do not launch Codex or Claude Code to run another agent inside ours.
How to setup BYOK in FixBugs
Important things first: Privacy
The privacy boundary is deliberately boring and auditable.
VS Code stores one key per provider in SecretStorage.
Choose a premium model for complex reasoning, a worker model for lighter tasks, and a multimodal model for images and videos.
The selected key stays in memory, and goes directly to the vendor.
We collect provider and plan status, never the key.
Caveat: You may lose Zero Data Retention guarantees that come with your FixBugs Plan.
We can't guarantee zero data retention in the BYOK model. You have to set this up yourself.
When you're subscribed to FixBugs, we enforce Zero Data Retention (ZDR) across all models.
Direct BYOK inherits the account policy the user negotiated or enabled. The details are uneven:
- Fireworks: Documents ZDR by default for open models.
- Together: Says it does not store inputs or outputs by default.
- OpenRouter: Can enforce ZDR per request.
- xAI: Offers a team setting with feature restrictions.
- Gemini: Documents paid-service and feature-specific conditions.
- OpenAI, Anthropic, and Mistral: Reserve some ZDR controls for approved or paid organizations. See the policies for OpenAI, Anthropic, and Mistral.
- Moonshot: We could not verify an equivalent guarantee in its public retention policy.
Should you build your own LLM provider layer?
We would not tell every team to do this.
FixBugs needed an ad-hoc API driver layer because we observed a lot of errors and unreliable behavior from LLM APIs; a workflow provided validation functor was therefore crucial.
I'd encourage you to give already mature and open source LLM adapter frameworks a try.
LiteLLM is the obvious first stop for this with broad provider coverage, gateway routing, budgets, fallbacks, and observability built in.
Sergey Inozemtsev's llm-api-adapter is interesting: it handles an extensive model provider set (including local models) while being open-source and avoiding bloat. We are looking to run some evaluations with it.