Models & Providers

agent-afk speaks to three provider families through a single abstraction in src/agent/providers/. The routing happens automatically based on the model name — no global AFK_PROVIDER needed in the common case.

Provider familiesCopy Anchor Link

anthropic-direct (default)Copy Anchor Link

Wraps the @anthropic-ai/sdk Messages API. Selected automatically for:

• claude-* model IDs
• Identity aliases (fixed): opus/opus_1m, opus-5.5/opus-5.5_1m, sonnet/sonnet_1m, haiku, fable
• 'anthropic' (silent alias)

openai-compatibleCopy Anchor Link

Talks directly to the OpenAI Chat Completions API (or any compatible endpoint via AFK_OPENAI_BASE_URL). Selected automatically for:

• gpt-*, o1*, o3*, o4*, codex-*
• deepseek-*, mistral-*, mixtral-*, llama-*, qwen-* — common third-party shim families (OpenRouter, Together, Fireworks, DeepSeek, etc.)
• HuggingFace-style org/model IDs (e.g. mlx-community/Qwen3-32B-4bit, Qwen/...) served by local OpenAI-shim runners — any ID containing /
• 'openai-codex' — deprecated alias from before 2026-05-18; the underlying @openai/codex-sdk harness has been removed and this now resolves to the same openai-compatible provider. Still accepted for back-compat; will be removed in a future major release.

xai / xai-oauthCopy Anchor Link

Grok model ids route to the first-class xAI provider. The grok short alias resolves to grok-4.5.

Choose API billing or supported SuperGrok, SuperGrok Heavy, or X Premium+ subscription OAuth:

afk chat "Summarize this diff" --model grok --provider xai

Without an explicit auth mode, a single available credential source is selected automatically. If both an API key and stored subscription OAuth exist, automatic resolution reports ambiguous authentication. Choose --provider xai or --provider xai-oauth, or bind the slot explicitly:

{
  "models": {
    "large": { "id": "grok-4.5", "provider": "xai-oauth" }
  }
}

xAI does not use OPENAI_API_KEY or AFK_OPENAI_BASE_URL. See authentication troubleshooting if the account or endpoint is rejected.

chatgpt-oauth (per-slot)Copy Anchor Link

Not a name-based routing rule like the two families above — an explicit per-slot provider override set in afk.config.json's models block (see Model Slots). A tier bound provider: "chatgpt-oauth" (or the "chatgpt" shorthand) still routes to openai-compatible, but additionally forces the ChatGPT-subscription OAuth credential (~/.codex/auth.json, written by codex login — see ChatGPT-subscription OAuth) for that tier only — regardless of OPENAI_API_KEY / CODEX_API_KEY.

This lets a ChatGPT-subscription model, a separately-keyed OpenAI model, and an Anthropic model coexist in the same session and /model-switch between them cleanly:

{
  "models": {
    // ChatGPT subscription — forces ~/.codex/auth.json for this tier only.
    "medium": { "id": "gpt-5.6", "provider": "chatgpt-oauth" },
    // A separately-keyed OpenAI model, unaffected by the tier above.
    "small": { "id": "qwen3.7-plus", "provider": "openai" }
    // (+ AFK_MODEL_SMALL_API_KEY / AFK_MODEL_SMALL_BASE_URL)
  }
}
// The main model stays Anthropic (sonnet/opus) — all three coexist and
// /model-switch cleanly between them.

Without a slot, ChatGPT-subscription OAuth is only the lowest tier of the global OpenAI auth chain: an ambient OPENAI_API_KEY wins, so a subscription model and a custom-keyed OpenAI model can't be used side by side. That global fallback needs no flag and still works for a single-model setup.

Selecting a modelCopy Anchor Link

Set the default for all sessions:

export AFK_MODEL=sonnet          # or: haiku, opus, fable, grok, gpt-5, etc.

Override for a single call:

afk chat "explain this stack trace" --model opus
afk i --model haiku
afk chat "summarise the staged diff" --model gpt-5

Switch mid-session in the REPL:

/model gpt-5.5       # next turn routes to openai-compatible
/model grok-4.6      # next turn routes to xai
/model grok          # same — short alias for the current Grok flagship
/model sonnet        # next turn routes back to anthropic-direct

Mid-session model switches keep cost totals and hooks. When a switch rebuilds the provider instance, the router prepends a one-turn notice to the next outbound prompt identifying the previous and new model/provider, so the new model does not assume the old provider is still active. This notice is not stored in the user's carried conversation history. Same-family forwards that do not rebuild the provider add no notice.

Cross-family history caveat: Anthropic thinking blocks and tool-call ID schemas differ between providers. When you switch provider families mid-session, the new model sees prior turns as plain text, not structured tool calls. Same-family switches keep full fidelity.

Forcing a providerCopy Anchor Link

AFK_PROVIDER (and --provider) force a single provider for the whole session, bypassing the per-model heuristic:

export AFK_PROVIDER=openai-compatible   # all models routed to OpenAI compat

Accepted values: anthropic, anthropic-direct, openai, openai-compatible, openai-codex, xai, xai-oauth. The --provider CLI flag wins when both are set.

AFK_PROVIDER is now an escape hatch, not a requirement. Omit it to let the router pick automatically per model.

Available modelsCopy Anchor Link

sonnet_1m and opus_1m resolve to the same fixed model ID as their base alias (sonnet/opus respectively) — like all identity aliases, they are not tiers, so "resolve to the same tier" no longer applies. The 1M context window is handled by the model itself; passing the _1m suffix routes to the same pinned model ID. grok resolves to GROK_FLAGSHIP_ID in src/agent/session/model-slots.ts; upgrade the flagship by updating that one constant. Source: DIRECT_MODEL_ALIASES in src/agent/session/model-slots.ts.

You can also pass any raw model ID accepted by the provider (e.g. claude-opus-5, gpt-5, claude-haiku-4-5-20251001).

Local OpenAI-compatible shims (MLX, ollama, llama.cpp, vLLM)Copy Anchor Link

Point AFK_OPENAI_BASE_URL at your local server to use any model served over an OpenAI-compatible API:

# mlx_lm.server, ollama (openai mode), vLLM, LM Studio, llama.cpp
export AFK_OPENAI_BASE_URL=http://127.0.0.1:8080/v1
export OPENAI_API_KEY=local          # placeholder; most shims accept any value
export AFK_MODEL=mlx-community/Qwen3-32B-4bit

afk i

The OpenAI SDK appends /chat/completions itself, so do not include that path in AFK_OPENAI_BASE_URL. A value ending in /chat/completions is stripped at config-load time with a one-shot warning.

For different endpoints per capability tier, use per-slot base URLs — see Model Slots.

OpenAI Responses APICopy Anchor Link

The OpenAI-compatible provider uses Chat Completions by default. To opt into the OpenAI Responses API surface instead:

export AFK_OPENAI_USE_RESPONSES=1

The ChatGPT-subscription OAuth path uses Responses automatically regardless of this flag. See API Keys for OAuth details.

Extended thinking and effortCopy Anchor Link

export AFK_THINKING=adaptive      # adaptive | disabled | max | enabled:<N> | enabled:max
export AFK_EFFORT=medium          # low | medium | high | xhigh | max

AFK_THINKING controls Anthropic extended thinking. AFK_EFFORT is forwarded as an effort hint (model-gated; ignored where unsupported). Both can also be set per-call with --thinking and --effort flags.

OpenAI o-series reasoning effort. For o-series models (o1, o3, o3-mini, o4, and provider/o* IDs), AFK_EFFORT maps to the OpenAI reasoning_effort parameter: low → low, medium → medium, and high/xhigh/max → high. The value is sent on the Chat Completions request (or as reasoning.effort on the Responses API). It is omitted entirely when AFK_EFFORT is unset or the model is not o-series.

Reliability and retriesCopy Anchor Link

The openai-compatible provider retries transient failures automatically. Requests that fail with 429, 500, 502, 503, or 529 are retried up to three times with exponential backoff (2s → 4s → 8s); both the initial connection and a dropped stream get their own retry budget. Non-transient statuses (400, 401, 403, 404) are surfaced immediately without retrying.

The short retry tier honors server retry-after or retry-after-ms hints (capped at 120 seconds) instead of blind backoff. An exhausted ChatGPT subscription window is not retried as a transient error; the outer usage-limit tier handles its pause and resume. A retry budget is bounded; persistent errors still surface.

Cost estimates for OpenAI-compatible modelsCopy Anchor Link

AFK estimates per-call USD cost from input, discounted cached input, and output tokens using a static table. Selected standard list rates per million tokens are:

Rates are a code-maintained snapshot, not a billing quote. Unrecognized hosted or local shim IDs show unknown cost, never a fabricated $0; even slash-form IDs can route to paid hosts. Long-context, batch, and cache-write surcharges are not modeled. Confirm current rates with your provider and use Usage and cost for session reporting.

Availability in /modelCopy Anchor Link

Bare /model opens a picker on the usual compositor REPL, but that picker currently has no authentication labels. On a non-compositor surface, the fallback alias listing appends (needs sign-in) to aliases without detected credentials. A label is an auth hint, not a guarantee that the provider will accept the account or model.

Output-token caps are threaded into the streaming request body: AFK_MAX_OUTPUT_TOKENS (or the resolved per-model default) is sent as max_tokens, or as max_completion_tokens for o-series models that require it.

Debug: what provider is active?Copy Anchor Link

afk config              # shows resolved model and provider
afk config --format json   # machine-readable configuration summary
afk provider auth diagnose  # targeted credential check per provider

RelatedCopy Anchor Link