Use Xantly with Continue.dev
Continue.dev's per-role model config pairs perfectly with Xantly, route chat/edit/apply to Xantly Quality, autocomplete to a free local Ollama. Best mix-and-match demo in the ecosystem.
Continue.dev is the most flexible AI coding extension for VS Code and JetBrains, its killer feature is per-role model config. You can run Xantly on the expensive chat role while a local Ollama model handles free autocomplete. This is the best "mix and match" demo in the ecosystem.
Prerequisites
- VS Code or a JetBrains IDE with the Continue extension installed
- A Xantly API key, create one
Setup
Continue uses ~/.continue/config.yaml (newer) or ~/.continue/config.json (legacy; still works). Drop this YAML block:
models:
- name: Xantly Quality
provider: openai
model: xantly/auto-quality
apiBase: https://api.xantly.com/v1
apiKey: xantly_sk_...
roles:
- chat
- edit
- apply
- name: Xantly Value
provider: openai
model: xantly/auto-value
apiBase: https://api.xantly.com/v1
apiKey: xantly_sk_...
roles:
- edit
- summarize
- name: Local Qwen (fast autocomplete)
provider: ollama
model: qwen2.5-coder:7b
roles:
- autocomplete
Save, restart the IDE. Continue will show three models in its model picker, each scoped to the roles you configured.
Why this config is the interesting one
Continue splits LLM calls by role:
chat: the big conversational model. Usexantly/auto-quality.edit: apply-diff operations. Usexantly/auto-value(cheaper, fine for this).apply: the final "apply these changes" step. Usexantly/auto-qualityfor reliability.autocomplete: inline ghost-text completions. This is the big-money role: fires on every keystroke. Run a local Ollama model here to avoid racking up API costs on noise.summarize: short summaries.xantly/auto-valueor evenxantly/auto-speed.embed: embeddings for Continue's @codebase context. Useopenai/text-embedding-3-smallvia Xantly, or a local embedding model.
The above config puts Xantly on the meaningful roles and Ollama on the autocomplete firehose. A typical week:
- Autocomplete: ~50,000 calls, all local, $0
- Chat: ~200 calls, Xantly auto-quality, ~-3
- Edit/Apply: ~500 calls mixed, ~-4/week for a heavy user.
- Total:
Model ID options
| Model ID | Best for |
|---|---|
xantly/auto-quality | chat, apply, complex reasoning |
xantly/auto-value | edit, summarize, balanced |
xantly/auto-speed | summarize only, not recommended for code roles |
anthropic/claude-sonnet-4.6 | Pin for deterministic choice |
openai/gpt-5.4 | Pin for OpenAI-first workflows |
groq/llama-3.3-70b | Ultra-fast for short edits |
Verify
- Open VS Code with a project.
Ctrl+I→ "Refactor this function to use async/await" → should hitxantly/auto-qualityfor chat, thenxantly/auto-valuefor edit/apply.- Type in a file, autocomplete should hit Ollama locally (no Xantly dashboard entries).
- Check your Xantly dashboard, only the meaningful requests appear, with role-by-role cost breakdown.
What you get
- Role-split cost optimization. Continue's architecture is purpose-built for this; Xantly is purpose-built to route intelligently inside each role.
- Semantic cache per role.
chatprompts often repeat ("what does this file do?"), L2 semantic cache returns instantly. - Memory. Continue's
@codebasecontext queries are enhanced by Xantly's L3 graph memory when you opt in. - Hybrid local+hosted. Ollama for autocomplete, Xantly for everything else, best of both worlds.
Gotchas
apiBase must include /v1. Same OpenAI-SDK rule.
YAML vs JSON. Continue supports both. If you paste the YAML above into a JSON config, it won't work, either convert to JSON manually or rename your config to .yaml.
Role fallbacks. If you don't configure a model for a role, Continue uses the default. Map every role explicitly to avoid surprises.
Next steps
- Cline, fully autonomous agent mode, simpler config, no role-split.
- Intelligence Modes, per-request memory/cache/routing control via headers, useful when you want some Continue chat calls to run in
agenticmode and others in fastproxymode.