Use Xantly alongside Ollama

Hybrid local + hosted pattern, Ollama for the keystroke firehose (autocomplete), Xantly for the hard stuff (agents, reasoning). The two systems split cleanly.

Run small models locally on Ollama for cheap/private work, route bigger requests through Xantly for heavy lifting. This page shows the hybrid pattern, the two systems don't fight; they split the workload cleanly.

Why hybrid?

WorkloadRun where
Inline autocomplete (fires per keystroke)Ollama (local): 0 cost, no network, private.
Simple chat / small completionsOllama: qwen2.5-coder 7B, llama 3.3 8B.
Agent mode / multi-file editXantly: BaRP picks the best T1 model.
Complex reasoning / codingXantly: Claude Sonnet 4.6, GPT-5.4.
Embedding large corporaXantly: cheaper at scale, better quality.
Tool-calling agentsXantly: local models often fail tool schemas.

Rule of thumb: Ollama for the firehose, Xantly for the hard stuff.

Prerequisites

Pattern 1, Continue.dev (per-role split)

The cleanest hybrid config in the ecosystem. ~/.continue/config.yaml:

models:
  - name: Xantly Quality
    provider: openai
    model: xantly/auto-quality
    apiBase: https://api.xantly.com/v1
    apiKey: xantly_sk_...
    roles: [chat, edit, apply]

  - name: Local Qwen (autocomplete)
    provider: ollama
    model: qwen2.5-coder:7b
    roles: [autocomplete]

  - name: Local Nomic (embed)
    provider: ollama
    model: nomic-embed-text
    roles: [embed]

Autocomplete fires locally (zero cost). Chat/edit/apply go to Xantly Quality. See the Continue.dev doc for the full config.

Pattern 2, Aider (role split)

aider \
  --openai-api-base https://api.xantly.com/v1 \
  --openai-api-key xantly_sk_... \
  --model xantly/auto-quality \
  --editor-model ollama_chat/qwen2.5-coder:7b \
  --weak-model ollama_chat/llama3.2:3b

A typical Aider session runs 70% locally, 30% on Xantly. The heaviest spend happens only when reasoning actually matters.

Pattern 3, Cline + Ollama companion

Cline itself is single-model, but you can run Ollama Coder as a VS Code inline-completion extension and use Cline only for agent mode.

Best of both: zero-cost keystroke completion, Xantly-powered agents.

Pattern 4, LangChain / LangGraph hybrid nodes

from langchain_openai import ChatOpenAI
from langchain_community.chat_models import ChatOllama

hosted = ChatOpenAI(
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
    model="xantly/auto-quality",
)

local = ChatOllama(model="qwen2.5-coder:7b")

# Use `hosted` for expensive reasoning nodes, `local` for classifiers

Bridging via LiteLLM (advanced)

If a tool only supports one OpenAI-compatible endpoint, run LiteLLM Gateway locally with two model configs, one for Ollama, one for Xantly. Point the tool at LiteLLM, let LiteLLM route based on model name.

Rarely needed, most good tools natively accept two providers.

Cost example

Heavy VS Code user, 200 keystroke-completions/hour, 20 agent runs/day:

Gotchas

Local model quality ceiling. Ollama 7B models struggle with complex tool-calling and long multi-file reasoning. Don't use them for agent mode, always route agents to Xantly.

Context window limits. Local 7B models have smaller effective context windows. If you paste a 50k-token file, it'll truncate; Xantly's waterfall escalates to a larger-context model automatically.

Ollama API key. Ollama doesn't require a key by default. If your tool demands one, pass ollama as a placeholder.

Cold-start latency. First Ollama request after idle loads the model into RAM, 2-5s. Keep ollama keep_alive set to prevent this in active sessions.

Next steps