# Use Xantly alongside Ollama (hybrid local + hosted)

Run small models locally on [Ollama](https://ollama.com) for cheap/private work, route bigger requests through Xantly for heavy lifting. This page shows the hybrid pattern, the two systems don't fight; they split the workload cleanly.

## Why hybrid?

| Workload | Run where |
|---|---|
| Inline autocomplete (fires per keystroke) | **Ollama (local)**: 0 cost, no network, private. |
| Simple chat / small completions | **Ollama**: qwen2.5-coder 7B, llama 3.3 8B. |
| Agent mode / multi-file edit | **Xantly**: BaRP picks the best T1 model. |
| Complex reasoning / coding | **Xantly**: Claude Sonnet 4.6, GPT-5.4. |
| Embedding large corpora | **Xantly**: cheaper at scale, better quality. |
| Tool-calling agents | **Xantly**: local models often fail tool schemas. |

Rule of thumb: **Ollama for the firehose, Xantly for the hard stuff.**

## Prerequisites

- [Ollama](https://ollama.com/download) installed and running (`ollama serve`)
- A Xantly API key, [create one](/dashboard/api-keys)
- An IDE/tool that supports per-role model config (Continue.dev, Aider, Cline with overrides, etc.)

## Pattern 1, Continue.dev (per-role split)

The cleanest hybrid config in the ecosystem. `~/.continue/config.yaml`:

```yaml
models:
  - name: Xantly Quality
    provider: openai
    model: xantly/auto-quality
    apiBase: https://api.xantly.com/v1
    apiKey: xantly_sk_...
    roles: [chat, edit, apply]

  - name: Local Qwen (autocomplete)
    provider: ollama
    model: qwen2.5-coder:7b
    roles: [autocomplete]

  - name: Local Nomic (embed)
    provider: ollama
    model: nomic-embed-text
    roles: [embed]
```

Autocomplete fires locally (zero cost). Chat/edit/apply go to Xantly Quality. See the [Continue.dev doc](/docs/use-with-continue-dev) for the full config.

## Pattern 2, Aider (role split)

```bash
aider \
  --openai-api-base https://api.xantly.com/v1 \
  --openai-api-key xantly_sk_... \
  --model xantly/auto-quality \
  --editor-model ollama_chat/qwen2.5-coder:7b \
  --weak-model ollama_chat/llama3.2:3b
```

- Main model (planning) → Xantly Quality.
- Editor model (apply diffs) → local Qwen.
- Weak model (commits, summaries) → local tiny Llama.

A typical Aider session runs 70% locally, 30% on Xantly. The heaviest spend happens only when reasoning actually matters.

## Pattern 3, Cline + Ollama companion

Cline itself is single-model, but you can run [Ollama Coder](https://github.com/ollama/ollama-js) as a VS Code inline-completion extension and use Cline only for agent mode.

- Ollama Coder → inline ghost-text (local).
- Cline (configured with Xantly) → agent mode / Plan / Act.

Best of both: zero-cost keystroke completion, Xantly-powered agents.

## Pattern 4, LangChain / LangGraph hybrid nodes

```python
from langchain_openai import ChatOpenAI
from langchain_community.chat_models import ChatOllama

hosted = ChatOpenAI(
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
    model="xantly/auto-quality",
)

local = ChatOllama(model="qwen2.5-coder:7b")

# Use `hosted` for expensive reasoning nodes, `local` for classifiers
```

## Bridging via LiteLLM (advanced)

If a tool only supports one OpenAI-compatible endpoint, run [LiteLLM Gateway](https://github.com/BerriAI/litellm) locally with two model configs, one for Ollama, one for Xantly. Point the tool at LiteLLM, let LiteLLM route based on model name.

Rarely needed, most good tools natively accept two providers.

## Cost example

Heavy VS Code user, 200 keystroke-completions/hour, 20 agent runs/day:

- Ollama (keystroke): 1,600 calls/day × $0 = **$0**
- Xantly (agent runs): 20 calls/day × $0.12 avg = **$2.40/day**
- **~$72/month**: down from $120-200/month all-cloud.

## Gotchas

**Local model quality ceiling.** Ollama 7B models struggle with complex tool-calling and long multi-file reasoning. Don't use them for agent mode, always route agents to Xantly.

**Context window limits.** Local 7B models have smaller effective context windows. If you paste a 50k-token file, it'll truncate; Xantly's waterfall escalates to a larger-context model automatically.

**Ollama API key.** Ollama doesn't require a key by default. If your tool demands one, pass `ollama` as a placeholder.

**Cold-start latency.** First Ollama request after idle loads the model into RAM, 2-5s. Keep `ollama keep_alive` set to prevent this in active sessions.

## Next steps

- [Continue.dev](/docs/use-with-continue-dev), the best per-role hybrid config.
- [Aider](/docs/use-with-aider), main/editor/weak role split.
- [Cline](/docs/use-with-cline), single-model IDE agent, pairs with local Ollama Coder.
- [Cost-Optimized Routing](/docs/cost-optimized-routing), when local isn't viable, use Xantly's speed tier.