# Use Xantly with AutoGen

[AutoGen](https://github.com/microsoft/autogen) is Microsoft's multi-agent conversation framework. Agents exchange messages, run code, call tools, and collaborate. Its `OpenAIChatCompletionClient` accepts a `base_url`, so pointing it at Xantly gives every agent conversation smart routing, semantic cache, and memory, without touching AutoGen internals.

## Prerequisites

- Python 3.10+
- `pip install "autogen-agentchat" "autogen-ext[openai]"` (AutoGen v0.4+)
- A Xantly API key, [create one](/dashboard/api-keys)

## Setup

```python
from autogen_ext.models.openai import OpenAIChatCompletionClient

model_client = OpenAIChatCompletionClient(
    model="xantly/auto-quality",
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
)
```

For AutoGen v0.2 (legacy):

```python
config_list = [{
    "model": "xantly/auto-quality",
    "api_key": "xantly_sk_...",
    "base_url": "https://api.xantly.com/v1",
    "api_type": "openai",
}]
```

## Two-agent conversation (v0.4)

```python
import asyncio
from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.conditions import TextMentionTermination
from autogen_ext.models.openai import OpenAIChatCompletionClient

async def main():
    client = OpenAIChatCompletionClient(
        model="xantly/auto-quality",
        base_url="https://api.xantly.com/v1",
        api_key="xantly_sk_...",
    )

    researcher = AssistantAgent(
        name="researcher",
        model_client=client,
        system_message="You research topics and present facts. Cite your sources.",
    )

    critic = AssistantAgent(
        name="critic",
        model_client=client,
        system_message="You critique the researcher's output. When satisfied, say APPROVED.",
    )

    team = RoundRobinGroupChat(
        [researcher, critic],
        termination_condition=TextMentionTermination("APPROVED"),
    )
    await team.run(task="Find 5 key facts about vector databases.")

asyncio.run(main())
```

## Tools

```python
from autogen_agentchat.agents import AssistantAgent

def multiply(a: int, b: int) -> int:
    """Multiply two integers."""
    return a * b

assistant = AssistantAgent(
    name="assistant",
    model_client=client,
    tools=[multiply],
)
```

Use `anthropic/claude-sonnet-4.6` as the model for tool-heavy agents.

## Per-agent models

Different agents on different Xantly tiers:

```python
researcher_client = OpenAIChatCompletionClient(
    model="xantly/auto-quality",
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
)

critic_client = OpenAIChatCompletionClient(
    model="xantly/auto-value",  # cheaper for critique
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
)
```

## Code execution

AutoGen's `CodeExecutorAgent` pairs with `DockerCommandLineCodeExecutor`. Model calls (reasoning about code) route through Xantly; actual execution happens locally, no Xantly involvement in the sandbox. Set the reasoning agent's model to `anthropic/claude-sonnet-4.6` for best code-reasoning quality.

## Model choice

| Model ID | When |
|---|---|
| `xantly/auto-quality` | Researcher / primary agents. |
| `xantly/auto-value` | Critics, formatters, cheaper. |
| `anthropic/claude-sonnet-4.6` | Code reasoning, tool-heavy agents. |
| `openai/gpt-5.4` | Structured JSON between agents. |
| `xantly/auto-speed` | High-throughput classifier agents. |

## Verify

```python
import asyncio
from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.messages import TextMessage
from autogen_core import CancellationToken
from autogen_ext.models.openai import OpenAIChatCompletionClient

async def main():
    client = OpenAIChatCompletionClient(
        model="xantly/auto-speed",
        base_url="https://api.xantly.com/v1",
        api_key="xantly_sk_...",
    )
    agent = AssistantAgent("pinger", model_client=client)
    resp = await agent.on_messages(
        [TextMessage(content="say pong", source="user")],
        cancellation_token=CancellationToken(),
    )
    print(resp.chat_message.content)

asyncio.run(main())
```

## What you get

- **Multi-agent cache wins.** AutoGen agents re-ask similar questions in round-robin conversations, L2 semantic cache turns repeats into near-zero cost.
- **Waterfall fallback per agent.** A single agent's model going down doesn't derail the whole team.
- **Per-agent cost breakdown** in your Xantly dashboard.
- **Memory across runs.** Each team's agents can share Xantly L3 memory via `X-Xantly-Memory-Mode: recall` in model-client headers.
- **Code-execution safety.** AutoGen sandboxes code in Docker; Xantly only sees the LLM calls, never executes anything.

## Gotchas

**v0.2 vs v0.4.** The config shape changed in v0.4. This doc shows v0.4. For v0.2 you use `config_list` with `base_url` per entry.

**`/v1` required.**

**AutoGen v0.4 requires `model_info` for non-OpenAI models.** If you see "unknown model" errors, pass `model_info={"vision": False, "function_calling": True, "json_output": True, "family": "unknown", "structured_output": True}` to `OpenAIChatCompletionClient`.

**Streaming.** v0.4's `on_messages_stream` works; Xantly delivers SSE deltas.

## Next steps

- [CrewAI](/docs/use-with-crewai), alternative multi-agent framework.
- [LangGraph](/docs/use-with-langgraph), graph-based agent runtime.
- [Multi-Agent Orchestration](/docs/multi-agent-orchestration), Xantly-native.
- [Cost-Optimized Routing](/docs/cost-optimized-routing), keeping team bills down.