# Use Xantly with LangChain (Python)

LangChain's `ChatOpenAI` and `ChatAnthropic` both accept custom base URLs. Point them at Xantly and every chain, agent, and RAG pipeline gets BaRP routing, semantic cache, and memory underneath, with zero LangChain-specific code changes.

## Prerequisites

- Python 3.9+
- `pip install langchain langchain-openai langchain-anthropic`
- A Xantly API key, [create one](/dashboard/api-keys)

## Setup (OpenAI-compatible)

```python
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
    model="xantly/auto-quality",
)

print(llm.invoke("What is LangChain?").content)
```

## Setup (Anthropic-compatible)

```python
from langchain_anthropic import ChatAnthropic

llm = ChatAnthropic(
    base_url="https://api.xantly.com",  # bare, no /v1
    api_key="xantly_sk_...",
    model="anthropic/claude-sonnet-4.6",
    max_tokens=1024,
)

print(llm.invoke("What is LangChain?").content)
```

Pick whichever shape matches the rest of your stack, Xantly speaks both natively.

## LCEL chain

```python
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
    model="xantly/auto-quality",
)

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a concise technical writer."),
    ("human", "Explain {topic} in one paragraph."),
])

chain = prompt | llm
print(chain.invoke({"topic": "vector databases"}).content)
```

## Tool-calling agent

```python
from langchain.agents import create_tool_calling_agent, AgentExecutor
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.tools import tool
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
    model="anthropic/claude-sonnet-4.6",  # strong tool calling
)

@tool
def multiply(a: int, b: int) -> int:
    """Multiply two integers."""
    return a * b

prompt = ChatPromptTemplate.from_messages([
    ("system", "Use tools when they apply."),
    ("human", "{input}"),
    ("placeholder", "{agent_scratchpad}"),
])

agent = create_tool_calling_agent(llm, [multiply], prompt)
executor = AgentExecutor(agent=agent, tools=[multiply])
print(executor.invoke({"input": "What is 17 * 23?"})["output"])
```

## RAG with Xantly embeddings

```python
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain_community.vectorstores import FAISS
from langchain_text_splitters import RecursiveCharacterTextSplitter

# Same base URL works for embeddings
embeddings = OpenAIEmbeddings(
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
    model="openai/text-embedding-3-small",
)

docs = ["Xantly is an AI gateway.", "LangChain is an LLM framework."]
splits = RecursiveCharacterTextSplitter().create_documents(docs)
store = FAISS.from_documents(splits, embeddings)

llm = ChatOpenAI(
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
    model="xantly/auto-quality",
)
# ... use store + llm in a retrieval chain
```

## Streaming

```python
for chunk in llm.stream("Stream a haiku."):
    print(chunk.content, end="", flush=True)
```

## Model choice

| Model ID | When |
|---|---|
| `xantly/auto-quality` | Chains, agents, RAG, default production pick. |
| `xantly/auto-value` | High-volume chains where quality tolerance is higher. |
| `anthropic/claude-sonnet-4.6` | Agent work, strong tool-calling via `ChatAnthropic` or `ChatOpenAI`. |
| `openai/gpt-5.4` | Classic LangChain workflows with tight JSON. |
| `openai/text-embedding-3-small` | Embeddings for RAG. |

## Verify

```python
llm = ChatOpenAI(
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
    model="xantly/auto-speed",
)
print(llm.invoke("say pong").content)
```

Check your Xantly dashboard, the call should log with routed-to model, cache status, cost.

## What you get

- **Every LangChain primitive.** Chains, agents, RAG, structured outputs, memory, all proxied through Xantly's routing.
- **Waterfall fallback.** Provider outages don't derail long-running agent executions.
- **Semantic cache.** LCEL `.invoke()` on repeat prompts hits cache at near-zero cost.
- **Memory across chains.** Xantly's L3 memory complements LangChain's `ConversationBufferMemory`, opt in per call via `model_kwargs={"extra_headers": {"X-Xantly-Memory-Mode": "recall"}}`.
- **Cost per chain step.** Every sub-call is logged individually.

## Gotchas

**`base_url` must match the shape.** OpenAI-style: `https://api.xantly.com/v1`. Anthropic-style: `https://api.xantly.com` (bare). Wrong combination = 404s.

**`ChatOpenAI` model field accepts provider-prefixed IDs.** `anthropic/claude-sonnet-4.6` through `ChatOpenAI` works, Xantly translates server-side.

**LangChain's retry config.** Set `max_retries=0` on the LLM to avoid double-retries (Xantly waterfalls internally).

**`openai` vs `openai-compatible` providers.** LangChain treats them interchangeably here, `ChatOpenAI` is fine; you don't need a separate `ChatOpenAICompatible` class.

## Next steps

- [LangChain (TypeScript)](/docs/use-with-langchain-typescript), langchain.js equivalent.
- [LangGraph](/docs/use-with-langgraph), graph-based agent runtime on top of LangChain.
- [LlamaIndex (Python)](/docs/use-with-llamaindex-python), the other big RAG framework.
- [Multi-Agent Orchestration](/docs/multi-agent-orchestration), Xantly's native agent chaining.