# Use Xantly with LangGraph

[LangGraph](https://github.com/langchain-ai/langgraph) is LangChain's graph-based agent runtime, build stateful, cyclic agent workflows as nodes + edges. Because it uses LangChain's `ChatOpenAI` / `ChatAnthropic` under the hood, pointing those models at Xantly routes every node's LLM call through smart routing, semantic cache, and memory.

## Prerequisites

- Python 3.9+
- `pip install langgraph langchain-openai`
- A Xantly API key, [create one](/dashboard/api-keys)

## Setup

```python
from langchain_openai import ChatOpenAI
from langgraph.graph import StateGraph, END

llm = ChatOpenAI(
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
    model="xantly/auto-quality",
)
```

## Minimal graph

```python
from typing import TypedDict
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage
from langgraph.graph import StateGraph, END

class State(TypedDict):
    messages: list

llm = ChatOpenAI(
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
    model="xantly/auto-quality",
)

def call_model(state: State):
    response = llm.invoke(state["messages"])
    return {"messages": state["messages"] + [response]}

graph = StateGraph(State)
graph.add_node("model", call_model)
graph.set_entry_point("model")
graph.add_edge("model", END)
app = graph.compile()

result = app.invoke({"messages": [HumanMessage(content="Hi there")]})
print(result["messages"][-1].content)
```

## ReAct agent (prebuilt)

```python
from langgraph.prebuilt import create_react_agent
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool

@tool
def multiply(a: int, b: int) -> int:
    """Multiply two integers."""
    return a * b

llm = ChatOpenAI(
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
    model="anthropic/claude-sonnet-4.6",  # strong tool-calling
)

agent = create_react_agent(llm, tools=[multiply])
result = agent.invoke({"messages": [("user", "What is 17 * 23?")]})
print(result["messages"][-1].content)
```

## Per-node models

LangGraph's killer move: different nodes can use different Xantly tiers.

```python
from langchain_openai import ChatOpenAI
from langgraph.graph import StateGraph, END

quality = ChatOpenAI(
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
    model="xantly/auto-quality",
)

value = ChatOpenAI(
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
    model="xantly/auto-value",
)

def plan(state):
    return {"plan": quality.invoke(f"Plan: {state['input']}").content}

def execute(state):
    return {"output": value.invoke(f"Execute {state['plan']}").content}
```

- **Planning node** → quality tier (reasoning matters).
- **Execution nodes** → value tier (cheaper).
- **Summarization nodes** → speed tier.

## Checkpointing + memory

LangGraph's `MemorySaver` preserves graph state across runs:

```python
from langgraph.checkpoint.memory import MemorySaver

checkpointer = MemorySaver()
app = graph.compile(checkpointer=checkpointer)

config = {"configurable": {"thread_id": "user-42"}}
app.invoke({"messages": [...]}, config=config)
# next call with same thread_id resumes state
```

Pair it with Xantly's L3 memory by setting `X-Xantly-Memory-Mode: recall` on the LLM's `default_headers`, LangGraph handles conversation state, Xantly handles cross-session knowledge.

## Streaming

```python
for event in app.stream({"messages": [HumanMessage(content="Stream a haiku")]}):
    for key, value in event.items():
        print(key, value)
```

## Model choice

| Model ID | When |
|---|---|
| `xantly/auto-quality` | Planning / reasoning nodes. |
| `xantly/auto-value` | Execution / formatting nodes. |
| `anthropic/claude-sonnet-4.6` | Tool-calling agents (`create_react_agent`). |
| `openai/gpt-5.4` | Structured-state nodes. |
| `xantly/auto-speed` | Summarizer / classifier nodes. |

## Verify

```python
llm = ChatOpenAI(
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
    model="xantly/auto-speed",
)
print(llm.invoke("say pong").content)
```

## What you get

- **Per-node routing.** Use Xantly Quality for expensive planning, Value for cheap executors.
- **Cycle safety.** Agent loops that re-ask similar questions hit semantic cache, cheaper + faster.
- **Waterfall fallback per node.** One provider outage doesn't derail a long-running graph.
- **Cost per node.** Each node's LLM call logs separately in your Xantly dashboard.
- **Checkpoint interop.** LangGraph's `MemorySaver` + Xantly's L3 memory, complementary, not competing.

## Gotchas

**LangGraph doesn't ship its own LLM client.** It uses LangChain's models. All base-URL config happens on `ChatOpenAI` / `ChatAnthropic`, see [LangChain (Python)](/docs/use-with-langchain-python) for details.

**Tool-calling failures in cycles.** If a model can't reliably emit tool calls, LangGraph will loop forever. Use `anthropic/claude-sonnet-4.6` or set a `recursion_limit` on `graph.compile(...)`.

**Async graphs.** `app.ainvoke(...)` and `app.astream(...)` work; the underlying OpenAI SDK handles async.

**State type hints.** `TypedDict` is recommended, LangGraph introspects it for the graph schema.

## Next steps

- [LangChain (Python)](/docs/use-with-langchain-python), the underlying library.
- [CrewAI](/docs/use-with-crewai), alternative multi-agent approach.
- [AutoGen](/docs/use-with-autogen), Microsoft's multi-agent framework.
- [Multi-Agent Orchestration](/docs/multi-agent-orchestration), Xantly-native graph orchestration.