# Use Xantly with LangChain.js

The `@langchain/openai` and `@langchain/anthropic` packages both accept a custom `configuration.baseURL`. Point them at Xantly and every chain, tool-calling agent, and RAG pipeline works, no LangChain-side code changes.

## Prerequisites

- Node 18+ (or Bun / Deno)
- `npm install langchain @langchain/openai @langchain/anthropic`
- A Xantly API key, [create one](/dashboard/api-keys)

## Setup (OpenAI-compatible)

```ts
import { ChatOpenAI } from '@langchain/openai'

const llm = new ChatOpenAI({
  model: 'xantly/auto-quality',
  apiKey: process.env.XANTLY_API_KEY,
  configuration: {
    baseURL: 'https://api.xantly.com/v1',
  },
})

const resp = await llm.invoke('What is LangChain?')
console.log(resp.content)
```

## Setup (Anthropic-compatible)

```ts
import { ChatAnthropic } from '@langchain/anthropic'

const llm = new ChatAnthropic({
  model: 'anthropic/claude-sonnet-4.6',
  apiKey: process.env.XANTLY_API_KEY,
  maxTokens: 1024,
  clientOptions: {
    baseURL: 'https://api.xantly.com', // bare, no /v1
  },
})
```

## LCEL chain

```ts
import { ChatOpenAI } from '@langchain/openai'
import { ChatPromptTemplate } from '@langchain/core/prompts'

const llm = new ChatOpenAI({
  model: 'xantly/auto-quality',
  apiKey: process.env.XANTLY_API_KEY,
  configuration: { baseURL: 'https://api.xantly.com/v1' },
})

const prompt = ChatPromptTemplate.fromMessages([
  ['system', 'You are a concise technical writer.'],
  ['human', 'Explain {topic} in one paragraph.'],
])

const chain = prompt.pipe(llm)
const out = await chain.invoke({ topic: 'vector databases' })
console.log(out.content)
```

## Tool-calling agent

```ts
import { ChatOpenAI } from '@langchain/openai'
import { createToolCallingAgent, AgentExecutor } from 'langchain/agents'
import { tool } from '@langchain/core/tools'
import { ChatPromptTemplate } from '@langchain/core/prompts'
import { z } from 'zod'

const llm = new ChatOpenAI({
  model: 'anthropic/claude-sonnet-4.6',
  apiKey: process.env.XANTLY_API_KEY,
  configuration: { baseURL: 'https://api.xantly.com/v1' },
})

const multiply = tool(
  async ({ a, b }) => (a * b).toString(),
  {
    name: 'multiply',
    description: 'Multiply two numbers.',
    schema: z.object({ a: z.number(), b: z.number() }),
  }
)

const prompt = ChatPromptTemplate.fromMessages([
  ['system', 'Use tools when they apply.'],
  ['human', '{input}'],
  ['placeholder', '{agent_scratchpad}'],
])

const agent = await createToolCallingAgent({ llm, tools: [multiply], prompt })
const executor = new AgentExecutor({ agent, tools: [multiply] })
const result = await executor.invoke({ input: 'What is 17 * 23?' })
console.log(result.output)
```

## RAG with Xantly embeddings

```ts
import { OpenAIEmbeddings, ChatOpenAI } from '@langchain/openai'
import { MemoryVectorStore } from 'langchain/vectorstores/memory'

const embeddings = new OpenAIEmbeddings({
  model: 'openai/text-embedding-3-small',
  apiKey: process.env.XANTLY_API_KEY,
  configuration: { baseURL: 'https://api.xantly.com/v1' },
})

const docs = ['Xantly is an AI gateway.', 'LangChain is an LLM framework.']
const store = await MemoryVectorStore.fromTexts(docs, [{}, {}], embeddings)
// ... use store in a retrieval chain
```

## Streaming

```ts
const stream = await llm.stream('Stream a haiku.')
for await (const chunk of stream) {
  process.stdout.write(String(chunk.content ?? ''))
}
```

## Model choice

| Model ID | When |
|---|---|
| `xantly/auto-quality` | Chains, agents, RAG, default. |
| `xantly/auto-value` | High-volume chains. |
| `anthropic/claude-sonnet-4.6` | Agents, strong tool-calling. |
| `openai/gpt-5.4` | Classic flows with tight JSON. |
| `openai/text-embedding-3-small` | RAG embeddings. |

## Verify

```ts
const llm = new ChatOpenAI({
  model: 'xantly/auto-speed',
  apiKey: process.env.XANTLY_API_KEY,
  configuration: { baseURL: 'https://api.xantly.com/v1' },
})
console.log((await llm.invoke('say pong')).content)
```

Check your Xantly dashboard, you'll see the call logged.

## What you get

- **Every LangChain.js primitive proxied.** Chains, agents, RAG, LangGraph nodes, all through Xantly's pipeline.
- **Waterfall fallback** during long-running agents.
- **Semantic cache** on repeat `.invoke()` calls.
- **Memory cascade** via `extra_headers: { 'X-Xantly-Memory-Mode': 'recall' }` passed through `modelKwargs`.
- **Edge-runtime support**: works in Cloudflare Workers, Vercel Edge, Bun.

## Gotchas

**Configuration vs clientOptions.** `ChatOpenAI` uses `configuration: { baseURL }`. `ChatAnthropic` uses `clientOptions: { baseURL }`. Don't mix them up.

**`baseURL` shape matters.** OpenAI side: `/v1`. Anthropic side: bare. 404s come from mismatches.

**Max retries.** Set `maxRetries: 0` on the LLM to avoid double-waterfalls.

**`langchain` vs `@langchain/core`.** You don't need `langchain` for basic chains, `@langchain/openai` + `@langchain/core` is enough. `langchain` adds agent executors and legacy helpers.

## Next steps

- [LangChain (Python)](/docs/use-with-langchain-python), Python equivalent.
- [LangGraph](/docs/use-with-langgraph), graph-based agents on top of LangChain.
- [Vercel AI SDK](/docs/use-with-vercel-ai-sdk), alternative TS agent framework.
- [Multi-Agent Orchestration](/docs/multi-agent-orchestration), Xantly's native chaining.