Use Xantly with LangChain.js

@langchain/openai and @langchain/anthropic both accept configuration.baseURL, route every chain, agent, and RAG pipeline through Xantly.

The @langchain/openai and @langchain/anthropic packages both accept a custom configuration.baseURL. Point them at Xantly and every chain, tool-calling agent, and RAG pipeline works, no LangChain-side code changes.

Prerequisites

Setup (OpenAI-compatible)

import { ChatOpenAI } from '@langchain/openai'

const llm = new ChatOpenAI({
  model: 'xantly/auto-quality',
  apiKey: process.env.XANTLY_API_KEY,
  configuration: {
    baseURL: 'https://api.xantly.com/v1',
  },
})

const resp = await llm.invoke('What is LangChain?')
console.log(resp.content)

Setup (Anthropic-compatible)

import { ChatAnthropic } from '@langchain/anthropic'

const llm = new ChatAnthropic({
  model: 'anthropic/claude-sonnet-4.6',
  apiKey: process.env.XANTLY_API_KEY,
  maxTokens: 1024,
  clientOptions: {
    baseURL: 'https://api.xantly.com', // bare, no /v1
  },
})

LCEL chain

import { ChatOpenAI } from '@langchain/openai'
import { ChatPromptTemplate } from '@langchain/core/prompts'

const llm = new ChatOpenAI({
  model: 'xantly/auto-quality',
  apiKey: process.env.XANTLY_API_KEY,
  configuration: { baseURL: 'https://api.xantly.com/v1' },
})

const prompt = ChatPromptTemplate.fromMessages([
  ['system', 'You are a concise technical writer.'],
  ['human', 'Explain {topic} in one paragraph.'],
])

const chain = prompt.pipe(llm)
const out = await chain.invoke({ topic: 'vector databases' })
console.log(out.content)

Tool-calling agent

import { ChatOpenAI } from '@langchain/openai'
import { createToolCallingAgent, AgentExecutor } from 'langchain/agents'
import { tool } from '@langchain/core/tools'
import { ChatPromptTemplate } from '@langchain/core/prompts'
import { z } from 'zod'

const llm = new ChatOpenAI({
  model: 'anthropic/claude-sonnet-4.6',
  apiKey: process.env.XANTLY_API_KEY,
  configuration: { baseURL: 'https://api.xantly.com/v1' },
})

const multiply = tool(
  async ({ a, b }) => (a * b).toString(),
  {
    name: 'multiply',
    description: 'Multiply two numbers.',
    schema: z.object({ a: z.number(), b: z.number() }),
  }
)

const prompt = ChatPromptTemplate.fromMessages([
  ['system', 'Use tools when they apply.'],
  ['human', '{input}'],
  ['placeholder', '{agent_scratchpad}'],
])

const agent = await createToolCallingAgent({ llm, tools: [multiply], prompt })
const executor = new AgentExecutor({ agent, tools: [multiply] })
const result = await executor.invoke({ input: 'What is 17 * 23?' })
console.log(result.output)

RAG with Xantly embeddings

import { OpenAIEmbeddings, ChatOpenAI } from '@langchain/openai'
import { MemoryVectorStore } from 'langchain/vectorstores/memory'

const embeddings = new OpenAIEmbeddings({
  model: 'openai/text-embedding-3-small',
  apiKey: process.env.XANTLY_API_KEY,
  configuration: { baseURL: 'https://api.xantly.com/v1' },
})

const docs = ['Xantly is an AI gateway.', 'LangChain is an LLM framework.']
const store = await MemoryVectorStore.fromTexts(docs, [{}, {}], embeddings)
// ... use store in a retrieval chain

Streaming

const stream = await llm.stream('Stream a haiku.')
for await (const chunk of stream) {
  process.stdout.write(String(chunk.content ?? ''))
}

Model choice

Model IDWhen
xantly/auto-qualityChains, agents, RAG, default.
xantly/auto-valueHigh-volume chains.
anthropic/claude-sonnet-4.6Agents, strong tool-calling.
openai/gpt-5.4Classic flows with tight JSON.
openai/text-embedding-3-smallRAG embeddings.

Verify

const llm = new ChatOpenAI({
  model: 'xantly/auto-speed',
  apiKey: process.env.XANTLY_API_KEY,
  configuration: { baseURL: 'https://api.xantly.com/v1' },
})
console.log((await llm.invoke('say pong')).content)

Check your Xantly dashboard, you'll see the call logged.

What you get

Gotchas

Configuration vs clientOptions. ChatOpenAI uses configuration: { baseURL }. ChatAnthropic uses clientOptions: { baseURL }. Don't mix them up.

baseURL shape matters. OpenAI side: /v1. Anthropic side: bare. 404s come from mismatches.

Max retries. Set maxRetries: 0 on the LLM to avoid double-waterfalls.

langchain vs @langchain/core. You don't need langchain for basic chains, @langchain/openai + @langchain/core is enough. langchain adds agent executors and legacy helpers.

Next steps