Use Xantly with the Vercel AI SDK

The Vercel AI SDK (ai + @ai-sdk/*) has a first-class @ai-sdk/openai-compatible provider for gateways like Xantly. Drop it in once and every generateText, streamText, generateObject, agent, and UI-hook call routes through Xantly, smart routing, semantic cache, memory, waterfall fallback.

Prerequisites

• Node 18+ (or Bun / Deno / Cloudflare Workers / Vercel Edge)
• npm install ai @ai-sdk/openai-compatible zod
• A Xantly API key, create one

Setup

import { createOpenAICompatible } from '@ai-sdk/openai-compatible'

export const xantly = createOpenAICompatible({
  name: 'xantly',
  baseURL: 'https://api.xantly.com/v1',
  apiKey: process.env.XANTLY_API_KEY!,
})

Now every model you reference via xantly('...') routes through Xantly:

import { generateText } from 'ai'
import { xantly } from './xantly'

const { text } = await generateText({
  model: xantly('xantly/auto-quality'),
  prompt: 'Write a bubble sort in TypeScript.',
})
console.log(text)

streamText with UI streaming

import { streamText } from 'ai'
import { xantly } from './xantly'

// In a Next.js route handler:
export async function POST(req: Request) {
  const { messages } = await req.json()

  const result = await streamText({
    model: xantly('xantly/auto-quality'),
    messages,
  })

  return result.toDataStreamResponse()
}

Pair with the useChat() React hook client-side, zero extra config.

Tool calling

import { generateText, tool } from 'ai'
import { z } from 'zod'
import { xantly } from './xantly'

const { text, toolCalls } = await generateText({
  model: xantly('anthropic/claude-sonnet-4.6'),
  tools: {
    getWeather: tool({
      description: 'Get weather for a city',
      parameters: z.object({ city: z.string() }),
      execute: async ({ city }) => ({ city, temp: 72 }),
    }),
  },
  prompt: 'Weather in Paris?',
  maxSteps: 5,
})
console.log(text, toolCalls)

generateObject (structured output)

import { generateObject } from 'ai'
import { z } from 'zod'
import { xantly } from './xantly'

const { object } = await generateObject({
  model: xantly('openai/gpt-5.4'),
  schema: z.object({
    name: z.string(),
    email: z.string(),
    role: z.string(),
  }),
  prompt: 'Extract: Jane Doe, [email protected], CTO',
})
console.log(object)

Agent loop

import { generateText, tool } from 'ai'
import { z } from 'zod'
import { xantly } from './xantly'

const result = await generateText({
  model: xantly('xantly/auto-quality'),
  maxSteps: 8, // agent loop budget
  tools: {
    readFile: tool({
      description: 'Read a file',
      parameters: z.object({ path: z.string() }),
      execute: async ({ path }) => ({ content: `<file ${path}>` }),
    }),
    writeFile: tool({
      description: 'Write a file',
      parameters: z.object({ path: z.string(), content: z.string() }),
      execute: async ({ path, content }) => ({ ok: true }),
    }),
  },
  prompt: 'Read config.json, add a "debug" field set to true, write it back.',
})
console.log(result.text)

Model choice

Verify

const { text } = await generateText({
  model: xantly('xantly/auto-speed'),
  prompt: 'say pong',
})
console.log(text)

Open your Xantly dashboard, call is logged with routing + cost.

What you get

• Full Vercel AI SDK surface. generateText, streamText, generateObject, streamObject, agents, useChat / useCompletion React hooks.
• Edge-runtime native. Works in Vercel Edge, Cloudflare Workers, Bun, Deno.
• Tool-call translation. Xantly converts tool-call format across providers, so swapping xantly('openai/...') for xantly('anthropic/...') Just Works.
• Semantic cache. Chat UIs with re-ask patterns get free replays.
• Waterfall fallback. Mid-stream provider 5xx → silently retried on next-best model.

Gotchas

Use @ai-sdk/openai-compatible, not @ai-sdk/openai. The @ai-sdk/openai package hard-codes OpenAI's endpoint and rejects custom base URLs in newer versions. openai-compatible is the purpose-built adapter.

name field is required. Pick a short string ('xantly'). It's used in telemetry + tool-call IDs.

Streaming with useChat. toDataStreamResponse() on the server matches the useChat() client expectation. Don't use toTextStreamResponse(), it breaks tool-call streaming.

maxSteps for agent loops. Default is 1 (single-turn). Set it to 3-10 for multi-step tool use.

Next steps

• OpenCode, CLI built on the Vercel AI SDK with the same provider.
• OpenAI SDK (TypeScript), lower-level alternative.
• Streaming Responses, SSE internals.
• Multi-Agent Orchestration, Xantly's server-side agent chains.