Use Xantly with Instructor

Instructor patches the OpenAI / Anthropic clients to return typed Pydantic objects with automatic retry on validation failures. Because it wraps the underlying client, pointing it at Xantly is a one-liner, you get smart routing + semantic cache + memory on every structured-output call.

Prerequisites

• Python 3.9+
• pip install instructor openai (or instructor anthropic)
• A Xantly API key, create one

Setup (OpenAI client)

import instructor
from openai import OpenAI
from pydantic import BaseModel

client = instructor.from_openai(
    OpenAI(
        base_url="https://api.xantly.com/v1",
        api_key="xantly_sk_...",
    )
)

class User(BaseModel):
    name: str
    email: str
    role: str

user = client.chat.completions.create(
    model="openai/gpt-5.4",
    response_model=User,
    messages=[{"role": "user", "content": "Extract: Jane Doe, [email protected], CTO"}],
)

assert isinstance(user, User)
print(user.name, user.email, user.role)

Setup (Anthropic client)

import instructor
from anthropic import Anthropic

client = instructor.from_anthropic(
    Anthropic(
        base_url="https://api.xantly.com",  # bare, no /v1
        api_key="xantly_sk_...",
    )
)

user = client.messages.create(
    model="anthropic/claude-sonnet-4.6",
    max_tokens=1024,
    response_model=User,
    messages=[{"role": "user", "content": "Extract: Jane Doe, [email protected], CTO"}],
)
print(user.name)

Async

import asyncio, instructor
from openai import AsyncOpenAI

aclient = instructor.from_openai(
    AsyncOpenAI(
        base_url="https://api.xantly.com/v1",
        api_key="xantly_sk_...",
    )
)

async def main():
    user = await aclient.chat.completions.create(
        model="openai/gpt-5.4",
        response_model=User,
        messages=[{"role": "user", "content": "Extract: ..."}],
    )
    print(user)

asyncio.run(main())

Streaming partial objects

from instructor import Partial

for partial in client.chat.completions.create_partial(
    model="openai/gpt-5.4",
    response_model=User,
    messages=[{"role": "user", "content": "Extract: Jane Doe, jane@..."}],
    stream=True,
):
    print(partial)

Instructor emits progressively-complete Pydantic objects as tokens stream in.

Nested / list outputs

from typing import List

class Address(BaseModel):
    street: str
    city: str

class PersonWithAddresses(BaseModel):
    name: str
    addresses: List[Address]

result = client.chat.completions.create(
    model="openai/gpt-5.4",
    response_model=PersonWithAddresses,
    messages=[{"role": "user", "content": "Jane Doe has two homes: 1 Main St, NYC and 5 Oak Ave, SF."}],
)
print(result)

Retries on validation failure

Instructor automatically reprompts when a response fails Pydantic validation:

client.chat.completions.create(
    model="openai/gpt-5.4",
    response_model=User,
    messages=[...],
    max_retries=3,  # Instructor will reprompt up to 3 times
)

When a specific model keeps failing, Xantly's waterfall will swap to a stronger model, combine both for rock-solid structured output.

Model choice

Verify

class Ping(BaseModel):
    response: str

out = client.chat.completions.create(
    model="openai/gpt-5.4",
    response_model=Ping,
    messages=[{"role": "user", "content": "say pong"}],
)
print(out.response)  # "pong"

What you get

• Typed outputs by default. Every call returns a Pydantic instance.
• Automatic retries on validation failure. Combined with Xantly's waterfall → rarely fails.
• Semantic cache hits on repeat schemas. Extract-X-from-Y patterns are highly repetitive.
• Multiple modes. instructor.Mode.TOOLS (default), Mode.JSON, Mode.MD_JSON, all work via Xantly.
• Cost per extraction logged per call.

Gotchas

Don't wrap AsyncOpenAI(...) with instructor.from_openai(...), use instructor.from_openai(AsyncOpenAI(...)). The function auto-detects sync vs async.

response_model is required. It's Instructor's killer feature; plain calls go through the unpatched client.

Provider-specific modes. Anthropic works best with Mode.ANTHROPIC_TOOLS. OpenAI works with Mode.TOOLS (default).

Pydantic v1 vs v2. Instructor requires Pydantic v2. If you're on v1, pip install -U pydantic.

Next steps

• PydanticAI, Pydantic's official agent framework.
• OpenAI SDK (Python), the underlying client.
• Anthropic SDK (Python), Anthropic-side setup.
• Chat Completions API, native structured output via response_format.