Outlines：结构化文本生成库，保证JSON/XML/代码输出，支持Pydantic与本地模型

outlines by orchestra-research/ai-research-skills

70 周安装量

8,000 GitHub Stars

GitHub

安装命令

npx skills add https://github.com/orchestra-research/ai-research-skills --skill outlines

AI/机器学习代码生成自然语言处理

🇨🇳中文介绍

Outlines: 结构化文本生成

何时使用此技能

在以下情况下使用 Outlines：

在生成过程中保证有效的 JSON/XML/代码结构
使用 Pydantic 模型进行类型安全的输出
支持本地模型（Transformers、llama.cpp、vLLM）
通过零开销的结构化生成最大化推理速度
自动根据 JSON 模式生成
在语法级别控制令牌采样

GitHub Stars : 8,000+ | 来自 : dottxt.ai (formerly .txt)

安装

# 基础安装
pip install outlines

# 安装特定后端
pip install outlines transformers  # Hugging Face 模型
pip install outlines llama-cpp-python  # llama.cpp
pip install outlines vllm  # vLLM 用于高吞吐量

快速开始

基础示例：分类

import outlines
from typing import Literal

# 加载模型
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")

# 使用类型约束生成
prompt = "Sentiment of 'This product is amazing!': "
generator = outlines.generate.choice(model, ["positive", "negative", "neutral"])
sentiment = generator(prompt)

print(sentiment)  # "positive" (保证是其中之一)

广告位招租

在这里展示您的产品或服务

触达数万 AI 开发者，精准高效

联系我们

使用 Pydantic 模型

from pydantic import BaseModel
import outlines

class User(BaseModel):
    name: str
    age: int
    email: str

model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")

# 生成结构化输出
prompt = "Extract user: John Doe, 30 years old, john@example.com"
generator = outlines.generate.json(model, User)
user = generator(prompt)

print(user.name)   # "John Doe"
print(user.age)    # 30
print(user.email)  # "john@example.com"

1. 受约束的令牌采样

Outlines 使用有限状态机在 logit 级别约束令牌生成。

将模式（JSON/Pydantic/正则表达式）转换为上下文无关语法
将 CFG 转换为有限状态机
在生成过程中的每一步过滤无效令牌
当只有一个有效令牌存在时快速前进

零开销：过滤发生在令牌级别
速度提升：快速通过确定性路径
保证有效性：不可能产生无效输出

import outlines

# Pydantic 模型 -> JSON 模式 -> CFG -> FSM
class Person(BaseModel):
    name: str
    age: int

model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")

# 幕后过程：
# 1. Person -> JSON 模式
# 2. JSON 模式 -> CFG
# 3. CFG -> FSM
# 4. FSM 在生成过程中过滤令牌

generator = outlines.generate.json(model, Person)
result = generator("Generate person: Alice, 25")

2. 结构化生成器

Outlines 为不同的输出类型提供了专门的生成器。

# 多项选择
generator = outlines.generate.choice(
    model,
    ["positive", "negative", "neutral"]
)

sentiment = generator("Review: This is great!")
# 结果：三个选项之一

from pydantic import BaseModel

class Product(BaseModel):
    name: str
    price: float
    in_stock: bool

# 生成符合模式的有效 JSON
generator = outlines.generate.json(model, Product)
product = generator("Extract: iPhone 15, $999, available")

# 保证是有效的 Product 实例
print(type(product))  # <class '__main__.Product'>

正则表达式生成器

# 生成匹配正则表达式的文本
generator = outlines.generate.regex(
    model,
    r"[0-9]{3}-[0-9]{3}-[0-9]{4}"  # 电话号码模式
)

phone = generator("Generate phone number:")
# 结果："555-123-4567" (保证匹配模式)

整数/浮点数生成器

# 生成特定数值类型
int_generator = outlines.generate.integer(model)
age = int_generator("Person's age:")  # 保证是整数

float_generator = outlines.generate.float(model)
price = float_generator("Product price:")  # 保证是浮点数

Outlines 支持多个本地和基于 API 的后端。

Transformers (Hugging Face)

import outlines

# 从 Hugging Face 加载
model = outlines.models.transformers(
    "microsoft/Phi-3-mini-4k-instruct",
    device="cuda"  # 或 "cpu"
)

# 与任何生成器一起使用
generator = outlines.generate.json(model, YourModel)

# 加载 GGUF 模型
model = outlines.models.llamacpp(
    "./models/llama-3.1-8b-instruct.Q4_K_M.gguf",
    n_gpu_layers=35
)

generator = outlines.generate.json(model, YourModel)

# 用于生产部署
model = outlines.models.vllm(
    "meta-llama/Llama-3.1-8B-Instruct",
    tensor_parallel_size=2  # 多 GPU
)

generator = outlines.generate.json(model, YourModel)

OpenAI (有限支持)

# 基础 OpenAI 支持
model = outlines.models.openai(
    "gpt-4o-mini",
    api_key="your-api-key"
)

# 注意：API 模型的部分功能受限
generator = outlines.generate.json(model, YourModel)

Outlines 具有一流的 Pydantic 支持，可自动进行模式转换。

from pydantic import BaseModel, Field

class Article(BaseModel):
    title: str = Field(description="Article title")
    author: str = Field(description="Author name")
    word_count: int = Field(description="Number of words", gt=0)
    tags: list[str] = Field(description="List of tags")

model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
generator = outlines.generate.json(model, Article)

article = generator("Generate article about AI")
print(article.title)
print(article.word_count)  # 保证 > 0

class Address(BaseModel):
    street: str
    city: str
    country: str

class Person(BaseModel):
    name: str
    age: int
    address: Address  # 嵌套模型

generator = outlines.generate.json(model, Person)
person = generator("Generate person in New York")

print(person.address.city)  # "New York"

from enum import Enum
from typing import Literal

class Status(str, Enum):
    PENDING = "pending"
    APPROVED = "approved"
    REJECTED = "rejected"

class Application(BaseModel):
    applicant: str
    status: Status  # 必须是枚举值之一
    priority: Literal["low", "medium", "high"]  # 必须是字面量之一

generator = outlines.generate.json(model, Application)
app = generator("Generate application")

print(app.status)  # Status.PENDING (或 APPROVED/REJECTED)

模式 1：数据提取

from pydantic import BaseModel
import outlines

class CompanyInfo(BaseModel):
    name: str
    founded_year: int
    industry: str
    employees: int

model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
generator = outlines.generate.json(model, CompanyInfo)

text = """
Apple Inc. was founded in 1976 in the technology industry.
The company employs approximately 164,000 people worldwide.
"""

prompt = f"Extract company information:\n{text}\n\nCompany:"
company = generator(prompt)

print(f"Name: {company.name}")
print(f"Founded: {company.founded_year}")
print(f"Industry: {company.industry}")
print(f"Employees: {company.employees}")

from typing import Literal
import outlines

model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")

# 二元分类
generator = outlines.generate.choice(model, ["spam", "not_spam"])
result = generator("Email: Buy now! 50% off!")

# 多类别分类
categories = ["technology", "business", "sports", "entertainment"]
category_gen = outlines.generate.choice(model, categories)
category = category_gen("Article: Apple announces new iPhone...")

# 带置信度
class Classification(BaseModel):
    label: Literal["positive", "negative", "neutral"]
    confidence: float

classifier = outlines.generate.json(model, Classification)
result = classifier("Review: This product is okay, nothing special")

模式 3：结构化表单

class UserProfile(BaseModel):
    full_name: str
    age: int
    email: str
    phone: str
    country: str
    interests: list[str]

model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
generator = outlines.generate.json(model, UserProfile)

prompt = """
Extract user profile from:
Name: Alice Johnson
Age: 28
Email: alice@example.com
Phone: 555-0123
Country: USA
Interests: hiking, photography, cooking
"""

profile = generator(prompt)
print(profile.full_name)
print(profile.interests)  # ["hiking", "photography", "cooking"]

模式 4：多实体提取

class Entity(BaseModel):
    name: str
    type: Literal["PERSON", "ORGANIZATION", "LOCATION"]

class DocumentEntities(BaseModel):
    entities: list[Entity]

model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
generator = outlines.generate.json(model, DocumentEntities)

text = "Tim Cook met with Satya Nadella at Microsoft headquarters in Redmond."
prompt = f"Extract entities from: {text}"

result = generator(prompt)
for entity in result.entities:
    print(f"{entity.name} ({entity.type})")

模式 5：代码生成

class PythonFunction(BaseModel):
    function_name: str
    parameters: list[str]
    docstring: str
    body: str

model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
generator = outlines.generate.json(model, PythonFunction)

prompt = "Generate a Python function to calculate factorial"
func = generator(prompt)

print(f"def {func.function_name}({', '.join(func.parameters)}):")
print(f'    """{func.docstring}"""')
print(f"    {func.body}")

模式 6：批处理

def batch_extract(texts: list[str], schema: type[BaseModel]):
    """从多个文本中提取结构化数据。"""
    model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
    generator = outlines.generate.json(model, schema)

    results = []
    for text in texts:
        result = generator(f"Extract from: {text}")
        results.append(result)

    return results

class Person(BaseModel):
    name: str
    age: int

texts = [
    "John is 30 years old",
    "Alice is 25 years old",
    "Bob is 40 years old"
]

people = batch_extract(texts, Person)
for person in people:
    print(f"{person.name}: {person.age}")

import outlines

# 基础用法
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")

# GPU 配置
model = outlines.models.transformers(
    "microsoft/Phi-3-mini-4k-instruct",
    device="cuda",
    model_kwargs={"torch_dtype": "float16"}
)

# 热门模型
model = outlines.models.transformers("meta-llama/Llama-3.1-8B-Instruct")
model = outlines.models.transformers("mistralai/Mistral-7B-Instruct-v0.3")
model = outlines.models.transformers("Qwen/Qwen2.5-7B-Instruct")

# 加载 GGUF 模型
model = outlines.models.llamacpp(
    "./models/llama-3.1-8b.Q4_K_M.gguf",
    n_ctx=4096,         # 上下文窗口
    n_gpu_layers=35,    # GPU 层数
    n_threads=8         # CPU 线程数
)

# 完全 GPU 卸载
model = outlines.models.llamacpp(
    "./models/model.gguf",
    n_gpu_layers=-1  # 所有层都在 GPU 上
)

# 单 GPU
model = outlines.models.vllm("meta-llama/Llama-3.1-8B-Instruct")

# 多 GPU
model = outlines.models.vllm(
    "meta-llama/Llama-3.1-70B-Instruct",
    tensor_parallel_size=4  # 4 个 GPU
)

# 使用量化
model = outlines.models.vllm(
    "meta-llama/Llama-3.1-8B-Instruct",
    quantization="awq"  # 或 "gptq"
)

1. 使用特定类型

# ✅ 好：特定类型
class Product(BaseModel):
    name: str
    price: float  # 不是 str
    quantity: int  # 不是 str
    in_stock: bool  # 不是 str

# ❌ 差：所有内容都用字符串
class Product(BaseModel):
    name: str
    price: str  # 应该是 float
    quantity: str  # 应该是 int

from pydantic import Field

# ✅ 好：带约束
class User(BaseModel):
    name: str = Field(min_length=1, max_length=100)
    age: int = Field(ge=0, le=120)
    email: str = Field(pattern=r"^[\w\.-]+@[\w\.-]+\.\w+$")

# ❌ 差：无约束
class User(BaseModel):
    name: str
    age: int
    email: str

3. 对类别使用枚举

# ✅ 好：枚举用于固定集合
class Priority(str, Enum):
    LOW = "low"
    MEDIUM = "medium"
    HIGH = "high"

class Task(BaseModel):
    title: str
    priority: Priority

# ❌ 差：自由格式字符串
class Task(BaseModel):
    title: str
    priority: str  # 可以是任何内容

4. 在提示中提供上下文

# ✅ 好：清晰的上下文
prompt = """
Extract product information from the following text.
Text: iPhone 15 Pro costs $999 and is currently in stock.
Product:
"""

# ❌ 差：最少的上下文
prompt = "iPhone 15 Pro costs $999 and is currently in stock."

5. 处理可选字段

from typing import Optional

# ✅ 好：可选字段用于不完整数据
class Article(BaseModel):
    title: str  # 必需
    author: Optional[str] = None  # 可选
    date: Optional[str] = None  # 可选
    tags: list[str] = []  # 默认空列表

# 即使缺少作者/日期也能成功

与替代方案的比较

功能	Outlines	Instructor	Guidance	LMQL
Pydantic 支持	✅ 原生	✅ 原生	❌ 无	❌ 无
JSON 模式	✅ 是	✅ 是	⚠️ 有限	✅ 是
正则表达式约束	✅ 是	❌ 无	✅ 是	✅ 是
本地模型	✅ 完整	⚠️ 有限	✅ 完整	✅ 完整
API 模型	⚠️ 有限	✅ 完整	✅ 完整	✅ 完整
零开销	✅ 是	❌ 无	⚠️ 部分	✅ 是
自动重试	❌ 无	✅ 是	❌ 无	❌ 无
学习曲线	低	低	低	高

何时选择 Outlines：

使用本地模型（Transformers、llama.cpp、vLLM）
需要最大推理速度
想要 Pydantic 模型支持
需要零开销的结构化生成
控制令牌采样过程

何时选择替代方案：

Instructor：需要具有自动重试功能的 API 模型
Guidance：需要令牌修复和复杂的工作流程
LMQL：偏好声明式查询语法

零开销：结构化生成与无约束生成一样快
快速前进优化：跳过确定性令牌
比生成后验证方法快 1.2-2 倍

FSM 每个模式编译一次（已缓存）
最小的运行时开销
与 vLLM 配合使用可实现高吞吐量

100% 有效输出（由 FSM 保证）
无需重试循环
确定性令牌过滤

references/json_generation.md - 全面的 JSON 和 Pydantic 模式
references/backends.md - 后端特定配置
references/examples.md - 生产就绪示例

🇺🇸English

Outlines: Structured Text Generation

When to Use This Skill

Use Outlines when you need to:

Guarantee valid JSON/XML/code structure during generation
Use Pydantic models for type-safe outputs
Support local models (Transformers, llama.cpp, vLLM)
Maximize inference speed with zero-overhead structured generation
Generate against JSON schemas automatically
Control token sampling at the grammar level

GitHub Stars : 8,000+ | From : dottxt.ai (formerly .txt)

Installation

# Base installation
pip install outlines

# With specific backends
pip install outlines transformers  # Hugging Face models
pip install outlines llama-cpp-python  # llama.cpp
pip install outlines vllm  # vLLM for high-throughput

Quick Start

Basic Example: Classification

import outlines
from typing import Literal

# Load model
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")

# Generate with type constraint
prompt = "Sentiment of 'This product is amazing!': "
generator = outlines.generate.choice(model, ["positive", "negative", "neutral"])
sentiment = generator(prompt)

print(sentiment)  # "positive" (guaranteed one of these)

With Pydantic Models

from pydantic import BaseModel
import outlines

class User(BaseModel):
    name: str
    age: int
    email: str

model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")

# Generate structured output
prompt = "Extract user: John Doe, 30 years old, john@example.com"
generator = outlines.generate.json(model, User)
user = generator(prompt)

print(user.name)   # "John Doe"
print(user.age)    # 30
print(user.email)  # "john@example.com"

Core Concepts

1. Constrained Token Sampling

Outlines uses Finite State Machines (FSM) to constrain token generation at the logit level.

How it works:

Convert schema (JSON/Pydantic/regex) to context-free grammar (CFG)
Transform CFG into Finite State Machine (FSM)
Filter invalid tokens at each step during generation
Fast-forward when only one valid token exists

Benefits:

Zero overhead : Filtering happens at token level
Speed improvement : Fast-forward through deterministic paths
Guaranteed validity : Invalid outputs impossible

import outlines

Pydantic model -> JSON schema -> CFG -> FSM

class Person(BaseModel): name: str age: int

model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")

Behind the scenes:

1. Person -> JSON schema

2. JSON schema -> CFG

3. CFG -> FSM

4. FSM filters tokens during generation

generator = outlines.generate.json(model, Person) result = generator("Generate person: Alice, 25")

2. Structured Generators

Outlines provides specialized generators for different output types.

Choice Generator

# Multiple choice selection
generator = outlines.generate.choice(
    model,
    ["positive", "negative", "neutral"]
)

sentiment = generator("Review: This is great!")
# Result: One of the three choices

JSON Generator

from pydantic import BaseModel

class Product(BaseModel):
    name: str
    price: float
    in_stock: bool

# Generate valid JSON matching schema
generator = outlines.generate.json(model, Product)
product = generator("Extract: iPhone 15, $999, available")

# Guaranteed valid Product instance
print(type(product))  # <class '__main__.Product'>

Regex Generator

# Generate text matching regex
generator = outlines.generate.regex(
    model,
    r"[0-9]{3}-[0-9]{3}-[0-9]{4}"  # Phone number pattern
)

phone = generator("Generate phone number:")
# Result: "555-123-4567" (guaranteed to match pattern)

Integer/Float Generators

# Generate specific numeric types
int_generator = outlines.generate.integer(model)
age = int_generator("Person's age:")  # Guaranteed integer

float_generator = outlines.generate.float(model)
price = float_generator("Product price:")  # Guaranteed float

3. Model Backends

Outlines supports multiple local and API-based backends.

Transformers (Hugging Face)

import outlines

# Load from Hugging Face
model = outlines.models.transformers(
    "microsoft/Phi-3-mini-4k-instruct",
    device="cuda"  # Or "cpu"
)

# Use with any generator
generator = outlines.generate.json(model, YourModel)

llama.cpp

# Load GGUF model
model = outlines.models.llamacpp(
    "./models/llama-3.1-8b-instruct.Q4_K_M.gguf",
    n_gpu_layers=35
)

generator = outlines.generate.json(model, YourModel)

vLLM (High Throughput)

# For production deployments
model = outlines.models.vllm(
    "meta-llama/Llama-3.1-8B-Instruct",
    tensor_parallel_size=2  # Multi-GPU
)

generator = outlines.generate.json(model, YourModel)

OpenAI (Limited Support)

# Basic OpenAI support
model = outlines.models.openai(
    "gpt-4o-mini",
    api_key="your-api-key"
)

# Note: Some features limited with API models
generator = outlines.generate.json(model, YourModel)

4. Pydantic Integration

Outlines has first-class Pydantic support with automatic schema translation.

Basic Models

from pydantic import BaseModel, Field

class Article(BaseModel):
    title: str = Field(description="Article title")
    author: str = Field(description="Author name")
    word_count: int = Field(description="Number of words", gt=0)
    tags: list[str] = Field(description="List of tags")

model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
generator = outlines.generate.json(model, Article)

article = generator("Generate article about AI")
print(article.title)
print(article.word_count)  # Guaranteed > 0

Nested Models

class Address(BaseModel):
    street: str
    city: str
    country: str

class Person(BaseModel):
    name: str
    age: int
    address: Address  # Nested model

generator = outlines.generate.json(model, Person)
person = generator("Generate person in New York")

print(person.address.city)  # "New York"

Enums and Literals

from enum import Enum
from typing import Literal

class Status(str, Enum):
    PENDING = "pending"
    APPROVED = "approved"
    REJECTED = "rejected"

class Application(BaseModel):
    applicant: str
    status: Status  # Must be one of enum values
    priority: Literal["low", "medium", "high"]  # Must be one of literals

generator = outlines.generate.json(model, Application)
app = generator("Generate application")

print(app.status)  # Status.PENDING (or APPROVED/REJECTED)

Common Patterns

Pattern 1: Data Extraction

from pydantic import BaseModel
import outlines

class CompanyInfo(BaseModel):
    name: str
    founded_year: int
    industry: str
    employees: int

model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
generator = outlines.generate.json(model, CompanyInfo)

text = """
Apple Inc. was founded in 1976 in the technology industry.
The company employs approximately 164,000 people worldwide.
"""

prompt = f"Extract company information:\n{text}\n\nCompany:"
company = generator(prompt)

print(f"Name: {company.name}")
print(f"Founded: {company.founded_year}")
print(f"Industry: {company.industry}")
print(f"Employees: {company.employees}")

Pattern 2: Classification

from typing import Literal
import outlines

model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")

# Binary classification
generator = outlines.generate.choice(model, ["spam", "not_spam"])
result = generator("Email: Buy now! 50% off!")

# Multi-class classification
categories = ["technology", "business", "sports", "entertainment"]
category_gen = outlines.generate.choice(model, categories)
category = category_gen("Article: Apple announces new iPhone...")

# With confidence
class Classification(BaseModel):
    label: Literal["positive", "negative", "neutral"]
    confidence: float

classifier = outlines.generate.json(model, Classification)
result = classifier("Review: This product is okay, nothing special")

Pattern 3: Structured Forms

class UserProfile(BaseModel):
    full_name: str
    age: int
    email: str
    phone: str
    country: str
    interests: list[str]

model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
generator = outlines.generate.json(model, UserProfile)

prompt = """
Extract user profile from:
Name: Alice Johnson
Age: 28
Email: alice@example.com
Phone: 555-0123
Country: USA
Interests: hiking, photography, cooking
"""

profile = generator(prompt)
print(profile.full_name)
print(profile.interests)  # ["hiking", "photography", "cooking"]

Pattern 4: Multi-Entity Extraction

class Entity(BaseModel):
    name: str
    type: Literal["PERSON", "ORGANIZATION", "LOCATION"]

class DocumentEntities(BaseModel):
    entities: list[Entity]

model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
generator = outlines.generate.json(model, DocumentEntities)

text = "Tim Cook met with Satya Nadella at Microsoft headquarters in Redmond."
prompt = f"Extract entities from: {text}"

result = generator(prompt)
for entity in result.entities:
    print(f"{entity.name} ({entity.type})")

Pattern 5: Code Generation

class PythonFunction(BaseModel):
    function_name: str
    parameters: list[str]
    docstring: str
    body: str

model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
generator = outlines.generate.json(model, PythonFunction)

prompt = "Generate a Python function to calculate factorial"
func = generator(prompt)

print(f"def {func.function_name}({', '.join(func.parameters)}):")
print(f'    """{func.docstring}"""')
print(f"    {func.body}")

Pattern 6: Batch Processing

def batch_extract(texts: list[str], schema: type[BaseModel]):
    """Extract structured data from multiple texts."""
    model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
    generator = outlines.generate.json(model, schema)

    results = []
    for text in texts:
        result = generator(f"Extract from: {text}")
        results.append(result)

    return results

class Person(BaseModel):
    name: str
    age: int

texts = [
    "John is 30 years old",
    "Alice is 25 years old",
    "Bob is 40 years old"
]

people = batch_extract(texts, Person)
for person in people:
    print(f"{person.name}: {person.age}")

Backend Configuration

Transformers

import outlines

# Basic usage
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")

# GPU configuration
model = outlines.models.transformers(
    "microsoft/Phi-3-mini-4k-instruct",
    device="cuda",
    model_kwargs={"torch_dtype": "float16"}
)

# Popular models
model = outlines.models.transformers("meta-llama/Llama-3.1-8B-Instruct")
model = outlines.models.transformers("mistralai/Mistral-7B-Instruct-v0.3")
model = outlines.models.transformers("Qwen/Qwen2.5-7B-Instruct")

llama.cpp

# Load GGUF model
model = outlines.models.llamacpp(
    "./models/llama-3.1-8b.Q4_K_M.gguf",
    n_ctx=4096,         # Context window
    n_gpu_layers=35,    # GPU layers
    n_threads=8         # CPU threads
)

# Full GPU offload
model = outlines.models.llamacpp(
    "./models/model.gguf",
    n_gpu_layers=-1  # All layers on GPU
)

vLLM (Production)

# Single GPU
model = outlines.models.vllm("meta-llama/Llama-3.1-8B-Instruct")

# Multi-GPU
model = outlines.models.vllm(
    "meta-llama/Llama-3.1-70B-Instruct",
    tensor_parallel_size=4  # 4 GPUs
)

# With quantization
model = outlines.models.vllm(
    "meta-llama/Llama-3.1-8B-Instruct",
    quantization="awq"  # Or "gptq"
)

Best Practices

1. Use Specific Types

# ✅ Good: Specific types
class Product(BaseModel):
    name: str
    price: float  # Not str
    quantity: int  # Not str
    in_stock: bool  # Not str

# ❌ Bad: Everything as string
class Product(BaseModel):
    name: str
    price: str  # Should be float
    quantity: str  # Should be int

2. Add Constraints

from pydantic import Field

# ✅ Good: With constraints
class User(BaseModel):
    name: str = Field(min_length=1, max_length=100)
    age: int = Field(ge=0, le=120)
    email: str = Field(pattern=r"^[\w\.-]+@[\w\.-]+\.\w+$")

# ❌ Bad: No constraints
class User(BaseModel):
    name: str
    age: int
    email: str

3. Use Enums for Categories

# ✅ Good: Enum for fixed set
class Priority(str, Enum):
    LOW = "low"
    MEDIUM = "medium"
    HIGH = "high"

class Task(BaseModel):
    title: str
    priority: Priority

# ❌ Bad: Free-form string
class Task(BaseModel):
    title: str
    priority: str  # Can be anything

4. Provide Context in Prompts

# ✅ Good: Clear context
prompt = """
Extract product information from the following text.
Text: iPhone 15 Pro costs $999 and is currently in stock.
Product:
"""

# ❌ Bad: Minimal context
prompt = "iPhone 15 Pro costs $999 and is currently in stock."

5. Handle Optional Fields

from typing import Optional

# ✅ Good: Optional fields for incomplete data
class Article(BaseModel):
    title: str  # Required
    author: Optional[str] = None  # Optional
    date: Optional[str] = None  # Optional
    tags: list[str] = []  # Default empty list

# Can succeed even if author/date missing

Comparison to Alternatives

Feature	Outlines	Instructor	Guidance	LMQL
Pydantic Support	✅ Native	✅ Native	❌ No	❌ No
JSON Schema	✅ Yes	✅ Yes	⚠️ Limited	✅ Yes
Regex Constraints	✅ Yes	❌ No	✅ Yes	✅ Yes
Local Models	✅ Full	⚠️ Limited	✅ Full	✅ Full
API Models	⚠️ Limited	✅ Full	✅ Full	✅ Full
Zero Overhead	✅ Yes	❌ No

When to choose Outlines:

Using local models (Transformers, llama.cpp, vLLM)
Need maximum inference speed
Want Pydantic model support
Require zero-overhead structured generation
Control token sampling process

When to choose alternatives:

Instructor: Need API models with automatic retrying
Guidance: Need token healing and complex workflows
LMQL: Prefer declarative query syntax

Performance Characteristics

Speed:

Zero overhead : Structured generation as fast as unconstrained
Fast-forward optimization : Skips deterministic tokens
1.2-2x faster than post-generation validation approaches

Memory:

FSM compiled once per schema (cached)
Minimal runtime overhead
Efficient with vLLM for high throughput

Accuracy:

100% valid outputs (guaranteed by FSM)
No retry loops needed
Deterministic token filtering

Resources

Documentation : https://outlines-dev.github.io/outlines
GitHub : https://github.com/outlines-dev/outlines (8k+ stars)
Discord : https://discord.gg/R9DSu34mGd
Blog : https://blog.dottxt.co

Outlines：结构化文本生成库，保证JSON/XML/代码输出，支持Pydantic与本地模型

🇨🇳中文介绍

Outlines: 结构化文本生成

何时使用此技能

安装

快速开始

基础示例：分类

相关 Skills

使用 Pydantic 模型

核心概念

1. 受约束的令牌采样

2. 结构化生成器

选择生成器

JSON 生成器

正则表达式生成器

整数/浮点数生成器

3. 模型后端

Transformers (Hugging Face)

llama.cpp

vLLM (高吞吐量)

OpenAI (有限支持)

4. Pydantic 集成

基础模型

嵌套模型

枚举和字面量

常见模式

模式 1：数据提取

模式 2：分类

模式 3：结构化表单

模式 4：多实体提取

模式 5：代码生成

模式 6：批处理

后端配置

Transformers

llama.cpp

vLLM (生产环境)

最佳实践

1. 使用特定类型

2. 添加约束

3. 对类别使用枚举

4. 在提示中提供上下文

5. 处理可选字段

与替代方案的比较

性能特征

资源

另请参阅

🇺🇸English

Outlines: Structured Text Generation

When to Use This Skill

Installation

Quick Start

Basic Example: Classification

With Pydantic Models

Core Concepts

1. Constrained Token Sampling

Pydantic model -> JSON schema -> CFG -> FSM

Behind the scenes:

1. Person -> JSON schema

2. JSON schema -> CFG

3. CFG -> FSM

4. FSM filters tokens during generation

2. Structured Generators

Choice Generator

JSON Generator

Regex Generator

Integer/Float Generators

3. Model Backends

Transformers (Hugging Face)

llama.cpp

vLLM (High Throughput)

OpenAI (Limited Support)

4. Pydantic Integration

Basic Models

Nested Models

Enums and Literals

Common Patterns

Pattern 1: Data Extraction

Pattern 2: Classification

Pattern 3: Structured Forms

Pattern 4: Multi-Entity Extraction