Guidance：Python库控制LLM输出语法，保证JSON/XML/代码生成，降低延迟

guidance by davila7/claude-code-templates

181 周安装量

18,000 GitHub Stars

GitHub

安装命令

npx skills add https://github.com/davila7/claude-code-templates --skill guidance

AI/机器学习代码生成提示工程

🇨🇳中文介绍

指南：受限的 LLM 生成

何时使用此技能

在以下情况下使用 Guidance：

使用正则表达式或语法控制 LLM 输出语法
保证生成有效的 JSON/XML/代码
与传统提示方法相比降低延迟
强制结构化格式（日期、电子邮件、ID 等）
使用 Pythonic 控制流构建多步骤工作流
通过语法约束防止无效输出

GitHub Stars : 18,000+ | 来自 : Microsoft Research

安装

# 基础安装
pip install guidance

# 安装特定后端
pip install guidance[transformers]  # Hugging Face 模型
pip install guidance[llama_cpp]     # llama.cpp 模型

快速开始

基础示例：结构化生成

from guidance import models, gen

# 加载模型（支持 OpenAI、Transformers、llama.cpp）
lm = models.OpenAI("gpt-4")

# 带约束生成
result = lm + "The capital of France is " + gen("capital", max_tokens=5)

print(result["capital"])  # "Paris"

使用 Anthropic Claude

from guidance import models, gen, system, user, assistant

# 配置 Claude
lm = models.Anthropic("claude-sonnet-4-5-20250929")

# 使用上下文管理器处理聊天格式
with system():
    lm += "You are a helpful assistant."

with user():
    lm += "What is the capital of France?"

with assistant():
    lm += gen(max_tokens=20)

广告位招租

在这里展示您的产品或服务

触达数万 AI 开发者，精准高效

联系我们

1. 上下文管理器

Guidance 使用 Pythonic 的上下文管理器来处理聊天式交互。

from guidance import system, user, assistant, gen

lm = models.Anthropic("claude-sonnet-4-5-20250929")

# 系统消息
with system():
    lm += "You are a JSON generation expert."

# 用户消息
with user():
    lm += "Generate a person object with name and age."

# 助手回复
with assistant():
    lm += gen("response", max_tokens=100)

print(lm["response"])

自然的聊天流程
清晰的角色分离
易于阅读和维护

Guidance 使用正则表达式或语法确保输出匹配指定的模式。

正则表达式约束

from guidance import models, gen

lm = models.Anthropic("claude-sonnet-4-5-20250929")

# 约束为有效的电子邮件格式
lm += "Email: " + gen("email", regex=r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}")

# 约束为日期格式 (YYYY-MM-DD)
lm += "Date: " + gen("date", regex=r"\d{4}-\d{2}-\d{2}")

# 约束为电话号码
lm += "Phone: " + gen("phone", regex=r"\d{3}-\d{3}-\d{4}")

print(lm["email"])  # 保证是有效的电子邮件
print(lm["date"])   # 保证是 YYYY-MM-DD 格式

正则表达式在词元级别转换为语法
生成过程中过滤无效词元
模型只能产生匹配的输出

from guidance import models, gen, select

lm = models.Anthropic("claude-sonnet-4-5-20250929")

# 约束为特定选项
lm += "Sentiment: " + select(["positive", "negative", "neutral"], name="sentiment")

# 多项选择
lm += "Best answer: " + select(
    ["A) Paris", "B) London", "C) Berlin", "D) Madrid"],
    name="answer"
)

print(lm["sentiment"])  # 其中之一：positive, negative, neutral
print(lm["answer"])     # 其中之一：A, B, C, 或 D

Guidance 自动"修复"提示和生成之间的词元边界。

问题： 词元化会产生不自然的边界。

# 没有词元修复
prompt = "The capital of France is "
# 最后一个词元：" is "
# 第一个生成的词元可能是 " Par"（带前导空格）
# 结果："The capital of France is  Paris"（双空格！）

解决方案： Guidance 回退一个词元并重新生成。

from guidance import models, gen

lm = models.Anthropic("claude-sonnet-4-5-20250929")

# 默认启用词元修复
lm += "The capital of France is " + gen("capital", max_tokens=5)
# 结果："The capital of France is Paris"（正确的间距）

自然的文本边界
没有尴尬的间距问题
更好的模型性能（看到自然的词元序列）

4. 基于语法的生成

使用上下文无关语法定义复杂结构。

from guidance import models, gen

lm = models.Anthropic("claude-sonnet-4-5-20250929")

# JSON 语法（简化版）
json_grammar = """
{
    "name": <gen name regex="[A-Za-z ]+" max_tokens=20>,
    "age": <gen age regex="[0-9]+" max_tokens=3>,
    "email": <gen email regex="[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}" max_tokens=50>
}
"""

# 生成有效的 JSON
lm += gen("person", grammar=json_grammar)

print(lm["person"])  # 保证是有效的 JSON 结构

复杂的结构化输出
嵌套数据结构
编程语言语法
领域特定语言

使用 @guidance 装饰器创建可重用的生成模式。

from guidance import guidance, gen, models

@guidance
def generate_person(lm):
    """生成一个包含姓名和年龄的人。"""
    lm += "Name: " + gen("name", max_tokens=20, stop="\n")
    lm += "\nAge: " + gen("age", regex=r"[0-9]+", max_tokens=3)
    return lm

# 使用该函数
lm = models.Anthropic("claude-sonnet-4-5-20250929")
lm = generate_person(lm)

print(lm["name"])
print(lm["age"])

有状态函数：

@guidance(stateless=False)
def react_agent(lm, question, tools, max_rounds=5):
    """使用工具的 ReAct 智能体。"""
    lm += f"Question: {question}\n\n"

    for i in range(max_rounds):
        # 思考
        lm += f"Thought {i+1}: " + gen("thought", stop="\n")

        # 行动
        lm += "\nAction: " + select(list(tools.keys()), name="action")

        # 执行工具
        tool_result = tools[lm["action"]]()
        lm += f"\nObservation: {tool_result}\n\n"

        # 检查是否完成
        lm += "Done? " + select(["Yes", "No"], name="done")
        if lm["done"] == "Yes":
            break

    # 最终答案
    lm += "\nFinal Answer: " + gen("answer", max_tokens=100)
    return lm

from guidance import models

lm = models.Anthropic(
    model="claude-sonnet-4-5-20250929",
    api_key="your-api-key"  # 或设置 ANTHROPIC_API_KEY 环境变量
)

lm = models.OpenAI(
    model="gpt-4o-mini",
    api_key="your-api-key"  # 或设置 OPENAI_API_KEY 环境变量
)

本地模型 (Transformers)

from guidance.models import Transformers

lm = Transformers(
    "microsoft/Phi-4-mini-instruct",
    device="cuda"  # 或 "cpu"
)

本地模型 (llama.cpp)

from guidance.models import LlamaCpp

lm = LlamaCpp(
    model_path="/path/to/model.gguf",
    n_ctx=4096,
    n_gpu_layers=35
)

模式 1：JSON 生成

from guidance import models, gen, system, user, assistant

lm = models.Anthropic("claude-sonnet-4-5-20250929")

with system():
    lm += "You generate valid JSON."

with user():
    lm += "Generate a user profile with name, age, and email."

with assistant():
    lm += """{
    "name": """ + gen("name", regex=r'"[A-Za-z ]+"', max_tokens=30) + """,
    "age": """ + gen("age", regex=r"[0-9]+", max_tokens=3) + """,
    "email": """ + gen("email", regex=r'"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}"', max_tokens=50) + """
}"""

print(lm)  # 保证是有效的 JSON

from guidance import models, gen, select

lm = models.Anthropic("claude-sonnet-4-5-20250929")

text = "This product is amazing! I love it."

lm += f"Text: {text}\n"
lm += "Sentiment: " + select(["positive", "negative", "neutral"], name="sentiment")
lm += "\nConfidence: " + gen("confidence", regex=r"[0-9]+", max_tokens=3) + "%"

print(f"Sentiment: {lm['sentiment']}")
print(f"Confidence: {lm['confidence']}%")

模式 3：多步推理

from guidance import models, gen, guidance

@guidance
def chain_of_thought(lm, question):
    """生成带有逐步推理的答案。"""
    lm += f"Question: {question}\n\n"

    # 生成多个推理步骤
    for i in range(3):
        lm += f"Step {i+1}: " + gen(f"step_{i+1}", stop="\n", max_tokens=100) + "\n"

    # 最终答案
    lm += "\nTherefore, the answer is: " + gen("answer", max_tokens=50)

    return lm

lm = models.Anthropic("claude-sonnet-4-5-20250929")
lm = chain_of_thought(lm, "What is 15% of 200?")

print(lm["answer"])

模式 4：ReAct 智能体

from guidance import models, gen, select, guidance

@guidance(stateless=False)
def react_agent(lm, question):
    """使用工具的 ReAct 智能体。"""
    tools = {
        "calculator": lambda expr: eval(expr),
        "search": lambda query: f"Search results for: {query}",
    }

    lm += f"Question: {question}\n\n"

    for round in range(5):
        # 思考
        lm += f"Thought: " + gen("thought", stop="\n") + "\n"

        # 行动选择
        lm += "Action: " + select(["calculator", "search", "answer"], name="action")

        if lm["action"] == "answer":
            lm += "\nFinal Answer: " + gen("answer", max_tokens=100)
            break

        # 行动输入
        lm += "\nAction Input: " + gen("action_input", stop="\n") + "\n"

        # 执行工具
        if lm["action"] in tools:
            result = tools[lm["action"]](lm["action_input"])
            lm += f"Observation: {result}\n\n"

    return lm

lm = models.Anthropic("claude-sonnet-4-5-20250929")
lm = react_agent(lm, "What is 25 * 4 + 10?")
print(lm["answer"])

模式 5：数据提取

from guidance import models, gen, guidance

@guidance
def extract_entities(lm, text):
    """从文本中提取结构化实体。"""
    lm += f"Text: {text}\n\n"

    # 提取人物
    lm += "Person: " + gen("person", stop="\n", max_tokens=30) + "\n"

    # 提取组织
    lm += "Organization: " + gen("organization", stop="\n", max_tokens=30) + "\n"

    # 提取日期
    lm += "Date: " + gen("date", regex=r"\d{4}-\d{2}-\d{2}", max_tokens=10) + "\n"

    # 提取地点
    lm += "Location: " + gen("location", stop="\n", max_tokens=30) + "\n"

    return lm

text = "Tim Cook announced at Apple Park on 2024-09-15 in Cupertino."

lm = models.Anthropic("claude-sonnet-4-5-20250929")
lm = extract_entities(lm, text)

print(f"Person: {lm['person']}")
print(f"Organization: {lm['organization']}")
print(f"Date: {lm['date']}")
print(f"Location: {lm['location']}")

1. 使用正则表达式进行格式验证

# ✅ 好：正则表达式确保有效格式
lm += "Email: " + gen("email", regex=r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}")

# ❌ 差：自由生成可能产生无效的电子邮件
lm += "Email: " + gen("email", max_tokens=50)

2. 对固定类别使用 select()

# ✅ 好：保证是有效的类别
lm += "Status: " + select(["pending", "approved", "rejected"], name="status")

# ❌ 差：可能生成拼写错误或无效值
lm += "Status: " + gen("status", max_tokens=20)

3. 利用词元修复

# 词元修复默认启用
# 无需特殊操作 - 只需自然连接
lm += "The capital is " + gen("capital")  # 自动修复

4. 使用停止序列

# ✅ 好：在换行处停止，用于单行输出
lm += "Name: " + gen("name", stop="\n")

# ❌ 差：可能生成多行
lm += "Name: " + gen("name", max_tokens=50)

5. 创建可重用函数

# ✅ 好：可重用模式
@guidance
def generate_person(lm):
    lm += "Name: " + gen("name", stop="\n")
    lm += "\nAge: " + gen("age", regex=r"[0-9]+")
    return lm

# 多次使用
lm = generate_person(lm)
lm += "\n\n"
lm = generate_person(lm)

# ✅ 好：合理的约束
lm += gen("name", regex=r"[A-Za-z ]+", max_tokens=30)

# ❌ 太严格：可能失败或非常慢
lm += gen("name", regex=r"^(John|Jane)$", max_tokens=10)

与其他方案的比较

功能	Guidance	Instructor	Outlines	LMQL
正则表达式约束	✅ 是	❌ 否	✅ 是	✅ 是
语法支持	✅ CFG	❌ 否	✅ CFG	✅ CFG
Pydantic 验证	❌ 否	✅ 是	✅ 是	❌ 否
词元修复	✅ 是	❌ 否	✅ 是	❌ 否
本地模型	✅ 是	⚠️ 有限	✅ 是	✅ 是
API 模型	✅ 是	✅ 是	⚠️ 有限	✅ 是
Pythonic 语法	✅ 是	✅ 是	✅ 是	❌ SQL-like
学习曲线	低	低	中	高

何时选择 Guidance：

需要正则表达式/语法约束
想要词元修复
使用控制流构建复杂工作流
使用本地模型（Transformers, llama.cpp）
偏好 Pythonic 语法

何时选择其他方案：

Instructor：需要带自动重试的 Pydantic 验证
Outlines：需要 JSON 模式验证
LMQL：偏好声明式查询语法

对于受限输出，比传统提示快 30-50%
词元修复减少了不必要的重新生成
语法约束防止了无效的词元生成

与无约束生成相比，开销最小
语法编译在首次使用后缓存
推理时进行高效的词元过滤

防止在无效输出上浪费词元
无需重试循环
直接生成有效输出

文档 : https://guidance.readthedocs.io
GitHub : https://github.com/guidance-ai/guidance (18k+ stars)
Notebooks : https://github.com/guidance-ai/guidance/tree/main/notebooks
Discord : 提供社区支持

references/constraints.md - 全面的正则表达式和语法模式
references/backends.md - 后端特定配置
references/examples.md - 生产就绪示例

🇺🇸English

Guidance: Constrained LLM Generation

When to Use This Skill

Use Guidance when you need to:

Control LLM output syntax with regex or grammars
Guarantee valid JSON/XML/code generation
Reduce latency vs traditional prompting approaches
Enforce structured formats (dates, emails, IDs, etc.)
Build multi-step workflows with Pythonic control flow
Prevent invalid outputs through grammatical constraints

GitHub Stars : 18,000+ | From : Microsoft Research

Installation

# Base installation
pip install guidance

# With specific backends
pip install guidance[transformers]  # Hugging Face models
pip install guidance[llama_cpp]     # llama.cpp models

Quick Start

Basic Example: Structured Generation

from guidance import models, gen

# Load model (supports OpenAI, Transformers, llama.cpp)
lm = models.OpenAI("gpt-4")

# Generate with constraints
result = lm + "The capital of France is " + gen("capital", max_tokens=5)

print(result["capital"])  # "Paris"

With Anthropic Claude

from guidance import models, gen, system, user, assistant

# Configure Claude
lm = models.Anthropic("claude-sonnet-4-5-20250929")

# Use context managers for chat format
with system():
    lm += "You are a helpful assistant."

with user():
    lm += "What is the capital of France?"

with assistant():
    lm += gen(max_tokens=20)

Core Concepts

1. Context Managers

Guidance uses Pythonic context managers for chat-style interactions.

from guidance import system, user, assistant, gen

lm = models.Anthropic("claude-sonnet-4-5-20250929")

# System message
with system():
    lm += "You are a JSON generation expert."

# User message
with user():
    lm += "Generate a person object with name and age."

# Assistant response
with assistant():
    lm += gen("response", max_tokens=100)

print(lm["response"])

Benefits:

Natural chat flow
Clear role separation
Easy to read and maintain

2. Constrained Generation

Guidance ensures outputs match specified patterns using regex or grammars.

Regex Constraints

from guidance import models, gen

lm = models.Anthropic("claude-sonnet-4-5-20250929")

# Constrain to valid email format
lm += "Email: " + gen("email", regex=r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}")

# Constrain to date format (YYYY-MM-DD)
lm += "Date: " + gen("date", regex=r"\d{4}-\d{2}-\d{2}")

# Constrain to phone number
lm += "Phone: " + gen("phone", regex=r"\d{3}-\d{3}-\d{4}")

print(lm["email"])  # Guaranteed valid email
print(lm["date"])   # Guaranteed YYYY-MM-DD format

How it works:

Regex converted to grammar at token level
Invalid tokens filtered during generation
Model can only produce matching outputs

Selection Constraints

from guidance import models, gen, select

lm = models.Anthropic("claude-sonnet-4-5-20250929")

# Constrain to specific choices
lm += "Sentiment: " + select(["positive", "negative", "neutral"], name="sentiment")

# Multiple-choice selection
lm += "Best answer: " + select(
    ["A) Paris", "B) London", "C) Berlin", "D) Madrid"],
    name="answer"
)

print(lm["sentiment"])  # One of: positive, negative, neutral
print(lm["answer"])     # One of: A, B, C, or D

3. Token Healing

Guidance automatically "heals" token boundaries between prompt and generation.

Problem: Tokenization creates unnatural boundaries.

# Without token healing
prompt = "The capital of France is "
# Last token: " is "
# First generated token might be " Par" (with leading space)
# Result: "The capital of France is  Paris" (double space!)

Solution: Guidance backs up one token and regenerates.

from guidance import models, gen

lm = models.Anthropic("claude-sonnet-4-5-20250929")

# Token healing enabled by default
lm += "The capital of France is " + gen("capital", max_tokens=5)
# Result: "The capital of France is Paris" (correct spacing)

Benefits:

Natural text boundaries
No awkward spacing issues
Better model performance (sees natural token sequences)

4. Grammar-Based Generation

Define complex structures using context-free grammars.

from guidance import models, gen

lm = models.Anthropic("claude-sonnet-4-5-20250929")

# JSON grammar (simplified)
json_grammar = """
{
    "name": <gen name regex="[A-Za-z ]+" max_tokens=20>,
    "age": <gen age regex="[0-9]+" max_tokens=3>,
    "email": <gen email regex="[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}" max_tokens=50>
}
"""

# Generate valid JSON
lm += gen("person", grammar=json_grammar)

print(lm["person"])  # Guaranteed valid JSON structure

Use cases:

Complex structured outputs
Nested data structures
Programming language syntax
Domain-specific languages

5. Guidance Functions

Create reusable generation patterns with the @guidance decorator.

from guidance import guidance, gen, models

@guidance
def generate_person(lm):
    """Generate a person with name and age."""
    lm += "Name: " + gen("name", max_tokens=20, stop="\n")
    lm += "\nAge: " + gen("age", regex=r"[0-9]+", max_tokens=3)
    return lm

# Use the function
lm = models.Anthropic("claude-sonnet-4-5-20250929")
lm = generate_person(lm)

print(lm["name"])
print(lm["age"])

Stateful Functions:

@guidance(stateless=False)
def react_agent(lm, question, tools, max_rounds=5):
    """ReAct agent with tool use."""
    lm += f"Question: {question}\n\n"

    for i in range(max_rounds):
        # Thought
        lm += f"Thought {i+1}: " + gen("thought", stop="\n")

        # Action
        lm += "\nAction: " + select(list(tools.keys()), name="action")

        # Execute tool
        tool_result = tools[lm["action"]]()
        lm += f"\nObservation: {tool_result}\n\n"

        # Check if done
        lm += "Done? " + select(["Yes", "No"], name="done")
        if lm["done"] == "Yes":
            break

    # Final answer
    lm += "\nFinal Answer: " + gen("answer", max_tokens=100)
    return lm

Backend Configuration

Anthropic Claude

from guidance import models

lm = models.Anthropic(
    model="claude-sonnet-4-5-20250929",
    api_key="your-api-key"  # Or set ANTHROPIC_API_KEY env var
)

OpenAI

lm = models.OpenAI(
    model="gpt-4o-mini",
    api_key="your-api-key"  # Or set OPENAI_API_KEY env var
)

Local Models (Transformers)

from guidance.models import Transformers

lm = Transformers(
    "microsoft/Phi-4-mini-instruct",
    device="cuda"  # Or "cpu"
)

Local Models (llama.cpp)

from guidance.models import LlamaCpp

lm = LlamaCpp(
    model_path="/path/to/model.gguf",
    n_ctx=4096,
    n_gpu_layers=35
)

Common Patterns

Pattern 1: JSON Generation

from guidance import models, gen, system, user, assistant

lm = models.Anthropic("claude-sonnet-4-5-20250929")

with system():
    lm += "You generate valid JSON."

with user():
    lm += "Generate a user profile with name, age, and email."

with assistant():
    lm += """{
    "name": """ + gen("name", regex=r'"[A-Za-z ]+"', max_tokens=30) + """,
    "age": """ + gen("age", regex=r"[0-9]+", max_tokens=3) + """,
    "email": """ + gen("email", regex=r'"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}"', max_tokens=50) + """
}"""

print(lm)  # Valid JSON guaranteed

Pattern 2: Classification

from guidance import models, gen, select

lm = models.Anthropic("claude-sonnet-4-5-20250929")

text = "This product is amazing! I love it."

lm += f"Text: {text}\n"
lm += "Sentiment: " + select(["positive", "negative", "neutral"], name="sentiment")
lm += "\nConfidence: " + gen("confidence", regex=r"[0-9]+", max_tokens=3) + "%"

print(f"Sentiment: {lm['sentiment']}")
print(f"Confidence: {lm['confidence']}%")

Pattern 3: Multi-Step Reasoning

from guidance import models, gen, guidance

@guidance
def chain_of_thought(lm, question):
    """Generate answer with step-by-step reasoning."""
    lm += f"Question: {question}\n\n"

    # Generate multiple reasoning steps
    for i in range(3):
        lm += f"Step {i+1}: " + gen(f"step_{i+1}", stop="\n", max_tokens=100) + "\n"

    # Final answer
    lm += "\nTherefore, the answer is: " + gen("answer", max_tokens=50)

    return lm

lm = models.Anthropic("claude-sonnet-4-5-20250929")
lm = chain_of_thought(lm, "What is 15% of 200?")

print(lm["answer"])

Pattern 4: ReAct Agent

from guidance import models, gen, select, guidance

@guidance(stateless=False)
def react_agent(lm, question):
    """ReAct agent with tool use."""
    tools = {
        "calculator": lambda expr: eval(expr),
        "search": lambda query: f"Search results for: {query}",
    }

    lm += f"Question: {question}\n\n"

    for round in range(5):
        # Thought
        lm += f"Thought: " + gen("thought", stop="\n") + "\n"

        # Action selection
        lm += "Action: " + select(["calculator", "search", "answer"], name="action")

        if lm["action"] == "answer":
            lm += "\nFinal Answer: " + gen("answer", max_tokens=100)
            break

        # Action input
        lm += "\nAction Input: " + gen("action_input", stop="\n") + "\n"

        # Execute tool
        if lm["action"] in tools:
            result = tools[lm["action"]](lm["action_input"])
            lm += f"Observation: {result}\n\n"

    return lm

lm = models.Anthropic("claude-sonnet-4-5-20250929")
lm = react_agent(lm, "What is 25 * 4 + 10?")
print(lm["answer"])

Pattern 5: Data Extraction

from guidance import models, gen, guidance

@guidance
def extract_entities(lm, text):
    """Extract structured entities from text."""
    lm += f"Text: {text}\n\n"

    # Extract person
    lm += "Person: " + gen("person", stop="\n", max_tokens=30) + "\n"

    # Extract organization
    lm += "Organization: " + gen("organization", stop="\n", max_tokens=30) + "\n"

    # Extract date
    lm += "Date: " + gen("date", regex=r"\d{4}-\d{2}-\d{2}", max_tokens=10) + "\n"

    # Extract location
    lm += "Location: " + gen("location", stop="\n", max_tokens=30) + "\n"

    return lm

text = "Tim Cook announced at Apple Park on 2024-09-15 in Cupertino."

lm = models.Anthropic("claude-sonnet-4-5-20250929")
lm = extract_entities(lm, text)

print(f"Person: {lm['person']}")
print(f"Organization: {lm['organization']}")
print(f"Date: {lm['date']}")
print(f"Location: {lm['location']}")

Best Practices

1. Use Regex for Format Validation

# ✅ Good: Regex ensures valid format
lm += "Email: " + gen("email", regex=r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}")

# ❌ Bad: Free generation may produce invalid emails
lm += "Email: " + gen("email", max_tokens=50)

2. Use select() for Fixed Categories

# ✅ Good: Guaranteed valid category
lm += "Status: " + select(["pending", "approved", "rejected"], name="status")

# ❌ Bad: May generate typos or invalid values
lm += "Status: " + gen("status", max_tokens=20)

3. Leverage Token Healing

# Token healing is enabled by default
# No special action needed - just concatenate naturally
lm += "The capital is " + gen("capital")  # Automatic healing

4. Use stop Sequences

# ✅ Good: Stop at newline for single-line outputs
lm += "Name: " + gen("name", stop="\n")

# ❌ Bad: May generate multiple lines
lm += "Name: " + gen("name", max_tokens=50)

5. Create Reusable Functions

# ✅ Good: Reusable pattern
@guidance
def generate_person(lm):
    lm += "Name: " + gen("name", stop="\n")
    lm += "\nAge: " + gen("age", regex=r"[0-9]+")
    return lm

# Use multiple times
lm = generate_person(lm)
lm += "\n\n"
lm = generate_person(lm)

6. Balance Constraints

# ✅ Good: Reasonable constraints
lm += gen("name", regex=r"[A-Za-z ]+", max_tokens=30)

# ❌ Too strict: May fail or be very slow
lm += gen("name", regex=r"^(John|Jane)$", max_tokens=10)

Comparison to Alternatives

Feature	Guidance	Instructor	Outlines	LMQL
Regex Constraints	✅ Yes	❌ No	✅ Yes	✅ Yes
Grammar Support	✅ CFG	❌ No	✅ CFG	✅ CFG
Pydantic Validation	❌ No	✅ Yes	✅ Yes	❌ No
Token Healing	✅ Yes	❌ No	✅ Yes	❌ No
Local Models	✅ Yes	⚠️ Limited	✅ Yes	✅ Yes
API Models	✅ Yes	✅ Yes	⚠️ Limited

When to choose Guidance:

Need regex/grammar constraints
Want token healing
Building complex workflows with control flow
Using local models (Transformers, llama.cpp)
Prefer Pythonic syntax

When to choose alternatives:

Instructor: Need Pydantic validation with automatic retrying
Outlines: Need JSON schema validation
LMQL: Prefer declarative query syntax

Performance Characteristics

Latency Reduction:

30-50% faster than traditional prompting for constrained outputs
Token healing reduces unnecessary regeneration
Grammar constraints prevent invalid token generation

Memory Usage:

Minimal overhead vs unconstrained generation
Grammar compilation cached after first use
Efficient token filtering at inference time

Token Efficiency:

Prevents wasted tokens on invalid outputs
No need for retry loops
Direct path to valid outputs

Resources

Documentation : https://guidance.readthedocs.io
GitHub : https://github.com/guidance-ai/guidance (18k+ stars)
Notebooks : https://github.com/guidance-ai/guidance/tree/main/notebooks
Discord : Community support available

Guidance：Python库控制LLM输出语法，保证JSON/XML/代码生成，降低延迟

🇨🇳中文介绍

指南：受限的 LLM 生成

何时使用此技能

安装

快速开始

基础示例：结构化生成

使用 Anthropic Claude

相关 Skills

核心概念

1. 上下文管理器

2. 受限生成

正则表达式约束

选择约束

3. 词元修复

4. 基于语法的生成

5. Guidance 函数

后端配置

Anthropic Claude

OpenAI

本地模型 (Transformers)

本地模型 (llama.cpp)

常见模式

模式 1：JSON 生成

模式 2：分类

模式 3：多步推理

模式 4：ReAct 智能体

模式 5：数据提取

最佳实践

1. 使用正则表达式进行格式验证

2. 对固定类别使用 select()

3. 利用词元修复

4. 使用停止序列

5. 创建可重用函数

6. 平衡约束

与其他方案的比较

性能特征

资源

另请参阅

🇺🇸English

Guidance: Constrained LLM Generation

When to Use This Skill

Installation

Quick Start

Basic Example: Structured Generation

With Anthropic Claude

Core Concepts

1. Context Managers

2. Constrained Generation

Regex Constraints

Selection Constraints

3. Token Healing

4. Grammar-Based Generation

5. Guidance Functions

Backend Configuration

Anthropic Claude

OpenAI

Local Models (Transformers)

Local Models (llama.cpp)

Common Patterns

Pattern 1: JSON Generation

Pattern 2: Classification

Pattern 3: Multi-Step Reasoning

Pattern 4: ReAct Agent

Pattern 5: Data Extraction

Best Practices

1. Use Regex for Format Validation

2. Use select() for Fixed Categories

3. Leverage Token Healing

4. Use stop Sequences

5. Create Reusable Functions

6. Balance Constraints

Comparison to Alternatives

Performance Characteristics

Resources

See Also

最新 Skills