LangSmith Trace 追踪工具：AI应用调试与性能监控的完整指南

langsmith-trace by langchain-ai/langsmith-skills

695 周安装量

76 GitHub Stars

安装命令

npx skills add https://github.com/langchain-ai/langsmith-skills --skill langsmith-trace

🇨🇳中文介绍

LANGSMITH_API_KEY=lsv2_pt_your_api_key_here # 必需 LANGSMITH_PROJECT=your-project-name # 可选：默认项目 LANGSMITH_WORKSPACE_ID=your-workspace-id # 可选：用于组织范围的密钥

重要提示： 在查询或与 LangSmith 交互之前，请务必检查环境变量或 .env 文件中的 LANGSMITH_PROJECT。这会告诉你哪个项目包含相关的追踪记录和数据。如果 LangSmith 项目不可用，请运用你的最佳判断来确定正确的项目。

CLI 工具

curl -sSL https://raw.githubusercontent.com/langchain-ai/langsmith-cli/main/scripts/install.sh | sh

<trace_langchain_oss> 对于 LangChain/LangGraph 应用，追踪是自动的。只需设置环境变量：

export LANGSMITH_TRACING=true
export LANGSMITH_API_KEY=<your-api-key>
export OPENAI_API_KEY=<your-openai-api-key>  # 或你的 LLM 提供商的密钥

可选变量：

LANGSMITH_PROJECT - 指定项目名称（默认为 "default"）
LANGCHAIN_CALLBACKS_BACKGROUND=false - 用于无服务器环境，确保在函数退出前完成追踪记录（Python）</trace_langchain_oss>

广告位招租

在这里展示您的产品或服务

触达数万 AI 开发者，精准高效

联系我们

相关 Skills

FlyClaw：零登录航班聚合查询工具，Python实现多源航班信息与价格搜索

4,000,000 周安装

find-skills 技能搜索工具 - Vercel Labs 开源智能体技能包管理器

749,400 周安装

Vercel React 最佳实践指南 | 58条Next.js性能优化规则与代码重构

255,700 周安装

Vercel Web界面规范检查工具 - 自动检测代码是否符合Web设计指南

205,600 周安装

<trace_other_frameworks> 对于非 LangChain 应用，如果框架原生支持 OpenTelemetry，请使用 LangSmith 的 OpenTelemetry 集成。

如果应用没有使用框架，或者使用的框架没有自动的 OTel 支持，请使用 traceable 装饰器/包装器来包装你的 LLM 客户端。

client = wrap_openai(OpenAI())

@traceable def my_llm_pipeline(question: str) -> str: resp = client.chat.completions.create( model="gpt-4o-mini", messages=[{"role": "user", "content": question}], ) return resp.choices[0].message.content

@traceable def rag_pipeline(question: str) -> str: docs = retrieve_docs(question) return generate_answer(question, docs)

@traceable(name="retrieve_docs") def retrieve_docs(query: str) -> list[str]: return docs

@traceable(name="generate_answer") def generate_answer(question: str, docs: list[str]) -> str: return client.chat.completions.create(...)

</python>

<typescript>
使用 traceable() 包装器和 wrapOpenAI() 实现自动追踪。
```typescript
import { traceable } from "langsmith/traceable";
import { wrapOpenAI } from "langsmith/wrappers";
import OpenAI from "openai";

const client = wrapOpenAI(new OpenAI());

const myLlmPipeline = traceable(async (question: string): Promise<string> => {
  const resp = await client.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: question }],
  });
  return resp.choices[0].message.content || "";
}, { name: "my_llm_pipeline" });

// 嵌套追踪示例
const retrieveDocs = traceable(async (query: string): Promise<string[]> => {
  return docs;
}, { name: "retrieve_docs" });

const generateAnswer = traceable(async (question: string, docs: string[]): Promise<string> => {
  const resp = await client.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: `${question}\nContext: ${docs.join("\n")}` }],
  });
  return resp.choices[0].message.content || "";
}, { name: "generate_answer" });

const ragPipeline = traceable(async (question: string): Promise<string> => {
  const docs = await retrieveDocs(question);
  return await generateAnswer(question, docs);
}, { name: "rag_pipeline" });

对所有你希望在 LangSmith 中可见的嵌套函数应用 traceable
包装后的客户端会自动追踪所有调用 — wrap_openai()/wrapOpenAI() 会记录每一次 LLM 调用
为你的追踪记录命名以便于筛选
添加元数据以提高可搜索性</trace_other_frameworks>

<traces_vs_runs> 使用 langsmith CLI 来查询追踪数据。

理解两者的区别至关重要：

追踪 = 一个完整的执行树（根运行 + 所有子运行）。一个追踪代表一次完整的代理调用，包含其所有的 LLM 调用、工具调用和嵌套操作。
运行 = 树中的一个单一节点（一次 LLM 调用、一次工具调用等）

通常，首先查询追踪 — 它们提供完整的上下文并保留轨迹分析和数据集生成所需的层次结构。</traces_vs_runs>

<command_structure> 两个命令组，行为一致：

langsmith
├── trace (对追踪树的操作 - 首先使用这个)
│   ├── list    - 列出追踪（过滤器应用于根运行）
│   ├── get     - 获取单个追踪及其完整层次结构
│   └── export  - 将追踪导出到 JSONL 文件（每个追踪一个文件）
│
├── run (对单个运行的操作 - 用于特定分析)
│   ├── list    - 列出运行（扁平列表，过滤器应用于任何运行）
│   ├── get     - 获取单个运行
│   └── export  - 将运行导出到单个 JSONL 文件（扁平）
│
├── dataset (数据集操作)
│   ├── list    - 列出数据集
│   ├── get     - 获取数据集详情
│   ├── create  - 创建空数据集
│   ├── delete  - 删除数据集
│   ├── export  - 将数据集导出到文件
│   └── upload  - 上传本地 JSON 作为数据集
│
├── example (示例操作)
│   ├── list    - 列出数据集中的示例
│   ├── create  - 向数据集添加示例
│   └── delete  - 删除示例
│
├── evaluator (评估器操作)
│   ├── list    - 列出评估器
│   ├── upload  - 上传评估器
│   └── delete  - 删除评估器
│
├── experiment (实验操作)
│   ├── list    - 列出实验
│   └── get     - 获取实验结果
│
├── thread (线程操作)
│   ├── list    - 列出对话线程
│   └── get     - 获取线程详情
│
└── project (项目操作)
    └── list    - 列出追踪项目

| traces * | runs *
---|---|---
过滤器应用于 | 仅根运行 | 任何匹配的运行
--run-type | 不可用 | 可用
返回结果 | 完整层次结构 | 扁平列表
导出输出 | 目录（每个追踪一个文件） | 单个文件
</command_structure> | |

<querying_traces> 使用 langsmith CLI 查询追踪。命令与语言无关。

# 列出最近的追踪（最常用的操作）
langsmith trace list --limit 10 --project my-project

# 列出包含元数据的追踪（时间、令牌数、成本）
langsmith trace list --limit 10 --include-metadata

# 按时间筛选追踪
langsmith trace list --last-n-minutes 60
langsmith trace list --since 2025-01-20T10:00:00Z

# 获取特定追踪及其完整层次结构
langsmith trace get <trace-id>

# 列出追踪并内联显示层次结构
langsmith trace list --limit 5 --show-hierarchy

# 将追踪导出到 JSONL（每个追踪一个文件，包含所有运行）
langsmith trace export ./traces --limit 20 --full

# 按性能筛选追踪
langsmith trace list --min-latency 5.0 --limit 10    # 慢追踪（>= 5秒）
langsmith trace list --error --last-n-minutes 60     # 失败的追踪

# 列出特定运行类型（扁平列表）
langsmith run list --run-type llm --limit 20

基本过滤器：

--trace-ids abc,def - 筛选到特定追踪
--limit N - 最大结果数
--project NAME - 项目名称
--last-n-minutes N - 时间过滤器
--since TIMESTAMP - 时间过滤器（ISO 格式）
--error / --no-error - 错误状态
--name PATTERN - 名称包含（不区分大小写）

性能过滤器：

--min-latency SECONDS - 最小延迟（例如，5 表示 >= 5秒）
--max-latency SECONDS - 最大延迟
--min-tokens N - 最小总令牌数
--tags tag1,tag2 - 具有这些标签中的任何一个

高级过滤器：

--filter QUERY - 原始 LangSmith 过滤器查询，用于复杂情况（反馈、元数据等）

使用原始 LangSmith 查询按反馈分数筛选追踪

langsmith trace list --filter 'and(eq(feedback_key, "correctness"), gte(feedback_score, 0.8))'

<export_format> 导出会创建 .jsonl 文件（每行一个运行），包含以下字段：

{"run_id": "...", "trace_id": "...", "name": "...", "run_type": "...", "parent_run_id": "...", "inputs": {...}, "outputs": {...}}

使用 --include-io 或 --full 来包含输入/输出（数据集生成所需）。</export_format>

🇺🇸English

LANGSMITH_API_KEY=lsv2_pt_your_api_key_here # Required LANGSMITH_PROJECT=your-project-name # Optional: default project LANGSMITH_WORKSPACE_ID=your-workspace-id # Optional: for org-scoped keys

IMPORTANT: Always check the environment variables or .env file for LANGSMITH_PROJECT before querying or interacting with LangSmith. This tells you which project contains the relevant traces and data. If the LangSmith project is not available, use your best judgement to identify the right one.

CLI Tool

curl -sSL https://raw.githubusercontent.com/langchain-ai/langsmith-cli/main/scripts/install.sh | sh

<trace_langchain_oss> For LangChain/LangGraph apps, tracing is automatic. Just set environment variables:

export LANGSMITH_TRACING=true
export LANGSMITH_API_KEY=<your-api-key>
export OPENAI_API_KEY=<your-openai-api-key>  # or your LLM provider's key

Optional variables:

LANGSMITH_PROJECT - specify project name (defaults to "default")
LANGCHAIN_CALLBACKS_BACKGROUND=false - use for serverless to ensure traces complete before function exit (Python) </trace_langchain_oss>

<trace_other_frameworks> For non-LangChain apps, if the framework has native OpenTelemetry support, use LangSmith's OpenTelemetry integration.

If the app is NOT using a framework, or using one without automatic OTel support, use the traceable decorator/wrapper and wrap your LLM client.

client = wrap_openai(OpenAI())

Nested tracing example

@traceable def rag_pipeline(question: str) -> str: docs = retrieve_docs(question) return generate_answer(question, docs)

@traceable(name="retrieve_docs") def retrieve_docs(query: str) -> list[str]: return docs

@traceable(name="generate_answer") def generate_answer(question: str, docs: list[str]) -> str: return client.chat.completions.create(...)

</python>

<typescript>
Use traceable() wrapper and wrapOpenAI() for automatic tracing.
```typescript
import { traceable } from "langsmith/traceable";
import { wrapOpenAI } from "langsmith/wrappers";
import OpenAI from "openai";

const client = wrapOpenAI(new OpenAI());

const myLlmPipeline = traceable(async (question: string): Promise<string> => {
  const resp = await client.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: question }],
  });
  return resp.choices[0].message.content || "";
}, { name: "my_llm_pipeline" });

// Nested tracing example
const retrieveDocs = traceable(async (query: string): Promise<string[]> => {
  return docs;
}, { name: "retrieve_docs" });

const generateAnswer = traceable(async (question: string, docs: string[]): Promise<string> => {
  const resp = await client.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: `${question}\nContext: ${docs.join("\n")}` }],
  });
  return resp.choices[0].message.content || "";
}, { name: "generate_answer" });

const ragPipeline = traceable(async (question: string): Promise<string> => {
  const docs = await retrieveDocs(question);
  return await generateAnswer(question, docs);
}, { name: "rag_pipeline" });

Best Practices:

Apply traceable to all nested functions you want visible in LangSmith
Wrapped clients auto-trace all calls — wrap_openai()/wrapOpenAI() records every LLM call
Name your traces for easier filtering
Add metadata for searchability </trace_other_frameworks>

<traces_vs_runs> Use the langsmith CLI to query trace data.

Understanding the difference is critical:

Trace = A complete execution tree (root run + all child runs). A trace represents one full agent invocation with all its LLM calls, tool calls, and nested operations.
Run = A single node in the tree (one LLM call, one tool call, etc.)

Generally, query traces first — they provide complete context and preserve hierarchy needed for trajectory analysis and dataset generation. </traces_vs_runs>

<command_structure> Two command groups with consistent behavior:

langsmith
├── trace (operations on trace trees - USE THIS FIRST)
│   ├── list    - List traces (filters apply to root run)
│   ├── get     - Get single trace with full hierarchy
│   └── export  - Export traces to JSONL files (one file per trace)
│
├── run (operations on individual runs - for specific analysis)
│   ├── list    - List runs (flat, filters apply to any run)
│   ├── get     - Get single run
│   └── export  - Export runs to single JSONL file (flat)
│
├── dataset (dataset operations)
│   ├── list    - List datasets
│   ├── get     - Get dataset details
│   ├── create  - Create empty dataset
│   ├── delete  - Delete dataset
│   ├── export  - Export dataset to file
│   └── upload  - Upload local JSON as dataset
│
├── example (example operations)
│   ├── list    - List examples in a dataset
│   ├── create  - Add example to a dataset
│   └── delete  - Delete an example
│
├── evaluator (evaluator operations)
│   ├── list    - List evaluators
│   ├── upload  - Upload evaluator
│   └── delete  - Delete evaluator
│
├── experiment (experiment operations)
│   ├── list    - List experiments
│   └── get     - Get experiment results
│
├── thread (thread operations)
│   ├── list    - List conversation threads
│   └── get     - Get thread details
│
└── project (project operations)
    └── list    - List tracing projects

Key differences:

| traces * | runs *
---|---|---
Filters apply to | Root run only | Any matching run
--run-type | Not available | Available
Returns | Full hierarchy | Flat list
Export output | Directory (one file/trace) | Single file
</command_structure> | |

<querying_traces> Query traces using the langsmith CLI. Commands are language-agnostic.

# List recent traces (most common operation)
langsmith trace list --limit 10 --project my-project

# List traces with metadata (timing, tokens, costs)
langsmith trace list --limit 10 --include-metadata

# Filter traces by time
langsmith trace list --last-n-minutes 60
langsmith trace list --since 2025-01-20T10:00:00Z

# Get specific trace with full hierarchy
langsmith trace get <trace-id>

# List traces and show hierarchy inline
langsmith trace list --limit 5 --show-hierarchy

# Export traces to JSONL (one file per trace, includes all runs)
langsmith trace export ./traces --limit 20 --full

# Filter traces by performance
langsmith trace list --min-latency 5.0 --limit 10    # Slow traces (>= 5s)
langsmith trace list --error --last-n-minutes 60     # Failed traces

# List specific run types (flat list)
langsmith run list --run-type llm --limit 20

</querying_traces>

Basic filters:

--trace-ids abc,def - Filter to specific traces
--limit N - Max results
--project NAME - Project name
--last-n-minutes N - Time filter
--since TIMESTAMP - Time filter (ISO format)
--error / --no-error - Error status
--name PATTERN - Name contains (case-insensitive)

Performance filters:

--min-latency SECONDS - Minimum latency (e.g., 5 for >= 5s)
--max-latency SECONDS - Maximum latency
--min-tokens N - Minimum total tokens
--tags tag1,tag2 - Has any of these tags

Advanced filter:

--filter QUERY - Raw LangSmith filter query for complex cases (feedback, metadata, etc.)

Filter traces by feedback score using raw LangSmith query

langsmith trace list --filter 'and(eq(feedback_key, "correctness"), gte(feedback_score, 0.8))'

<export_format> Export creates .jsonl files (one run per line) with these fields:

{"run_id": "...", "trace_id": "...", "name": "...", "run_type": "...", "parent_run_id": "...", "inputs": {...}, "outputs": {...}}

Use --include-io or --full to include inputs/outputs (required for dataset generation). </export_format>

Weekly Installs

695

Repository

langchain-ai/la…h-skills

GitHub Stars

First Seen

Mar 4, 2026

Security Audits

Gen Agent Trust HubPass SocketPass SnykPass

Installed on

claude-code621

codex514

cursor451

gemini-cli438

github-copilot437

opencode437

agent-browser 浏览器自动化工具 - Vercel Labs 命令行网页操作与测试

136,300 周安装

LangSmith Trace 追踪工具：AI应用调试与性能监控的完整指南

🇨🇳中文介绍

相关 Skills

嵌套追踪示例

使用原始 LangSmith 查询按反馈分数筛选追踪

🇺🇸English

Nested tracing example

Filter traces by feedback score using raw LangSmith query

最新 Skills