langsmith-trace by langchain-ai/langsmith-skills
npx skills add https://github.com/langchain-ai/langsmith-skills --skill langsmith-traceLANGSMITH_API_KEY=lsv2_pt_your_api_key_here # 必需 LANGSMITH_PROJECT=your-project-name # 可选:默认项目 LANGSMITH_WORKSPACE_ID=your-workspace-id # 可选:用于组织范围的密钥
重要提示: 在查询或与 LangSmith 交互之前,请务必检查环境变量或 .env 文件中的 LANGSMITH_PROJECT。这会告诉你哪个项目包含相关的追踪记录和数据。如果 LangSmith 项目不可用,请运用你的最佳判断来确定正确的项目。
CLI 工具
curl -sSL https://raw.githubusercontent.com/langchain-ai/langsmith-cli/main/scripts/install.sh | sh
<trace_langchain_oss> 对于 LangChain/LangGraph 应用,追踪是自动的。只需设置环境变量:
export LANGSMITH_TRACING=true
export LANGSMITH_API_KEY=<your-api-key>
export OPENAI_API_KEY=<your-openai-api-key> # 或你的 LLM 提供商的密钥
可选变量:
LANGSMITH_PROJECT - 指定项目名称(默认为 "default")LANGCHAIN_CALLBACKS_BACKGROUND=false - 用于无服务器环境,确保在函数退出前完成追踪记录(Python)</trace_langchain_oss>广告位招租
在这里展示您的产品或服务
触达数万 AI 开发者,精准高效
<trace_other_frameworks> 对于非 LangChain 应用,如果框架原生支持 OpenTelemetry,请使用 LangSmith 的 OpenTelemetry 集成。
如果应用没有使用框架,或者使用的框架没有自动的 OTel 支持,请使用 traceable 装饰器/包装器来包装你的 LLM 客户端。
client = wrap_openai(OpenAI())
@traceable def my_llm_pipeline(question: str) -> str: resp = client.chat.completions.create( model="gpt-4o-mini", messages=[{"role": "user", "content": question}], ) return resp.choices[0].message.content
@traceable def rag_pipeline(question: str) -> str: docs = retrieve_docs(question) return generate_answer(question, docs)
@traceable(name="retrieve_docs") def retrieve_docs(query: str) -> list[str]: return docs
@traceable(name="generate_answer") def generate_answer(question: str, docs: list[str]) -> str: return client.chat.completions.create(...)
</python>
<typescript>
使用 traceable() 包装器和 wrapOpenAI() 实现自动追踪。
```typescript
import { traceable } from "langsmith/traceable";
import { wrapOpenAI } from "langsmith/wrappers";
import OpenAI from "openai";
const client = wrapOpenAI(new OpenAI());
const myLlmPipeline = traceable(async (question: string): Promise<string> => {
const resp = await client.chat.completions.create({
model: "gpt-4o-mini",
messages: [{ role: "user", content: question }],
});
return resp.choices[0].message.content || "";
}, { name: "my_llm_pipeline" });
// 嵌套追踪示例
const retrieveDocs = traceable(async (query: string): Promise<string[]> => {
return docs;
}, { name: "retrieve_docs" });
const generateAnswer = traceable(async (question: string, docs: string[]): Promise<string> => {
const resp = await client.chat.completions.create({
model: "gpt-4o-mini",
messages: [{ role: "user", content: `${question}\nContext: ${docs.join("\n")}` }],
});
return resp.choices[0].message.content || "";
}, { name: "generate_answer" });
const ragPipeline = traceable(async (question: string): Promise<string> => {
const docs = await retrieveDocs(question);
return await generateAnswer(question, docs);
}, { name: "rag_pipeline" });
最佳实践:
wrap_openai()/wrapOpenAI() 会记录每一次 LLM 调用<traces_vs_runs> 使用 langsmith CLI 来查询追踪数据。
理解两者的区别至关重要:
通常,首先查询追踪 — 它们提供完整的上下文并保留轨迹分析和数据集生成所需的层次结构。</traces_vs_runs>
<command_structure> 两个命令组,行为一致:
langsmith
├── trace (对追踪树的操作 - 首先使用这个)
│ ├── list - 列出追踪(过滤器应用于根运行)
│ ├── get - 获取单个追踪及其完整层次结构
│ └── export - 将追踪导出到 JSONL 文件(每个追踪一个文件)
│
├── run (对单个运行的操作 - 用于特定分析)
│ ├── list - 列出运行(扁平列表,过滤器应用于任何运行)
│ ├── get - 获取单个运行
│ └── export - 将运行导出到单个 JSONL 文件(扁平)
│
├── dataset (数据集操作)
│ ├── list - 列出数据集
│ ├── get - 获取数据集详情
│ ├── create - 创建空数据集
│ ├── delete - 删除数据集
│ ├── export - 将数据集导出到文件
│ └── upload - 上传本地 JSON 作为数据集
│
├── example (示例操作)
│ ├── list - 列出数据集中的示例
│ ├── create - 向数据集添加示例
│ └── delete - 删除示例
│
├── evaluator (评估器操作)
│ ├── list - 列出评估器
│ ├── upload - 上传评估器
│ └── delete - 删除评估器
│
├── experiment (实验操作)
│ ├── list - 列出实验
│ └── get - 获取实验结果
│
├── thread (线程操作)
│ ├── list - 列出对话线程
│ └── get - 获取线程详情
│
└── project (项目操作)
└── list - 列出追踪项目
关键区别:
| traces * | runs *
---|---|---
过滤器应用于 | 仅根运行 | 任何匹配的运行
--run-type | 不可用 | 可用
返回结果 | 完整层次结构 | 扁平列表
导出输出 | 目录(每个追踪一个文件) | 单个文件
</command_structure> | |
<querying_traces> 使用 langsmith CLI 查询追踪。命令与语言无关。
# 列出最近的追踪(最常用的操作)
langsmith trace list --limit 10 --project my-project
# 列出包含元数据的追踪(时间、令牌数、成本)
langsmith trace list --limit 10 --include-metadata
# 按时间筛选追踪
langsmith trace list --last-n-minutes 60
langsmith trace list --since 2025-01-20T10:00:00Z
# 获取特定追踪及其完整层次结构
langsmith trace get <trace-id>
# 列出追踪并内联显示层次结构
langsmith trace list --limit 5 --show-hierarchy
# 将追踪导出到 JSONL(每个追踪一个文件,包含所有运行)
langsmith trace export ./traces --limit 20 --full
# 按性能筛选追踪
langsmith trace list --min-latency 5.0 --limit 10 # 慢追踪(>= 5秒)
langsmith trace list --error --last-n-minutes 60 # 失败的追踪
# 列出特定运行类型(扁平列表)
langsmith run list --run-type llm --limit 20
</querying_traces>
基本过滤器:
--trace-ids abc,def - 筛选到特定追踪--limit N - 最大结果数--project NAME - 项目名称--last-n-minutes N - 时间过滤器--since TIMESTAMP - 时间过滤器(ISO 格式)--error / --no-error - 错误状态--name PATTERN - 名称包含(不区分大小写)性能过滤器:
--min-latency SECONDS - 最小延迟(例如,5 表示 >= 5秒)--max-latency SECONDS - 最大延迟--min-tokens N - 最小总令牌数--tags tag1,tag2 - 具有这些标签中的任何一个高级过滤器:
--filter QUERY - 原始 LangSmith 过滤器查询,用于复杂情况(反馈、元数据等)
langsmith trace list --filter 'and(eq(feedback_key, "correctness"), gte(feedback_score, 0.8))'
<export_format> 导出会创建 .jsonl 文件(每行一个运行),包含以下字段:
{"run_id": "...", "trace_id": "...", "name": "...", "run_type": "...", "parent_run_id": "...", "inputs": {...}, "outputs": {...}}
使用 --include-io 或 --full 来包含输入/输出(数据集生成所需)。</export_format>
每周安装次数
695
代码仓库
GitHub 星标数
76
首次出现
Mar 4, 2026
安全审计
安装于
claude-code621
codex514
cursor451
gemini-cli438
github-copilot437
opencode437
LANGSMITH_API_KEY=lsv2_pt_your_api_key_here # Required LANGSMITH_PROJECT=your-project-name # Optional: default project LANGSMITH_WORKSPACE_ID=your-workspace-id # Optional: for org-scoped keys
IMPORTANT: Always check the environment variables or .env file for LANGSMITH_PROJECT before querying or interacting with LangSmith. This tells you which project contains the relevant traces and data. If the LangSmith project is not available, use your best judgement to identify the right one.
CLI Tool
curl -sSL https://raw.githubusercontent.com/langchain-ai/langsmith-cli/main/scripts/install.sh | sh
<trace_langchain_oss> For LangChain/LangGraph apps, tracing is automatic. Just set environment variables:
export LANGSMITH_TRACING=true
export LANGSMITH_API_KEY=<your-api-key>
export OPENAI_API_KEY=<your-openai-api-key> # or your LLM provider's key
Optional variables:
LANGSMITH_PROJECT - specify project name (defaults to "default")LANGCHAIN_CALLBACKS_BACKGROUND=false - use for serverless to ensure traces complete before function exit (Python) </trace_langchain_oss><trace_other_frameworks> For non-LangChain apps, if the framework has native OpenTelemetry support, use LangSmith's OpenTelemetry integration.
If the app is NOT using a framework, or using one without automatic OTel support, use the traceable decorator/wrapper and wrap your LLM client.
client = wrap_openai(OpenAI())
@traceable def my_llm_pipeline(question: str) -> str: resp = client.chat.completions.create( model="gpt-4o-mini", messages=[{"role": "user", "content": question}], ) return resp.choices[0].message.content
@traceable def rag_pipeline(question: str) -> str: docs = retrieve_docs(question) return generate_answer(question, docs)
@traceable(name="retrieve_docs") def retrieve_docs(query: str) -> list[str]: return docs
@traceable(name="generate_answer") def generate_answer(question: str, docs: list[str]) -> str: return client.chat.completions.create(...)
</python>
<typescript>
Use traceable() wrapper and wrapOpenAI() for automatic tracing.
```typescript
import { traceable } from "langsmith/traceable";
import { wrapOpenAI } from "langsmith/wrappers";
import OpenAI from "openai";
const client = wrapOpenAI(new OpenAI());
const myLlmPipeline = traceable(async (question: string): Promise<string> => {
const resp = await client.chat.completions.create({
model: "gpt-4o-mini",
messages: [{ role: "user", content: question }],
});
return resp.choices[0].message.content || "";
}, { name: "my_llm_pipeline" });
// Nested tracing example
const retrieveDocs = traceable(async (query: string): Promise<string[]> => {
return docs;
}, { name: "retrieve_docs" });
const generateAnswer = traceable(async (question: string, docs: string[]): Promise<string> => {
const resp = await client.chat.completions.create({
model: "gpt-4o-mini",
messages: [{ role: "user", content: `${question}\nContext: ${docs.join("\n")}` }],
});
return resp.choices[0].message.content || "";
}, { name: "generate_answer" });
const ragPipeline = traceable(async (question: string): Promise<string> => {
const docs = await retrieveDocs(question);
return await generateAnswer(question, docs);
}, { name: "rag_pipeline" });
Best Practices:
wrap_openai()/wrapOpenAI() records every LLM call<traces_vs_runs> Use the langsmith CLI to query trace data.
Understanding the difference is critical:
Generally, query traces first — they provide complete context and preserve hierarchy needed for trajectory analysis and dataset generation. </traces_vs_runs>
<command_structure> Two command groups with consistent behavior:
langsmith
├── trace (operations on trace trees - USE THIS FIRST)
│ ├── list - List traces (filters apply to root run)
│ ├── get - Get single trace with full hierarchy
│ └── export - Export traces to JSONL files (one file per trace)
│
├── run (operations on individual runs - for specific analysis)
│ ├── list - List runs (flat, filters apply to any run)
│ ├── get - Get single run
│ └── export - Export runs to single JSONL file (flat)
│
├── dataset (dataset operations)
│ ├── list - List datasets
│ ├── get - Get dataset details
│ ├── create - Create empty dataset
│ ├── delete - Delete dataset
│ ├── export - Export dataset to file
│ └── upload - Upload local JSON as dataset
│
├── example (example operations)
│ ├── list - List examples in a dataset
│ ├── create - Add example to a dataset
│ └── delete - Delete an example
│
├── evaluator (evaluator operations)
│ ├── list - List evaluators
│ ├── upload - Upload evaluator
│ └── delete - Delete evaluator
│
├── experiment (experiment operations)
│ ├── list - List experiments
│ └── get - Get experiment results
│
├── thread (thread operations)
│ ├── list - List conversation threads
│ └── get - Get thread details
│
└── project (project operations)
└── list - List tracing projects
Key differences:
| traces * | runs *
---|---|---
Filters apply to | Root run only | Any matching run
--run-type | Not available | Available
Returns | Full hierarchy | Flat list
Export output | Directory (one file/trace) | Single file
</command_structure> | |
<querying_traces> Query traces using the langsmith CLI. Commands are language-agnostic.
# List recent traces (most common operation)
langsmith trace list --limit 10 --project my-project
# List traces with metadata (timing, tokens, costs)
langsmith trace list --limit 10 --include-metadata
# Filter traces by time
langsmith trace list --last-n-minutes 60
langsmith trace list --since 2025-01-20T10:00:00Z
# Get specific trace with full hierarchy
langsmith trace get <trace-id>
# List traces and show hierarchy inline
langsmith trace list --limit 5 --show-hierarchy
# Export traces to JSONL (one file per trace, includes all runs)
langsmith trace export ./traces --limit 20 --full
# Filter traces by performance
langsmith trace list --min-latency 5.0 --limit 10 # Slow traces (>= 5s)
langsmith trace list --error --last-n-minutes 60 # Failed traces
# List specific run types (flat list)
langsmith run list --run-type llm --limit 20
</querying_traces>
Basic filters:
--trace-ids abc,def - Filter to specific traces--limit N - Max results--project NAME - Project name--last-n-minutes N - Time filter--since TIMESTAMP - Time filter (ISO format)--error / --no-error - Error status--name PATTERN - Name contains (case-insensitive)Performance filters:
--min-latency SECONDS - Minimum latency (e.g., 5 for >= 5s)--max-latency SECONDS - Maximum latency--min-tokens N - Minimum total tokens--tags tag1,tag2 - Has any of these tagsAdvanced filter:
--filter QUERY - Raw LangSmith filter query for complex cases (feedback, metadata, etc.)
langsmith trace list --filter 'and(eq(feedback_key, "correctness"), gte(feedback_score, 0.8))'
<export_format> Export creates .jsonl files (one run per line) with these fields:
{"run_id": "...", "trace_id": "...", "name": "...", "run_type": "...", "parent_run_id": "...", "inputs": {...}, "outputs": {...}}
Use --include-io or --full to include inputs/outputs (required for dataset generation). </export_format>
Weekly Installs
695
Repository
GitHub Stars
76
First Seen
Mar 4, 2026
Security Audits
Gen Agent Trust HubPassSocketPassSnykPass
Installed on
claude-code621
codex514
cursor451
gemini-cli438
github-copilot437
opencode437
agent-browser 浏览器自动化工具 - Vercel Labs 命令行网页操作与测试
136,300 周安装