langchain-rag by langchain-ai/langchain-skills
npx skills add https://github.com/langchain-ai/langchain-skills --skill langchain-rag流程管道:
关键组件:
| 向量存储 | 使用场景 | 持久性 |
|---|---|---|
| InMemory | 测试 | 仅内存 |
| FAISS | 本地,高性能 | 磁盘 |
| Chroma | 开发 | 磁盘 |
| Pinecone | 生产,托管 |
广告位招租
在这里展示您的产品或服务
触达数万 AI 开发者,精准高效
| 云 |
docs = [ Document(page_content="LangChain is a framework for LLM apps.", metadata={}), Document(page_content="RAG = Retrieval Augmented Generation.", metadata={}), ]
splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50) splits = splitter.split_documents(docs)
embeddings = OpenAIEmbeddings(model="text-embedding-3-small") vectorstore = InMemoryVectorStore.from_documents(splits, embeddings)
retriever = vectorstore.as_retriever(search_kwargs={"k": 4})
model = ChatOpenAI(model="gpt-4.1") query = "What is RAG?" relevant_docs = retriever.invoke(query)
context = "\n\n".join([doc.page_content for doc in relevant_docs]) response = model.invoke([ {"role": "system", "content": f"Use this context:\n\n{context}"}, {"role": "user", "content": query}, ])
</python>
<typescript>
端到端 RAG 流程:加载文档、分割成块、嵌入、存储、检索并生成响应。
```typescript
import { ChatOpenAI, OpenAIEmbeddings } from "@langchain/openai";
import { MemoryVectorStore } from "@langchain/classic/vectorstores/memory";
import { RecursiveCharacterTextSplitter } from "@langchain/textsplitters";
import { Document } from "@langchain/core/documents";
// 1. 加载文档
const docs = [
new Document({ pageContent: "LangChain is a framework for LLM apps.", metadata: {} }),
new Document({ pageContent: "RAG = Retrieval Augmented Generation.", metadata: {} }),
];
// 2. 分割文档
const splitter = new RecursiveCharacterTextSplitter({ chunkSize: 500, chunkOverlap: 50 });
const splits = await splitter.splitDocuments(docs);
// 3. 创建嵌入并存储
const embeddings = new OpenAIEmbeddings({ model: "text-embedding-3-small" });
const vectorstore = await MemoryVectorStore.fromDocuments(splits, embeddings);
// 4. 创建检索器
const retriever = vectorstore.asRetriever({ k: 4 });
// 5. 在 RAG 中使用
const model = new ChatOpenAI({ model: "gpt-4.1" });
const query = "What is RAG?";
const relevantDocs = await retriever.invoke(query);
const context = relevantDocs.map(doc => doc.pageContent).join("\n\n");
const response = await model.invoke([
{ role: "system", content: `Use this context:\n\n${context}` },
{ role: "user", content: query },
]);
loader = PyPDFLoader("./document.pdf") docs = loader.load() print(f"Loaded {len(docs)} pages")
</python>
<typescript>
加载 PDF 文件并将每一页提取为单独的文档。
```typescript
import { PDFLoader } from "@langchain/community/document_loaders/fs/pdf";
const loader = new PDFLoader("./document.pdf");
const docs = await loader.load();
console.log(`Loaded ${docs.length} pages`);
loader = WebBaseLoader("https://docs.langchain.com") docs = loader.load()
</python>
<typescript>
使用 Cheerio 从网页 URL 获取内容并解析为文档。
```typescript
import { CheerioWebBaseLoader } from "@langchain/community/document_loaders/web/cheerio";
const loader = new CheerioWebBaseLoader("https://docs.langchain.com");
const docs = await loader.load();
loader = DirectoryLoader( "path/to/documents", glob="**/*.txt", # 要加载文件的模式 loader_cls=TextLoader ) docs = loader.load()
</python>
</ex-loading-directory>
---
## 文本分割
<ex-text-splitting>
<python>
使用可配置大小和重叠的 RecursiveCharacterTextSplitter 将文档分割成块。
```python
from langchain_text_splitters import RecursiveCharacterTextSplitter
splitter = RecursiveCharacterTextSplitter(
chunk_size=1000, # 每块字符数
chunk_overlap=200, # 用于上下文连续性的重叠
separators=["\n\n", "\n", " ", ""], # 分割层次结构
)
splits = splitter.split_documents(docs)
vectorstore = Chroma.from_documents( documents=splits, embedding=OpenAIEmbeddings(), persist_directory="./chroma_db", collection_name="my-collection", )
vectorstore = Chroma( persist_directory="./chroma_db", embedding_function=OpenAIEmbeddings(), collection_name="my-collection", )
</python>
<typescript>
创建一个连接到正在运行的 Chroma 服务器的 Chroma 向量存储。
```typescript
import { Chroma } from "@langchain/community/vectorstores/chroma";
import { OpenAIEmbeddings } from "@langchain/openai";
const vectorstore = await Chroma.fromDocuments(
splits,
new OpenAIEmbeddings(),
{ collectionName: "my-collection", url: "http://localhost:8000" }
);
vectorstore = FAISS.from_documents(splits, embeddings) vectorstore.save_local("./faiss_index")
loaded = FAISS.load_local( "./faiss_index", embeddings, allow_dangerous_deserialization=True )
</python>
<typescript>
创建一个 FAISS 向量存储,保存到磁盘,然后重新加载。
```typescript
import { FaissStore } from "@langchain/community/vectorstores/faiss";
const vectorstore = await FaissStore.fromDocuments(splits, embeddings);
await vectorstore.save("./faiss_index");
const loaded = await FaissStore.load("./faiss_index", embeddings);
results_with_score = vectorstore.similarity_search_with_score(query, k=5) for doc, score in results_with_score: print(f"Score: {score}, Content: {doc.page_content}")
</python>
<typescript>
执行相似性搜索并检索带有相关性分数的结果。
```typescript
// 基本搜索
const results = await vectorstore.similaritySearch(query, 5);
// 带分数
const resultsWithScore = await vectorstore.similaritySearchWithScore(query, 5);
for (const [doc, score] of resultsWithScore) {
console.log(`Score: ${score}, Content: ${doc.pageContent}`);
}
results = vectorstore.similarity_search( "programming", k=5, filter={"language": "python"} # 仅限 Python 文档 )
</python>
</ex-metadata-filtering>
<ex-rag-with-agent>
<python>
创建一个使用 RAG 作为工具来回答问题的智能体。
```python
from langchain.agents import create_agent
from langchain.tools import tool
@tool
def search_docs(query: str) -> str:
"""搜索文档以获取相关信息。"""
docs = retriever.invoke(query)
return "\n\n".join([d.page_content for d in docs])
agent = create_agent(
model="gpt-4.1",
tools=[search_docs],
)
result = agent.invoke({
"messages": [{"role": "user", "content": "How do I create an agent?"}]
})
const searchDocs = tool( async (input) => { const docs = await retriever.invoke(input.query); return docs.map(d => d.pageContent).join("\n\n"); }, { name: "search_docs", description: "Search documentation for relevant information.", schema: z.object({ query: z.string() }), } );
const agent = createAgent({ model: "gpt-4.1", tools: [searchDocs], });
const result = await agent.invoke({ messages: [{ role: "user", content: "How do I create an agent?" }], });
</typescript>
</ex-rag-with-agent>
<boundaries>
### 您可以配置的内容
- 块大小/重叠
- 嵌入模型
- 结果数量 (k)
- 元数据过滤器
- 搜索算法:相似性,MMR
### 您无法配置的内容
- 嵌入维度(每个模型)
- 在同一存储中混合使用不同模型的嵌入
</boundaries>
<fix-chunk-size>
<python>
块大小 500-1500 通常比较合适。
```python
# 错误:太小(丢失上下文)或太大(超出限制)
splitter = RecursiveCharacterTextSplitter(chunk_size=50)
splitter = RecursiveCharacterTextSplitter(chunk_size=10000)
# 正确
splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
// 正确 const splitter = new RecursiveCharacterTextSplitter({ chunkSize: 1000, chunkOverlap: 200 });
</typescript>
</fix-chunk-size>
<fix-chunk-overlap>
<python>
使用重叠(块大小的 10-20%)以在边界处保持上下文。
```python
# 错误:无重叠 - 上下文在边界处中断
splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
# 正确:10-20% 重叠
splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
vectorstore = Chroma.from_documents(docs, embeddings, persist_directory="./chroma_db")
</python>
<typescript>
使用持久化向量存储而非内存存储,以避免数据丢失。
```typescript
// 错误:内存存储 - 重启后丢失
const vectorstore = await MemoryVectorStore.fromDocuments(docs, embeddings);
// 正确
const vectorstore = await Chroma.fromDocuments(docs, embeddings, { collectionName: "my-collection" });
embeddings = OpenAIEmbeddings(model="text-embedding-3-small") vectorstore = Chroma.from_documents(docs, embeddings) retriever = vectorstore.as_retriever() # 使用相同的嵌入
</python>
<typescript>
为索引和查询使用相同的嵌入模型。
```typescript
const embeddings = new OpenAIEmbeddings({ model: "text-embedding-3-small" });
const vectorstore = await Chroma.fromDocuments(docs, embeddings);
const retriever = vectorstore.asRetriever(); // 使用相同的嵌入
loaded_store = FAISS.load_local("./faiss_index", embeddings, allow_dangerous_deserialization=True)
</python>
</fix-faiss-deserialization>
<fix-dimension-mismatch>
<python>
确保嵌入维度与向量存储索引维度匹配。
```python
# 错误:索引有 1536 维但使用 512 维嵌入
pc.create_index(name="idx", dimension=1536, metric="cosine")
vectorstore = PineconeVectorStore.from_documents(
docs, OpenAIEmbeddings(model="text-embedding-3-small", dimensions=512), index=pc.Index("idx")
) # 错误:维度不匹配!
# 正确:匹配维度
embeddings = OpenAIEmbeddings() # 默认 1536
每周安装量
2.5K
代码仓库
GitHub 星标数
423
首次出现
2026年3月4日
安全审计
安装于
claude-code2.0K
codex1.9K
cursor1.9K
github-copilot1.9K
opencode1.9K
gemini-cli1.9K
Pipeline:
Key Components:
| Vector Store | Use Case | Persistence |
|---|---|---|
| InMemory | Testing | Memory only |
| FAISS | Local, high performance | Disk |
| Chroma | Development | Disk |
| Pinecone | Production, managed | Cloud |
docs = [ Document(page_content="LangChain is a framework for LLM apps.", metadata={}), Document(page_content="RAG = Retrieval Augmented Generation.", metadata={}), ]
splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50) splits = splitter.split_documents(docs)
embeddings = OpenAIEmbeddings(model="text-embedding-3-small") vectorstore = InMemoryVectorStore.from_documents(splits, embeddings)
retriever = vectorstore.as_retriever(search_kwargs={"k": 4})
model = ChatOpenAI(model="gpt-4.1") query = "What is RAG?" relevant_docs = retriever.invoke(query)
context = "\n\n".join([doc.page_content for doc in relevant_docs]) response = model.invoke([ {"role": "system", "content": f"Use this context:\n\n{context}"}, {"role": "user", "content": query}, ])
</python>
<typescript>
End-to-end RAG pipeline: load documents, split into chunks, embed, store, retrieve, and generate a response.
```typescript
import { ChatOpenAI, OpenAIEmbeddings } from "@langchain/openai";
import { MemoryVectorStore } from "@langchain/classic/vectorstores/memory";
import { RecursiveCharacterTextSplitter } from "@langchain/textsplitters";
import { Document } from "@langchain/core/documents";
// 1. Load documents
const docs = [
new Document({ pageContent: "LangChain is a framework for LLM apps.", metadata: {} }),
new Document({ pageContent: "RAG = Retrieval Augmented Generation.", metadata: {} }),
];
// 2. Split documents
const splitter = new RecursiveCharacterTextSplitter({ chunkSize: 500, chunkOverlap: 50 });
const splits = await splitter.splitDocuments(docs);
// 3. Create embeddings and store
const embeddings = new OpenAIEmbeddings({ model: "text-embedding-3-small" });
const vectorstore = await MemoryVectorStore.fromDocuments(splits, embeddings);
// 4. Create retriever
const retriever = vectorstore.asRetriever({ k: 4 });
// 5. Use in RAG
const model = new ChatOpenAI({ model: "gpt-4.1" });
const query = "What is RAG?";
const relevantDocs = await retriever.invoke(query);
const context = relevantDocs.map(doc => doc.pageContent).join("\n\n");
const response = await model.invoke([
{ role: "system", content: `Use this context:\n\n${context}` },
{ role: "user", content: query },
]);
loader = PyPDFLoader("./document.pdf") docs = loader.load() print(f"Loaded {len(docs)} pages")
</python>
<typescript>
Load a PDF file and extract each page as a separate document.
```typescript
import { PDFLoader } from "@langchain/community/document_loaders/fs/pdf";
const loader = new PDFLoader("./document.pdf");
const docs = await loader.load();
console.log(`Loaded ${docs.length} pages`);
loader = WebBaseLoader("https://docs.langchain.com") docs = loader.load()
</python>
<typescript>
Fetch and parse content from a web URL into a document using Cheerio.
```typescript
import { CheerioWebBaseLoader } from "@langchain/community/document_loaders/web/cheerio";
const loader = new CheerioWebBaseLoader("https://docs.langchain.com");
const docs = await loader.load();
loader = DirectoryLoader( "path/to/documents", glob="**/*.txt", # Pattern for files to load loader_cls=TextLoader ) docs = loader.load()
</python>
</ex-loading-directory>
---
## Text Splitting
<ex-text-splitting>
<python>
Split documents into chunks using RecursiveCharacterTextSplitter with configurable size and overlap.
```python
from langchain_text_splitters import RecursiveCharacterTextSplitter
splitter = RecursiveCharacterTextSplitter(
chunk_size=1000, # Characters per chunk
chunk_overlap=200, # Overlap for context continuity
separators=["\n\n", "\n", " ", ""], # Split hierarchy
)
splits = splitter.split_documents(docs)
vectorstore = Chroma.from_documents( documents=splits, embedding=OpenAIEmbeddings(), persist_directory="./chroma_db", collection_name="my-collection", )
vectorstore = Chroma( persist_directory="./chroma_db", embedding_function=OpenAIEmbeddings(), collection_name="my-collection", )
</python>
<typescript>
Create a Chroma vector store connected to a running Chroma server.
```typescript
import { Chroma } from "@langchain/community/vectorstores/chroma";
import { OpenAIEmbeddings } from "@langchain/openai";
const vectorstore = await Chroma.fromDocuments(
splits,
new OpenAIEmbeddings(),
{ collectionName: "my-collection", url: "http://localhost:8000" }
);
vectorstore = FAISS.from_documents(splits, embeddings) vectorstore.save_local("./faiss_index")
loaded = FAISS.load_local( "./faiss_index", embeddings, allow_dangerous_deserialization=True )
</python>
<typescript>
Create a FAISS vector store, save it to disk, and reload it.
```typescript
import { FaissStore } from "@langchain/community/vectorstores/faiss";
const vectorstore = await FaissStore.fromDocuments(splits, embeddings);
await vectorstore.save("./faiss_index");
const loaded = await FaissStore.load("./faiss_index", embeddings);
results_with_score = vectorstore.similarity_search_with_score(query, k=5) for doc, score in results_with_score: print(f"Score: {score}, Content: {doc.page_content}")
</python>
<typescript>
Perform similarity search and retrieve results with relevance scores.
```typescript
// Basic search
const results = await vectorstore.similaritySearch(query, 5);
// With scores
const resultsWithScore = await vectorstore.similaritySearchWithScore(query, 5);
for (const [doc, score] of resultsWithScore) {
console.log(`Score: ${score}, Content: ${doc.pageContent}`);
}
results = vectorstore.similarity_search( "programming", k=5, filter={"language": "python"} # Only Python docs )
</python>
</ex-metadata-filtering>
<ex-rag-with-agent>
<python>
Create an agent that uses RAG as a tool for answering questions.
```python
from langchain.agents import create_agent
from langchain.tools import tool
@tool
def search_docs(query: str) -> str:
"""Search documentation for relevant information."""
docs = retriever.invoke(query)
return "\n\n".join([d.page_content for d in docs])
agent = create_agent(
model="gpt-4.1",
tools=[search_docs],
)
result = agent.invoke({
"messages": [{"role": "user", "content": "How do I create an agent?"}]
})
const searchDocs = tool( async (input) => { const docs = await retriever.invoke(input.query); return docs.map(d => d.pageContent).join("\n\n"); }, { name: "search_docs", description: "Search documentation for relevant information.", schema: z.object({ query: z.string() }), } );
const agent = createAgent({ model: "gpt-4.1", tools: [searchDocs], });
const result = await agent.invoke({ messages: [{ role: "user", content: "How do I create an agent?" }], });
</typescript>
</ex-rag-with-agent>
<boundaries>
### What You CAN Configure
- Chunk size/overlap
- Embedding model
- Number of results (k)
- Metadata filters
- Search algorithms: Similarity, MMR
### What You CANNOT Configure
- Embedding dimensions (per model)
- Mix embeddings from different models in same store
</boundaries>
<fix-chunk-size>
<python>
Chunk size 500-1500 is typically good.
```python
# WRONG: Too small (loses context) or too large (hits limits)
splitter = RecursiveCharacterTextSplitter(chunk_size=50)
splitter = RecursiveCharacterTextSplitter(chunk_size=10000)
# CORRECT
splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
// CORRECT const splitter = new RecursiveCharacterTextSplitter({ chunkSize: 1000, chunkOverlap: 200 });
</typescript>
</fix-chunk-size>
<fix-chunk-overlap>
<python>
Use overlap (10-20% of chunk size) to maintain context at boundaries.
```python
# WRONG: No overlap - context breaks at boundaries
splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
# CORRECT: 10-20% overlap
splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
vectorstore = Chroma.from_documents(docs, embeddings, persist_directory="./chroma_db")
</python>
<typescript>
Use persistent vector store instead of in-memory to avoid data loss.
```typescript
// WRONG: Memory - lost on restart
const vectorstore = await MemoryVectorStore.fromDocuments(docs, embeddings);
// CORRECT
const vectorstore = await Chroma.fromDocuments(docs, embeddings, { collectionName: "my-collection" });
embeddings = OpenAIEmbeddings(model="text-embedding-3-small") vectorstore = Chroma.from_documents(docs, embeddings) retriever = vectorstore.as_retriever() # Uses same embeddings
</python>
<typescript>
Use the same embedding model for indexing and querying.
```typescript
const embeddings = new OpenAIEmbeddings({ model: "text-embedding-3-small" });
const vectorstore = await Chroma.fromDocuments(docs, embeddings);
const retriever = vectorstore.asRetriever(); // Uses same embeddings
loaded_store = FAISS.load_local("./faiss_index", embeddings, allow_dangerous_deserialization=True)
</python>
</fix-faiss-deserialization>
<fix-dimension-mismatch>
<python>
Ensure embedding dimensions match the vector store index dimensions.
```python
# WRONG: Index has 1536 dimensions but using 512-dim embeddings
pc.create_index(name="idx", dimension=1536, metric="cosine")
vectorstore = PineconeVectorStore.from_documents(
docs, OpenAIEmbeddings(model="text-embedding-3-small", dimensions=512), index=pc.Index("idx")
) # Error: dimension mismatch!
# CORRECT: Match dimensions
embeddings = OpenAIEmbeddings() # Default 1536
Weekly Installs
2.5K
Repository
GitHub Stars
423
First Seen
Mar 4, 2026
Security Audits
Gen Agent Trust HubPassSocketWarnSnykWarn
Installed on
claude-code2.0K
codex1.9K
cursor1.9K
github-copilot1.9K
opencode1.9K
gemini-cli1.9K
React 组合模式指南:Vercel 组件架构最佳实践,提升代码可维护性
102,200 周安装