LaminDB：面向生物学的开源数据框架，实现FAIR原则与数据谱系追踪

lamindb by davila7/claude-code-templates

182 周安装量

24,300 GitHub Stars

GitHub

安装命令

npx skills add https://github.com/davila7/claude-code-templates --skill lamindb

科研工具生物信息学数据处理

🇨🇳中文介绍

LaminDB

概述

LaminDB 是一个面向生物学的开源数据框架，旨在使数据可查询、可追溯、可重现且符合 FAIR（可查找、可访问、可互操作、可重用）原则。它通过单一的 Python API 提供了一个统一平台，结合了湖仓架构、谱系追踪、特征存储、生物本体论、LIMS（实验室信息管理系统）和 ELN（电子实验记录本）功能。

核心价值主张：

可查询性：通过元数据、特征和本体术语搜索和筛选数据集
可追溯性：从原始数据到分析再到结果的自动谱系追踪
可重现性：数据、代码和环境的版本控制
FAIR 合规性：使用生物本体论进行标准化标注

何时使用此技能

在以下情况下使用此技能：

管理生物数据集：scRNA-seq、bulk RNA-seq、空间转录组学、流式细胞术、多模态数据、EHR 数据
追踪计算工作流：Notebook、脚本、管道执行（Nextflow、Snakemake、Redun）
管理和验证数据：模式验证、标准化、基于本体论的标注
处理生物本体论：基因、蛋白质、细胞类型、组织、疾病、通路（通过 Bionty）
构建数据湖仓：跨多个数据集的统一查询接口
确保可重现性：自动版本控制、谱系追踪、环境捕获
集成 ML 管道：与 Weights & Biases、MLflow、HuggingFace、scVI-tools 连接
部署数据基础设施：设置本地或基于云的数据管理系统
数据集协作：使用标准化元数据共享经过管理和标注的数据

核心能力

LaminDB 提供六个相互关联的能力领域，每个领域在参考资料文件夹中都有详细文档。

1. 核心概念与数据谱系

核心实体：

广告位招租

在这里展示您的产品或服务

触达数万 AI 开发者，精准高效

联系我们

2. 数据管理与查询

使用自动完成功能进行注册表探索和查找
使用 get()、one()、one_or_none() 进行单条记录检索
使用比较运算符进行筛选（__gt、__lte、__contains、__startswith）
基于特征的查询（通过标注的元数据进行查询）
使用双下划线语法进行跨注册表遍历
跨注册表的全文搜索
使用 Q 对象进行高级逻辑查询（AND、OR、NOT）
无需加载到内存即可流式传输大型数据集

关键工作流：

使用筛选器和排序浏览 Artifacts
按特征、创建日期、创建者、大小等进行查询
分块或使用数组切片流式传输大文件
使用分层键组织数据
将 Artifacts 分组到集合中

参考： references/data-management.md - 阅读此文件以获取全面的查询模式、筛选示例、流式传输策略和数据组织最佳实践。

验证：确认数据集符合期望的模式
标准化：修正拼写错误，将同义词映射到规范术语
标注：将数据集链接到元数据实体以实现可查询性

灵活模式：仅验证已知列，允许额外的元数据
最小必需模式：指定必需列，允许额外列
严格模式：完全控制结构和值

支持的数据类型：

DataFrames（Parquet、CSV）
AnnData（单细胞基因组学）
MuData（多模态）
SpatialData（空间转录组学）
TileDB-SOMA（可扩展数组）

关键工作流：

为数据验证定义特征和模式
使用 DataFrameCurator 或 AnnDataCurator 进行验证
使用 .cat.standardize() 标准化值
使用 .cat.add_ontology() 映射到本体论
保存带有模式链接的管理后 Artifacts
通过特征查询已验证的数据集

参考： references/annotation-validation.md - 阅读此文件以获取详细的管理工作流、模式设计模式、处理验证错误和最佳实践。

可用的本体论（通过 Bionty）：

基因（Ensembl）、蛋白质（UniProt）
细胞类型（CL）、细胞系（CLO）
组织（Uberon）、疾病（Mondo、DOID）
表型（HPO）、通路（GO）
实验因子（EFO）、发育阶段
生物体（NCBItaxon）、药物（DrugBank）

关键工作流：

使用 bt.CellType.import_source() 导入公共本体论
使用关键词或精确匹配搜索本体论
使用同义词映射标准化术语
探索层次关系（父级、子级、祖先）
根据本体术语验证数据
使用本体记录标注数据集
创建自定义术语和层次结构
处理多生物体上下文（人类、小鼠等）

参考： references/ontologies.md - 阅读此文件以获取全面的本体论操作、标准化策略、层次结构导航和标注工作流。

工作流管理器：

Nextflow：追踪管道流程和输出
Snakemake：集成到 Snakemake 规则中
Redun：与 Redun 任务追踪结合

Weights & Biases：将实验与数据 Artifacts 链接
MLflow：追踪模型和实验
HuggingFace：追踪模型微调
scVI-tools：单细胞分析工作流

本地文件系统、AWS S3、Google Cloud Storage
S3 兼容（MinIO、Cloudflare R2）
HTTP/HTTPS 端点（只读）
HuggingFace 数据集

TileDB-SOMA（支持 cellxgene）
DuckDB 用于对 Parquet 文件进行 SQL 查询

Vitessce 用于交互式空间/单细胞可视化

Git 集成用于源代码追踪

参考： references/integrations.md - 阅读此文件以获取第三方系统的集成模式、代码示例和故障排除。

基础：uv pip install lamindb
带额外功能：uv pip install 'lamindb[gcp,zarr,fcs]'
模块：bionty、wetlab、clinical

本地 SQLite（开发）
云存储 + SQLite（小型团队）
云存储 + PostgreSQL（生产）

本地文件系统
AWS S3，具有可配置的区域和权限
Google Cloud Storage
S3 兼容端点（MinIO、Cloudflare R2）

云文件的缓存管理
多用户系统配置
Git 仓库同步
环境变量

本地开发 → 云生产迁移
多区域部署
共享存储与个人实例

参考： references/setup-deployment.md - 阅读此文件以获取详细的安装、配置、存储设置、数据库管理、安全最佳实践和故障排除。

常见用例工作流

用例 1：使用本体论验证的单细胞 RNA-seq 分析

import lamindb as ln
import bionty as bt
import anndata as ad

# 开始追踪
ln.track(params={"analysis": "scRNA-seq QC and annotation"})

# 导入细胞类型本体论
bt.CellType.import_source()

# 加载数据
adata = ad.read_h5ad("raw_counts.h5ad")

# 验证并标准化细胞类型
adata.obs["cell_type"] = bt.CellType.standardize(adata.obs["cell_type"])

# 使用模式进行管理
curator = ln.curators.AnnDataCurator(adata, schema)
curator.validate()
artifact = curator.save_artifact(key="scrna/validated.h5ad")

# 链接本体论标注
cell_types = bt.CellType.from_values(adata.obs.cell_type)
artifact.feature_sets.add_ontology(cell_types)

ln.finish()

用例 2：构建可查询的数据湖仓

import lamindb as ln

# 注册多个实验
for i, file in enumerate(data_files):
    artifact = ln.Artifact.from_anndata(
        ad.read_h5ad(file),
        key=f"scrna/batch_{i}.h5ad",
        description=f"scRNA-seq batch {i}"
    ).save()

    # 使用特征进行标注
    artifact.features.add_values({
        "batch": i,
        "tissue": tissues[i],
        "condition": conditions[i]
    })

# 跨所有实验查询
immune_datasets = ln.Artifact.filter(
    key__startswith="scrna/",
    tissue="PBMC",
    condition="treated"
).to_dataframe()

# 加载特定数据集
for artifact in immune_datasets:
    adata = artifact.load()
    # 分析

用例 3：与 W&B 集成的 ML 管道

import lamindb as ln
import wandb

# 初始化两个系统
wandb.init(project="drug-response", name="exp-42")
ln.track(params={"model": "random_forest", "n_estimators": 100})

# 从 LaminDB 加载训练数据
train_artifact = ln.Artifact.get(key="datasets/train.parquet")
train_data = train_artifact.load()

# 训练模型
model = train_model(train_data)

# 记录到 W&B
wandb.log({"accuracy": 0.95})

# 在 LaminDB 中保存模型并链接到 W&B
import joblib
joblib.dump(model, "model.pkl")
model_artifact = ln.Artifact("model.pkl", key="models/exp-42.pkl").save()
model_artifact.features.add_values({"wandb_run_id": wandb.run.id})

ln.finish()
wandb.finish()

用例 4：Nextflow 管道集成

# 在 Nextflow 进程脚本中
import lamindb as ln

ln.track()

# 加载输入 Artifact
input_artifact = ln.Artifact.get(key="raw/batch_${batch_id}.fastq.gz")
input_path = input_artifact.cache()

# 处理（比对、定量等）
# ... Nextflow 进程逻辑 ...

# 保存输出
output_artifact = ln.Artifact(
    "counts.csv",
    key="processed/batch_${batch_id}_counts.csv"
).save()

ln.finish()

要开始有效使用 LaminDB：

安装与设置 (references/setup-deployment.md)
- 安装 LaminDB 和所需的额外功能
- 使用 lamin login 进行身份验证
- 使用 lamin init --storage ... 初始化实例
学习核心概念 (references/core-concepts.md)
- 理解 Artifacts、Records、Runs、Transforms
- 练习创建和检索 Artifacts
- 在工作流中实现 ln.track() 和 ln.finish()
掌握查询 (references/data-management.md)
- 练习筛选和搜索注册表
- 学习基于特征的查询
- 尝试流式传输大文件
设置验证 (references/annotation-validation.md)
- 定义与研究领域相关的特征
- 为数据类型创建模式
- 练习管理工作流
集成本体论 (references/ontologies.md)
- 导入相关的生物本体论（基因、细胞类型等）
- 验证现有标注
- 使用本体术语标准化元数据
连接工具 (references/integrations.md)
- 与现有的工作流管理器集成
- 链接 ML 平台以进行实验追踪
- 配置云存储和计算

在使用 LaminDB 时遵循以下原则：

追踪一切：在每次分析开始时使用 ln.track() 以自动捕获谱系
尽早验证：在深入分析之前定义模式并验证数据
使用本体论：利用公共生物本体论进行标准化标注
使用键组织：分层结构组织 Artifact 键（例如 project/experiment/batch/file.h5ad）
先查询元数据：在加载大文件之前进行筛选和搜索
版本化，不复制：使用内置版本控制，而不是为修改创建新键
使用特征标注：定义类型化特征以实现可查询的元数据
充分记录：为 Artifacts、模式和 Transforms 添加描述
利用谱系：使用 view_lineage() 理解数据来源
从本地开始，扩展到云端：使用 SQLite 在本地开发，使用 PostgreSQL 部署到云端

此技能包含按能力组织的全面参考文档：

references/core-concepts.md - Artifacts、Records、Runs、Transforms、Features、版本控制、谱系
references/data-management.md - 查询、筛选、搜索、流式传输、数据组织
references/annotation-validation.md - 模式设计、管理工作流、验证策略
references/ontologies.md - 生物本体论管理、标准化、层次结构
references/integrations.md - 工作流管理器、MLOps 平台、存储系统、工具
references/setup-deployment.md - 安装、配置、部署、故障排除

根据手头任务所需的特定 LaminDB 能力，阅读相关的参考文件。

官方文档：https://docs.lamin.ai
API 参考：https://docs.lamin.ai/api
GitHub 仓库：https://github.com/laminlabs/lamindb
教程：https://docs.lamin.ai/tutorial
常见问题：https://docs.lamin.ai/faq

🇺🇸English

LaminDB

Overview

LaminDB is an open-source data framework for biology designed to make data queryable, traceable, reproducible, and FAIR (Findable, Accessible, Interoperable, Reusable). It provides a unified platform that combines lakehouse architecture, lineage tracking, feature stores, biological ontologies, LIMS (Laboratory Information Management System), and ELN (Electronic Lab Notebook) capabilities through a single Python API.

Core Value Proposition:

Queryability : Search and filter datasets by metadata, features, and ontology terms
Traceability : Automatic lineage tracking from raw data through analysis to results
Reproducibility : Version control for data, code, and environment
FAIR Compliance : Standardized annotations using biological ontologies

When to Use This Skill

Use this skill when:

Managing biological datasets : scRNA-seq, bulk RNA-seq, spatial transcriptomics, flow cytometry, multi-modal data, EHR data
Tracking computational workflows : Notebooks, scripts, pipeline execution (Nextflow, Snakemake, Redun)
Curating and validating data : Schema validation, standardization, ontology-based annotation
Working with biological ontologies : Genes, proteins, cell types, tissues, diseases, pathways (via Bionty)
Building data lakehouses : Unified query interface across multiple datasets
Ensuring reproducibility : Automatic versioning, lineage tracking, environment capture
Integrating ML pipelines : Connecting with Weights & Biases, MLflow, HuggingFace, scVI-tools
Deploying data infrastructure : Setting up local or cloud-based data management systems
Collaborating on datasets : Sharing curated, annotated data with standardized metadata

Core Capabilities

LaminDB provides six interconnected capability areas, each documented in detail in the references folder.

1. Core Concepts and Data Lineage

Core entities:

Artifacts : Versioned datasets (DataFrame, AnnData, Parquet, Zarr, etc.)
Records : Experimental entities (samples, perturbations, instruments)
Runs & Transforms: Computational lineage tracking (what code produced what data)
Features : Typed metadata fields for annotation and querying

Key workflows:

Create and version artifacts from files or Python objects
Track notebook/script execution with ln.track() and ln.finish()
Annotate artifacts with typed features
Visualize data lineage graphs with artifact.view_lineage()
Query by provenance (find all outputs from specific code/inputs)

Reference: references/core-concepts.md - Read this for detailed information on artifacts, records, runs, transforms, features, versioning, and lineage tracking.

2. Data Management and Querying

Query capabilities:

Registry exploration and lookup with auto-complete
Single record retrieval with get(), one(), one_or_none()
Filtering with comparison operators (__gt, __lte, __contains, __startswith)
Feature-based queries (query by annotated metadata)
Cross-registry traversal with double-underscore syntax
Full-text search across registries
Advanced logical queries with Q objects (AND, OR, NOT)
Streaming large datasets without loading into memory

Key workflows:

Browse artifacts with filters and ordering
Query by features, creation date, creator, size, etc.
Stream large files in chunks or with array slicing
Organize data with hierarchical keys
Group artifacts into collections

Reference: references/data-management.md - Read this for comprehensive query patterns, filtering examples, streaming strategies, and data organization best practices.

3. Annotation and Validation

Curation process:

Validation : Confirm datasets match desired schemas
Standardization : Fix typos, map synonyms to canonical terms
Annotation : Link datasets to metadata entities for queryability

Schema types:

Flexible schemas : Validate only known columns, allow additional metadata
Minimal required schemas : Specify essential columns, permit extras
Strict schemas : Complete control over structure and values

Supported data types:

DataFrames (Parquet, CSV)
AnnData (single-cell genomics)
MuData (multi-modal)
SpatialData (spatial transcriptomics)
TileDB-SOMA (scalable arrays)

Key workflows:

Define features and schemas for data validation
Use DataFrameCurator or AnnDataCurator for validation
Standardize values with .cat.standardize()
Map to ontologies with .cat.add_ontology()
Save curated artifacts with schema linkage
Query validated datasets by features

Reference: references/annotation-validation.md - Read this for detailed curation workflows, schema design patterns, handling validation errors, and best practices.

4. Biological Ontologies

Available ontologies (via Bionty):

Genes (Ensembl), Proteins (UniProt)
Cell types (CL), Cell lines (CLO)
Tissues (Uberon), Diseases (Mondo, DOID)
Phenotypes (HPO), Pathways (GO)
Experimental factors (EFO), Developmental stages
Organisms (NCBItaxon), Drugs (DrugBank)

Key workflows:

Import public ontologies with bt.CellType.import_source()
Search ontologies with keyword or exact matching
Standardize terms using synonym mapping
Explore hierarchical relationships (parents, children, ancestors)
Validate data against ontology terms
Annotate datasets with ontology records
Create custom terms and hierarchies
Handle multi-organism contexts (human, mouse, etc.)

Reference: references/ontologies.md - Read this for comprehensive ontology operations, standardization strategies, hierarchy navigation, and annotation workflows.

5. Integrations

Workflow managers:

Nextflow: Track pipeline processes and outputs
Snakemake: Integrate into Snakemake rules
Redun: Combine with Redun task tracking

MLOps platforms:

Weights & Biases: Link experiments with data artifacts
MLflow: Track models and experiments
HuggingFace: Track model fine-tuning
scVI-tools: Single-cell analysis workflows

Storage systems:

Local filesystem, AWS S3, Google Cloud Storage
S3-compatible (MinIO, Cloudflare R2)
HTTP/HTTPS endpoints (read-only)
HuggingFace datasets

Array stores:

TileDB-SOMA (with cellxgene support)
DuckDB for SQL queries on Parquet files

Visualization:

Vitessce for interactive spatial/single-cell visualization

Version control:

Git integration for source code tracking

Reference: references/integrations.md - Read this for integration patterns, code examples, and troubleshooting for third-party systems.

6. Setup and Deployment

Installation:

Basic: uv pip install lamindb
With extras: uv pip install 'lamindb[gcp,zarr,fcs]'
Modules: bionty, wetlab, clinical

Instance types:

Local SQLite (development)
Cloud storage + SQLite (small teams)
Cloud storage + PostgreSQL (production)

Storage options:

Local filesystem
AWS S3 with configurable regions and permissions
Google Cloud Storage
S3-compatible endpoints (MinIO, Cloudflare R2)

Configuration:

Cache management for cloud files
Multi-user system configurations
Git repository sync
Environment variables

Deployment patterns:

Local dev → Cloud production migration
Multi-region deployments
Shared storage with personal instances

Reference: references/setup-deployment.md - Read this for detailed installation, configuration, storage setup, database management, security best practices, and troubleshooting.

Common Use Case Workflows

Use Case 1: Single-Cell RNA-seq Analysis with Ontology Validation

import lamindb as ln
import bionty as bt
import anndata as ad

# Start tracking
ln.track(params={"analysis": "scRNA-seq QC and annotation"})

# Import cell type ontology
bt.CellType.import_source()

# Load data
adata = ad.read_h5ad("raw_counts.h5ad")

# Validate and standardize cell types
adata.obs["cell_type"] = bt.CellType.standardize(adata.obs["cell_type"])

# Curate with schema
curator = ln.curators.AnnDataCurator(adata, schema)
curator.validate()
artifact = curator.save_artifact(key="scrna/validated.h5ad")

# Link ontology annotations
cell_types = bt.CellType.from_values(adata.obs.cell_type)
artifact.feature_sets.add_ontology(cell_types)

ln.finish()

Use Case 2: Building a Queryable Data Lakehouse

import lamindb as ln

# Register multiple experiments
for i, file in enumerate(data_files):
    artifact = ln.Artifact.from_anndata(
        ad.read_h5ad(file),
        key=f"scrna/batch_{i}.h5ad",
        description=f"scRNA-seq batch {i}"
    ).save()

    # Annotate with features
    artifact.features.add_values({
        "batch": i,
        "tissue": tissues[i],
        "condition": conditions[i]
    })

# Query across all experiments
immune_datasets = ln.Artifact.filter(
    key__startswith="scrna/",
    tissue="PBMC",
    condition="treated"
).to_dataframe()

# Load specific datasets
for artifact in immune_datasets:
    adata = artifact.load()
    # Analyze

Use Case 3: ML Pipeline with W&B Integration

import lamindb as ln
import wandb

# Initialize both systems
wandb.init(project="drug-response", name="exp-42")
ln.track(params={"model": "random_forest", "n_estimators": 100})

# Load training data from LaminDB
train_artifact = ln.Artifact.get(key="datasets/train.parquet")
train_data = train_artifact.load()

# Train model
model = train_model(train_data)

# Log to W&B
wandb.log({"accuracy": 0.95})

# Save model in LaminDB with W&B linkage
import joblib
joblib.dump(model, "model.pkl")
model_artifact = ln.Artifact("model.pkl", key="models/exp-42.pkl").save()
model_artifact.features.add_values({"wandb_run_id": wandb.run.id})

ln.finish()
wandb.finish()

Use Case 4: Nextflow Pipeline Integration

# In Nextflow process script
import lamindb as ln

ln.track()

# Load input artifact
input_artifact = ln.Artifact.get(key="raw/batch_${batch_id}.fastq.gz")
input_path = input_artifact.cache()

# Process (alignment, quantification, etc.)
# ... Nextflow process logic ...

# Save output
output_artifact = ln.Artifact(
    "counts.csv",
    key="processed/batch_${batch_id}_counts.csv"
).save()

ln.finish()

Getting Started Checklist

To start using LaminDB effectively:

Installation & Setup (references/setup-deployment.md)
- Install LaminDB and required extras
- Authenticate with lamin login
- Initialize instance with lamin init --storage ...
Learn Core Concepts (references/core-concepts.md)
- Understand Artifacts, Records, Runs, Transforms
- Practice creating and retrieving artifacts
- Implement ln.track() and ln.finish() in workflows
Master Querying (references/data-management.md)

Key Principles

Follow these principles when working with LaminDB:

Track everything : Use ln.track() at the start of every analysis for automatic lineage capture
Validate early : Define schemas and validate data before extensive analysis
Use ontologies : Leverage public biological ontologies for standardized annotations
Organize with keys : Structure artifact keys hierarchically (e.g., project/experiment/batch/file.h5ad)
Query metadata first : Filter and search before loading large files
Version, don't duplicate : Use built-in versioning instead of creating new keys for modifications
Annotate with features : Define typed features for queryable metadata
Document thoroughly : Add descriptions to artifacts, schemas, and transforms
Leverage lineage : Use view_lineage() to understand data provenance
Start local, scale cloud : Develop locally with SQLite, deploy to cloud with PostgreSQL

Reference Files

This skill includes comprehensive reference documentation organized by capability:

references/core-concepts.md - Artifacts, records, runs, transforms, features, versioning, lineage
references/data-management.md - Querying, filtering, searching, streaming, organizing data
references/annotation-validation.md - Schema design, curation workflows, validation strategies
references/ontologies.md - Biological ontology management, standardization, hierarchies
references/integrations.md - Workflow managers, MLOps platforms, storage systems, tools
references/setup-deployment.md - Installation, configuration, deployment, troubleshooting

Read the relevant reference file(s) based on the specific LaminDB capability needed for the task at hand.

Additional Resources

Official Documentation : https://docs.lamin.ai
API Reference : https://docs.lamin.ai/api
GitHub Repository : https://github.com/laminlabs/lamindb
Tutorial : https://docs.lamin.ai/tutorial
FAQ : https://docs.lamin.ai/faq

Weekly Installs

122

Repository

davila7/claude-…emplates

GitHub Stars

22.6K

First Seen

Jan 21, 2026

Security Audits

Gen Agent Trust HubPass SocketPass SnykWarn

Installed on

claude-code104

opencode98

gemini-cli93

cursor93

antigravity84

codex82

免费AI数据抓取智能体：自动化收集、丰富与存储网站/API数据

1,100 周安装

Practice filtering and searching registries
Learn feature-based queries
Experiment with streaming large files

Set Up Validation (references/annotation-validation.md)

Define features relevant to research domain
Create schemas for data types
Practice curation workflows

Integrate Ontologies (references/ontologies.md)

Import relevant biological ontologies (genes, cell types, etc.)
Validate existing annotations
Standardize metadata with ontology terms

Connect Tools (references/integrations.md)

Integrate with existing workflow managers
Link ML platforms for experiment tracking
Configure cloud storage and compute

LaminDB：面向生物学的开源数据框架，实现FAIR原则与数据谱系追踪

🇨🇳中文介绍

LaminDB

概述

何时使用此技能

核心能力

1. 核心概念与数据谱系

相关 Skills

2. 数据管理与查询

3. 标注与验证

4. 生物本体论

5. 集成

6. 设置与部署

常见用例工作流

用例 1：使用本体论验证的单细胞 RNA-seq 分析

用例 2：构建可查询的数据湖仓

用例 3：与 W&B 集成的 ML 管道

用例 4：Nextflow 管道集成

入门清单

关键原则

参考文件

其他资源

🇺🇸English

LaminDB

Overview

When to Use This Skill

Core Capabilities

1. Core Concepts and Data Lineage

2. Data Management and Querying

3. Annotation and Validation

4. Biological Ontologies

5. Integrations

6. Setup and Deployment

Common Use Case Workflows

Use Case 1: Single-Cell RNA-seq Analysis with Ontology Validation

Use Case 2: Building a Queryable Data Lakehouse

Use Case 3: ML Pipeline with W&B Integration

Use Case 4: Nextflow Pipeline Integration

Getting Started Checklist

Key Principles

Reference Files

Additional Resources

最新 Skills