npx skills add https://github.com/bsene/skills --skill gitlab-dag使用此技能来设计、生成、审查和修复利用 DAG (needs:) 和并行处理 (parallel:, parallel:matrix) 的 GitLab CI/CD 流水线配置。
不使用 DAG:作业严格按照阶段顺序执行。阶段 N 中的所有作业必须在阶段 N+1 开始前完成——即使该阶段中只有一个作业是真正的先决条件。
使用 DAG (needs:):作业在其特定依赖项完成后立即开始,跳过了阶段顺序的等待。这可以显著减少流水线总时间。
# 标准(慢):test 等待 ALL build 作业完成
stages: [build, test, deploy]
build-frontend:
stage: build
build-backend:
stage: build # 与 build-frontend 并行运行
test-frontend:
stage: test # 等待 BOTH build 作业,即使它只需要 build-frontend
# DAG(快):test-frontend 在 build-frontend 完成后立即开始
test-frontend:
stage: test
needs: [build-frontend] # 完全忽略 build-backend 的状态
needs: 关键字参考Use this skill to design, generate, review, and fix GitLab CI/CD pipeline configurations that leverage DAG (needs:) and parallelism (parallel:, parallel:matrix).
Without DAG : Jobs execute strictly stage-by-stage. All jobs in stage N must complete before stage N+1 starts — even if only one job in a stage is the actual prerequisite.
With DAG (needs:): Jobs start as soon as their specific dependencies finish, skipping the stage-order wait. This can dramatically reduce total pipeline time.
# Standard (slow): test waits for ALL build jobs to finish
stages: [build, test, deploy]
build-frontend:
stage: build
build-backend:
stage: build # runs in parallel with build-frontend
test-frontend:
stage: test # waits for BOTH build jobs even though it only needs build-frontend
# DAG (fast): test-frontend starts the moment build-frontend finishes
test-frontend:
stage: test
needs: [build-frontend] # ignores build-backend's status entirely
广告位招租
在这里展示您的产品或服务
触达数万 AI 开发者,精准高效
job-b:
stage: deploy
needs:
- job: job-a # 必须指定作业名称
- job: job-c
artifacts: false # 可选:不从 job-c 下载产物
needs: 可以引用任何阶段中的作业,包括同一阶段甚至更晚的阶段(使用 needs: [] 表示立即开始,无依赖)needs: 条目数:50(可由管理员配置,默认为 50)needs: 为空 (needs: []),作业在流水线开始时立即启动needs: 的作业默认跳过从非列出作业下载产物needs: 与 stage: 结合使用以保持组织清晰——阶段仍有助于 UI 显示lint:
stage: test
needs: [] # 流水线一开始就运行,无论其他作业如何
script: ./run-lint.sh
parallel: — 简单的测试分片将单个作业拆分为 N 个并发运行的克隆。GitLab 会自动注入 $CI_NODE_INDEX(从 1 开始)和 $CI_NODE_TOTAL。
test:
stage: test
parallel: 5 # 创建 test 1/5, test 2/5, ... test 5/5
script:
- bundle exec rspec --format progress \
$(ruby -e "
files = Dir['spec/**/*_spec.rb']
chunk = files.each_slice((files.size.to_f / $CI_NODE_TOTAL).ceil).to_a
puts chunk[$CI_NODE_INDEX - 1].join(' ')
")
最大值:200 个并行实例。
parallel:matrix — 多维作业在一组变量组合上运行相同的作业配置。每个组合成为一个单独的作业,命名为 job-name: [VAL1, VAL2]。
build:
stage: build
image: python:$VERSION
script: ./build.sh
parallel:
matrix:
- VERSION: ["3.10", "3.11", "3.12"]
PLATFORM: [linux, macos]
这将创建 6 个作业:每个 VERSION × PLATFORM 的排列组合。
限制:矩阵生成的总作业数最大为 200。请相应规划你的维度——在添加值之前检查计算。
needs: + parallel:matrix — 带矩阵的 DAGdeploy:
stage: deploy
needs:
- job: build
parallel:
matrix:
- VERSION: ["3.10", "3.11", "3.12"]
PLATFORM: [linux, macos]
Deploy 等待所有 6 个 build 矩阵作业。
deploy-linux-310:
stage: deploy
needs:
- job: build
parallel:
matrix:
- VERSION: "3.10"
PLATFORM: linux
使用 $[[ matrix.IDENTIFIER ]] 创建自动的 1:1 依赖关系——每个测试实例只等待其匹配的构建实例:
build:
stage: build
parallel:
matrix:
- OS: [ubuntu, alpine]
ARCH: [amd64, arm64]
script: ./build.sh $OS $ARCH
test:
stage: test
parallel:
matrix:
- OS: [ubuntu, alpine]
ARCH: [amd64, arm64]
needs:
- job: build
parallel:
matrix:
- OS: ["$[[ matrix.OS ]]"]
ARCH: ["$[[ matrix.ARCH ]]"]
script: ./test.sh $OS $ARCH
test: [ubuntu, amd64] 只等待 build: [ubuntu, amd64]——而不是其他 3 个构建作业。这是大型矩阵流水线最强大的模式。
!reference 的 DRY 模式避免在多个作业中重复相同的 parallel:matrix 块。
.matrix: &matrix
parallel:
matrix:
- OS: [ubuntu, alpine]
ARCH: [amd64, arm64]
build:
stage: build
<<: *matrix
script: ./build.sh
test:
stage: test
<<: *matrix
needs:
- job: build
parallel:
matrix:
- OS: ["$[[ matrix.OS ]]"]
ARCH: ["$[[ matrix.ARCH ]]"]
script: ./test.sh
⚠️
extends:合并哈希但不合并数组——对于script:块或数组覆盖,建议使用 YAML 锚点或!reference。
stages:
- lint
- build
- test
- deploy
# 立即开始——无依赖
lint:
stage: lint
needs: []
script: ./run-linters.sh
# 矩阵构建:6 个作业 (3 个版本 × 2 个平台)
build:
stage: build
needs: [lint]
parallel:
matrix:
- LANG_VERSION: ["3.10", "3.11", "3.12"]
PLATFORM: [linux, windows]
script: ./build.sh $LANG_VERSION $PLATFORM
artifacts:
paths: [dist/]
# 通过矩阵表达式实现 1:1 的测试-构建对应
unit-test:
stage: test
parallel:
matrix:
- LANG_VERSION: ["3.10", "3.11", "3.12"]
PLATFORM: [linux, windows]
needs:
- job: build
parallel:
matrix:
- LANG_VERSION: ["$[[ matrix.LANG_VERSION ]]"]
PLATFORM: ["$[[ matrix.PLATFORM ]]"]
script: ./test.sh $LANG_VERSION $PLATFORM
# 集成测试的并行分片(独立)
integration-test:
stage: test
needs: [] # 立即开始,不等待构建
parallel: 4
script: ./integration-test.sh
# 仅在所有单元测试和集成测试通过后部署
deploy:
stage: deploy
needs:
- job: unit-test
parallel:
matrix:
- LANG_VERSION: ["3.10", "3.11", "3.12"]
PLATFORM: [linux, windows]
- job: integration-test
script: ./deploy.sh
| 问题 | 原因 | 修复方法 |
|---|---|---|
| 作业仍在等待不相关的作业 | 未使用 needs: | 添加 needs: 仅列出直接先决条件 |
parallel:matrix 生成过多作业 | 笛卡尔积超过 200 | 减少维度或拆分为单独的作业组 |
needs: 中断产物下载 | 跳过非列出作业的产物 | 为需要的产物显式添加 artifacts: true |
矩阵表达式 $[[ matrix.X ]] 不工作 | GitLab 版本 < 17 或标识符拼写错误 | 检查 GitLab 版本;标识符区分大小写 |
extends: 静默丢弃数组值 | extends 合并哈希,不合并数组 | 对数组使用 YAML 锚点 (&anchor / <<: *anchor) |
| 作业启动过早 | 无意中使用了 needs: [] | 如果作业应等待某个阶段,则移除 needs: [] |
需要并行化吗?
├── 相同作业,拆分工作负载 → parallel: N (使用 $CI_NODE_INDEX/$CI_NODE_TOTAL)
├── 相同作业,不同环境/版本 → parallel:matrix
└── 不同作业 → 定义单独的作业,用 needs: 链接
需要跨矩阵依赖吗?
├── 一个作业需要 ALL 矩阵实例 → needs: 带完整矩阵列表
├── 一个作业需要 ONE 特定实例 → needs: 带精确矩阵值
└── N:1 相同形状映射 → 使用 $[[ matrix.IDENTIFIER ]] 表达式 (GitLab 17+)
需要减少流水线时间吗?
├── 没有真实依赖的作业 → needs: [] (立即运行)
├── 长的测试套件 → parallel: N 分片
└── 多环境构建 + 测试 → 矩阵 + 矩阵表达式实现 1:1 DAG
编写或审查 .gitlab-ci.yml 片段时:
stages: 声明以保持清晰needs: 关系$[[ matrix.X ]] 表达式,而非手动列出所有组合artifacts: 块)parallel:matrix 时列出总作业数(必须 ≤ 200)更多详情,请在处理复杂的嵌套 DAG、带有子流水线的 trigger: 或跨流水线依赖 needs:pipeline 时阅读 references/gitlab-dag-advanced.md。
每周安装数
1
代码仓库
首次出现
1 天前
安全审计
安装于
amp1
cline1
opencode1
cursor1
kimi-cli1
codex1
needs: Keyword Referencejob-b:
stage: deploy
needs:
- job: job-a # must specify job name
- job: job-c
artifacts: false # optional: don't download artifacts from job-c
needs: can reference jobs in any stage , including the same stage or even a later stage (use needs: [] to start immediately with no dependencies)needs: entries per job: 50 (configurable by admins, default 50)needs: is empty (needs: []), the job starts immediately when the pipeline beginsneeds: skips artifact download from non-listed jobs by defaultneeds: with stage: for organizational clarity — stages still help the UI displaylint:
stage: test
needs: [] # runs as soon as the pipeline starts, regardless of other jobs
script: ./run-lint.sh
parallel: — Simple Test ShardingSplit a single job into N clones running concurrently. GitLab injects $CI_NODE_INDEX (1-based) and $CI_NODE_TOTAL automatically.
test:
stage: test
parallel: 5 # creates test 1/5, test 2/5, ... test 5/5
script:
- bundle exec rspec --format progress \
$(ruby -e "
files = Dir['spec/**/*_spec.rb']
chunk = files.each_slice((files.size.to_f / $CI_NODE_TOTAL).ceil).to_a
puts chunk[$CI_NODE_INDEX - 1].join(' ')
")
Max value: 200 parallel instances.
parallel:matrix — Multi-Dimensional JobsRun the same job configuration across a set of variable combinations. Each combination becomes a separate job named job-name: [VAL1, VAL2].
build:
stage: build
image: python:$VERSION
script: ./build.sh
parallel:
matrix:
- VERSION: ["3.10", "3.11", "3.12"]
PLATFORM: [linux, macos]
This creates 6 jobs: every VERSION × PLATFORM permutation.
Limit : max 200 jobs total from a matrix. Plan your dimensions accordingly — check the math before adding values.
needs: + parallel:matrix — DAG with Matrixdeploy:
stage: deploy
needs:
- job: build
parallel:
matrix:
- VERSION: ["3.10", "3.11", "3.12"]
PLATFORM: [linux, macos]
Deploy waits for all 6 build matrix jobs.
deploy-linux-310:
stage: deploy
needs:
- job: build
parallel:
matrix:
- VERSION: "3.10"
PLATFORM: linux
Use $[[ matrix.IDENTIFIER ]] to create automatic 1:1 dependencies — each test instance only waits for its matching build instance:
build:
stage: build
parallel:
matrix:
- OS: [ubuntu, alpine]
ARCH: [amd64, arm64]
script: ./build.sh $OS $ARCH
test:
stage: test
parallel:
matrix:
- OS: [ubuntu, alpine]
ARCH: [amd64, arm64]
needs:
- job: build
parallel:
matrix:
- OS: ["$[[ matrix.OS ]]"]
ARCH: ["$[[ matrix.ARCH ]]"]
script: ./test.sh $OS $ARCH
test: [ubuntu, amd64] only waits for build: [ubuntu, amd64] — not the other 3 build jobs. This is the most powerful pattern for large matrix pipelines.
!referenceAvoid repeating the same parallel:matrix block across multiple jobs.
.matrix: &matrix
parallel:
matrix:
- OS: [ubuntu, alpine]
ARCH: [amd64, arm64]
build:
stage: build
<<: *matrix
script: ./build.sh
test:
stage: test
<<: *matrix
needs:
- job: build
parallel:
matrix:
- OS: ["$[[ matrix.OS ]]"]
ARCH: ["$[[ matrix.ARCH ]]"]
script: ./test.sh
⚠️
extends:merges hashes but not arrays — forscript:blocks or array overrides, prefer YAML anchors or!reference.
stages:
- lint
- build
- test
- deploy
# Starts immediately — no dependencies
lint:
stage: lint
needs: []
script: ./run-linters.sh
# Matrix build: 6 jobs (3 versions × 2 platforms)
build:
stage: build
needs: [lint]
parallel:
matrix:
- LANG_VERSION: ["3.10", "3.11", "3.12"]
PLATFORM: [linux, windows]
script: ./build.sh $LANG_VERSION $PLATFORM
artifacts:
paths: [dist/]
# 1:1 test-per-build via matrix expressions
unit-test:
stage: test
parallel:
matrix:
- LANG_VERSION: ["3.10", "3.11", "3.12"]
PLATFORM: [linux, windows]
needs:
- job: build
parallel:
matrix:
- LANG_VERSION: ["$[[ matrix.LANG_VERSION ]]"]
PLATFORM: ["$[[ matrix.PLATFORM ]]"]
script: ./test.sh $LANG_VERSION $PLATFORM
# Parallel sharding of integration tests (independent)
integration-test:
stage: test
needs: [] # starts immediately, doesn't wait for build
parallel: 4
script: ./integration-test.sh
# Deploy only after all unit-tests and integration-tests pass
deploy:
stage: deploy
needs:
- job: unit-test
parallel:
matrix:
- LANG_VERSION: ["3.10", "3.11", "3.12"]
PLATFORM: [linux, windows]
- job: integration-test
script: ./deploy.sh
| Problem | Cause | Fix |
|---|---|---|
| Job still waits for unrelated jobs | Not using needs: | Add needs: listing only direct prerequisites |
parallel:matrix generates too many jobs | Cartesian product exceeds 200 | Reduce dimensions or split into separate job groups |
needs: breaks artifact download | Non-listed jobs' artifacts are skipped | Add artifacts: true explicitly for needed artifacts |
Matrix expression $[[ matrix.X ]] not working | GitLab version < 17 or typo in identifier | Check GitLab version; identifiers are case-sensitive |
extends: silently drops array values | extends merges hashes, not arrays | Use YAML anchors (&anchor / <<: *anchor) for arrays |
| Job starts too early | needs: [] used unintentionally | Remove needs: [] if the job should wait for a stage |
Need to parallelize?
├── Same job, split workload → parallel: N (use $CI_NODE_INDEX/$CI_NODE_TOTAL)
├── Same job, different envs/versions → parallel:matrix
└── Different jobs → define separate jobs, link with needs:
Need cross-matrix dependencies?
├── One job needs ALL matrix instances → needs: with full matrix list
├── One job needs ONE specific instance → needs: with exact matrix values
└── N:1 same-shape mapping → use $[[ matrix.IDENTIFIER ]] expressions (GitLab 17+)
Need to reduce pipeline time?
├── Jobs without real dependencies → needs: [] (run immediately)
├── Long test suite → parallel: N sharding
└── Multi-env builds + tests → matrix + matrix expressions for 1:1 DAG
When writing or reviewing .gitlab-ci.yml snippets:
stages: declaration for clarityneeds: relationships$[[ matrix.X ]] expressions over manually listing all combinationsartifacts: block) when jobs produce outputs consumed by dependentsparallel:matrix (must be ≤ 200)For more details, read references/gitlab-dag-advanced.md when handling complex nested DAGs, trigger: with child pipelines, or needs:pipeline cross-pipeline dependencies.
Weekly Installs
1
Repository
First Seen
1 day ago
Security Audits
Installed on
amp1
cline1
opencode1
cursor1
kimi-cli1
codex1
Azure Data Explorer (Kusto) 查询技能:KQL数据分析、日志遥测与时间序列处理
114,200 周安装