CI/CD测试故障排查指南：如何复现和修复CI中的不稳定测试失败

triage-ci-flake by payloadcms/payload

114 周安装量

41,400 GitHub Stars

GitHub

安装命令

npx skills add https://github.com/payloadcms/payload --skill triage-ci-flake

自动化开发运维测试

🇨🇳中文介绍

排查 CI 故障

概述

用于在 CI 中排查和修复测试失败的系统化工作流程，特别是那些在本地通过但在 CI 中失败的“不稳定”测试。那些能进入 main 分支的测试通常由于时序、打包或环境差异而变得不稳定。

关键规则：在提出任何修复方案之前，你必须运行复现工作流程。没有例外。

何时使用

PR 合并后，main 分支上的 CI 测试失败
测试在本地通过但在 CI 中失败
测试失败被标记为“不稳定”或间歇性失败
仅在 CI 中端到端或集成测试超时

强制性的第一步

你必须执行这些命令。阅读代码或分析日志不算作复现。

提取：从 CI 日志中提取套件名称、测试名称和错误信息
执行：终止端口 3000 以避免冲突
执行：pnpm dev $SUITE_NAME（使用 run_in_background=true）
执行：等待服务器准备就绪（使用 curl 检查或等待）
执行：直接使用 Playwright 运行特定的失败测试（npx playwright test test/TEST_SUITE_NAME/e2e.spec.ts:31:3 --headed -g "TEST_DESCRIPTION_TARGET_GOES_HERE"）
如果测试通过，执行：pnpm prepare-run-test-against-prod

广告位招租

在这里展示您的产品或服务

触达数万 AI 开发者，精准高效

联系我们

1. 提取 CI 详细信息

从 CI 日志或 GitHub Actions URL 中识别：

套件名称：目录名称（例如 i18n、fields、lexical）
测试文件：完整路径（例如 test/i18n/e2e.spec.ts）
测试名称：确切的测试描述
错误信息：完整的堆栈跟踪
测试类型：端到端测试（Playwright）或集成测试（Vitest）

2. 使用开发代码复现

关键：始终按名称运行特定测试，而不是整个套件。

服务器管理规则：

在启动新服务器之前，始终终止所有服务器
永远不要假设端口是空闲的
在运行测试之前，始终等待服务器就绪确认

# ========================================

# 步骤 2A：停止所有服务器
# ========================================
lsof -ti:3000 | xargs kill -9 2>/dev/null || echo "Port 3000 clear"

# ========================================
# 步骤 2B：启动开发服务器
# ========================================
# 使用套件启动开发服务器（在后台使用 run_in_background=true）
pnpm dev $SUITE_NAME

# ========================================
# 步骤 2C：等待服务器就绪
# ========================================
# 等待服务器准备就绪（必需 - 不要跳过）
until curl -s http://localhost:3000/admin > /dev/null 2>&1; do sleep 1; done && echo "Server ready"

# ========================================
# 步骤 2D：运行特定测试
# ========================================
# 直接使用 Playwright 运行特定的失败测试
# 对于端到端测试（不要使用 pnpm test:e2e，因为它会启动自己的服务器）：
pnpm exec playwright test test/$SUITE_NAME/e2e.spec.ts -g "exact test name"

# 对于集成测试：
pnpm test:int $SUITE_NAME -t "exact test name"

测试失败了吗？

✅ 是：你复现了它！继续使用开发代码进行调试。
❌ 否：继续步骤 3（打包代码测试）。

3. 使用打包代码复现

如果测试在开发代码下通过，问题很可能出在打包/生产代码中。

重要：在启动生产服务器之前，你必须停止开发服务器。

# ========================================
# 步骤 3A：停止所有服务器（包括步骤 2 中的开发服务器）
# ========================================
lsof -ti:3000 | xargs kill -9 2>/dev/null || echo "Port 3000 clear"

# ========================================
# 步骤 3B：为生产环境构建和打包
# ========================================
# 构建所有包并打包它们（这需要时间 - 请耐心等待）
pnpm prepare-run-test-against-prod

# ========================================
# 步骤 3C：启动生产服务器
# ========================================
# 启动生产开发服务器（在后台使用 run_in_background=true）
pnpm dev:prod $SUITE_NAME

# ========================================
# 步骤 3D：等待服务器就绪
# ========================================
# 等待服务器准备就绪（必需 - 不要跳过）
until curl -s http://localhost:3000/admin > /dev/null 2>&1; do sleep 1; done && echo "Server ready"

# ========================================
# 步骤 3E：运行特定测试
# ========================================
# 再次直接使用 Playwright 运行特定测试
pnpm exec playwright test test/$SUITE_NAME/e2e.spec.ts -g "exact test name"
# 或者对于集成测试：
pnpm test:int $SUITE_NAME -t "exact test name"

现在测试失败了吗？

✅ 是：打包或生产构建问题。查找：
- package.json 中缺少导出
- 构建配置问题
- 打包后行为不同的代码
❌ 否：无法在本地复现。继续步骤 4。

如果在两次尝试后仍无法在本地复现：

更仔细地审查 CI 日志以查找环境差异
检查竞态条件（多次运行测试：for i in {1..10}; do pnpm test:e2e...; done）
查找 CI 特定的限制（内存、CPU、时序）
考虑是否是高度依赖时序的真正竞态条件

常见的不稳定测试模式

页面在断言运行时进行导航
网络请求在断言之前未完成
状态更新未完成

使用 Playwright 的 Web 优先断言（toBeVisible()、toHaveText()）
等待特定条件，而不是任意的超时
使用带有条件检查的 waitForFunction()

测试在数据库中留下数据
测试之间的共享状态
afterEach 中缺少清理

跟踪创建的 ID 并在 afterEach 中清理
使用隔离的测试数据
不要使用会影响其他测试的 deleteAll

使用 setTimeout/sleep 而不是基于条件的等待
未等待页面稳定
动画/过渡未完成

使用 waitForPageStability() 辅助函数
等待特定的 DOM 状态
使用 Playwright 内置的等待机制

代码检查注意事项

修复端到端测试时，请注意这些 eslint 规则：

playwright/no-networkidle - 避免使用 waitForLoadState('networkidle')（改用基于条件的等待）
payload/no-wait-function - 避免使用自定义的 wait() 函数（使用 Playwright 内置的等待）
payload/no-flaky-assertions - 避免使用不可重试的断言
playwright/prefer-web-first-assertions - 使用内置的 Playwright 断言

现有代码可能违反这些规则 - 在添加新代码时，即使现有代码不遵守，也要遵循这些规则。

# 确保开发服务器在端口 3000 上运行
# 多次运行测试以确认稳定性
for i in {1..10}; do
  pnpm exec playwright test test/$SUITE_NAME/e2e.spec.ts -g "exact test name" || break
done

# 运行完整套件
pnpm exec playwright test test/$SUITE_NAME/e2e.spec.ts

# 如果你修改了打包代码，使用生产构建进行测试
lsof -ti:3000 | xargs kill -9 2>/dev/null
pnpm prepare-run-test-against-prod
pnpm dev:prod $SUITE_NAME
until curl -s http://localhost:3000/admin > /dev/null; do sleep 1; done
pnpm exec playwright test test/$SUITE_NAME/e2e.spec.ts

没有复现，就没有修复

如果你在完成工作流程的步骤 1-3 之前就提出修复方案，你就违反了这项技能。

即使出现以下情况，这也适用：

从日志看修复方案似乎很明显
你以前见过这个错误
团队有来自时间的压力
你对根本原因很有信心
日志显示了清晰的堆栈跟踪

没有例外。首先运行复现工作流程。

每一个跳过复现的借口，以及为什么它是错误的：

合理化理由	现实情况
“日志显示了确切的错误”	日志显示的是症状，而不是根本原因。进行复现。
“我能在代码中看到问题”	你是在猜测。进行复现以确认。
“这显然是一个竞态条件”	也许吧。进行复现以确定。
“我以前见过这个错误”	这次可能不同。进行复现。
“堆栈跟踪很清晰”	堆栈跟踪显示的是位置，而不是原因。进行复现。
“时间紧迫 - 需要快速修复”	复现本身就是快速的。猜测会浪费时间。
“测试文件显示了问题”	阅读 ≠ 运行。执行命令。
“我先分析一下代码”	代码分析应在复现之后进行。
“让我调查一下根本原因”	复现本身就是调查。
“我需要理解这个错误”	理解来自于看到它失败。

危险信号 - 停止

这些想法意味着你即将违反这项技能：

“让我直接修复这个明显的问题” → 停止。首先执行复现。
“我增加一下超时时间” → 停止。通过复现找到根本原因。
“从日志看这像是 X” → 停止。运行测试。
“我运行一下整个套件” → 停止。首先运行特定测试。
“让我检查一下代码” → 停止。首先运行复现。
“我能在堆栈跟踪中看到问题” → 停止。进行复现以确认。
“日志足够清晰了” → 停止。执行命令。
“让我分析一下...” → 停止。分析应在复现之后进行。

所有这些都意味着：立即停止。执行复现命令。没有执行就没有分析。

正确的方法 - 首先执行命令：

User: CI failed on i18n suite - https://github.com/.../actions/runs/XXX

Agent:
1. Extracted details from CI logs:
   - Suite: i18n
   - Test: "ensure translations update correctly when switching language"
   - File: test/i18n/e2e.spec.ts:124
   - Error: "Error loading language: en" TypeError: network error

2. Executing reproduction workflow:
   [Uses Bash tool to kill ports]
   $ lsof -ti:3000 | xargs kill -9 2>/dev/null || echo "Port 3000 clear"

   [Uses Bash tool with run_in_background=true]
   $ pnpm dev i18n

   [Waits for server to start with curl check]
   $ until curl -s http://localhost:3000/admin > /dev/null; do sleep 1; done && echo "Server ready"

   [Uses Bash tool to run Playwright directly]
   $ pnpm exec playwright test test/i18n/e2e.spec.ts -g "ensure translations update correctly when switching language"

   Output shows: Test PASSED locally

3. Test passed with dev code, trying bundled code:
   [Kills ports again]
   $ lsof -ti:3000 | xargs kill -9 2>/dev/null

   [Uses Bash tool]
   $ pnpm prepare-run-test-against-prod

   [Uses Bash tool with run_in_background=true]
   $ pnpm dev:prod i18n

   [Waits for server]
   $ until curl -s http://localhost:3000/admin > /dev/null; do sleep 1; done

   [Uses Bash tool]
   $ pnpm exec playwright test test/i18n/e2e.spec.ts -g "ensure translations update correctly when switching language"

   Output shows: Test FAILED - reproduced!

4. Now that I've reproduced it, analyzing the failure...
   [Code analysis, root cause identification]

5. Implementing fix: [specific changes]

6. Verification:
   [Uses Bash tool]
   $ for i in {1..10}; do pnpm test:e2e i18n -g "exact test"; done
   All 10 runs passed!

错误的方法 - 不执行就进行分析：

User: CI failed on i18n suite

Agent: Looking at the error logs, I can see this is a race condition...
[Analyzes code, proposes fix without running any commands]

❌ WRONG - This violates the skill. Must execute reproduction commands first.

错误	修复方法
首先运行完整测试套件	按名称运行特定测试
跳过开发代码复现	始终先尝试开发代码
不使用打包代码测试	如果开发代码通过，使用 `prepare-run-test-against-prod` 测试
未复现就提出修复方案	遵循工作流程 - 先复现
在新代码中使用 `networkidle`	使用带有 `waitForFunction()` 的基于条件的等待
添加任意的 `wait()` 调用	使用 Playwright 内置的断言和等待

修复前先复现：在未复现问题之前，绝不提出修复方案
针对性测试：运行确切的失败测试，而不是整个套件
先开发，后生产：在检查打包代码之前先检查开发代码
遵循工作流程：没有捷径 - 这些步骤的存在是为了节省时间
验证稳定性：多次运行测试以确认修复

在你完成以下步骤之后：

✅ 复现了问题
✅ 实施了修复
✅ 验证了修复在本地通过（多次运行）
✅ 使用生产构建进行了测试（如果适用）

你必须提示用户创建 PR：

The fix has been verified and is ready for review. Would you like me to create a PR with these changes?

Summary of changes:
- [List files modified]
- [Brief description of the fix]
- [Verification results]

不要自动创建 PR - 始终先询问用户
提供清晰的更改内容和原因摘要
包含验证结果（测试运行次数、通过率）
让用户决定是立即创建 PR 还是先进行其他更改

这确保了用户对提交审查的内容具有可见性和控制权。

🇺🇸English

Triage CI Failure

Overview

Systematic workflow for triaging and fixing test failures in CI, especially flaky tests that pass locally but fail in CI. Tests that made it to main are usually flaky due to timing, bundling, or environment differences.

CRITICAL RULE: You MUST run the reproduction workflow before proposing any fixes. No exceptions.

When to Use

CI test fails on main branch after PR was merged
Test passes locally but fails in CI
Test failure labeled as "flaky" or intermittent
E2E or integration test timing out in CI only

MANDATORY First Steps

YOU MUST EXECUTE THESE COMMANDS. Reading code or analyzing logs does NOT count as reproduction.

Extract suite name, test name, and error from CI logs
EXECUTE : Kill port 3000 to avoid conflicts
EXECUTE : pnpm dev $SUITE_NAME (use run_in_background=true)
EXECUTE : Wait for server to be ready (check with curl or sleep)
EXECUTE : Run the specific failing test with Playwright directly (npx playwright test test/TEST_SUITE_NAME/e2e.spec.ts:31:3 --headed -g "TEST_DESCRIPTION_TARGET_GOES_HERE")
If test passes , EXECUTE : pnpm prepare-run-test-against-prod
EXECUTE : pnpm dev:prod $SUITE_NAME and run test again

Only after EXECUTING these commands and seeing their output can you proceed to analysis and fixes.

"Analysis from logs" is NOT reproduction. You must RUN the commands.

Core Workflow

digraph triage_ci {
    "CI failure reported" [shape=box];
    "Extract details from CI logs" [shape=box];
    "Identify suite and test name" [shape=box];
    "Run dev server: pnpm dev $SUITE" [shape=box];
    "Run specific test by name" [shape=box];
    "Did test fail?" [shape=diamond];
    "Debug with dev code" [shape=box];
    "Run prepare-run-test-against-prod" [shape=box];
    "Run: pnpm dev:prod $SUITE" [shape=box];
    "Run specific test again" [shape=box];
    "Did test fail now?" [shape=diamond];
    "Debug bundling issue" [shape=box];
    "Unable to reproduce - check logs" [shape=box];
    "Fix and verify" [shape=box];

    "CI failure reported" -> "Extract details from CI logs";
    "Extract details from CI logs" -> "Identify suite and test name";
    "Identify suite and test name" -> "Run dev server: pnpm dev $SUITE";
    "Run dev server: pnpm dev $SUITE" -> "Run specific test by name";
    "Run specific test by name" -> "Did test fail?";
    "Did test fail?" -> "Debug with dev code" [label="yes"];
    "Did test fail?" -> "Run prepare-run-test-against-prod" [label="no"];
    "Run prepare-run-test-against-prod" -> "Run: pnpm dev:prod $SUITE";
    "Run: pnpm dev:prod $SUITE" -> "Run specific test again";
    "Run specific test again" -> "Did test fail now?";
    "Did test fail now?" -> "Debug bundling issue" [label="yes"];
    "Did test fail now?" -> "Unable to reproduce - check logs" [label="no"];
    "Debug with dev code" -> "Fix and verify";
    "Debug bundling issue" -> "Fix and verify";
}

Step-by-Step Process

1. Extract CI Details

From CI logs or GitHub Actions URL, identify:

Suite name : Directory name (e.g., i18n, fields, lexical)
Test file : Full path (e.g., test/i18n/e2e.spec.ts)
Test name : Exact test description
Error message : Full stack trace
Test type : E2E (Playwright) or integration (Vitest)

2. Reproduce with Dev Code

CRITICAL: Always run the specific test by name, not the full suite.

SERVER MANAGEMENT RULES:

ALWAYS kill all servers before starting a new one
NEVER assume ports are free
ALWAYS wait for server ready confirmation before running tests

# ========================================

# STEP 2A: STOP ALL SERVERS
# ========================================
lsof -ti:3000 | xargs kill -9 2>/dev/null || echo "Port 3000 clear"

# ========================================
# STEP 2B: START DEV SERVER
# ========================================
# Start dev server with the suite (in background with run_in_background=true)
pnpm dev $SUITE_NAME

# ========================================
# STEP 2C: WAIT FOR SERVER READY
# ========================================
# Wait for server to be ready (REQUIRED - do not skip)
until curl -s http://localhost:3000/admin > /dev/null 2>&1; do sleep 1; done && echo "Server ready"

# ========================================
# STEP 2D: RUN SPECIFIC TEST
# ========================================
# Run ONLY the specific failing test using Playwright directly
# For E2E tests (DO NOT use pnpm test:e2e as it spawns its own server):
pnpm exec playwright test test/$SUITE_NAME/e2e.spec.ts -g "exact test name"

# For integration tests:
pnpm test:int $SUITE_NAME -t "exact test name"

Did the test fail?

✅ YES : You reproduced it! Proceed to debug with dev code.
❌ NO : Continue to step 3 (bundled code test).

3. Reproduce with Bundled Code

If test passed with dev code, the issue is likely in bundled/production code.

IMPORTANT: You MUST stop the dev server before starting prod server.

# ========================================
# STEP 3A: STOP ALL SERVERS (INCLUDING DEV SERVER FROM STEP 2)
# ========================================
lsof -ti:3000 | xargs kill -9 2>/dev/null || echo "Port 3000 clear"

# ========================================
# STEP 3B: BUILD AND PACK FOR PROD
# ========================================
# Build all packages and pack them (this takes time - be patient)
pnpm prepare-run-test-against-prod

# ========================================
# STEP 3C: START PROD SERVER
# ========================================
# Start prod dev server (in background with run_in_background=true)
pnpm dev:prod $SUITE_NAME

# ========================================
# STEP 3D: WAIT FOR SERVER READY
# ========================================
# Wait for server to be ready (REQUIRED - do not skip)
until curl -s http://localhost:3000/admin > /dev/null 2>&1; do sleep 1; done && echo "Server ready"

# ========================================
# STEP 3E: RUN SPECIFIC TEST
# ========================================
# Run the specific test again using Playwright directly
pnpm exec playwright test test/$SUITE_NAME/e2e.spec.ts -g "exact test name"
# OR for integration tests:
pnpm test:int $SUITE_NAME -t "exact test name"

Did the test fail now?

✅ YES : Bundling or production build issue. Look for:
- Missing exports in package.json
- Build configuration problems
- Code that behaves differently when bundled
❌ NO : Unable to reproduce locally. Proceed to step 4.

4. Unable to Reproduce

If you cannot reproduce locally after both attempts:

Review CI logs more carefully for environment differences
Check for race conditions (run test multiple times: for i in {1..10}; do pnpm test:e2e...; done)
Look for CI-specific constraints (memory, CPU, timing)
Consider if it's a true race condition that's highly timing-dependent

Common Flaky Test Patterns

Race Conditions

Page navigating while assertions run
Network requests not settled before assertions
State updates not completed

Fix patterns:

Use Playwright's web-first assertions (toBeVisible(), toHaveText())
Wait for specific conditions, not arbitrary timeouts
Use waitForFunction() with condition checks

Test Pollution

Tests leaving data in database
Shared state between tests
Missing cleanup in afterEach

Fix patterns:

Track created IDs and clean up in afterEach
Use isolated test data
Don't use deleteAll that affects other tests

Timing Issues

setTimeout/sleep instead of condition-based waiting
Not waiting for page stability
Animations/transitions not complete

Fix patterns:

Use waitForPageStability() helper
Wait for specific DOM states
Use Playwright's built-in waiting mechanisms

Linting Considerations

When fixing e2e tests, be aware of these eslint rules:

playwright/no-networkidle - Avoid waitForLoadState('networkidle') (use condition-based waiting instead)
payload/no-wait-function - Avoid custom wait() functions (use Playwright's built-in waits)
payload/no-flaky-assertions - Avoid non-retryable assertions
playwright/prefer-web-first-assertions - Use built-in Playwright assertions

Existing code may violate these rules - when adding new code, follow the rules even if existing code doesn't.

Verification

After fixing:

# Ensure dev server is running on port 3000
# Run test multiple times to confirm stability
for i in {1..10}; do
  pnpm exec playwright test test/$SUITE_NAME/e2e.spec.ts -g "exact test name" || break
done

# Run full suite
pnpm exec playwright test test/$SUITE_NAME/e2e.spec.ts

# If you modified bundled code, test with prod build
lsof -ti:3000 | xargs kill -9 2>/dev/null
pnpm prepare-run-test-against-prod
pnpm dev:prod $SUITE_NAME
until curl -s http://localhost:3000/admin > /dev/null; do sleep 1; done
pnpm exec playwright test test/$SUITE_NAME/e2e.spec.ts

The Iron Law

NO FIX WITHOUT REPRODUCTION FIRST

If you propose a fix before completing steps 1-3 of the workflow, you've violated this skill.

This applies even when:

The fix seems obvious from the logs
You've seen this error before
Time pressure from the team
You're confident about the root cause
The logs show clear stack traces

No exceptions. Run the reproduction workflow first.

Rationalization Table

Every excuse for skipping reproduction, and why it's wrong:

Rationalization	Reality
"The logs show the exact error"	Logs show symptoms, not root cause. Reproduce.
"I can see the problem in the code"	You're guessing. Reproduce to confirm.
"This is obviously a race condition"	Maybe. Reproduce to be sure.
"I've seen this error before"	This might be different. Reproduce.
"The stack trace is clear"	Stack trace shows where, not why. Reproduce.
"Time pressure - need to fix fast"	Reproducing IS fast. Guessing wastes time.
"The test file shows the issue"	Reading ≠ running. Execute the commands.
"I'll analyze the code first"	Code analysis comes AFTER reproduction.
"Let me investigate the root cause"	Reproduction IS the investigation.
"I need to understand the error"	Understanding comes from seeing it fail.

Red Flags - STOP

These thoughts mean you're about to violate the skill:

"Let me just fix the obvious issue" → STOP. Execute reproduction first.
"I'll increase the timeout" → STOP. Find root cause via reproduction.
"This looks like X from the logs" → STOP. Run the test.
"I'll run the full suite" → STOP. Run specific test first.
"Let me check the code" → STOP. Run reproduction first.
"I can see the problem in the stack trace" → STOP. Reproduce to confirm.
"The logs are clear enough" → STOP. Execute the commands.
"Let me analyze..." → STOP. Analysis comes after reproduction.

All of these mean: Stop immediately. Execute the reproduction commands. No analysis without execution.

Example Session

CORRECT approach - executes commands first:

User: CI failed on i18n suite - https://github.com/.../actions/runs/XXX

Agent:
1. Extracted details from CI logs:
   - Suite: i18n
   - Test: "ensure translations update correctly when switching language"
   - File: test/i18n/e2e.spec.ts:124
   - Error: "Error loading language: en" TypeError: network error

2. Executing reproduction workflow:
   [Uses Bash tool to kill ports]
   $ lsof -ti:3000 | xargs kill -9 2>/dev/null || echo "Port 3000 clear"

   [Uses Bash tool with run_in_background=true]
   $ pnpm dev i18n

   [Waits for server to start with curl check]
   $ until curl -s http://localhost:3000/admin > /dev/null; do sleep 1; done && echo "Server ready"

   [Uses Bash tool to run Playwright directly]
   $ pnpm exec playwright test test/i18n/e2e.spec.ts -g "ensure translations update correctly when switching language"

   Output shows: Test PASSED locally

3. Test passed with dev code, trying bundled code:
   [Kills ports again]
   $ lsof -ti:3000 | xargs kill -9 2>/dev/null

   [Uses Bash tool]
   $ pnpm prepare-run-test-against-prod

   [Uses Bash tool with run_in_background=true]
   $ pnpm dev:prod i18n

   [Waits for server]
   $ until curl -s http://localhost:3000/admin > /dev/null; do sleep 1; done

   [Uses Bash tool]
   $ pnpm exec playwright test test/i18n/e2e.spec.ts -g "ensure translations update correctly when switching language"

   Output shows: Test FAILED - reproduced!

4. Now that I've reproduced it, analyzing the failure...
   [Code analysis, root cause identification]

5. Implementing fix: [specific changes]

6. Verification:
   [Uses Bash tool]
   $ for i in {1..10}; do pnpm test:e2e i18n -g "exact test"; done
   All 10 runs passed!

INCORRECT approach - analyzes without executing:

User: CI failed on i18n suite

Agent: Looking at the error logs, I can see this is a race condition...
[Analyzes code, proposes fix without running any commands]

❌ WRONG - This violates the skill. Must execute reproduction commands first.

Common Mistakes

Mistake	Fix
Running full test suite first	Run specific test by name
Skipping dev code reproduction	Always try dev code first
Not testing with bundled code	If dev passes, test with `prepare-run-test-against-prod`
Proposing fix without reproducing	Follow the workflow - reproduce first
Using `networkidle` in new code	Use condition-based waiting with `waitForFunction()`
Adding arbitrary `wait()` calls	Use Playwright's built-in assertions and waits

Key Principles

Reproduce before fixing : Never propose a fix without reproducing the issue
Test specifically : Run the exact failing test, not the full suite
Dev first, prod second : Check dev code before bundled code
Follow the workflow : No shortcuts - the steps exist to save time
Verify stability : Run tests multiple times to confirm fix

Completion: Creating a PR

After you have:

✅ Reproduced the issue
✅ Implemented a fix
✅ Verified the fix passes locally (multiple runs)
✅ Tested with prod build (if applicable)

You MUST prompt the user to create a PR:

The fix has been verified and is ready for review. Would you like me to create a PR with these changes?

Summary of changes:
- [List files modified]
- [Brief description of the fix]
- [Verification results]

IMPORTANT:

DO NOT automatically create a PR - always ask the user first
Provide a clear summary of what was changed and why
Include verification results (number of test runs, pass rate)
Let the user decide whether to create the PR immediately or make additional changes first

This ensures the user has visibility and control over what gets submitted for review.

Weekly Installs

114

Repository

payloadcms/payload

GitHub Stars

41.4K

First Seen

Jan 21, 2026

Security Audits

Gen Agent Trust HubPass SocketPass SnykPass

Installed on

codex102

opencode101

gemini-cli98

github-copilot94

cursor93

claude-code92

CI/CD测试故障排查指南：如何复现和修复CI中的不稳定测试失败

🇨🇳中文介绍

排查 CI 故障

概述

何时使用

强制性的第一步

相关 Skills

核心工作流程

分步流程

1. 提取 CI 详细信息

2. 使用开发代码复现

3. 使用打包代码复现

4. 无法复现

常见的不稳定测试模式

竞态条件

测试污染

时序问题

代码检查注意事项

验证

铁律

合理化对照表

危险信号 - 停止

示例会话

常见错误

关键原则

完成：创建 PR

🇺🇸English

Triage CI Failure

Overview

When to Use

MANDATORY First Steps

Core Workflow

Step-by-Step Process

1. Extract CI Details

2. Reproduce with Dev Code

3. Reproduce with Bundled Code

4. Unable to Reproduce

Common Flaky Test Patterns

Race Conditions

Test Pollution

Timing Issues

Linting Considerations

Verification

The Iron Law

Rationalization Table

Red Flags - STOP

Example Session

Common Mistakes

Key Principles

Completion: Creating a PR

最新 Skills