HyperShift 集群调试指南：解决托管集群删除、卡住资源与节点池问题

Debug Cluster by openshift/hypershift

515 GitHub Stars

GitHub

安装命令

npx skills add https://github.com/openshift/hypershift --skill 'Debug Cluster'

云服务开发运维监控

🇨🇳中文介绍

HyperShift 集群调试指南

此技能为常见的 HyperShift 托管集群问题提供结构化的调试工作流。

何时使用此技能

此技能在以下情况会自动应用：

调查托管集群删除问题
调试卡住的资源或终结器
排查控制平面问题
分析节点池生命周期问题
审查集群问题的操作员日志

特定于供应商的故障排除

对于特定于供应商的问题和详细的故障排除步骤，请参考以下子技能：

AWS : aws-troubleshooting.md - AWS 特定的 CAPI 问题、基础设施清理和重新安装

下面的主要技能提供与供应商无关的调试工作流。当您遇到特定于供应商的问题时，请查阅相关的子技能以获取详细的解决步骤。

需要理解的关键组件

资源层次结构

HostedCluster (HC) : 管理集群中的主要集群资源
HostedControlPlane (HCP) : HCP 命名空间中 HC 的控制平面表示
NodePool (NP) : 工作节点池资源
CAPI 资源 : HCP 命名空间中的集群 API 资源（Cluster、Machine 等）

操作员

hypershift-operator (HO) : 管理 HC 和 NP 资源
control-plane-operator (CPO) : 管理 HCP 和控制平面组件
hosted-cluster-config-operator (HCCO) : 管理托管集群的配置和集群内资源

命名空间

广告位招租

在这里展示您的产品或服务

触达数万 AI 开发者，精准高效

联系我们

场景：托管集群卡在删除状态

当托管集群卡在删除状态时，请遵循此系统化的调试流程：

检查并验证 NodePool 删除是否正在进行：

# 检查 HC 命名空间中的 NodePool 资源
kubectl get nodepool -n <hc-namespace>

# 检查 HCP 命名空间中的 CAPI 集群资源状态
kubectl get cluster -n <hcp-namespace> -o yaml

# 检查 CAPI 提供者 Pod 日志
kubectl logs -n <hcp-namespace> deployment/capi-provider

# 检查 HCP 命名空间中的 CAPI 机器状态
kubectl get machines -n <hcp-namespace>
kubectl describe machines -n <hcp-namespace>

# 审查 HyperShift 操作员日志中关于 NodePool 的问题
kubectl logs -n hypershift deployment/operator --tail=100 | grep -i nodepool
kubectl logs -n hypershift deployment/operator --tail=100 | grep -i <cluster-name>

需要关注的内容：

阻止 NodePool 删除的终结器
未终止的 CAPI 机器
阻止机器删除的提供者错误
显示协调错误的 HO 日志

2. HostedControlPlane 资源删除

验证 HCP 资源和 Pod 是否正在被清理：

# 检查 HCP 资源状态
kubectl get hostedcontrolplane -n <hcp-namespace> -o yaml

# 检查 HCP 命名空间中的 Pod
kubectl get pods -n <hcp-namespace>

# 检查卡住的 Pod
kubectl get pods -n <hcp-namespace> --field-selector=status.phase!=Running

# 审查 control-plane-operator 日志
kubectl logs -n <hcp-namespace> deployment/control-plane-operator --tail=100

需要关注的内容：

阻止删除的 HCP 终结器
带有终结器或处于 Terminating 状态的 Pod
显示资源清理错误的 CPO 日志
阻止命名空间删除的 PVC 或其他资源

3. HCP 命名空间删除

调查 HCP 命名空间未被删除的原因：

# 检查命名空间状态
kubectl get namespace <hcp-namespace> -o yaml

# 列出命名空间中所有剩余的资源
kubectl api-resources --verbs=list --namespaced -o name | \
  xargs -n 1 kubectl get --show-kind --ignore-not-found -n <hcp-namespace>

# 检查带有终结器的资源
kubectl get all -n <hcp-namespace> -o json | \
  jq '.items[] | select(.metadata.finalizers != null) | {kind: .kind, name: .metadata.name, finalizers: .metadata.finalizers}'

# 审查 HO 日志中关于命名空间清理的信息
kubectl logs -n hypershift deployment/operator --tail=100 | grep -i namespace

需要关注的内容：

阻止删除的带有终结器的资源
本应由 HO 清理的 API 资源
Webhook 或准入控制器错误
卡在 Terminating 状态的命名空间

4. HostedCluster 资源删除

对 HostedCluster 资源本身的最终检查：

# 检查 HostedCluster 状态
kubectl get hostedcluster -n <hc-namespace> <cluster-name> -o yaml

# 检查 HostedCluster 终结器
kubectl get hostedcluster -n <hc-namespace> <cluster-name> -o jsonpath='{.metadata.finalizers}'

# 审查 HO 日志中关于 HostedCluster 删除的信息
kubectl logs -n hypershift deployment/operator --tail=200 | grep -i "hostedcluster.*<cluster-name>"

需要关注的内容：

阻止 HostedCluster 删除的终结器
HO 协调循环中的错误
尚未清理的依赖项
未能删除的云资源

调查集群删除问题时：

检查资源状态和条件
审查相关操作员日志（HO、CPO）
检查卡住资源上的终结器
验证 CAPI 资源是否正在协调
检查指示失败的事件
查找提供者错误
验证命名空间清理进度
检查 Webhook 或准入错误

常见问题及解决方法

问题：NodePool 无法删除

原因：由于提供者错误导致 CAPI 机器卡住
解决方法：检查提供者凭据，调查 HO 日志中的机器删除错误

问题：机器卡在 "Deleting" 阶段并显示 "WaitingForInfrastructureDeletion"

原因：集群资源在机器资源之前被删除，阻塞了 CAPI 控制器的协调
根本原因：CAPI 机器控制器需要集群资源处于 "ready" 状态才能继续删除
症状：
- 机器显示 InfrastructureReady: 1 of 2 completed
- CAPI 日志显示集群资源未就绪
- 实例仍在运行但控制器无法终止它们
解决方法：需要特定于提供者的清理。请参阅特定于提供者的故障排除：
- AWS : 参见 aws-troubleshooting.md 获取详细的解决步骤

问题：HCP 命名空间卡在 Terminating 状态

原因：资源带有终结器或 Webhook 失败
解决方法：列出命名空间中的所有资源，从孤立资源中移除终结器，检查 Webhook 可用性

问题：HostedCluster 卡住并带有终结器

原因：依赖项未完全清理（NodePools、HCP、云资源）
解决方法：验证所有依赖资源是否已删除，检查 HO 日志中的协调错误

问题：控制平面 Pod 未终止

原因：PVC 或其他资源带有保护性终结器
解决方法：检查 PVC 状态，审查存储类终结器，检查 CPO 日志

场景：HyperShift CRD 缺失或损坏

当 HyperShift CRD 被意外删除或损坏时（例如，在使用 hypershift destroy infra 之后），您需要重新安装 HyperShift。

⚠️ 警告：重新安装 HyperShift 应是最后的手段。 仅当 CRD 确实缺失或损坏且无法通过其他方式恢复时才进行。重新安装将导致停机，并可能影响现有的托管集群。

像 kubectl get hostedclusters 这样的命令失败，并显示：error: the server doesn't have a resource type "hostedclusters"
HyperShift 操作员日志显示有关缺失 CRD 的错误
HostedCluster、NodePool 或 CAPI 资源的 CRD 缺失

检查缺失的 CRD：

# 检查关键的 HyperShift CRD 是否存在
kubectl get crd hostedclusters.hypershift.openshift.io
kubectl get crd nodepools.hypershift.openshift.io

# 统计 HyperShift CRD 数量（应该大约为 9 个）
kubectl get crd | grep hypershift | wc -l

# 统计 CAPI CRD 数量（应该大约为 50 个）
kubectl get crd | grep cluster.x-k8s.io | wc -l

解决方法：重新安装 HyperShift

🤖 AI 助手说明： 当遇到此场景时，Claude 应指导并建议用户重新安装步骤，但绝不自行执行重新安装命令。用户必须明确运行这些命令。提供清晰的说明和解释，但不要使用 Bash 工具执行实际的重新安装。

步骤 1：收集所需参数

# 重新安装需要以下内容：
# - OIDC 存储提供者配置（特定于提供者，见下文）
# - 提供者凭据（如果适用）
# - 原始安装中使用的任何自定义配置标志

特定于提供者的参数：

AWS : 参见 aws-troubleshooting.md 获取 AWS 特定的 OIDC S3 参数

步骤 2：完全卸载 HyperShift

hypershift install render | kubectl delete -f -

移除 HyperShift 操作员部署
删除所有 HyperShift CRD（HostedCluster、NodePool 等）
清理 RBAC 资源、Webhook 和其他组件

步骤 3：重新安装 HyperShift

hypershift install \
  [provider-specific-flags] \
  --enable-defaulting-webhook true

添加原始安装中包含的任何其他标志。

特定于提供者的安装：

AWS : 参见 aws-troubleshooting.md 获取 AWS 特定的安装示例

步骤 4：验证安装

# 检查操作员是否在运行
kubectl get deploy -n hypershift
kubectl get pods -n hypershift

# 验证 CRD 已安装
kubectl get crd | grep hostedcluster
kubectl get crd | grep nodepool
kubectl get crd | grep cluster.x-k8s.io | wc -l

# 测试 API 可访问性
kubectl get hostedclusters -A

# 检查操作员日志中的错误
kubectl logs -n hypershift deployment/operator --tail=50

# 验证控制器正在运行
kubectl logs -n hypershift deployment/operator --tail=100 | grep "Starting workers"

重新安装后的预期结果：

操作员部署：2/2 READY
HostedCluster CRD：存在
NodePool CRD：存在
CAPI CRD：约 50 个已安装
HyperShift CRD：总计约 9 个
控制器：HostedCluster、NodePool 和其他控制器已启动工作进程
Webhook：已配置 Mutating webhook
API：kubectl get hostedclusters -A 成功返回（即使没有集群存在）

如果现有托管集群的资源仍然存在，重新安装不会影响它们
如果 CRD 被删除，任何现有的 HostedCluster/NodePool 资源都将丢失
如果集群定义丢失，您可能需要重新创建托管集群
确保使用与原始安装相同的配置标志

# 列出资源上的所有终结器
kubectl get <resource-type> <name> -n <namespace> -o jsonpath='{.metadata.finalizers}'

# 移除特定的终结器（谨慎使用！）
kubectl patch <resource-type> <name> -n <namespace> -p '{"metadata":{"finalizers":null}}' --type=merge

# 带上下文的 HyperShift 操作员日志
kubectl logs -n hypershift deployment/operator --tail=500 --timestamps

# 控制平面操作员日志
kubectl logs -n <hcp-namespace> deployment/control-plane-operator --tail=500 --timestamps

# 实时跟踪日志
kubectl logs -n hypershift deployment/operator -f

# 获取特定资源的事件
kubectl describe <resource-type> <name> -n <namespace>

# 获取命名空间中的所有事件，按时间排序
kubectl get events -n <namespace> --sort-by='.lastTimestamp'

# 检查资源条件
kubectl get <resource-type> <name> -n <namespace> -o jsonpath='{.status.conditions}' | jq .

# 检查特定条件
kubectl get hostedcluster <name> -n <namespace> -o jsonpath='{.status.conditions[?(@.type=="Available")]}'

HyperShift 操作员代码：hypershift-operator/controllers/hostedcluster/
控制平面操作员代码：control-plane-operator/controllers/
API 定义：api/hypershift/v1beta1/
E2E 测试示例：test/e2e/

🇺🇸English

HyperShift Cluster Debugging Guide

This skill provides structured debugging workflows for common HyperShift hosted-cluster issues.

When to Use This Skill

This skill automatically applies when:

Investigating hosted-cluster deletion issues
Debugging stuck resources or finalizers
Troubleshooting control plane problems
Analyzing NodePool lifecycle issues
Reviewing operator logs for cluster problems

Provider-Specific Troubleshooting

For provider-specific issues and detailed troubleshooting steps, refer to these subskills:

AWS : aws-troubleshooting.md - AWS-specific CAPI issues, infrastructure cleanup, and reinstallation

The main skill below provides provider-agnostic debugging workflows. When you encounter provider-specific issues, consult the relevant subskill for detailed resolution steps.

Key Components to Understand

Resource Hierarchy

HostedCluster (HC) : Main cluster resource in the management cluster
HostedControlPlane (HCP) : Control plane representation of the HC in HCP namespace
NodePool (NP) : Worker node pool resources
CAPI Resources : Cluster API resources (Cluster, Machine, etc.) in HCP namespace

Operators

hypershift-operator (HO) : Manages HC and NP resources
control-plane-operator (CPO) : Manages HCP and control plane components
hosted-cluster-config-operator (HCCO) : Manages configuration and in-cluster resources for hosted clusters

Namespaces

HC namespace : Where HostedCluster and NodePool resources live (e.g., default, clusters)
HCP namespace : Where control plane pods and CAPI resources run (e.g., clusters-<cluster-name>)

Common Debugging Scenarios

Scenario: Hosted Cluster Stuck Deleting

When a hosted-cluster is stuck in deleting state, follow this systematic debugging process:

1. Node Pools Deletion

Check and verify NodePool deletion is progressing:

# Check NodePool resources in HC namespace
kubectl get nodepool -n <hc-namespace>

# Check CAPI cluster resource status in HCP namespace
kubectl get cluster -n <hcp-namespace> -o yaml

# Check CAPI provider pod logs
kubectl logs -n <hcp-namespace> deployment/capi-provider

# Check CAPI machines status in HCP namespace
kubectl get machines -n <hcp-namespace>
kubectl describe machines -n <hcp-namespace>

# Review HyperShift operator logs for NodePool issues
kubectl logs -n hypershift deployment/operator --tail=100 | grep -i nodepool
kubectl logs -n hypershift deployment/operator --tail=100 | grep -i <cluster-name>

What to look for:

Finalizers blocking NodePool deletion
CAPI machines that aren't terminating
Provider errors preventing machine deletion
HO logs showing reconciliation errors

2. HostedControlPlane Resource Deletion

Verify HCP resource and pods are being cleaned up:

# Check HCP resource status
kubectl get hostedcontrolplane -n <hcp-namespace> -o yaml

# Check pods in HCP namespace
kubectl get pods -n <hcp-namespace>

# Check for stuck pods
kubectl get pods -n <hcp-namespace> --field-selector=status.phase!=Running

# Review control-plane-operator logs
kubectl logs -n <hcp-namespace> deployment/control-plane-operator --tail=100

What to look for:

HCP finalizers blocking deletion
Pods with finalizers or in Terminating state
CPO logs showing errors in resource cleanup
PVC or other resources preventing namespace deletion

3. HCP Namespace Deletion

Investigate why the HCP namespace isn't being removed:

# Check namespace status
kubectl get namespace <hcp-namespace> -o yaml

# List all remaining resources in namespace
kubectl api-resources --verbs=list --namespaced -o name | \
  xargs -n 1 kubectl get --show-kind --ignore-not-found -n <hcp-namespace>

# Check for resources with finalizers
kubectl get all -n <hcp-namespace> -o json | \
  jq '.items[] | select(.metadata.finalizers != null) | {kind: .kind, name: .metadata.name, finalizers: .metadata.finalizers}'

# Review HO logs for namespace cleanup
kubectl logs -n hypershift deployment/operator --tail=100 | grep -i namespace

What to look for:

Resources with finalizers preventing deletion
API resources that HO should have cleaned up
Webhook or admission controller errors
Namespace stuck in Terminating state

4. HostedCluster Resource Deletion

Final check on the HostedCluster resource itself:

# Check HostedCluster status
kubectl get hostedcluster -n <hc-namespace> <cluster-name> -o yaml

# Check HostedCluster finalizers
kubectl get hostedcluster -n <hc-namespace> <cluster-name> -o jsonpath='{.metadata.finalizers}'

# Review HO logs for HostedCluster deletion
kubectl logs -n hypershift deployment/operator --tail=200 | grep -i "hostedcluster.*<cluster-name>"

What to look for:

Finalizers blocking HostedCluster deletion
HO errors in reconciliation loop
Dependencies that haven't been cleaned up
Cloud resources that failed to delete

Quick Debugging Checklist

When investigating cluster deletion issues:

Check resource status and conditions
Review relevant operator logs (HO, CPO)
Inspect finalizers on stuck resources
Verify CAPI resources are reconciling
Check for events indicating failures
Look for provider errors
Verify namespace cleanup progress
Check for webhook or admission errors

Common Issues and Resolutions

Issue: NodePool won't delete

Cause : CAPI machines stuck due to provider errors
Resolution : Check provider credentials, investigate machine deletion errors in HO logs

Issue: Machines stuck in "Deleting" phase with "WaitingForInfrastructureDeletion"

Cause : Cluster resource deleted before machine resources, blocking CAPI controller reconciliation
Root Cause : CAPI machine controllers require the cluster resource to be "ready" to proceed with deletion
Symptoms :
- Machines show InfrastructureReady: 1 of 2 completed
- CAPI logs show cluster resource is not ready
- Instances still running but controller can't terminate them
Resolution : Provider-specific cleanup is required. See provider-specific troubleshooting:
- AWS : See aws-troubleshooting.md for detailed resolution steps

Issue: HCP namespace stuck in Terminating

Cause : Resources with finalizers or failing webhooks
Resolution : List all resources in namespace, remove finalizers from orphaned resources, check webhook availability

Issue: HostedCluster stuck with finalizers

Cause : Dependencies not fully cleaned up (NodePools, HCP, cloud resources)
Resolution : Verify all dependent resources are deleted, check HO logs for reconciliation errors

Issue: Control plane pods not terminating

Cause : PVCs or other resources with protection finalizers
Resolution : Check PVC status, review storage class finalizers, inspect CPO logs

Scenario: HyperShift CRDs Missing or Corrupted

When HyperShift CRDs are accidentally deleted or corrupted (e.g., after using hypershift destroy infra), you'll need to reinstall HyperShift.

⚠️ WARNING: HyperShift reinstallation should be a last resort. Only proceed if CRDs are genuinely missing or corrupted and cannot be recovered through other means. Reinstallation will cause downtime and may impact existing hosted-clusters.

Symptoms:

Commands like kubectl get hostedclusters fail with: error: the server doesn't have a resource type "hostedclusters"
HyperShift operator logs show errors about missing CRDs
CRDs for HostedCluster, NodePool, or CAPI resources are missing

Check for Missing CRDs:

# Check if critical HyperShift CRDs exist
kubectl get crd hostedclusters.hypershift.openshift.io
kubectl get crd nodepools.hypershift.openshift.io

# Count HyperShift CRDs (should be ~9)
kubectl get crd | grep hypershift | wc -l

# Count CAPI CRDs (should be ~50)
kubectl get crd | grep cluster.x-k8s.io | wc -l

Resolution: Reinstall HyperShift

🤖 AI Assistant Note: When this scenario is encountered, Claude should guide and suggest the reinstallation steps to the user but NEVER execute the reinstallation commands itself. The user must explicitly run these commands. Provide clear instructions and explanations, but do not use the Bash tool to perform the actual reinstallation.

Step 1: Gather Required Parameters

# You'll need these for reinstallation:
# - OIDC storage provider configuration (provider-specific, see below)
# - Provider credentials (if applicable)
# - Any custom configuration flags used in original installation

Provider-specific parameters:

AWS : See aws-troubleshooting.md for AWS-specific OIDC S3 parameters

Step 2: Completely Uninstall HyperShift

hypershift install render | kubectl delete -f -

This will:

Remove the HyperShift operator deployment
Delete all HyperShift CRDs (HostedCluster, NodePool, etc.)
Clean up RBAC resources, webhooks, and other components

Step 3: Reinstall HyperShift

hypershift install \
  [provider-specific-flags] \
  --enable-defaulting-webhook true

Add any other flags that were part of your original installation.

Provider-specific installation:

AWS : See aws-troubleshooting.md for AWS-specific installation example

Step 4: Verify Installation

# Check operator is running
kubectl get deploy -n hypershift
kubectl get pods -n hypershift

# Verify CRDs are installed
kubectl get crd | grep hostedcluster
kubectl get crd | grep nodepool
kubectl get crd | grep cluster.x-k8s.io | wc -l

# Test API accessibility
kubectl get hostedclusters -A

# Check operator logs for errors
kubectl logs -n hypershift deployment/operator --tail=50

# Verify controllers are running
kubectl logs -n hypershift deployment/operator --tail=100 | grep "Starting workers"

Expected Results After Reinstallation:

Operator deployment: 2/2 READY
HostedCluster CRD: Present
NodePool CRD: Present
CAPI CRDs: ~50 installed
HyperShift CRDs: ~9 total
Controllers: HostedCluster, NodePool, and other controllers with workers started
Webhooks: Mutating webhook configured
API: kubectl get hostedclusters -A returns successfully (even if no clusters exist)

Important Notes:

Reinstallation does NOT affect existing hosted-clusters if their resources still exist
If CRDs were deleted, any existing HostedCluster/NodePool resources are gone
You may need to recreate hosted-clusters if their definitions were lost
Ensure you use the same configuration flags as the original installation

General Debugging Tips

Checking Finalizers

# List all finalizers on a resource
kubectl get <resource-type> <name> -n <namespace> -o jsonpath='{.metadata.finalizers}'

# Remove a specific finalizer (use with caution!)
kubectl patch <resource-type> <name> -n <namespace> -p '{"metadata":{"finalizers":null}}' --type=merge

Operator Logs

# HyperShift operator logs with context
kubectl logs -n hypershift deployment/operator --tail=500 --timestamps

# Control plane operator logs
kubectl logs -n <hcp-namespace> deployment/control-plane-operator --tail=500 --timestamps

# Follow logs in real-time
kubectl logs -n hypershift deployment/operator -f

Resource Events

# Get events for a specific resource
kubectl describe <resource-type> <name> -n <namespace>

# Get all events in a namespace, sorted by time
kubectl get events -n <namespace> --sort-by='.lastTimestamp'

Conditions and Status

# Check resource conditions
kubectl get <resource-type> <name> -n <namespace> -o jsonpath='{.status.conditions}' | jq .

# Check specific condition
kubectl get hostedcluster <name> -n <namespace> -o jsonpath='{.status.conditions[?(@.type=="Available")]}'

Additional Resources

HyperShift operator code: hypershift-operator/controllers/hostedcluster/
Control plane operator code: control-plane-operator/controllers/
API definitions: api/hypershift/v1beta1/
E2E test examples: test/e2e/

Weekly Installs

–

Repository

openshift/hypershift

GitHub Stars

515

First Seen

–

Security Audits

Gen Agent Trust HubPass SocketPass SnykPass

HyperShift 集群调试指南：解决托管集群删除、卡住资源与节点池问题

🇨🇳中文介绍

HyperShift 集群调试指南

何时使用此技能

特定于供应商的故障排除

需要理解的关键组件

资源层次结构

操作员

命名空间

相关 Skills

常见调试场景

场景：托管集群卡在删除状态

1. 节点池删除

2. HostedControlPlane 资源删除

3. HCP 命名空间删除

4. HostedCluster 资源删除

快速调试清单

常见问题及解决方法

问题：NodePool 无法删除

问题：机器卡在 "Deleting" 阶段并显示 "WaitingForInfrastructureDeletion"

问题：HCP 命名空间卡在 Terminating 状态

问题：HostedCluster 卡住并带有终结器

问题：控制平面 Pod 未终止

场景：HyperShift CRD 缺失或损坏

症状：

检查缺失的 CRD：

解决方法：重新安装 HyperShift

步骤 1：收集所需参数

步骤 2：完全卸载 HyperShift

步骤 3：重新安装 HyperShift

步骤 4：验证安装

通用调试技巧

检查终结器

操作员日志

资源事件

条件和状态

其他资源

🇺🇸English

HyperShift Cluster Debugging Guide

When to Use This Skill

Provider-Specific Troubleshooting

Key Components to Understand

Resource Hierarchy

Operators

Namespaces

Common Debugging Scenarios

Scenario: Hosted Cluster Stuck Deleting

1. Node Pools Deletion

2. HostedControlPlane Resource Deletion

3. HCP Namespace Deletion

4. HostedCluster Resource Deletion

Quick Debugging Checklist

Common Issues and Resolutions

Issue: NodePool won't delete

Issue: Machines stuck in "Deleting" phase with "WaitingForInfrastructureDeletion"

Issue: HCP namespace stuck in Terminating

Issue: HostedCluster stuck with finalizers

Issue: Control plane pods not terminating

Scenario: HyperShift CRDs Missing or Corrupted

Symptoms:

Check for Missing CRDs:

Resolution: Reinstall HyperShift

Step 1: Gather Required Parameters

Step 2: Completely Uninstall HyperShift

Step 3: Reinstall HyperShift

Step 4: Verify Installation

General Debugging Tips

Checking Finalizers

Operator Logs

Resource Events

Conditions and Status

Additional Resources

最新 Skills