Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 28 additions & 3 deletions tidb-cloud/monitor-built-in-alerting.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,9 +124,34 @@ TiDB Cloud provides different alert rules for each [TiDB Cloud plan](/tidb-cloud

</CustomContent>

<CustomContent plan="essential, premium">
<CustomContent plan="premium">

### Performance overview alerts

| Condition | Recommended Action |
|:--- |:--- |
| Request units per second (RU/s) exceed 80% of the maximum RCU | <ol><li>Review RU metrics to determine whether the increase is gradual or a sudden spike.</li><li>If the increase is gradual, check whether query duration has increased. If so, the current maximum RCU might be insufficient.</li><li>Scale capacity by manually increasing the maximum RCU in the TiDB Cloud console.</li></ol><br/>If you cannot resolve the issue, contact [TiDB Cloud Support](/tidb-cloud/tidb-cloud-support.md).|
| QPS drops by 80% | <ol><li>Check whether the drop is caused by increasing query latency.</li><li>Verify that your application is operating normally. If the drop is intentional, ignore this alert. If the drop is unintentional and you cannot identify the root cause, contact [TiDB Cloud Support](/tidb-cloud/tidb-cloud-support.md) immediately.</li></ol>|
| Query P99 latency exceeds 200 ms | <ol><li>Investigate slow queries: go to the Slow Query page and filter by a recent time range to identify newly introduced or slower-running queries.</li><li>Review recent changes, such as application deployments, schema changes, or data import jobs, that might have affected traffic patterns.</li></ol><br/>If you cannot identify the root cause, contact [TiDB Cloud Support](/tidb-cloud/tidb-cloud-support.md) immediately.|
| Query P95 latency exceeds 200 ms | <ol><li>Investigate slow queries: go to the Slow Query page and filter by a recent time range to identify newly introduced or slower-running queries.</li><li>Review recent changes, such as application deployments, schema changes, or data import jobs, that might have affected traffic patterns.</li></ol><br/>If you cannot identify the root cause, contact [TiDB Cloud Support](/tidb-cloud/tidb-cloud-support.md) immediately.|
| Request error rate exceeds 10% | Review recent errors and the overall statement execution status for the cluster.|
| More than 20 SQL statements exceeded 256 ms within 1 minute | Check for recent schema or index changes. Review the slow query list in the TiDB Cloud console. It might take several minutes for the corresponding slow query records to appear. If you cannot identify the root cause, contact [TiDB Cloud Support](/tidb-cloud/tidb-cloud-support.md).|
| More than 20 SQL statements exceeded 512 ms within 1 minute | Check for recent schema or index changes. Review the slow query list in the TiDB Cloud console. It might take several minutes for the corresponding slow query records to appear. If you cannot identify the root cause, contact [TiDB Cloud Support](/tidb-cloud/tidb-cloud-support.md).|
| More than 20 SQL statements exceeded 4096 ms within 1 minute | Check for recent schema or index changes. Review the slow query list in the TiDB Cloud console. It might take several minutes for the corresponding slow query records to appear. If you cannot identify the root cause, contact [TiDB Cloud Support](/tidb-cloud/tidb-cloud-support.md).|

### Changefeed alerts for {{{ .premium }}}

| Condition | Recommended Action |
|:--------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| The changefeed latency exceeds 600 seconds. | Check the changefeed status on the **Changefeed** page and **Changefeed Detail** page of the TiDB Cloud console, where you can find some error messages to help diagnose this issue. <br/> Possible reasons that can trigger this alert include:<ul><li>The overall traffic in the upstream has increased, causing the existing changefeed specification to be insufficient to handle it. If the traffic increase is temporary, the changefeed latency will automatically recover after the traffic returns to normal. If the traffic increase is continuous, you need to scale up the changefeed.</li><li>The downstream or network is abnormal. In this case, resolve this abnormality first.</li><li>Tables lack indexes if the downstream is RDS, which might cause low write performance and high latency. In this case, you need to add the necessary indexes to the upstream or downstream.</li></ul>If the problem cannot be fixed from your side, contact [TiDB Cloud Support](/tidb-cloud/tidb-cloud-support.md) for further assistance. |
| The changefeed status is `FAILED`. | Check the changefeed status on the **Changefeed** page and **Changefeed Detail** page of the TiDB Cloud console, where you can find some error messages to help diagnose this issue. <br/> If the problem cannot be fixed from your side, contact [TiDB Cloud Support](/tidb-cloud/tidb-cloud-support.md) for further assistance.|
| The changefeed status is `WARNING`. | Check the changefeed status on the **Changefeed** page and **Changefeed Detail** page of the TiDB Cloud console, where you can find some error messages to help diagnose this issue. <br/> If the problem cannot be fixed from your side, contact [TiDB Cloud Support](/tidb-cloud/tidb-cloud-support.md) for further assistance.|

</CustomContent>

<CustomContent plan="essential">

### Performance overview alerts
### Performance overview alerts

| Condition | Recommended Action |
|:--- |:--- |
Expand All @@ -136,7 +161,7 @@ TiDB Cloud provides different alert rules for each [TiDB Cloud plan](/tidb-cloud
| Query P95 latency exceeds 200 ms | <ol><li>Investigate slow queries: go to the Slow Query page and filter by a recent time range to identify newly introduced or slower-running queries.</li><li>Review recent changes, such as application deployments, schema changes, or data import jobs, that might have affected traffic patterns.</li></ol><br/>If you cannot identify the root cause, contact [TiDB Cloud Support](/tidb-cloud/tidb-cloud-support.md) immediately.|
| Request error rate exceeds 10% | Review recent errors and the overall statement execution status for the cluster.|

### Changefeed alerts for {{{ .essential }}} and {{{ .premium }}}
### Changefeed alerts for {{{ .essential }}}

| Condition | Recommended Action |
|:--------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
Expand Down
Loading