Skip to content

lake: expand TiDB Cloud Lake overview - #23932

Open
lilin90 wants to merge 7 commits into
pingcap:release-8.5from
lilin90:why-lake
Open

lilin90 wants to merge 7 commits into
pingcap:release-8.5from
lilin90:why-lake

Conversation

@lilin90

@lilin90 lilin90 commented Sep 22, 2026 •

Copy link
Copy Markdown
Member

What is changed, added or deleted? (Required)

Expand the Why TiDB Cloud Lake? section in the TiDB Cloud Lake overview to explain:

  • independent scaling of compute and storage
  • usage-based pricing and cost controls
  • analytics performance and optimization options
  • built-in data pipelines and continuous ingestion
  • full and incremental data replication from TiDB Cloud through Data Pipeline
  • structured and semi-structured data processing
  • unified analytics, search, vector, geospatial, and AI workloads

The update links each capability to the relevant detailed documentation and keeps the overview focused on documented product behavior. This PR depends on #23910 for the Data Pipeline documentation and link target.

Which TiDB version(s) do your changes apply to? (Required)

Tips for choosing the affected version(s):

By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.

For details, see tips for choosing the affected versions.

  • master (the latest development version)
  • v8.5 (TiDB 8.5 versions)
  • v8.4 (TiDB 8.4 versions)
  • v8.3 (TiDB 8.3 versions)
  • v8.2 (TiDB 8.2 versions)
  • v8.1 (TiDB 8.1 versions)
  • v7.5 (TiDB 7.5 versions)
  • v7.1 (TiDB 7.1 versions)
  • v6.5 (TiDB 6.5 versions)

What is the related PR or file link(s)?

AI agent involvement

  • The changes in this PR were primarily made by an AI agent on behalf of the PR author.

Do your changes match any of the following descriptions?

  • Delete files
  • Change aliases
  • Need modification after applied to another branch
  • Might cause conflicts after applied to another branch

Summary by CodeRabbit

  • Documentation
    • Updated the TiDB Cloud Lake overview to cover analytics, data engineering, search, and AI workloads, including built-in multimodal capabilities.
    • Replaced the feature-comparison table with six sections covering scalable compute and storage, predictable costs, high-performance analytics, data pipelines, structured and semi-structured data, and combined analytics, search, and AI workloads.
    • Added links to related architecture, pricing, performance, governance, integration, and multimodal analytics documentation.

Copilot AI lite review requested due to automatic review settings September 22, 2026 09:06
@ti-chi-bot ti-chi-bot Bot added missing-translation-status This PR does not have translation status info. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Sep 22, 2026
@coderabbitai

coderabbitai Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: pingcap/docs/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: bf08e97c-cd68-4f2a-bb0a-9b4b463c5f5c

📥 Commits

Reviewing files that changed from the base of the PR and between 7081c6f and f7079e4.

📒 Files selected for processing (1)
  • tidb-cloud-lake/lake-overview.md

Included review availability: Your plan provides up to 4 included reviews per hour; 1 remains after this review.


📝 Walkthrough

Walkthrough

The .lake overview now describes analytics, data engineering, search, and AI workloads. The “Why .lake?” section replaces its feature table with six linked prose subsections covering compute, costs, analytics, pipelines, semi-structured data, and multimodal workloads.

Changes

.lake overview

Layer / File(s) Summary
Overview capability narrative
tidb-cloud-lake/lake-overview.md
The introduction now covers independently scalable compute, object storage, SQL access, and multimodal capabilities. The feature table is replaced with six subsections covering compute and storage, costs, analytics performance, data pipelines, VARIANT and JSON, and combined analytics, search, and AI workloads.

Priority: ⬇️ Low

Estimated code review effort: 1 (Trivial) | ~5 minutes

Change: Other

Merge Risk: ⚪ Minimal · up to f7079

This documentation-only update is ready to merge with normal checks.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely describes the main change: expanding the TiDB Cloud Lake overview.
Description check ✅ Passed The description explains the documentation changes, affected version, related PR, AI involvement, and applicable change categories. It is complete enough for review.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@lilin90 lilin90 self-assigned this Sep 22, 2026
@lilin90 lilin90 added the translation/no-need No need to translate this PR. label Sep 22, 2026
@ti-chi-bot ti-chi-bot Bot removed the missing-translation-status This PR does not have translation status info. label Sep 22, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Address the documented terminology, consistency, access, and linking issues before approval.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 4 Low severity

Open (4)
What changed in this PR

Expands the TiDB Cloud Lake overview with documented benefits and links for scaling, cost controls, analytics, pipelines, semi-structured data, and multimodal workloads.

Changes:

  • Adds compute/storage scaling and cost-management guidance.
  • Describes analytics optimization and ingestion pipelines.
  • Covers search, vector, geospatial, and AI workloads.
File Summary
tidb-cloud-lake/​lake-overview.md Expands the “Why TiDB Cloud Lake?” section.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.


The main cost components are straightforward:

- **Warehouse compute** is billed per second while a warehouse is running. By default, a warehouse automatically suspends after five minutes of inactivity. A suspended warehouse does not consume compute resources.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@awxxxxxx PTAL

Comment thread tidb-cloud-lake/lake-overview.md Outdated
Comment thread tidb-cloud-lake/lake-overview.md Outdated
Comment thread tidb-cloud-lake/lake-overview.md Outdated
@lilin90 lilin90 added the lake Related to TiDB Cloud Lake docs. label Sep 22, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3


ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: pingcap/docs/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: fe856a80-39d4-46dc-8c19-8ac0eb678305

📥 Commits

Reviewing files that changed from the base of the PR and between ff43a4e and 4d94a53.

📒 Files selected for processing (1)
  • tidb-cloud-lake/lake-overview.md

Included review availability: Your plan provides up to 4 included reviews per hour; 3 remain after this review.

Comment thread tidb-cloud-lake/lake-overview.md Outdated

The main cost components are straightforward:

- **Warehouse compute** is billed per second while a warehouse is running. By default, a warehouse automatically suspends after five minutes of inactivity. A suspended warehouse does not consume compute resources.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

🔎 Supported by static analysis

🏁 Script executed:

rg -n -i 'AUTO_SUSPEND|auto.?suspend|suspend.*warehouse|warehouse.*suspend|inactive.*cluster|release.*cluster' tidb-cloud-lake
sed -n '20,36p' tidb-cloud-lake/lake-overview.md

Repository: pingcap/docs

Length of output: 12397


🏁 Script executed:

printf '%s\n' '--- create warehouse SQL reference ---'
sed -n '24,40p' tidb-cloud-lake/sql/create-warehouse.md
printf '%s\n' '--- warehouse guide ---'
sed -n '34,52p' tidb-cloud-lake/guides/warehouse.md
sed -n '128,150p' tidb-cloud-lake/guides/warehouse.md
printf '%s\n' '--- lake architecture ---'
fd -i '.*lake.*architecture.*' tidb-cloud-lake
for f in $(fd -i '.*lake.*architecture.*' tidb-cloud-lake); do
  sed -n '1,180p' "$f"
done
printf '%s\n' '--- pricing and overview references ---'
sed -n '26,38p' tidb-cloud-lake/guides/pricing-billing.md
sed -n '26,33p' tidb-cloud-lake/lake-overview.md

Repository: pingcap/docs

Length of output: 13392


🤖 get_repo_knowledge executed:

get_repo_knowledge pingcap/docs /tmp/coderabbit-repo-knowledge/pingcap-docs-9c73f70c/learnings

Length of output: 1584


Use the documented warehouse auto-suspend default.

The warehouse reference defines the default AUTO_SUSPEND value as 600 seconds, or ten minutes. Replace “five minutes” with “ten minutes.”

Committable replacement
-**Warehouse compute** is billed per second while a warehouse is running. By default, a warehouse automatically suspends after five minutes of inactivity. A suspended warehouse does not consume compute resources.
+**Warehouse compute** is billed per second while a warehouse is running. By default, a warehouse automatically suspends after ten minutes of inactivity. A suspended warehouse does not consume compute resources.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- **Warehouse compute** is billed per second while a warehouse is running. By default, a warehouse automatically suspends after five minutes of inactivity. A suspended warehouse does not consume compute resources.
- **Warehouse compute** is billed per second while a warehouse is running. By default, a warehouse automatically suspends after ten minutes of inactivity. A suspended warehouse does not consume compute resources.

Comment thread tidb-cloud-lake/lake-overview.md Outdated
Clarified the description of working with structured and semi-structured data, emphasizing the role of JSON path expressions, virtual columns, and inverted indexes.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@ti-chi-bot

ti-chi-bot Bot commented Sep 22, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from lilin90. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

lilin90 and others added 2 commits September 22, 2026 17:15
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
@lilin90
lilin90 requested a review from awxxxxxx September 22, 2026 09:19

This is useful for application events, logs, user activity data, API payloads, and AI agent traces, which often have evolving schemas and can be costly to flatten up front.

To protect sensitive data, you can apply row access policies (an experimental feature) to filter rows at query time and masking policies to redact column values, including selected keys in `VARIANT` data. For more information, see [JSON & Search](/tidb-cloud-lake/guides/json-search.md), [Row Access Policy](/tidb-cloud-lake/guides/row-access-policy.md), and [Masking Policy](/tidb-cloud-lake/guides/masking-policy.md).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@awxxxxxx Since row access policy is still experimental, please help confirm whether we should keep it here or remove it and adjust wording?

Added a new overview section describing TiDB Cloud Data Pipeline replication from TiDB Cloud to TiDB Cloud Lake. It explains the full snapshot plus incremental CDC flow, common use cases, and the external-stage dependency and plan availability restrictions.
@lilin90 lilin90 changed the title cloud: expand TiDB Cloud Lake overview lake: expand TiDB Cloud Lake overview Sep 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lake Related to TiDB Cloud Lake docs. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. translation/no-need No need to translate this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants