Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions sql-statements/sql-statement-import-into.md
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,29 @@ SET GLOBAL tidb_server_memory_limit='75%';
> - If the KV range overlap in a source data file is low, enabling Global Sort might decrease import performance. This is because when Global Sort is enabled, TiDB needs to wait for the completion of local sorting in all sub-jobs before proceeding with the Global Sort operations and subsequent import.
> - After an import job using Global Sort completes, the files stored in the cloud storage for Global Sort are cleaned up asynchronously in a background thread.

### Asynchronous preparation

<CustomContent platform="tidb-cloud" plan="starter,essential,premium">

On TiDB X instances, `IMPORT INTO ... FROM FILE` uses [Global Sort](#global-sort). TiDB creates the import job first and then prepares it asynchronously in the background. The preparation work includes:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🔎 Supported by static analysis

🏁 Script executed:

sed -n '180,270p' sql-statements/sql-statement-import-into.md
rg -n -i 'global sort|starter|essential|premium' sql-statements/sql-statement-import-into.md

Repository: pingcap/docs

Length of output: 14540


Scope the Global Sort statement to asynchronous-preparation instances.

The section applies to Starter, Essential, and Premium instances. Global Sort is unavailable on Starter and Essential, and Starter uses synchronous preparation. Replace the incorrect universal Global Sort claim with asynchronous-preparation wording.

Suggested wording
-On TiDB X instances, `IMPORT INTO ... FROM FILE` uses [Global Sort](`#global-sort`). TiDB creates the import job first and then prepares it asynchronously in the background. The preparation work includes:
+On TiDB X instances that use asynchronous preparation, `IMPORT INTO ... FROM FILE` creates the import job first and then prepares it asynchronously in the background. The preparation work includes:
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
On TiDB X instances, `IMPORT INTO ... FROM FILE` uses [Global Sort](#global-sort). TiDB creates the import job first and then prepares it asynchronously in the background. The preparation work includes:
On TiDB X instances that use asynchronous preparation, `IMPORT INTO ... FROM FILE` creates the import job first and then prepares it asynchronously in the background. The preparation work includes:


- Listing the source files that match `fileLocation`.
- Detecting the file format when the `FORMAT` clause is omitted.
- Calculating resource-related parameters, such as the import concurrency.
- Splitting the source files into chunks for parallel processing.

Before the preparation finishes, the job is in the `preparing` phase. Because the source files have not been listed yet, the `Source_File_Size` field in [`SHOW IMPORT JOB(s)`](/sql-statements/sql-statement-show-import-job.md) is `N/A` until the preparation succeeds.

If you execute `IMPORT INTO ... FROM FILE` with the `DETACHED` option, the statement returns immediately after the import job is created. You can then use [`SHOW IMPORT JOB(s)`](/sql-statements/sql-statement-show-import-job.md) to check whether the preparation is still running.

> **Note:**
>
> The preparation time grows with the number and size of the source files. For a large single-table import, such as an import of tens of TiB, the `preparing` phase can take a noticeable amount of time even when you execute the statement with the `DETACHED` option, and the import job does not import any data during this period. Before you take any action on the job, such as canceling it, check the job phase with [`SHOW IMPORT JOB(s)`](/sql-statements/sql-statement-show-import-job.md).

For {{{ .starter }}} instances, TiDB prepares the import job synchronously, so the `preparing` phase is not shown in [`SHOW IMPORT JOB(s)`](/sql-statements/sql-statement-show-import-job.md).

</CustomContent>

### Output

When `IMPORT INTO ... FROM FILE` completes the import or when the `DETACHED` mode is enabled, TiDB returns the current job information in the output, as shown in the following examples. For the description of each field, see [`SHOW IMPORT JOB(s)`](/sql-statements/sql-statement-show-import-job.md).
Expand Down
4 changes: 2 additions & 2 deletions sql-statements/sql-statement-show-import-job.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,9 @@ The output fields of the `SHOW IMPORT` statement are described as follows:
| Job_ID | The ID of the task |
| Data_Source | Information about the data source |
| Target_Table | The name of the target table |
| Phase | The current phase of the job, including `importing`, `validating`, and `add-index` |
| Phase | The current phase of the job, including `preparing`, `global-sorting`, `importing`, `resolving-conflicts`, and `validating` |
| Status | The current status of the job, including `pending` (means created but not started yet), `running`, `canceled`, `failed`, and `finished` |
| Source_File_Size | The size of the source file |
| Source_File_Size | The total size of the source files. If the source files have not been listed yet, this column is `N/A`, which can happen while an import job on a TiDB X instance is waiting for or running the [asynchronous preparation](/sql-statements/sql-statement-import-into.md#asynchronous-preparation) |
| Imported_Rows | The number of data rows that have been read and written to the target table |
| Result_Message | If the import fails, this field returns the error message. Otherwise, it is empty.|
| Create_Time | The time when the task is created |
Expand Down
Loading