From 7d05fb0f3e8ad552bba5780f5c3692fc782a4733 Mon Sep 17 00:00:00 2001 From: Jianjun Liao Date: Mon, 10 Mar 2025 12:35:56 +0800 Subject: [PATCH 1/4] draft Signed-off-by: Jianjun Liao --- br/backup-and-restore-use-cases.md | 4 +++- br/br-pitr-guide.md | 4 +++- br/br-pitr-manual.md | 4 +++- br/br-snapshot-guide.md | 25 ++++++++++++++++++++++--- br/br-snapshot-manual.md | 4 +++- 5 files changed, 34 insertions(+), 7 deletions(-) diff --git a/br/backup-and-restore-use-cases.md b/br/backup-and-restore-use-cases.md index affaf6684f701..86c99f0909ae3 100644 --- a/br/backup-and-restore-use-cases.md +++ b/br/backup-and-restore-use-cases.md @@ -144,7 +144,9 @@ tiup br restore point --pd="${PD_IP}:2379" \ --full-backup-storage='s3://tidb-pitr-bucket/backup-data/snapshot-20220514000000' \ --restored-ts '2022-05-15 18:00:00+0800' -Full Restore <--------------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% +Split&Scatter Region <--------------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% +Download&Ingest SST <--------------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% +Restore Pipeline <--------------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% [2022/05/29 18:15:39.132 +08:00] [INFO] [collector.go:69] ["Full Restore success summary"] [total-ranges=12] [ranges-succeed=xxx] [ranges-failed=0] [split-region=xxx.xxxµs] [restore-ranges=xxx] [total-take=xxx.xxxs] [restore-data-size(after-compressed)=xxx.xxx] [Size=xxxx] [BackupTS={TS}] [total-kv=xxx] [total-kv-size=xxx] [average-speed=xxx] Restore Meta Files <--------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% Restore KV Files <----------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% diff --git a/br/br-pitr-guide.md b/br/br-pitr-guide.md index 27aecb8b9e196..c428e57bf20c8 100644 --- a/br/br-pitr-guide.md +++ b/br/br-pitr-guide.md @@ -93,7 +93,9 @@ tiup br restore point --pd "${PD_IP}:2379" \ During data restore, you can view the progress through the progress bar in the terminal. The restore is divided into two phases, full restore and log restore (restore meta files and restore KV files). After each phase is completed, `br` outputs information such as restore time and data size. ```shell -Full Restore <--------------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% +Split&Scatter Region <--------------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% +Download&Ingest SST <--------------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% +Restore Pipeline <--------------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% *** ["Full Restore success summary"] ****** [total-take=xxx.xxxs] [restore-data-size(after-compressed)=xxx.xxx] [Size=xxxx] [BackupTS={TS}] [total-kv=xxx] [total-kv-size=xxx] [average-speed=xxx] Restore Meta Files <--------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% Restore KV Files <----------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% diff --git a/br/br-pitr-manual.md b/br/br-pitr-manual.md index c873fb2789725..019403fcf1d46 100644 --- a/br/br-pitr-manual.md +++ b/br/br-pitr-manual.md @@ -458,7 +458,9 @@ tiup br restore point --pd="${PD_IP}:2379" --storage='s3://backup-101/logbackup?access-key=${access-key}&secret-access-key=${secret-access-key}' --full-backup-storage='s3://backup-101/snapshot-202205120000?access-key=${access-key}&secret-access-key=${secret-access-key}' -Full Restore <--------------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% +Split&Scatter Region <--------------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% +Download&Ingest SST <--------------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% +Restore Pipeline <--------------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% *** ***["Full Restore success summary"] ****** [total-take=3.112928252s] [restore-data-size(after-compressed)=5.056kB] [Size=5056] [BackupTS=434693927394607136] [total-kv=4] [total-kv-size=290B] [average-speed=93.16B/s] Restore Meta Files <--------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% Restore KV Files <----------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% diff --git a/br/br-snapshot-guide.md b/br/br-snapshot-guide.md index 80e04a2207e4c..86a70d925079c 100644 --- a/br/br-snapshot-guide.md +++ b/br/br-snapshot-guide.md @@ -36,10 +36,17 @@ In the preceding command: During backup, a progress bar is displayed in the terminal as shown below. When the progress bar advances to 100%, the backup task is completed and statistics such as total backup time, average backup speed, and backup data size are displayed. +- `total-ranges` indicates the total number of files backed up +- `ranges-succeed` indicates the number of files successfully backed up +- `ranges-failed` indicates the number of files that failed to be backed up +- `backup-total-ranges` indicates the number of tables (including partitions) and indexes backed up +- `write-CF-files` indicates the number of backed up SSTs containing `write CF` data +- `default-CF-files` indicates the number of backed up SSTs containing `default CF` data + ```shell Full Backup <-------------------------------------------------------------------------------> 100.00% Checksum <----------------------------------------------------------------------------------> 100.00% -*** ["Full Backup success summary"] *** [backup-checksum=3.597416ms] [backup-fast-checksum=2.36975ms] *** [total-take=4.715509333s] [BackupTS=435844546560000000] [total-kv=1131] [total-kv-size=250kB] [average-speed=53.02kB/s] [backup-data-size(after-compressed)=71.33kB] [Size=71330] +*** ["Full Backup success summary"] *** [total-ranges=20] [ranges-succeed=20] [ranges-failed=0] [backup-checksum=3.597416ms] [backup-fast-checksum=2.36975ms] [backup-total-ranges=11] [backup-total-regions=10] [write-CF-files=14] [default-CF-files=6] [total-take=4.715509333s] [BackupTS=435844546560000000] [total-kv=1131] [total-kv-size=250kB] [average-speed=53.02kB/s] [backup-data-size(after-compressed)=71.33kB] [Size=71330] ``` ## Get the backup time point of a snapshot backup @@ -78,9 +85,21 @@ tiup br restore full --pd "${PD_IP}:2379" \ During restore, a progress bar is displayed in the terminal as shown below. When the progress bar advances to 100%, the restore task is completed and statistics such as total restore time, average restore speed, and total data size are displayed. +- `total-ranges` indicates the total number of files restored +- `ranges-succeed` indicates the number of files successfully restored +- `ranges-failed` indicates the number of files that failed to be restored +- `merge-ranges` indicates the time taken to merge the data range +- `split-region` indicates the time taken to split and scatter regions +- `restore-files` indicates the time it takes TiKV to download and ingest SSTs +- `write-CF-files` indicates the number of restored SSTs containing `write CF` data +- `default-CF-files` indicates the number of restored SSTs containing `default CF` data +- `split-keys` indicates the number of keys generated for splitting regions + ```shell -Full Restore <------------------------------------------------------------------------------> 100.00% -*** ["Full Restore success summary"] *** [total-take=4.344617542s] [total-kv=5] [total-kv-size=327B] [average-speed=75.27B/s] [restore-data-size(after-compressed)=4.813kB] [Size=4813] [BackupTS=435844901803917314] +Split&Scatter Region <--------------------------------------------------------------------> 100.00% +Download&Ingest SST <---------------------------------------------------------------------> 100.00% +Restore Pipeline <------------------------------------------------------------------------> 100.00% +*** ["Full Restore success summary"] [total-ranges=20] [ranges-succeed=20] [ranges-failed=0] [merge-ranges=7.546971ms] [split-region=343.594072ms] [restore-files=1.57662s] [default-CF-files=6] [write-CF-files=14] [split-keys=9] [total-take=4.344617542s] [total-kv=5] [total-kv-size=327B] [average-speed=75.27B/s] [restore-data-size(after-compressed)=4.813kB] [Size=4813] [BackupTS=435844901803917314] ``` ### Restore a database or a table diff --git a/br/br-snapshot-manual.md b/br/br-snapshot-manual.md index c6805f6ec0342..05b77255f9a9e 100644 --- a/br/br-snapshot-manual.md +++ b/br/br-snapshot-manual.md @@ -176,7 +176,9 @@ In the preceding command: During restore, a progress bar is displayed in the terminal as shown below. When the progress bar advances to 100%, the restore task is completed. Then `br` will verify the restored data to ensure data security. ```shell -Full Restore <---------/...............................................> 17.12%. +Split&Scatter Region <--------------------------------------------------------------------> 100.00% +Download&Ingest SST <---------------------------------------------------------------------> 100.00% +Restore Pipeline <-------------------------/...............................................> 17.12% ``` ## Restore a database or a table From 4ca92311e07665f28a664c1c26492dff7ad0ed15 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Fri, 14 Mar 2025 09:15:19 +0800 Subject: [PATCH 2/4] Apply suggestions from code review --- br/br-snapshot-guide.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/br/br-snapshot-guide.md b/br/br-snapshot-guide.md index 86a70d925079c..0ef1e698827ec 100644 --- a/br/br-snapshot-guide.md +++ b/br/br-snapshot-guide.md @@ -85,15 +85,15 @@ tiup br restore full --pd "${PD_IP}:2379" \ During restore, a progress bar is displayed in the terminal as shown below. When the progress bar advances to 100%, the restore task is completed and statistics such as total restore time, average restore speed, and total data size are displayed. -- `total-ranges` indicates the total number of files restored -- `ranges-succeed` indicates the number of files successfully restored -- `ranges-failed` indicates the number of files that failed to be restored -- `merge-ranges` indicates the time taken to merge the data range -- `split-region` indicates the time taken to split and scatter regions -- `restore-files` indicates the time it takes TiKV to download and ingest SSTs -- `write-CF-files` indicates the number of restored SSTs containing `write CF` data -- `default-CF-files` indicates the number of restored SSTs containing `default CF` data -- `split-keys` indicates the number of keys generated for splitting regions +- `total-ranges`: indicates the total number of files that are restored +- `ranges-succeed`: indicates the number of files that are successfully restored +- `ranges-failed`: indicates the number of files that failed to be restored +- `merge-ranges`: indicates the time taken to merge the data range +- `split-region`: indicates the time taken to split and scatter regions +- `restore-files`: indicates the time it takes TiKV to download and ingest SST files +- `write-CF-files`: indicates the number of restored SST files that contain `write CF` data +- `default-CF-files`: indicates the number of restored SST files that contain `default CF` data +- `split-keys`: indicates the number of keys generated for splitting regions ```shell Split&Scatter Region <--------------------------------------------------------------------> 100.00% From 4a7c70cf802c0d9fdd32e8b59596ca894e28268d Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Fri, 14 Mar 2025 09:15:33 +0800 Subject: [PATCH 3/4] Apply suggestions from code review --- br/br-snapshot-guide.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/br/br-snapshot-guide.md b/br/br-snapshot-guide.md index 0ef1e698827ec..0c6bf19cc87cf 100644 --- a/br/br-snapshot-guide.md +++ b/br/br-snapshot-guide.md @@ -36,12 +36,12 @@ In the preceding command: During backup, a progress bar is displayed in the terminal as shown below. When the progress bar advances to 100%, the backup task is completed and statistics such as total backup time, average backup speed, and backup data size are displayed. -- `total-ranges` indicates the total number of files backed up -- `ranges-succeed` indicates the number of files successfully backed up -- `ranges-failed` indicates the number of files that failed to be backed up -- `backup-total-ranges` indicates the number of tables (including partitions) and indexes backed up -- `write-CF-files` indicates the number of backed up SSTs containing `write CF` data -- `default-CF-files` indicates the number of backed up SSTs containing `default CF` data +- `total-ranges`: indicates the total number of files that are backed up +- `ranges-succeed`: indicates the number of files that are successfully backed up +- `ranges-failed`: indicates the number of files that failed to be backed up +- `backup-total-ranges`: indicates the number of tables (including partitions) and indexes that are backed up +- `write-CF-files`: indicates the number of backup SST files that contain `write CF` data +- `default-CF-files`: indicates the number of backup SST files that contain `default CF` data ```shell Full Backup <-------------------------------------------------------------------------------> 100.00% From 027cf4cc36758d175cb60edd4c1a28e008b308d6 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Mon, 17 Mar 2025 11:50:24 +0800 Subject: [PATCH 4/4] Apply suggestions from code review Co-authored-by: Grace Cai --- br/br-snapshot-guide.md | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/br/br-snapshot-guide.md b/br/br-snapshot-guide.md index 0c6bf19cc87cf..551272a134b8b 100644 --- a/br/br-snapshot-guide.md +++ b/br/br-snapshot-guide.md @@ -36,12 +36,12 @@ In the preceding command: During backup, a progress bar is displayed in the terminal as shown below. When the progress bar advances to 100%, the backup task is completed and statistics such as total backup time, average backup speed, and backup data size are displayed. -- `total-ranges`: indicates the total number of files that are backed up -- `ranges-succeed`: indicates the number of files that are successfully backed up -- `ranges-failed`: indicates the number of files that failed to be backed up -- `backup-total-ranges`: indicates the number of tables (including partitions) and indexes that are backed up -- `write-CF-files`: indicates the number of backup SST files that contain `write CF` data -- `default-CF-files`: indicates the number of backup SST files that contain `default CF` data +- `total-ranges`: indicates the total number of files to be backed up. +- `ranges-succeed`: indicates the number of files that are successfully backed up. +- `ranges-failed`: indicates the number of files that failed to be backed up. +- `backup-total-ranges`: indicates the number of tables (including partitions) and indexes that are to be backed up. +- `write-CF-files`: indicates the number of backup SST files that contain `write CF` data. +- `default-CF-files`: indicates the number of backup SST files that contain `default CF` data. ```shell Full Backup <-------------------------------------------------------------------------------> 100.00% @@ -85,15 +85,15 @@ tiup br restore full --pd "${PD_IP}:2379" \ During restore, a progress bar is displayed in the terminal as shown below. When the progress bar advances to 100%, the restore task is completed and statistics such as total restore time, average restore speed, and total data size are displayed. -- `total-ranges`: indicates the total number of files that are restored -- `ranges-succeed`: indicates the number of files that are successfully restored -- `ranges-failed`: indicates the number of files that failed to be restored -- `merge-ranges`: indicates the time taken to merge the data range -- `split-region`: indicates the time taken to split and scatter regions -- `restore-files`: indicates the time it takes TiKV to download and ingest SST files -- `write-CF-files`: indicates the number of restored SST files that contain `write CF` data -- `default-CF-files`: indicates the number of restored SST files that contain `default CF` data -- `split-keys`: indicates the number of keys generated for splitting regions +- `total-ranges`: indicates the total number of files that are to be restored. +- `ranges-succeed`: indicates the number of files that are successfully restored. +- `ranges-failed`: indicates the number of files that failed to be restored. +- `merge-ranges`: indicates the time taken to merge the data range. +- `split-region`: indicates the time taken to split and scatter Regions. +- `restore-files`: indicates the time TiKV takes to download and ingest SST files. +- `write-CF-files`: indicates the number of restored SST files that contain `write CF` data. +- `default-CF-files`: indicates the number of restored SST files that contain `default CF` data. +- `split-keys`: indicates the number of keys generated for splitting Regions. ```shell Split&Scatter Region <--------------------------------------------------------------------> 100.00%