diff --git a/dm/dm-compatibility-catalog.md b/dm/dm-compatibility-catalog.md index c7a28d31ed256..293b0bae56797 100644 --- a/dm/dm-compatibility-catalog.md +++ b/dm/dm-compatibility-catalog.md @@ -27,10 +27,25 @@ DM supports migrating data from different sources to TiDB clusters. Based on the | MariaDB 10.1.2 ~ 10.5.10 | Experimental | | | MariaDB > 10.5.10 | Not tested | Expected to work in most cases after bypassing the [precheck](/dm/dm-precheck.md). See [MariaDB notes](#mariadb-notes). | -### Incompatibility with foreign key CASCADE operations +### Foreign key `CASCADE` operations -- DM creates foreign key **constraints** on the target, but they are not enforced while applying transactions because DM sets the session variable [`foreign_key_checks=OFF`](/system-variables.md#foreign_key_checks). -- DM does **not** support `ON DELETE CASCADE` or `ON UPDATE CASCADE` behavior by default, and enabling `foreign_key_checks` via a DM task session variable is not recommended. If your workload relies on cascades, **do not assume** that cascade effects will be replicated. +Starting from v8.5.6, DM has **experimental** support for replicating tables with foreign key constraints. This includes two improvements: + +- **Safe mode**: DM sets `foreign_key_checks=0` per batch during safe mode execution and skips the redundant `DELETE` for `UPDATE` statements that do not change primary key or unique key values. This prevents `REPLACE INTO` (internally `DELETE` + `INSERT`) from triggering unintended `ON DELETE CASCADE` side effects on child rows. For details, see [DM Safe Mode](/dm/dm-safe-mode.md#foreign-key-handling-experimental). +- **Multi-worker causality**: When `worker-count > 1`, DM discovers foreign key relations from the downstream schema at task start and injects causality keys so that parent row DML operations complete before dependent child row operations, preserving binlog order across workers. + +> **Warning:** +> +> This feature is experimental. The following constraints apply: +> +> - `UPDATE` statements that change primary key or unique key values in safe mode are rejected. The task pauses with the error: `safe-mode update with foreign_key_checks=1 and PK/UK changes is not supported`. To replicate these UPDATEs, use `safe-mode: false`. +> - DDL operations that create, alter, or drop foreign key constraints during replication are rejected when `foreign_key_checks=1`. +> - Table routing combined with `worker-count > 1` is not supported. When using table routing with foreign key tables, you must set `worker-count` to `1`. +> - Your block-allow-list must include all ancestor tables in the foreign key chain. If you filter out ancestor tables, the task pauses with an error during incremental sync. +> - Your source and downstream foreign key metadata must match. If you detect a mismatch, use `binlog-schema update --from-target` to resync. +> - `ON UPDATE CASCADE` is not replicated correctly in safe mode when the `UPDATE` changes a primary key or unique key value. DM rewrites such UPDATEs as `DELETE` + `REPLACE`, which would trigger `ON DELETE` actions instead of `ON UPDATE` actions. The guardrail rejects the PK/UK change and pauses the task. Non-key `UPDATE` statements on tables with `ON UPDATE CASCADE` replicate without issues. + +In versions earlier than v8.5.6, DM creates foreign key constraints on the target but does not enforce them because it sets the session variable [`foreign_key_checks=OFF`](/system-variables.md#foreign_key_checks). Cascading operations are not replicated, so do not assume that cascade effects will be applied in the downstream. ### MariaDB notes diff --git a/dm/dm-precheck.md b/dm/dm-precheck.md index 48b41f16a6e09..e1f7ab2b16b57 100644 --- a/dm/dm-precheck.md +++ b/dm/dm-precheck.md @@ -52,7 +52,7 @@ Regardless of the migration mode you choose, the precheck always includes the fo - Compatibility of the upstream MySQL table schema - - Check whether the upstream tables have foreign keys, which are not supported by TiDB. A warning is returned if a foreign key is found in the precheck. + - Checks whether the upstream tables have foreign keys. TiDB supports foreign keys (GA since v8.5.0), and DM has experimental support for replicating tables with foreign key constraints starting from v8.5.6. DM returns a warning if it finds a foreign key in the precheck. For details on supported scenarios and limitations, see [DM Compatibility Catalog](/dm/dm-compatibility-catalog.md#foreign-key-cascade-operations). - Check whether the upstream tables use character sets that are incompatible with TiDB. For more information, see [TiDB Supported Character Sets](/character-set-and-collation.md). - Check whether the upstream tables have primary key constraints or unique key constraints (introduced from v1.0.7). diff --git a/dm/dm-safe-mode.md b/dm/dm-safe-mode.md index 52b7dd10c169d..187a5d00d81c6 100644 --- a/dm/dm-safe-mode.md +++ b/dm/dm-safe-mode.md @@ -22,7 +22,7 @@ In safe mode, DM can rewrite SQL statements to resolve the preceding issues. In safe mode, DM guarantees the idempotency of binlog events by rewriting SQL statements. Specifically, the following SQL statements are rewritten: * `INSERT` statements are rewritten to `REPLACE` statements. -* `UPDATE` statements are analyzed to obtain the value of the primary key or the unique index of the row updated. `UPDATE` statements are then rewritten to `DELETE` + `REPLACE` statements in the following two steps: DM deletes the old record using the primary key or unique index, and inserts the new record using the `REPLACE` statement. +* `UPDATE` statements are analyzed to obtain the value of the primary key or the unique index of the row updated. `UPDATE` statements are then rewritten to `DELETE` + `REPLACE` statements in the following two steps: DM deletes the old record using the primary key or unique index, and inserts the new record using the `REPLACE` statement. Starting from v8.5.6, when you set `foreign_key_checks=1` in the task session, DM skips the `DELETE` step for `UPDATE` statements that do not change any primary key or unique key value. See [Foreign key handling](#foreign-key-handling-experimental) for details. `REPLACE` is a MySQL-specific syntax for inserting data. When you insert data using `REPLACE`, and the new data and existing data have a primary key or unique constraint conflict, MySQL deletes all the conflicting records and executes the insert operation, which is equivalent to "force insert". For details, see [`REPLACE` statement](https://dev.mysql.com/doc/refman/8.0/en/replace.html) in MySQL documentation. @@ -91,6 +91,53 @@ mysql-instances: syncer-config-name: "global" # Name of the syncers configuration. ``` +## Foreign key handling (experimental) + +> **Note:** +> +> This feature is available starting from v8.5.6 and is experimental. + +When you enable safe mode and set `foreign_key_checks=1` in the downstream task session, the standard `DELETE` + `REPLACE` rewrite for `UPDATE` statements can trigger unintended `ON DELETE CASCADE` side effects on child rows. Starting from v8.5.6, DM includes two improvements to address this: + +### Non-key `UPDATE` optimization + +For `UPDATE` statements that do not change any primary key or unique key value, DM skips the `DELETE` step and emits only `REPLACE INTO`. Because the primary key is unchanged, `REPLACE INTO` overwrites the existing row without triggering a cascade delete. This optimization applies automatically in safe mode. + +For example, given the following upstream statement where `id` is the primary key: + +```sql +UPDATE dummydb.dummytbl SET int_value = 888999 WHERE id = 123; +``` + +In previous versions, safe mode rewrites this as: + +```sql +DELETE FROM dummydb.dummytbl WHERE id = 123; -- triggers ON DELETE CASCADE +REPLACE INTO dummydb.dummytbl (id, int_value, ...) VALUES (123, 888999, ...); +``` + +Starting from v8.5.6, safe mode rewrites this as: + +```sql +REPLACE INTO dummydb.dummytbl (id, int_value, ...) VALUES (123, 888999, ...); -- no cascade +``` + +> **Warning:** +> +> `UPDATE` statements that change primary key or unique key values are rejected by a guardrail when `foreign_key_checks=1`. The task pauses with the error: `safe-mode update with foreign_key_checks=1 and PK/UK changes is not supported`. To replicate PK/UK-changing UPDATEs on tables with foreign keys, use `safe-mode: false`. + +### Session-level `foreign_key_checks` + +During safe mode batch execution, DM sets `SET SESSION foreign_key_checks=0` before executing `INSERT` and `UPDATE` batches, and restores the original value afterward. This prevents `REPLACE INTO` (internally `DELETE` + `INSERT`) from triggering cascade operations in the downstream. + +The session variable toggle adds a small overhead per batch flush (two `SET SESSION` round-trips). This overhead is negligible for most workloads. + +### Multi-worker foreign key causality + +When using `worker-count > 1` with foreign key tables, DM discovers foreign key relations from the downstream `CREATE TABLE` schema at task start. For each DML operation, DM injects causality keys derived from the foreign key relationships, ensuring that parent row operations and dependent child row operations are assigned to the same DML worker queue. This preserves binlog order across workers and prevents foreign key constraint violations. + +For detailed constraints, see [DM Compatibility Catalog](/dm/dm-compatibility-catalog.md#foreign-key-cascade-operations). + ## Notes for safe mode If you want to enable safe mode during the entire replication process for safety reasons, be aware of the following: diff --git a/foreign-key.md b/foreign-key.md index aea4246cc9cc8..34aa626149292 100644 --- a/foreign-key.md +++ b/foreign-key.md @@ -303,7 +303,7 @@ Create Table | CREATE TABLE `child` ( -- [DM](/dm/dm-overview.md) does not support foreign keys. DM disables the [`foreign_key_checks`](/system-variables.md#foreign_key_checks) of the downstream TiDB when replicating data to TiDB. Therefore, the cascading operations caused by foreign keys are not replicated from the upstream to the downstream, which might cause data inconsistency. +- [DM](/dm/dm-overview.md): Starting from v8.5.6, DM has experimental support for replicating tables with foreign key constraints. In safe mode, DM sets `foreign_key_checks=0` per batch and optimizes non-key `UPDATE` rewrites to prevent unintended `ON DELETE CASCADE` side effects. When you use `worker-count > 1`, DM injects causality keys so that parent row operations complete before child row operations. This feature is experimental; DM rejects PK/UK-changing UPDATEs and FK-related DDL in safe mode. For the full list of constraints, see [DM Safe Mode](/dm/dm-safe-mode.md#foreign-key-handling-experimental) and [DM Compatibility Catalog](/dm/dm-compatibility-catalog.md#foreign-key-cascade-operations). In versions earlier than v8.5.6, DM disables [`foreign_key_checks`](/system-variables.md#foreign_key_checks) when replicating data to TiDB, so cascading operations are not replicated from upstream to downstream. - [TiCDC](/ticdc/ticdc-overview.md) v6.6.0 is compatible with foreign keys. The previous versions of TiCDC might report an error when replicating tables with foreign keys. It is recommended to disable the `foreign_key_checks` of the downstream TiDB cluster when using a TiCDC version earlier than v6.6.0. - [BR](/br/backup-and-restore-overview.md) v6.6.0 is compatible with foreign keys. The previous versions of BR might report an error when restoring tables with foreign keys to a v6.6.0 or later cluster. It is recommended to disable the `foreign_key_checks` of the downstream TiDB cluster before restoring the cluster when using a BR earlier than v6.6.0. - When you use [TiDB Lightning](/tidb-lightning/tidb-lightning-overview.md), if the target table uses a foreign key, it is recommended to disable the `foreign_key_checks` of the downstream TiDB cluster before importing data. For versions earlier than v6.6.0, disabling this system variable does not take effect, and you need to grant the `REFERENCES` privilege for the downstream database user, or manually create the target table in the downstream database in advance to ensure smooth data import.