Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 18 additions & 3 deletions dm/dm-compatibility-catalog.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,25 @@ DM supports migrating data from different sources to TiDB clusters. Based on the
| MariaDB 10.1.2 ~ 10.5.10 | Experimental | |
| MariaDB > 10.5.10 | Not tested | Expected to work in most cases after bypassing the [precheck](/dm/dm-precheck.md). See [MariaDB notes](#mariadb-notes). |

### Incompatibility with foreign key CASCADE operations
### Foreign key `CASCADE` operations

- DM creates foreign key **constraints** on the target, but they are not enforced while applying transactions because DM sets the session variable [`foreign_key_checks=OFF`](/system-variables.md#foreign_key_checks).
- DM does **not** support `ON DELETE CASCADE` or `ON UPDATE CASCADE` behavior by default, and enabling `foreign_key_checks` via a DM task session variable is not recommended. If your workload relies on cascades, **do not assume** that cascade effects will be replicated.
Starting from v8.5.6, DM has **experimental** support for replicating tables with foreign key constraints. This includes two improvements:

- **Safe mode**: DM sets `foreign_key_checks=0` per batch during safe mode execution and skips the redundant `DELETE` for `UPDATE` statements that do not change primary key or unique key values. This prevents `REPLACE INTO` (internally `DELETE` + `INSERT`) from triggering unintended `ON DELETE CASCADE` side effects on child rows. For details, see [DM Safe Mode](/dm/dm-safe-mode.md#foreign-key-handling-experimental).
- **Multi-worker causality**: When `worker-count > 1`, DM discovers foreign key relations from the downstream schema at task start and injects causality keys so that parent row DML operations complete before dependent child row operations, preserving binlog order across workers.

> **Warning:**
>
> This feature is experimental. The following constraints apply:
>
> - `UPDATE` statements that change primary key or unique key values in safe mode are rejected. The task pauses with the error: `safe-mode update with foreign_key_checks=1 and PK/UK changes is not supported`. To replicate these UPDATEs, use `safe-mode: false`.
> - DDL operations that create, alter, or drop foreign key constraints during replication are rejected when `foreign_key_checks=1`.
> - Table routing combined with `worker-count > 1` is not supported. When using table routing with foreign key tables, you must set `worker-count` to `1`.
> - Your block-allow-list must include all ancestor tables in the foreign key chain. If you filter out ancestor tables, the task pauses with an error during incremental sync.
> - Your source and downstream foreign key metadata must match. If you detect a mismatch, use `binlog-schema update --from-target` to resync.
> - `ON UPDATE CASCADE` is not replicated correctly in safe mode when the `UPDATE` changes a primary key or unique key value. DM rewrites such UPDATEs as `DELETE` + `REPLACE`, which would trigger `ON DELETE` actions instead of `ON UPDATE` actions. The guardrail rejects the PK/UK change and pauses the task. Non-key `UPDATE` statements on tables with `ON UPDATE CASCADE` replicate without issues.

In versions earlier than v8.5.6, DM creates foreign key constraints on the target but does not enforce them because it sets the session variable [`foreign_key_checks=OFF`](/system-variables.md#foreign_key_checks). Cascading operations are not replicated, so do not assume that cascade effects will be applied in the downstream.

### MariaDB notes

Expand Down
2 changes: 1 addition & 1 deletion dm/dm-precheck.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ Regardless of the migration mode you choose, the precheck always includes the fo

- Compatibility of the upstream MySQL table schema

- Check whether the upstream tables have foreign keys, which are not supported by TiDB. A warning is returned if a foreign key is found in the precheck.
- Checks whether the upstream tables have foreign keys. TiDB supports foreign keys (GA since v8.5.0), and DM has experimental support for replicating tables with foreign key constraints starting from v8.5.6. DM returns a warning if it finds a foreign key in the precheck. For details on supported scenarios and limitations, see [DM Compatibility Catalog](/dm/dm-compatibility-catalog.md#foreign-key-cascade-operations).
- Check whether the upstream tables use character sets that are incompatible with TiDB. For more information, see [TiDB Supported Character Sets](/character-set-and-collation.md).
- Check whether the upstream tables have primary key constraints or unique key constraints (introduced from v1.0.7).

Expand Down
49 changes: 48 additions & 1 deletion dm/dm-safe-mode.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ In safe mode, DM can rewrite SQL statements to resolve the preceding issues.
In safe mode, DM guarantees the idempotency of binlog events by rewriting SQL statements. Specifically, the following SQL statements are rewritten:

* `INSERT` statements are rewritten to `REPLACE` statements.
* `UPDATE` statements are analyzed to obtain the value of the primary key or the unique index of the row updated. `UPDATE` statements are then rewritten to `DELETE` + `REPLACE` statements in the following two steps: DM deletes the old record using the primary key or unique index, and inserts the new record using the `REPLACE` statement.
* `UPDATE` statements are analyzed to obtain the value of the primary key or the unique index of the row updated. `UPDATE` statements are then rewritten to `DELETE` + `REPLACE` statements in the following two steps: DM deletes the old record using the primary key or unique index, and inserts the new record using the `REPLACE` statement. Starting from v8.5.6, when you set `foreign_key_checks=1` in the task session, DM skips the `DELETE` step for `UPDATE` statements that do not change any primary key or unique key value. See [Foreign key handling](#foreign-key-handling-experimental) for details.

`REPLACE` is a MySQL-specific syntax for inserting data. When you insert data using `REPLACE`, and the new data and existing data have a primary key or unique constraint conflict, MySQL deletes all the conflicting records and executes the insert operation, which is equivalent to "force insert". For details, see [`REPLACE` statement](https://dev.mysql.com/doc/refman/8.0/en/replace.html) in MySQL documentation.

Expand Down Expand Up @@ -91,6 +91,53 @@ mysql-instances:
syncer-config-name: "global" # Name of the syncers configuration.
```

## Foreign key handling (experimental)

> **Note:**
>
> This feature is available starting from v8.5.6 and is experimental.

When you enable safe mode and set `foreign_key_checks=1` in the downstream task session, the standard `DELETE` + `REPLACE` rewrite for `UPDATE` statements can trigger unintended `ON DELETE CASCADE` side effects on child rows. Starting from v8.5.6, DM includes two improvements to address this:

### Non-key `UPDATE` optimization

For `UPDATE` statements that do not change any primary key or unique key value, DM skips the `DELETE` step and emits only `REPLACE INTO`. Because the primary key is unchanged, `REPLACE INTO` overwrites the existing row without triggering a cascade delete. This optimization applies automatically in safe mode.

For example, given the following upstream statement where `id` is the primary key:

```sql
UPDATE dummydb.dummytbl SET int_value = 888999 WHERE id = 123;
```

In previous versions, safe mode rewrites this as:

```sql
DELETE FROM dummydb.dummytbl WHERE id = 123; -- triggers ON DELETE CASCADE
REPLACE INTO dummydb.dummytbl (id, int_value, ...) VALUES (123, 888999, ...);
```

Starting from v8.5.6, safe mode rewrites this as:

```sql
REPLACE INTO dummydb.dummytbl (id, int_value, ...) VALUES (123, 888999, ...); -- no cascade
```

> **Warning:**
>
> `UPDATE` statements that change primary key or unique key values are rejected by a guardrail when `foreign_key_checks=1`. The task pauses with the error: `safe-mode update with foreign_key_checks=1 and PK/UK changes is not supported`. To replicate PK/UK-changing UPDATEs on tables with foreign keys, use `safe-mode: false`.

### Session-level `foreign_key_checks`

During safe mode batch execution, DM sets `SET SESSION foreign_key_checks=0` before executing `INSERT` and `UPDATE` batches, and restores the original value afterward. This prevents `REPLACE INTO` (internally `DELETE` + `INSERT`) from triggering cascade operations in the downstream.

The session variable toggle adds a small overhead per batch flush (two `SET SESSION` round-trips). This overhead is negligible for most workloads.

### Multi-worker foreign key causality

When using `worker-count > 1` with foreign key tables, DM discovers foreign key relations from the downstream `CREATE TABLE` schema at task start. For each DML operation, DM injects causality keys derived from the foreign key relationships, ensuring that parent row operations and dependent child row operations are assigned to the same DML worker queue. This preserves binlog order across workers and prevents foreign key constraint violations.

For detailed constraints, see [DM Compatibility Catalog](/dm/dm-compatibility-catalog.md#foreign-key-cascade-operations).

## Notes for safe mode

If you want to enable safe mode during the entire replication process for safety reasons, be aware of the following:
Expand Down
2 changes: 1 addition & 1 deletion foreign-key.md
Original file line number Diff line number Diff line change
Expand Up @@ -303,7 +303,7 @@ Create Table | CREATE TABLE `child` (

<CustomContent platform="tidb">

- [DM](/dm/dm-overview.md) does not support foreign keys. DM disables the [`foreign_key_checks`](/system-variables.md#foreign_key_checks) of the downstream TiDB when replicating data to TiDB. Therefore, the cascading operations caused by foreign keys are not replicated from the upstream to the downstream, which might cause data inconsistency.
- [DM](/dm/dm-overview.md): Starting from v8.5.6, DM has experimental support for replicating tables with foreign key constraints. In safe mode, DM sets `foreign_key_checks=0` per batch and optimizes non-key `UPDATE` rewrites to prevent unintended `ON DELETE CASCADE` side effects. When you use `worker-count > 1`, DM injects causality keys so that parent row operations complete before child row operations. This feature is experimental; DM rejects PK/UK-changing UPDATEs and FK-related DDL in safe mode. For the full list of constraints, see [DM Safe Mode](/dm/dm-safe-mode.md#foreign-key-handling-experimental) and [DM Compatibility Catalog](/dm/dm-compatibility-catalog.md#foreign-key-cascade-operations). In versions earlier than v8.5.6, DM disables [`foreign_key_checks`](/system-variables.md#foreign_key_checks) when replicating data to TiDB, so cascading operations are not replicated from upstream to downstream.
- [TiCDC](/ticdc/ticdc-overview.md) v6.6.0 is compatible with foreign keys. The previous versions of TiCDC might report an error when replicating tables with foreign keys. It is recommended to disable the `foreign_key_checks` of the downstream TiDB cluster when using a TiCDC version earlier than v6.6.0.
- [BR](/br/backup-and-restore-overview.md) v6.6.0 is compatible with foreign keys. The previous versions of BR might report an error when restoring tables with foreign keys to a v6.6.0 or later cluster. It is recommended to disable the `foreign_key_checks` of the downstream TiDB cluster before restoring the cluster when using a BR earlier than v6.6.0.
- When you use [TiDB Lightning](/tidb-lightning/tidb-lightning-overview.md), if the target table uses a foreign key, it is recommended to disable the `foreign_key_checks` of the downstream TiDB cluster before importing data. For versions earlier than v6.6.0, disabling this system variable does not take effect, and you need to grant the `REFERENCES` privilege for the downstream database user, or manually create the target table in the downstream database in advance to ensure smooth data import.
Expand Down
Loading