Skip to content

schemastore: fix cross-schema create view with unqualified source table#5027

Open
lidezhu wants to merge 6 commits into
masterfrom
ldz/fix-schema-store051202
Open

schemastore: fix cross-schema create view with unqualified source table#5027
lidezhu wants to merge 6 commits into
masterfrom
ldz/fix-schema-store051202

Conversation

@lidezhu
Copy link
Copy Markdown
Collaborator

@lidezhu lidezhu commented May 12, 2026

What problem does this PR solve?

Issue Number: close #5026

What is changed and how it works?

This pull request addresses an issue where cross-schema CREATE VIEW statements could fail or be incorrectly processed due to unqualified source table references. By leveraging the normalized SELECT statement stored in the table metadata, the system can now correctly reconstruct the view definition with fully qualified table names. This ensures consistency and accuracy in downstream replication for views that span multiple schemas.

Highlights

  • Cross-Schema View Normalization: Introduced logic to normalize CREATE VIEW queries by utilizing the stored SELECT statement from the table metadata, ensuring that cross-schema table references are correctly resolved.
  • AST Table Schema Extraction: Added a helper utility to extract schema qualifiers from AST nodes, allowing the system to identify when a view's SELECT statement references tables outside the current schema.
  • Integration Testing: Expanded integration tests to verify that cross-schema views are correctly handled and that original queries are preserved when only the current schema is involved.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Questions

Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?

Release note

Please refer to [Release Notes Language Style Guide](https://pingcap.github.io/tidb-dev-guide/contribute-to-tidb/release-notes-style-guide.html) to write a quality release note.

If you don't think this PR needs a release note then fill it with `None`.

Summary by CodeRabbit

  • New Features

    • Enhanced persistence and replication of database views
    • Improved handling for views that reference tables across different schemas with proper DDL query normalization
  • Tests

    • Added validation tests for view creation DDL events
    • Added test coverage for schema extraction and query normalization logic

Review Change Stack

@ti-chi-bot
Copy link
Copy Markdown

ti-chi-bot Bot commented May 12, 2026

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@ti-chi-bot ti-chi-bot Bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. labels May 12, 2026
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 12, 2026

📝 Walkthrough

Walkthrough

This PR fixes TiCDC's replication of CREATE VIEW statements that reference unqualified table names across database boundaries. It adds schema extraction utilities, normalization logic to restore database qualifiers in view DDL, and integration tests to validate the fix.

Changes

CREATE VIEW Normalization for Cross-Schema References

Layer / File(s) Summary
Table schema extraction utility
logservice/schemastore/utils.go, logservice/schemastore/utils_test.go
Added AST visitor tableSchemaExtractor and extractTableSchemas(ast.Node) to extract schema qualifiers from table references in SQL statements, identifying which schemas a view's SELECT statement references.
CREATE VIEW normalization and qualification check
logservice/schemastore/persist_storage_ddl_handlers.go, logservice/schemastore/persist_storage_test.go
buildPersistedDDLEventForCreateView now calls normalizeCreateViewQueryWithStoredSelect to replace the CREATE VIEW statement's SELECT clause with the fully parsed stored select when the view references tables across multiple schemas, preserving database qualifiers and preventing downstream resolution errors. Two unit tests validate normalization with cross-schema and same-schema view scenarios.
Integration test setup and downstream verification
tests/integration_tests/common_1/data/test.sql, tests/integration_tests/common_1/data/test_finish.sql, tests/integration_tests/common_1/run.sh
Added a users table and a cross-schema view (common.v selecting from users), with downstream existence checks to validate the normalization produces correct replicated objects.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

  • pingcap/ticdc#4270: Both PRs modify the schemastore DDL event persist pipeline by adding/updating builders in persist_storage_ddl_handlers.go and related types.

Suggested reviewers

  • wk989898
  • tenfyzhong

Poem

🐰 A view's tables were lost in the haze,
Unqualified names in cross-schema maze,
Now schemas restored, the path becomes clear,
Qualified references replicate near! 🌟

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ldz/fix-schema-store051202

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.12.2)

Command failed

Warning

Review ran into problems

🔥 Problems

Timed out fetching pipeline failures after 30000ms


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@ti-chi-bot ti-chi-bot Bot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label May 12, 2026
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces integration tests for cross-schema views by creating a table in one database and a corresponding view in another. It updates the test configuration and execution scripts to verify these objects in the downstream environment. A reviewer suggested enhancing the test by inserting sample data into the source table to verify that data is correctly replicated and accessible through the cross-schema view.

Comment thread tests/integration_tests/common_1/data/test.sql Outdated
@ti-chi-bot ti-chi-bot Bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. do-not-merge/needs-linked-issue labels May 12, 2026
@lidezhu lidezhu changed the title add test schemastore: fix cross-schema create view with unqualified source table May 12, 2026
@ti-chi-bot ti-chi-bot Bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels May 12, 2026
@lidezhu lidezhu marked this pull request as ready for review May 12, 2026 09:01
@ti-chi-bot ti-chi-bot Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 12, 2026
@lidezhu
Copy link
Copy Markdown
Collaborator Author

lidezhu commented May 12, 2026

/gemini review

@lidezhu
Copy link
Copy Markdown
Collaborator Author

lidezhu commented May 12, 2026

@coderabbitai review

@lidezhu
Copy link
Copy Markdown
Collaborator Author

lidezhu commented May 12, 2026

/gemini summary

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 12, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements normalization for CREATE VIEW DDL events by utilizing TiDB's stored SelectStmt to ensure table references are fully resolved, particularly for cross-schema views. It introduces a new AST visitor to extract schema names and includes comprehensive unit and integration tests. Feedback was provided to improve the efficiency of the normalization logic by reusing a single parser instance for multiple parsing operations within the same function.

Comment thread logservice/schemastore/persist_storage_ddl_handlers.go
@gemini-code-assist
Copy link
Copy Markdown

Summary of Changes

This pull request addresses an issue where cross-schema CREATE VIEW statements could fail or be incorrectly processed due to unqualified source table references. By leveraging the normalized SELECT statement stored in the table metadata, the system can now correctly reconstruct the view definition with fully qualified table names. This ensures consistency and accuracy in downstream replication for views that span multiple schemas.

Highlights

  • Cross-Schema View Normalization: Introduced logic to normalize CREATE VIEW queries by utilizing the stored SELECT statement from the table metadata, ensuring that cross-schema table references are correctly resolved.
  • AST Table Schema Extraction: Added a helper utility to extract schema qualifiers from AST nodes, allowing the system to identify when a view's SELECT statement references tables outside the current schema.
  • Integration Testing: Expanded integration tests to verify that cross-schema views are correctly handled and that original queries are preserved when only the current schema is involved.
New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Activity
  • Pull request created by lidezhu.
  • Initial automated review triggered by CodeRabbit.
  • Review comment provided by gemini-code-assist regarding test data coverage.
  • Review comment provided by gemini-code-assist regarding parser instance reuse.
  • Author requested summary and review.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/integration_tests/common_1/run.sh`:
- Around line 68-69: The two calls to check_table_exists use unquoted expansions
which can break on word-splitting/globbing; update the calls to quote each
argument so they pass literal table names and host/port safely, e.g. call
check_table_exists "common_1.users" "$DOWN_TIDB_HOST" "$DOWN_TIDB_PORT" and
check_table_exists "common.v" "$DOWN_TIDB_HOST" "$DOWN_TIDB_PORT" (identify the
two occurrences of check_table_exists in the diff and add quotes around the
table names and the $DOWN_TIDB_HOST/$DOWN_TIDB_PORT expansions).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 92823661-0c11-449d-a2a3-fcadbb56e18b

📥 Commits

Reviewing files that changed from the base of the PR and between ba9fcf5 and 9951108.

📒 Files selected for processing (7)
  • logservice/schemastore/persist_storage_ddl_handlers.go
  • logservice/schemastore/persist_storage_test.go
  • logservice/schemastore/utils.go
  • logservice/schemastore/utils_test.go
  • tests/integration_tests/common_1/data/test.sql
  • tests/integration_tests/common_1/data/test_finish.sql
  • tests/integration_tests/common_1/run.sh

Comment thread tests/integration_tests/common_1/run.sh
@lidezhu
Copy link
Copy Markdown
Collaborator Author

lidezhu commented May 12, 2026

/test all

@lidezhu
Copy link
Copy Markdown
Collaborator Author

lidezhu commented May 12, 2026

/retest

@ti-chi-bot
Copy link
Copy Markdown

ti-chi-bot Bot commented May 12, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: 3AceShowHand

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label May 12, 2026
@ti-chi-bot
Copy link
Copy Markdown

ti-chi-bot Bot commented May 12, 2026

[LGTM Timeline notifier]

Timeline:

  • 2026-05-12 11:01:04.121012769 +0000 UTC m=+177032.653792088: ☑️ agreed by 3AceShowHand.

@ti-chi-bot ti-chi-bot Bot added the approved label May 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved needs-1-more-lgtm Indicates a PR needs 1 more LGTM. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CDC may fail to replicate CREATE VIEW with unqualified table references

2 participants