Skip to content

Support Databricks query tags in session properties#5794

Open
WhammyLeaf wants to merge 1 commit into
SQLMesh:mainfrom
trygforsikring:add-databricks-query-tags-session-properties
Open

Support Databricks query tags in session properties#5794
WhammyLeaf wants to merge 1 commit into
SQLMesh:mainfrom
trygforsikring:add-databricks-query-tags-session-properties

Conversation

@WhammyLeaf
Copy link
Copy Markdown

@WhammyLeaf WhammyLeaf commented May 15, 2026

Description

The Databricks SQL connector supports per-statement query_tags, but SQLMesh did not previously pass model/session-level session_properties.query_tags through to the connector. This change stores validated query tags for the active adapter session and forwards them to Databricks SQL connector executions.

Query tags are only applied on the Databricks SQL connector path. They are not passed when execution is routed through Spark session / Databricks Connect paths, since those cursors do not support the query_tags argument.

ISSUE: #5795

Test Plan

  • Added Databricks engine adapter tests covering:
    • forwarding session query tags to Databricks SQL connector executions
    • clearing query tags after the session ends
    • preserving explicit per-statement query_tags
    • accepting NULL query tag values
    • rejecting invalid session_properties.query_tags
    • not applying query tags to Spark session / Databricks Connect execution paths
  • Verified the installed Databricks SQL connector exposes Cursor.execute(..., query_tags=...).
  • Built the package wheel successfully.
  • Ran:
pytest -q -o addopts='' tests/core/engine_adapter/test_databricks.py
python -m pip wheel . --no-deps -w /tmp/sqlmesh-wheel-final

@WhammyLeaf WhammyLeaf force-pushed the add-databricks-query-tags-session-properties branch 3 times, most recently from ad8c237 to d9881ce Compare May 15, 2026 15:05
@WhammyLeaf
Copy link
Copy Markdown
Author

Will fix the failing style tests.

Signed-off-by: Christian Troelsen <christian.troelsen@tryg.dk>
@WhammyLeaf WhammyLeaf force-pushed the add-databricks-query-tags-session-properties branch from d9881ce to 17b3af7 Compare May 15, 2026 18:19
@WhammyLeaf
Copy link
Copy Markdown
Author

@izeigerman Perhaps you can have a look at this 😄 ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant