feat: Implement row filter functionality#1294
Conversation
Introduces support for Unity Catalog row filters across table, incremental, materialized view, and streaming table materializations to enable fine-grained row-level security ### Checklist - [x] Code has been tested in development - [x] Relevant tests have been added - [ ] CHANGELOG.md has been updated with this feature
sd-db
changed the title
Coverage report
Click to see where and how coverage changed
This report was generated by python-coverage-comment-action |
||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Let's target the 1.12.latest branch since this is a significant feature. |
|
|
||
| {#-- ===== CREATE CLAUSE MACROS ===== --#} | ||
|
|
||
| {%- macro qualify_row_filter_function(function, relation) -%} |
There was a problem hiding this comment.
We should probably not do this in a macro, rather in python.
There was a problem hiding this comment.
We already do have this in Python as well for the ALTER flow, issue is with tables which require setting row-filters in the CREATE flow, which is why the duplication of code. I have updated the comment to note this more specifically as well. I don't think it is possible but if there is a cleaner approach let me know will explore...
There was a problem hiding this comment.
we can do pretty much anything in python by putting the method on the adapter, and then calling it from jinja with adapter.<method_name>. You can keep this sort of stuff as jinja, as there is no existing consistency as far as what MUST be jinja or MUST be python, other than Jinja templates the final SQL. Your call.
- Refactor materialization logic to not use incremental_processing for tables. - Add changes to properly handle case of dropping row-filters when there is a change in the model. - Misc fixes around review comments - Update tests to cover new functionality.
2 participants
Introduces support for Unity Catalog row filters across table, incremental, materialized view, and streaming table materializations to enable fine-grained row-level security
One Pager/PRD [Internal Doc]
Implementation Plan [Internal Doc]
Testing
Checklist