diff --git a/doc/user/content/headless/materialize-window-functions.md b/doc/user/content/headless/materialize-window-functions.md new file mode 100644 index 0000000000000..4e11449c0ac8a --- /dev/null +++ b/doc/user/content/headless/materialize-window-functions.md @@ -0,0 +1,18 @@ +--- +headless: true +--- +For indexed views and materialized views that contain [window +functions](/sql/functions/#window-functions) (including aggregate functions used +with an `OVER` clause), when an input record in a partition is +added/removed/changed, Materialize **recomputes the results from scratch** for +that partition (instead of using incremental computation). + +The `PARTITION BY` clause of your window function determines your partitions. If +`PARTITION BY` is omitted, all records belong to a single partition (i.e., any +record change results in a recomputation from scratch over the whole input). + +To avoid performance issues that may arise as the number of records grows, +consider rewriting your indexed views and materialized views to use idiomatic +Materialize SQL instead of window functions. If your view definitions cannot be +rewritten without the window functions and the performance of window functions +is insufficient for your use case, please [contact our team](/support/). diff --git a/doc/user/content/transform-data/idiomatic-materialize-sql/appendix/idiomatic-sql-chart.md b/doc/user/content/transform-data/idiomatic-materialize-sql/appendix/idiomatic-sql-chart.md index e0797b6985c50..996ce44af9003 100644 --- a/doc/user/content/transform-data/idiomatic-materialize-sql/appendix/idiomatic-sql-chart.md +++ b/doc/user/content/transform-data/idiomatic-materialize-sql/appendix/idiomatic-sql-chart.md @@ -28,7 +28,7 @@ performance. ### Materialize and window functions -{{< idiomatic-sql/materialize-window-functions >}} +{{% include-headless "/headless/materialize-window-functions" %}} {{}} diff --git a/doc/user/content/transform-data/idiomatic-materialize-sql/appendix/window-function-to-materialize.md b/doc/user/content/transform-data/idiomatic-materialize-sql/appendix/window-function-to-materialize.md index ecd9eee896468..189d866174886 100644 --- a/doc/user/content/transform-data/idiomatic-materialize-sql/appendix/window-function-to-materialize.md +++ b/doc/user/content/transform-data/idiomatic-materialize-sql/appendix/window-function-to-materialize.md @@ -22,7 +22,7 @@ can provide better performance. ### Materialize and window functions -{{< idiomatic-sql/materialize-window-functions >}} +{{% include-headless "/headless/materialize-window-functions" %}} {{}} diff --git a/doc/user/content/transform-data/idiomatic-materialize-sql/first-value.md b/doc/user/content/transform-data/idiomatic-materialize-sql/first-value.md index 99abc8e669b01..689b89ccc6f7c 100644 --- a/doc/user/content/transform-data/idiomatic-materialize-sql/first-value.md +++ b/doc/user/content/transform-data/idiomatic-materialize-sql/first-value.md @@ -17,7 +17,7 @@ to some ordering, in each group. ### Materialize and window functions -{{< idiomatic-sql/materialize-window-functions >}} +{{% include-headless "/headless/materialize-window-functions" %}} {{}} diff --git a/doc/user/content/transform-data/idiomatic-materialize-sql/lag.md b/doc/user/content/transform-data/idiomatic-materialize-sql/lag.md index 6d151cf24cee4..b1f7aad0269a0 100644 --- a/doc/user/content/transform-data/idiomatic-materialize-sql/lag.md +++ b/doc/user/content/transform-data/idiomatic-materialize-sql/lag.md @@ -22,7 +22,7 @@ function. ### Materialize and window functions -{{< idiomatic-sql/materialize-window-functions >}} +{{% include-headless "/headless/materialize-window-functions" %}} {{}} diff --git a/doc/user/content/transform-data/idiomatic-materialize-sql/last-value.md b/doc/user/content/transform-data/idiomatic-materialize-sql/last-value.md index 67c6350f058fe..40d8d6885f51a 100644 --- a/doc/user/content/transform-data/idiomatic-materialize-sql/last-value.md +++ b/doc/user/content/transform-data/idiomatic-materialize-sql/last-value.md @@ -17,7 +17,7 @@ to some ordering, in each group. ### Materialize and window functions -{{< idiomatic-sql/materialize-window-functions >}} +{{% include-headless "/headless/materialize-window-functions" %}} {{}} diff --git a/doc/user/content/transform-data/idiomatic-materialize-sql/lead.md b/doc/user/content/transform-data/idiomatic-materialize-sql/lead.md index a25ea85c5554b..d8cc5a69dd148 100644 --- a/doc/user/content/transform-data/idiomatic-materialize-sql/lead.md +++ b/doc/user/content/transform-data/idiomatic-materialize-sql/lead.md @@ -22,7 +22,7 @@ function. ### Materialize and window functions -{{< idiomatic-sql/materialize-window-functions >}} +{{% include-headless "/headless/materialize-window-functions" %}} {{}} diff --git a/doc/user/content/transform-data/idiomatic-materialize-sql/top-k.md b/doc/user/content/transform-data/idiomatic-materialize-sql/top-k.md index 57de678a6333d..6b080b1163789 100644 --- a/doc/user/content/transform-data/idiomatic-materialize-sql/top-k.md +++ b/doc/user/content/transform-data/idiomatic-materialize-sql/top-k.md @@ -22,7 +22,7 @@ elements within each group according to some ordering. ### Materialize and window functions -{{< idiomatic-sql/materialize-window-functions >}} +{{% include-headless "/headless/materialize-window-functions" %}} {{}} diff --git a/doc/user/data/explain_plan_operators.yml b/doc/user/data/explain_plan_operators.yml index 4750bb653df32..6a53caaa24032 100644 --- a/doc/user/data/explain_plan_operators.yml +++ b/doc/user/data/explain_plan_operators.yml @@ -242,7 +242,7 @@ operators: 1. `Distinct GroupAggregate` corresponds to the SQL `DISTINCT` operator. - 2. `Accumulable GroupAggregate` (e.g., `SUM`, `COUNT`) corresponds to several easy to implement aggregations that can be executed simultaneously and efficiently. + 2. `Accumulable GroupAggregate` (e.g., `SUM`, `COUNT`) corresponds to several easy to implement aggregations that can be executed efficiently. 3. `Hierarchical GroupAggregate` (e.g., `MIN`, `MAX`) corresponds to an aggregation requiring a tower of arrangements. These can be either monotonic (more efficient) or bucketed. These may benefit from a hint; [see `mz_introspection.mz_expected_group_size_advice`](/reference/system-catalog/mz_introspection/#mz_expected_group_size_advice). These may either be bucketed or monotonic (more efficient). diff --git a/doc/user/data/sql_funcs.yml b/doc/user/data/sql_funcs.yml index 567248a19cf20..0628df329a635 100644 --- a/doc/user/data/sql_funcs.yml +++ b/doc/user/data/sql_funcs.yml @@ -881,7 +881,7 @@ ROW`. {{< note >}} - {{% idiomatic-sql/materialize-window-functions %}} + {{% include-headless "/headless/materialize-window-functions" %}} See [Idiomatic Materialize SQL](/transform-data/idiomatic-materialize-sql/) for examples of rewriting window functions. diff --git a/doc/user/layouts/shortcodes/idiomatic-sql/materialize-window-functions.html b/doc/user/layouts/shortcodes/idiomatic-sql/materialize-window-functions.html deleted file mode 100644 index b2d4d9c53db2b..0000000000000 --- a/doc/user/layouts/shortcodes/idiomatic-sql/materialize-window-functions.html +++ /dev/null @@ -1,17 +0,0 @@ -For [window functions](/sql/functions/#window-functions), when an input record -in a partition (as determined by the `PARTITION BY` clause of your window -function) is added/removed/changed, Materialize recomputes the results for the -entire window partition. This means that when a new batch of input data arrives -(that is, every second), **the amount of computation performed is proportional -to the total size of the touched partitions**. - -For example, assume that in a given second, 20 input records change, and these -records belong to **10** different partitions, where the average size of each -partition is **100**. Then, amount of work to perform is proportional to -computing the window function results for **10\*100=1000** rows. - -To avoid performance issues that may arise as the number of records grows, -consider rewriting your query to use idiomatic Materialize SQL instead of window -functions. If your query cannot be rewritten without the window functions and -the performance of window functions is insufficient for your use case, please -[contact our team](/support/).