From 1a12bf0217b8b4c4562df9a9531681e7b4a1751f Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Mon, 2 Mar 2026 15:47:15 -0800 Subject: [PATCH 01/23] Add user quotas and align existing doc with how-to topic template --- .../manage-throughput.adoc | 240 ++++++++++++++---- 1 file changed, 192 insertions(+), 48 deletions(-) diff --git a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc index dab8e18374..fd675b307e 100644 --- a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc +++ b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc @@ -1,6 +1,10 @@ = Manage Throughput -:description: Learn how to manage the throughput of Kafka traffic. +:description: Configure broker-wide and client-specific throughput quotas to prevent resource exhaustion and enable multi-tenant isolation. :page-categories: Management, Networking +:page-topic-type: how-to +:learning-objective-1: Configure user-based throughput quotas with authentication +:learning-objective-2: Set client-based quotas for backward compatibility +:learning-objective-3: Monitor quota usage and throttling behavior // tag::single-source[] ifdef::env-cloud[] @@ -12,18 +16,28 @@ ifndef::env-cloud[] :connected-clients-api-doc-ref: link:/api/doc/admin/v2/operation/operation-redpanda-core-admin-v2-clusterservice-listkafkaconnections endif::[] -Redpanda supports throughput throttling on both ingress and egress independently, and allows configuration at the broker and client levels. This helps prevent clients from causing unbounded network and disk usage on brokers. You can configure limits at two levels: +Redpanda supports throughput throttling on both ingress and egress independently, and allows configuration at the broker and client levels. This helps prevent clients from causing unbounded network and disk usage on brokers. -* *Broker limits*: These apply to all clients connected to the broker and restrict total traffic on the broker. See <>. +After reading this page, you will be able to: + +* [ ] {learning-objective-1} +* [ ] {learning-objective-2} +* [ ] {learning-objective-3} + +You can configure limits at two levels: + +* Broker limits: These apply to all clients connected to the broker and restrict total traffic on the broker. See <>. ifndef::env-cloud[] -* *Client limits*: These apply to a set of clients defined by their `client_id` and help prevent a set of clients from starving other clients using the same broker. You can manage client quotas with xref:reference:rpk/rpk-cluster/rpk-cluster-quotas.adoc[`rpk cluster quotas`], with {ui}, or with the Kafka API. When no quotas apply, the client has unlimited throughput. +* Client limits: These apply to authenticated users or clients defined by their `client_id`. You can manage client quotas with xref:reference:rpk/rpk-cluster/rpk-cluster-quotas.adoc[`rpk cluster quotas`], with {ui}, or with the Kafka API. When no quotas apply, the client has unlimited throughput. endif::[] ifdef::env-cloud[] -* *Client limits*: These apply to a set of clients defined by their `client_id` and help prevent a set of clients from starving other clients using the same broker. You can manage client quotas with xref:reference:rpk/rpk-cluster/rpk-cluster-quotas.adoc[`rpk cluster quotas`], with the {ui} UI, with the link:https://docs.redpanda.com/api/doc/cloud-dataplane/operation/operation-quotaservice_listquotas[Redpanda Cloud Data Plane API], or with the Kafka API. When no quotas apply, the client has unlimited throughput. +* Client limits: These apply to authenticated users or clients defined by their `client_id`. You can manage client quotas with xref:reference:rpk/rpk-cluster/rpk-cluster-quotas.adoc[`rpk cluster quotas`], with the {ui} UI, with the link:https://docs.redpanda.com/api/doc/cloud-dataplane/operation/operation-quotaservice_listquotas[Redpanda Cloud Data Plane API], or with the Kafka API. When no quotas apply, the client has unlimited throughput. NOTE: Throughput throttling is supported for BYOC and Dedicated clusters only. endif::[] +TIP: For detailed information about quota types, precedence, and tracking behavior, see xref:manage:cluster-maintenance/about-throughput-quotas.adoc[]. + == View connected client details You may find it helpful to check the xref:{monitor-doc}[current produce and consume throughput] of a client before you configure throughput quotas. @@ -258,6 +272,8 @@ UID STATE USER CLIENT-ID b41584f3-2662-4185-a4b8-0d8510f5c780 OPEN UNAUTHENTICATED perf-producer-client 127.0.0.1:55002 0 0 8s 7.743592270s 0B 0B 1 b20601a3-624c-4a8c-ab88-717643f01d56 OPEN UNAUTHENTICATED perf-producer-client 127.0.0.1:55012 0 0 9s 0s 78.9MB 0B 292 ---- + +The `USER` field in the connection list shows the authenticated principal. Unauthenticated connections show `UNAUTHENTICATED`, which corresponds to an empty user principal (`user=""`) in quota configurations, not `user=`. -- ifndef::env-cloud[] @@ -417,10 +433,30 @@ curl \ } ---- ==== +The user principal field in the connection list shows the authenticated principal. Unauthenticated connections show `AUTHENTICATION_STATE_UNAUTHENTICATED`, which corresponds to an empty user principal (`user=""`) in quota configurations, not `user=`. -- endif::[] ====== +To view connections for a specific authenticated user: + +[,bash] +---- +rpk cluster connections list --user alice +---- + +This shows all connections from user `alice`, useful for monitoring clients that are subject to user-based quotas. + +== About throughput quotas + +Redpanda supports two types of throughput quotas: + +* Client-based quotas: Limit throughput based on the self-declared `client_id` field. No authentication required. +* User-based quotas: Limit throughput based on authenticated user principal. Requires xref:manage:security/authentication.adoc[authentication]. + +You can also combine both types for fine-grained control (for example, limiting a specific user when using a specific client application). + +For details about quota types, precedence rules, and tracking behavior, see xref:manage:cluster-maintenance/about-throughput-quotas.adoc[]. == Throughput throttling enforcement @@ -473,43 +509,144 @@ endif::[] == Client throughput limits -Redpanda provides configurable throughput quotas that apply to an individual client or a group of clients. You can apply a quota for an individual client based on an exact match with its `client_id`, or a group of clients based on IDs that start with a given prefix. +Redpanda provides configurable throughput quotas based on authenticated user principals, client IDs, or combinations of both. Quotas are managed through the Kafka-compatible AlterClientQuotas and DescribeClientQuotas APIs, accessible via rpk, Redpanda Console, or Kafka client libraries. -As of v24.2, client throughput quotas are compatible with the https://cwiki.apache.org/confluence/display/KAFKA/KIP-546%3A+Add+Client+Quota+APIs+to+the+Admin+Client[AlterClientQuotas and DescribeClientQuotas^] Kafka APIs, and are separate from quotas configured through cluster configuration in earlier Redpanda versions. The client throughput quotas no longer apply on a per-shard basis, and now limit the rates across a Redpanda broker's node. The quotas are neither shared nor balanced between brokers. +As of v26.1, Redpanda supports: -Redpanda supports the following Kafka API-based quota types on clients: +* User-based quotas: Limit specific authenticated users +* Client-based quotas: Limit clients by their `client_id` field +* Combined quotas: Limit specific (user, client) combinations +* Prefix matching: Group users or clients by name prefix +* Default quotas: Set fallback limits for unmatched users or clients + +Redpanda supports the following Kafka API-based quota types: |=== | Quota type | Description | `producer_byte_rate` -| Limit throughput of produce requests +| Limit throughput of produce requests (bytes per second) | `consumer_byte_rate` -| Limit throughput of fetch requests +| Limit throughput of fetch requests (bytes per second) | `controller_mutation_rate` -| Limit rate of topic mutation requests, including create, add, and delete partition, in number of partitions per second +| Limit rate of topic mutation requests (partitions created or deleted per second) |=== -You can also apply a default quota for all other client requests that don't have a specific quota based on an exact match or `client_id` prefix. +All quota types can be applied to user-based entities, client-based entities, or combinations of both. -It is possible to create conflicting quotas if you configure the same quotas through both the Kafka API and a cluster configuration. Redpanda resolves these conflicts by following an order of preference in finding a matching quota for a request: +NOTE: When multiple quotas could apply to a request, Redpanda uses a precedence hierarchy where the most specific match wins. For details about the precedence order and quota tracking behavior, see xref:manage:cluster-maintenance/about-throughput-quotas.adoc#quota-entity-hierarchy[Quota entity hierarchy]. -. Quota configured through the Kafka API for an exact match on `client_id` -. Quota configured through the Kafka API for a prefix match on `client_id` -ifndef::env-cloud[] -. Quota configured through cluster configuration properties (`kafka_client_group_byte_rate_quota`, `kafka_client_group_fetch_byte_rate_quota`-deprecated in v24.2) for a prefix match on `client_id` -endif::[] -. Default quota configured through the Kafka API on `client_id` -ifndef::env-cloud[] -. Default quota configured through cluster configuration properties (`target_quota_byte_rate`, `target_fetch_quota_byte_rate`, `kafka_admin_topic_api_rate`-deprecated in v24.2) on `client_id` +=== Set user-based quotas -Redpanda recommends <> over from cluster configuration-managed quotas to Kafka-compatible quotas. You can re-create the configuration-based quotas with `rpk`, and then remove the cluster configurations. -endif::[] +IMPORTANT: User-based quotas require authentication to be enabled. To set up client authentication, see xref:manage:security/authentication.adoc[]. + +==== Quota for a specific user + +To limit throughput for a specific authenticated user across all clients: + +[,bash] +---- +rpk cluster quotas alter --add producer_byte_rate=2000000 --name user=alice +---- + +This limits user `alice` to 2 MB/s for produce requests, regardless of which `client_id` they use. + +To view quotas for a user: + +[,bash] +---- +rpk cluster quotas describe --name user=alice +---- -=== Individual client throughput limit +Expected output: + +[,bash,role=no-copy] +---- +user=alice + producer_byte_rate=2000000 +---- + +==== Default quota for all users + +To set a fallback quota for any user without a more specific quota: + +[,bash] +---- +rpk cluster quotas alter --add consumer_byte_rate=5000000 --name user=default +---- + +This applies a 5 MB/s fetch quota to all authenticated users who don't have a more specific quota configured. + +==== Remove a user quota + +To remove a quota for a specific user: + +[,bash] +---- +rpk cluster quotas alter --delete consumer_byte_rate --name user=alice +---- + +To remove all quotas for a user: + +[,bash] +---- +rpk cluster quotas delete --name user=alice +---- + +=== Set combined user and client quotas + +You can set quotas for specific (user, client_id) combinations for fine-grained control. + +==== User with specific client + +To limit a specific user when using a specific client: + +[,bash] +---- +rpk cluster quotas alter --add consumer_byte_rate=1000000 --name user=alice,client-id=consumer-1 +---- + +User `alice` using `client_id="consumer-1"` is limited to 1 MB/s. The same user with a different `client_id` would use a different quota (or fall back to less specific matches). + +To view combined quotas: + +[,bash] +---- +rpk cluster quotas describe --name user=alice,client-id=consumer-1 +---- + +==== User with client prefix + +To set a shared quota for a user across multiple clients matching a prefix: + +[,bash] +---- +rpk cluster quotas alter --add producer_byte_rate=3000000 --name user=bob,client-id-prefix=producer- +---- + +All clients used by user `bob` with `client_id` starting with "producer-" share a combined 3 MB/s quota. + +==== Default user with specific client + +To set a quota for a specific client across all users: + +[,bash] +---- +rpk cluster quotas alter --add producer_byte_rate=500000 --name user=default,client-id=payment-processor +---- + +Any user using `client_id="payment-processor"` is limited to 500 KB/s, unless they have a more specific quota configured. + +=== Set client-based quotas + +Client-based quotas apply to all users using a specific `client_id`. These quotas do not require authentication and provide no security isolation between users or tenants. + +NOTE: For multi-tenant environments, Redpanda recommends user-based quotas to provide true security isolation. + +==== Individual client throughput limit ifdef::env-cloud[] NOTE: The following sections show how to manage throughput with `rpk`. You can also manage throughput with the link:https://docs.redpanda.com/api/doc/cloud-dataplane/operation/operation-quotaservice_listquotas[Redpanda Cloud Data Plane API]. @@ -531,7 +668,7 @@ client-id=consumer-1 ---- -To set a throughput quota for a single client, use the xref:reference:rpk/rpk-cluster/rpk-cluster-quotas-alter.adoc[`rpk cluster quotas alter`] command. +To set a throughput quota for a single client, use the xref:reference:rpk/rpk-cluster/rpk-cluster-quotas-alter.adoc[`rpk cluster quotas alter`] command. [,bash] ---- @@ -544,7 +681,7 @@ ENTITY STATUS client-id=consumer-1 OK ---- -=== Group of clients throughput limit +==== Group of clients throughput limit Alternatively, you can view or configure throughput quotas for a group of clients based on a match on client ID prefix. The following example sets the `consumer_byte_rate` quota to client IDs prefixed with `consumer-`: @@ -553,10 +690,9 @@ Alternatively, you can view or configure throughput quotas for a group of client rpk cluster quotas alter --add consumer_byte_rate=200000 --name client-id-prefix=consumer- ---- -NOTE: A client group specified with `client-id-prefix` is not the equivalent of a Kafka consumer group. It is used only to match requests based on the `client_id` prefix. The `client_id` field is typically a configurable property when you create a client with Kafka libraries. +NOTE: A client group specified with `client-id-prefix` is not the equivalent of a Kafka consumer group. It is used only to match requests based on the `client_id` prefix. - -=== Default client throughput limit +==== Default client throughput limit You can apply default throughput limits to clients. Redpanda applies the default limits if no quotas are configured for a specific `client_id` or prefix. @@ -590,47 +726,50 @@ You can also save the JSON or YAML output to a file and pass the file path in th [[migrate]] === Migrate cluster configuration quotas to Kafka API-based quotas +Prior to v24.2, quotas were configured through cluster configuration properties. These properties were deprecated in v24.2 in favor of the Kafka API-based approach. This section shows how to migrate from cluster configuration-based quotas to Kafka API-based quotas. + . Use xref:reference:rpk/rpk-cluster/rpk-cluster-config-get.adoc[`rpk cluster config get`] to view current client quotas managed with cluster configuration. The following example shows how to retrieve the `kafka_client_group_byte_rate_quota` for two groups of producers: + [,bash] ---- rpk cluster config get kafka_client_group_byte_rate_quota - ---- + [,bash,role=no-copy] ---- -"kafka_client_group_byte_rate_quota": [ - { - "group_name": "group_1", - "clients_prefix": "producer_group_alone_producer", - "quota": 10240 - }, - { "group_name": "group_2", - "clients_prefix": "producer_group_multiple", - "quota": 20480 - } +"kafka_client_group_byte_rate_quota": [ + { + "group_name": "group_1", + "clients_prefix": "producer_group_alone_producer", + "quota": 10240 + }, + { + "group_name": "group_2", + "clients_prefix": "producer_group_multiple", + "quota": 20480 + } ] ---- + ifndef::env-cloud[] . Each client quota cluster property (xref:upgrade:deprecated/index.adoc[deprecated in v24.2]) corresponds to a quota type in Kafka. Check the corresponding `rpk` arguments to use when setting the new quota values: + |=== | Cluster configuration property | `rpk cluster quotas` arguments -| `target_quota_byte_rate` +| `target_quota_byte_rate` | `--default client-id --add producer_byte_rate=` -| `target_fetch_quota_byte_rate` +| `target_fetch_quota_byte_rate` | `--default client-id --add consumer_byte_rate=` -| `kafka_admin_topic_api_rate` +| `kafka_admin_topic_api_rate` | `--default client-id --add controller_mutation_rate=` -| `kafka_client_group_byte_rate_quota` +| `kafka_client_group_byte_rate_quota` | `--name client-id-prefix= --add producer_byte_rate=` -| `kafka_client_group_fetch_byte_rate_quota` +| `kafka_client_group_fetch_byte_rate_quota` | `--name client-id-prefix= --add consumer_byte_rate=` |=== @@ -642,8 +781,8 @@ endif::[] + [,bash] ---- -rpk cluster quotas alter --name client-id-prefix=producer_group_alone_producer --add producer_byte_rate= -rpk cluster quotas alter --name client-id-prefix=producer_group_multiple --add producer_byte_rate= +rpk cluster quotas alter --name client-id-prefix=producer_group_alone_producer --add producer_byte_rate= +rpk cluster quotas alter --name client-id-prefix=producer_group_multiple --add producer_byte_rate= ---- + Replace the placeholder values with the new quota values, accounting for the conversion to per-broker limits. For example, 10240 * broker core count = new quota. @@ -674,6 +813,9 @@ ifndef::env-cloud[] ** `/metrics` - xref:reference:internal-metrics-reference.adoc#vectorized_kafka_quotas_client_quota_throttle_time[`vectorized_kafka_quotas_client_quota_throttle_time`] endif::[] +// TODO: Verify the exact quota_rule label values for user-based quotas in v26.1. +Quota metrics use the `quota_rule` label to identify which quota was applied to a request. The label distinguishes between different entity types (user, client, or combinations). See the `quota_rule` label values in xref:reference:public-metrics-reference.adoc#redpanda_kafka_quotas_client_quota_throughput[]. + The `kafka_quotas` logger provides details at the trace level on client quota throttling: [,bash] @@ -687,6 +829,8 @@ TRACE 2024-06-14 15:37:59,195 [shard 2:main] kafka_quotas - connection_context. == See also -- xref:manage:cluster-maintenance/configure-client-connections.adoc[Configure Client Connections] +- xref:manage:cluster-maintenance/about-throughput-quotas.adoc[] +- xref:manage:cluster-maintenance/configure-client-connections.adoc[] +- xref:manage:security/authentication.adoc[] // end::single-source[] From 5d261b838fce0305fac0ad39410980e5483ddce1 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Mon, 2 Mar 2026 17:29:47 -0800 Subject: [PATCH 02/23] Move more content out into new concepts doc --- .../manage-throughput.adoc | 53 ++++--------------- 1 file changed, 9 insertions(+), 44 deletions(-) diff --git a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc index fd675b307e..894df15c45 100644 --- a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc +++ b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc @@ -10,10 +10,12 @@ ifdef::env-cloud[] :monitor-doc: manage:monitor-cloud.adoc#throughput :connected-clients-api-doc-ref: link:/api/doc/cloud-dataplane/operation/operation-monitoringservice_listkafkaconnections +:authentication-doc: security:cloud-authentication.adoc endif::[] ifndef::env-cloud[] :monitor-doc: manage:monitoring.adoc#throughput :connected-clients-api-doc-ref: link:/api/doc/admin/v2/operation/operation-redpanda-core-admin-v2-clusterservice-listkafkaconnections +:authentication-doc: manage:security/authentication.adoc endif::[] Redpanda supports throughput throttling on both ingress and egress independently, and allows configuration at the broker and client levels. This helps prevent clients from causing unbounded network and disk usage on brokers. @@ -440,6 +442,7 @@ endif::[] To view connections for a specific authenticated user: +// TODO: Confirm syntax for user/principal flag [,bash] ---- rpk cluster connections list --user alice @@ -452,25 +455,13 @@ This shows all connections from user `alice`, useful for monitoring clients that Redpanda supports two types of throughput quotas: * Client-based quotas: Limit throughput based on the self-declared `client_id` field. No authentication required. -* User-based quotas: Limit throughput based on authenticated user principal. Requires xref:manage:security/authentication.adoc[authentication]. +* User-based quotas: Limit throughput based on authenticated user principal. Requires xref:{authentication-doc}[authentication]. You can also combine both types for fine-grained control (for example, limiting a specific user when using a specific client application). For details about quota types, precedence rules, and tracking behavior, see xref:manage:cluster-maintenance/about-throughput-quotas.adoc[]. -== Throughput throttling enforcement - -NOTE: As of v24.2, Redpanda enforces all throughput limits per broker, including client throughput. - -Throughput limits are enforced by applying backpressure to clients. When a connection is in breach of the throughput limit, the throttler advises the client about the delay (throttle time) that would bring the rate back to the allowed level. Redpanda starts by adding a `throttle_time_ms` field to responses. If that isn't honored, delays are inserted on the connection's next read operation. - -ifdef::env-cloud[] -In Redpanda Cloud, the throttling delay is set to 30 seconds. -endif::[] - -ifndef::env-cloud[] -The throttling delay may not exceed the limit set by xref:reference:tunable-properties.adoc#max_kafka_throttle_delay_ms[`max_kafka_throttle_delay_ms`]. -endif::[] +TIP: For information about how Redpanda enforces quotas through throttling and backpressure, see xref:manage:cluster-maintenance/about-throughput-quotas.adoc#throttling-enforcement[Throttling enforcement]. == Broker-wide throughput limits @@ -509,39 +500,13 @@ endif::[] == Client throughput limits -Redpanda provides configurable throughput quotas based on authenticated user principals, client IDs, or combinations of both. Quotas are managed through the Kafka-compatible AlterClientQuotas and DescribeClientQuotas APIs, accessible via rpk, Redpanda Console, or Kafka client libraries. - -As of v26.1, Redpanda supports: - -* User-based quotas: Limit specific authenticated users -* Client-based quotas: Limit clients by their `client_id` field -* Combined quotas: Limit specific (user, client) combinations -* Prefix matching: Group users or clients by name prefix -* Default quotas: Set fallback limits for unmatched users or clients - -Redpanda supports the following Kafka API-based quota types: - -|=== -| Quota type | Description - -| `producer_byte_rate` -| Limit throughput of produce requests (bytes per second) - -| `consumer_byte_rate` -| Limit throughput of fetch requests (bytes per second) - -| `controller_mutation_rate` -| Limit rate of topic mutation requests (partitions created or deleted per second) - -|=== - -All quota types can be applied to user-based entities, client-based entities, or combinations of both. +Redpanda provides configurable throughput quotas for individual clients or authenticated users. Quotas are managed through the Kafka-compatible AlterClientQuotas and DescribeClientQuotas APIs, accessible via rpk, Redpanda Console, or Kafka client libraries. -NOTE: When multiple quotas could apply to a request, Redpanda uses a precedence hierarchy where the most specific match wins. For details about the precedence order and quota tracking behavior, see xref:manage:cluster-maintenance/about-throughput-quotas.adoc#quota-entity-hierarchy[Quota entity hierarchy]. +TIP: For conceptual information about quota types, entity hierarchy, precedence rules, and tracking behavior, see xref:manage:cluster-maintenance/about-throughput-quotas.adoc[]. === Set user-based quotas -IMPORTANT: User-based quotas require authentication to be enabled. To set up client authentication, see xref:manage:security/authentication.adoc[]. +IMPORTANT: User-based quotas require authentication to be enabled. To set up authentication, see xref:{authentication-doc}[]. ==== Quota for a specific user @@ -831,6 +796,6 @@ TRACE 2024-06-14 15:37:59,195 [shard 2:main] kafka_quotas - connection_context. - xref:manage:cluster-maintenance/about-throughput-quotas.adoc[] - xref:manage:cluster-maintenance/configure-client-connections.adoc[] -- xref:manage:security/authentication.adoc[] +- xref:{authentication-doc}[] // end::single-source[] From 024a66e023a519eb1c46e85524d6a6c4bf76e2f6 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Mon, 2 Mar 2026 18:08:52 -0800 Subject: [PATCH 03/23] Add rpk cluster config set example for broker limits --- .../pages/cluster-maintenance/manage-throughput.adoc | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc index 894df15c45..27630a19b9 100644 --- a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc +++ b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc @@ -496,6 +496,17 @@ The properties for broker-wide throughput quota balancing are configured at the ==== By default, both `kafka_throughput_limit_node_in_bps` and `kafka_throughput_limit_node_out_bps` are disabled, and no throughput limits are applied. You must manually set them to enable throughput throttling. ==== + +To set broker-wide throughput limits, use xref:reference:rpk/rpk-cluster/rpk-cluster-config.set[`rpk cluster config set`] to configure the cluster properties: + +[,bash] +---- +# Set ingress limit to 100 MB/s per broker +rpk cluster config set kafka_throughput_limit_node_in_bps 100000000 + +# Set egress limit to 200 MB/s per broker +rpk cluster config set kafka_throughput_limit_node_out_bps 200000000 +---- endif::[] == Client throughput limits From c0f54ba4e22d41897f719ce7cc37a031dc6976b5 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Mon, 2 Mar 2026 18:27:09 -0800 Subject: [PATCH 04/23] Minor edit --- modules/manage/pages/cluster-maintenance/manage-throughput.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc index 27630a19b9..a29f6d9523 100644 --- a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc +++ b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc @@ -511,7 +511,7 @@ endif::[] == Client throughput limits -Redpanda provides configurable throughput quotas for individual clients or authenticated users. Quotas are managed through the Kafka-compatible AlterClientQuotas and DescribeClientQuotas APIs, accessible via rpk, Redpanda Console, or Kafka client libraries. +Redpanda provides configurable throughput quotas for individual clients or authenticated users. Quotas are managed through the Kafka-compatible AlterClientQuotas and DescribeClientQuotas APIs, accessible via `rpk`, Redpanda Console, or Kafka client libraries. TIP: For conceptual information about quota types, entity hierarchy, precedence rules, and tracking behavior, see xref:manage:cluster-maintenance/about-throughput-quotas.adoc[]. From d8ef762c2b65126bbc58f1280831b7e7c032d1d2 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Mon, 2 Mar 2026 18:57:28 -0800 Subject: [PATCH 05/23] Reorganize how-to --- .../manage-throughput.adoc | 30 ++++++++----------- 1 file changed, 12 insertions(+), 18 deletions(-) diff --git a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc index a29f6d9523..5c50993e22 100644 --- a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc +++ b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc @@ -38,7 +38,16 @@ ifdef::env-cloud[] NOTE: Throughput throttling is supported for BYOC and Dedicated clusters only. endif::[] -TIP: For detailed information about quota types, precedence, and tracking behavior, see xref:manage:cluster-maintenance/about-throughput-quotas.adoc[]. +== About throughput quotas + +Redpanda supports two types of throughput quotas: + +* Client-based quotas: Limit throughput based on the self-declared `client_id` field. No authentication required. +* User-based quotas: Limit throughput based on authenticated user principal. Requires xref:{authentication-doc}[authentication]. + +You can also combine both types for fine-grained control (for example, limiting a specific user when using a specific client application). + +For conceptual information about quota types, entity hierarchy, precedence rules, and how Redpanda tracks and enforces quotas through throttling, see xref:manage:cluster-maintenance/about-throughput-quotas.adoc[]. == View connected client details @@ -450,19 +459,6 @@ rpk cluster connections list --user alice This shows all connections from user `alice`, useful for monitoring clients that are subject to user-based quotas. -== About throughput quotas - -Redpanda supports two types of throughput quotas: - -* Client-based quotas: Limit throughput based on the self-declared `client_id` field. No authentication required. -* User-based quotas: Limit throughput based on authenticated user principal. Requires xref:{authentication-doc}[authentication]. - -You can also combine both types for fine-grained control (for example, limiting a specific user when using a specific client application). - -For details about quota types, precedence rules, and tracking behavior, see xref:manage:cluster-maintenance/about-throughput-quotas.adoc[]. - -TIP: For information about how Redpanda enforces quotas through throttling and backpressure, see xref:manage:cluster-maintenance/about-throughput-quotas.adoc#throttling-enforcement[Throttling enforcement]. - == Broker-wide throughput limits Broker-wide throughput limits account for all Kafka API traffic going into or out of the broker, as data is produced to or consumed from a topic. The limit values represent the allowed rate of data in bytes per second passing through in each direction. Redpanda also provides administrators the ability to exclude clients from throughput throttling and to fine-tune which Kafka request types are subject to throttling limits. @@ -497,7 +493,7 @@ The properties for broker-wide throughput quota balancing are configured at the By default, both `kafka_throughput_limit_node_in_bps` and `kafka_throughput_limit_node_out_bps` are disabled, and no throughput limits are applied. You must manually set them to enable throughput throttling. ==== -To set broker-wide throughput limits, use xref:reference:rpk/rpk-cluster/rpk-cluster-config.set[`rpk cluster config set`] to configure the cluster properties: +To set broker-wide throughput limits, use xref:reference:rpk/rpk-cluster/rpk-cluster-config-set.adoc[`rpk cluster config set`] to configure the cluster properties: [,bash] ---- @@ -513,8 +509,6 @@ endif::[] Redpanda provides configurable throughput quotas for individual clients or authenticated users. Quotas are managed through the Kafka-compatible AlterClientQuotas and DescribeClientQuotas APIs, accessible via `rpk`, Redpanda Console, or Kafka client libraries. -TIP: For conceptual information about quota types, entity hierarchy, precedence rules, and tracking behavior, see xref:manage:cluster-maintenance/about-throughput-quotas.adoc[]. - === Set user-based quotas IMPORTANT: User-based quotas require authentication to be enabled. To set up authentication, see xref:{authentication-doc}[]. @@ -620,7 +614,7 @@ Any user using `client_id="payment-processor"` is limited to 500 KB/s, unless th Client-based quotas apply to all users using a specific `client_id`. These quotas do not require authentication and provide no security isolation between users or tenants. -NOTE: For multi-tenant environments, Redpanda recommends user-based quotas to provide true security isolation. +For multi-tenant environments, Redpanda recommends user-based quotas to provide true security isolation. ==== Individual client throughput limit From a59ab7adcb41d268b6ff43294a5182cacf941174 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Mon, 2 Mar 2026 18:57:47 -0800 Subject: [PATCH 06/23] Add new conceptual doc about throughput quotas --- .../about-throughput-quotas.adoc | 203 ++++++++++++++++++ 1 file changed, 203 insertions(+) create mode 100644 modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc diff --git a/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc new file mode 100644 index 0000000000..b615fadb67 --- /dev/null +++ b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc @@ -0,0 +1,203 @@ += About Throughput Quotas +:description: Understand how Redpanda's user-based and client-based throughput quotas work, including entity hierarchy, precedence rules, and quota tracking behavior. +:page-topic-type: concepts +:page-aliases: +:learning-objective-1: Understand the difference between user-based and client-based quotas +:learning-objective-2: Determine which quota type to use for your use case +:learning-objective-3: Understand quota precedence and tracking behavior + +// tag::single-source[] +ifdef::env-cloud[] +:authentication-doc: security:cloud-authentication.adoc +endif::[] +ifndef::env-cloud[] +:authentication-doc: manage:security/authentication.adoc +endif::[] + +Redpanda provides throughput quotas to limit the rate of produce and consume requests from clients. Understanding how quotas work helps you design effective multi-tenant isolation and prevent resource exhaustion. + +After reading this page, you will be able to: + +* [ ] {learning-objective-1} +* [ ] {learning-objective-2} +* [ ] {learning-objective-3} + +To configure and manage throughput quotas, see xref:manage:cluster-maintenance/manage-throughput.adoc[]. + +== Throughput control overview + +Redpanda provides two types of throughput limits: broker-wide limits (configured using cluster properties) and client quotas (configured using Kafka API). + +This page covers client quotas, which enable per-user and per-client rate limiting with fine-grained control through entity hierarchy and precedence rules. For information about broker-wide limits, see xref:manage:cluster-maintenance/manage-throughput.adoc#broker-wide-throughput-limits[Broker-wide throughput limits]. + +== Supported quota types + +Redpanda supports three Kafka API-based quota types: + +|=== +| Quota type | Description + +| `producer_byte_rate` +| Limit throughput of produce requests (bytes per second) + +| `consumer_byte_rate` +| Limit throughput of fetch requests (bytes per second) + +| `controller_mutation_rate` +| Limit rate of topic mutation requests (partitions created or deleted per second) +|=== + +All quota types can be applied to user-based entities, client-based entities, or combinations of both. + +== Quota entities + +Redpanda supports two primary entity types for client quotas: + +* Client-based quotas (`client_id` or `client_id` prefix): Limit throughput based on the `client_id` field that clients self-declare. Multiple clients using the same `client_id` share the quota. This quota does not require authentication. +* User-based quotas: Limit throughput based on authenticated user principal (from SASL, mTLS, or OIDC). + +You can also combine both types for fine-grained control, such as limiting a specific user when using a specific client application. + +=== Client-based quotas + +Client-based quotas apply to clients identified by their `client_id` field, which is set by the client application. The `client_id` field is typically a configurable property when you create a client with Kafka libraries. Multiple clients using the same `client_id` share the same quota tracking. + +Client-based quotas rely on clients honestly reporting their identity. Because `client_id` is self-declared, any client can claim any identity. This makes client-based quotas unsuitable for security isolation between tenants. + +Use client-based quotas when: + +* Authentication is not enabled. +* Grouping by application or service name is sufficient. +* You operate a single-tenant environment where all clients are trusted. +* You need simple rate limiting without user-level isolation. + +=== User-based quotas + +IMPORTANT: User-based quotas require xref:manage:security/authentication.adoc[authentication] to be enabled on your cluster. + +User-based quotas apply to authenticated user principals (from SASL, mTLS, or OIDC authentication). Each user has a separate quota, providing true multi-tenant isolation. + +User-based quotas rely on Redpanda's authentication system to verify user identity. The user principal is extracted from SASL credentials, mTLS certificates, or OIDC tokens and cannot be forged by clients. This provides true security isolation between tenants. + +Use user-based quotas when: + +* You operate a multi-tenant environment, such as SaaS platforms or enterprises with departments. +* You require security isolation between users or tenants, including for compliance requirements. +* You need per-user billing or metering. + +=== Combined user and client quotas + +You can combine user and client identities for fine-grained control over specific (user, client) combinations. + +Use combined quotas when: + +* You need fine-grained control, for example: "user `alice` using specific application". +* Different rate limits apply to different apps used by the same user, for example: "`alice`'s payment-processor gets 10 MB/s, but `alice`'s analytics-consumer gets 50 MB/s". (See <> for examples.) + +== Quota entity hierarchy + +When a request arrives, Redpanda resolves which quota to apply by matching the request's authenticated user principal and `client_id` against configured quotas. Redpanda applies the most specific match, using the following precedence order (highest priority first): + +. Exact user + exact client: `/config/users//clients/` +. Exact user + client prefix: `/config/users//client-id-prefix/` +. Exact user + default client: `/config/users//clients/` +. Exact user only: `/config/users/` (applies to all clients for that user) +. Default user + exact client: `/config/users//clients/` +. Default user + client prefix: `/config/users//client-id-prefix/` +. Default user + default client: `/config/users//clients/` +. Default user only: `/config/users/` (applies to all users and clients) +. Exact client only: `/config/clients/` (applies to all users) +. Client prefix only: `/config/client-id-prefix/` (applies to all users) +. Default client only: `/config/clients/` (applies to all users and clients) +. No quota configured: Unlimited throughput (no throttling) + +IMPORTANT: The `` entity matches any user or client that doesn't have a more specific quota configured. This is different from an empty/unauthenticated user (`user=""`), which is treated as a specific entity. + +=== Example: Precedence resolution + +Given these configured quotas: + +[,bash] +---- +rpk cluster quotas alter --add consumer_byte_rate=5000000 --name user=alice,client-id=app-1 +rpk cluster quotas alter --add consumer_byte_rate=10000000 --name user=alice +rpk cluster quotas alter --add consumer_byte_rate=20000000 --name client-id=app-1 +---- + +|=== +| User + Client ID | Precedence match | Resulting quota + +| `user="alice"`, `client_id="app-1"` +| Level 1: Exact user + exact client +| 5 MB/s + +| `user="alice"`, `client_id="app-2"` +| Level 4: Exact user only +| 10 MB/s + +| `user="bob"`, `client_id="app-1"` +| Level 9: Exact client only +| 20 MB/s + +| `user="bob"`, `client_id="app-2"` +| Level 12: No quota configured +| Unlimited +|=== + +=== Unauthenticated connections + +Unauthenticated connections have an empty user principal (`user=""`), which is treated as a specific entity, not as `user=`. + +Unauthenticated connections: + +* Fall back to client-only quotas (precedence levels 9-11). +* Have unlimited throughput only if no client-only quota matches. + +== Quota tracking and isolation + +Each unique (user principal + `client_id`) combination is tracked independently for quota enforcement, even if multiple combinations have the same quota limit configured. This means: + +* Different users using the same `client_id` each get their own quota tracking. +* The same user with different `client_id` values also gets separate quota tracking with each client. +* Quota usage is not shared between different (user, client) pairs. + +=== Example: Separate quota tracking per user + +The following command sets a default 10 MB/s quota for all users: + +[,bash] +---- +rpk cluster quotas alter --add producer_byte_rate=10000000 --name user=default +---- + +* If user `alice` authenticates and uses `client_id="app1"`, Redpanda enforces a 10 MB/s quota. +* If user `bob` authenticates and uses `client_id="app1"`, Redpanda also enforces a 10 MB/s quota, but this is tracked separately from user `alice`. + +The combined throughput is 20 MB/s, and each user has their own independent 10 MB/s limit. + +[[throttling-enforcement]] +== Throughput throttling enforcement + +Redpanda enforces throughput limits by applying backpressure to clients. When a connection exceeds its throughput limit, Redpanda throttles the connection to bring the rate back within the allowed level: + +. Redpanda adds a `throttle_time_ms` field to responses, indicating how long the client should wait. +. If the client doesn't honor the throttle time, Redpanda inserts delays on the connection's next read operation. + +ifndef::env-cloud[] +The throttling delay may not exceed the limit set by the `max_kafka_throttle_delay_ms` tunable property. +endif::[] + +ifdef::env-cloud[] +In Redpanda Cloud, the throttling delay is set to 30 seconds. +endif::[] + +NOTE: As of v24.2, Redpanda enforces all throughput limits per broker, including client throughput. + +== Default behavior + +Quotas are opt-in restrictions and not enforced by default. When no quotas are configured, clients have unlimited throughput. + +== Next steps + +* xref:manage:cluster-maintenance/manage-throughput.adoc[Configure throughput quotas] +* xref:{authentication-doc}[Enable authentication for user-based quotas] From 6d15ae96023957720f20dd7d98b795eb9a7fc842 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Mon, 2 Mar 2026 18:57:59 -0800 Subject: [PATCH 07/23] Add new doc to nav --- modules/ROOT/nav.adoc | 1 + 1 file changed, 1 insertion(+) diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc index 5eb34695dc..7312f19fce 100644 --- a/modules/ROOT/nav.adoc +++ b/modules/ROOT/nav.adoc @@ -163,6 +163,7 @@ *** xref:manage:audit-logging.adoc[Audit Logging] **** xref:manage:audit-logging/audit-log-samples.adoc[Sample Audit Log Messages] *** xref:manage:cluster-maintenance/disk-utilization.adoc[] +*** xref:manage:cluster-maintenance/about-throughput-quotas.adoc[] *** xref:manage:cluster-maintenance/manage-throughput.adoc[Manage Throughput] *** xref:manage:cluster-maintenance/compaction-settings.adoc[Compaction Settings] *** xref:manage:cluster-maintenance/configure-client-connections.adoc[] From b3f4f2c2ae6b85b69d4668d9f867f910f365d488 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Tue, 3 Mar 2026 17:29:56 -0800 Subject: [PATCH 08/23] Add user quotas to 26.1 What's New --- .../pages/release-notes/redpanda.adoc | 169 +----------------- 1 file changed, 3 insertions(+), 166 deletions(-) diff --git a/modules/get-started/pages/release-notes/redpanda.adoc b/modules/get-started/pages/release-notes/redpanda.adoc index 34765f5ded..9dafbe0ee1 100644 --- a/modules/get-started/pages/release-notes/redpanda.adoc +++ b/modules/get-started/pages/release-notes/redpanda.adoc @@ -7,171 +7,8 @@ This topic includes new content added in version {page-component-version}. For a * xref:redpanda-cloud:get-started:whats-new-cloud.adoc[] * xref:redpanda-cloud:get-started:cloud-overview.adoc#redpanda-cloud-vs-self-managed-feature-compatibility[Redpanda Cloud vs Self-Managed feature compatibility] -NOTE: Redpanda v25.3 introduces breaking schema changes for Iceberg topics. If you are using Iceberg topics and want to retain the data in the corresponding Iceberg tables, review xref:upgrade:iceberg-schema-changes-and-migration-guide.adoc[] before upgrading your cluster, and follow the required migration steps to avoid sending new records to a dead-letter queue table. +== User-based throughput quotas -== Iceberg topics with GCP BigLake +Redpanda now supports throughput quotas based on authenticated user principals. Unlike client-based quotas (which rely on self-declared `client_id` values), user-based quotas enforce limits using verified identities from SASL, mTLS, or OIDC authentication. -A new xref:manage:iceberg/iceberg-topics-gcp-biglake.adoc[REST catalog integration] with Google Cloud BigLake allows you to add Redpanda topics as Iceberg tables in your data lakehouse. - -See xref:manage:iceberg/use-iceberg-catalogs.adoc[] for details on configuring Iceberg REST catalog integrations with Redpanda. - -== Shadowing - -Redpanda v25.3 introduces xref:deploy:redpanda/manual/disaster-recovery/shadowing/index.adoc[], an enterprise-licensed disaster recovery solution that provides asynchronous, offset-preserving replication between distinct Redpanda clusters. Shadowing enables cross-region data protection by replicating topic data, configurations, consumer group offsets, ACLs, and Schema Registry data with byte-level fidelity. - -The shadow cluster operates in read-only mode while continuously receiving updates from the source cluster. During a disaster, you can failover individual topics or an entire shadow link to make resources fully writable for production traffic. See xref:deploy:redpanda/manual/disaster-recovery/shadowing/failover-runbook.adoc[] for emergency procedures. - -Shadowing includes comprehensive metrics for monitoring replication health. See xref:manage:disaster-recovery/shadowing/monitor.adoc[] and xref:reference:public-metrics-reference.adoc#shadow-link-metrics[Shadow Link metrics reference]. - -== Connected client monitoring - -You can view details about Kafka client connections using `rpk` or the Admin API ListKafkaConnections endpoint. This allows you to view detailed information about active client connections on a cluster, and identify and troubleshoot problematic clients. For more information, see the xref:manage:cluster-maintenance/manage-throughput.adoc#view-connected-client-details[connected client details] example in the Manage Throughput guide. - -== New Admin API style - -Redpanda v25.3 introduces a new API style for the Admin API, powered by https://connectrpc.com/docs/introduction[ConnectRPC]. New Redpanda features and operations in v25.3 are available as ConnectRPC services, allowing you to use autogenerated Protobuf clients in addition to using HTTP clients such as `curl`. - -Use the new ConnectRPC endpoints with the following v25.3 features: - -* Shadowing -* Connected client monitoring - -Existing Admin API endpoints from versions earlier than 25.3 remain supported, and you can continue to use them as usual. See xref:manage:use-admin-api.adoc[Manage Redpanda with the Admin API] to learn more about Admin API, and the link:/api/doc/admin/v2/[Admin API reference] to view the new endpoints. - -== Schema Registry import mode - -Redpanda Schema Registry now supports an import mode that allows you to import existing schemas and retain their current IDs and version numbers. Import mode is useful when migrating from another schema registry. - -Starting with this release, import mode must be used when importing schemas. Read-write mode no longer allows specifying a schema ID and version when registering a schema. -See xref:manage:schema-reg/schema-reg-api.adoc#set-schema-registry-mode[Use the Schema Registry API]. - -== Security report - -You can now generate a security report for your Redpanda cluster using the link:/api/doc/admin/operation/operation-get_security_report[`/v1/security/report`] Admin API endpoint. The report provides detailed information about TLS configuration, authentication methods, authorization status, and security alerts across all Redpanda interfaces (Kafka, RPC, Admin, Schema Registry, HTTP Proxy). - -== Topic identifiers - -Redpanda v25.3 implements topic identifiers using 16 byte UUIDs as proposed in https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers[KIP-516^]. - -== Shadowing metrics - -Redpanda v25.3 introduces comprehensive xref:reference:public-metrics-reference.adoc#shadow-link-metrics[Shadowing metrics] for monitoring disaster recovery replication: - -* xref:reference:public-metrics-reference.adoc#redpanda_shadow_link_client_errors[`redpanda_shadow_link_client_errors`] - Track Kafka client errors during shadow link operations -* xref:reference:public-metrics-reference.adoc#redpanda_shadow_link_shadow_lag[`redpanda_shadow_link_shadow_lag`] - Monitor replication lag between source and shadow partitions -* xref:reference:public-metrics-reference.adoc#redpanda_shadow_link_shadow_topic_state[`redpanda_shadow_link_shadow_topic_state`] - Track shadow topic state distribution across links -* xref:reference:public-metrics-reference.adoc#redpanda_shadow_link_total_bytes_fetched[`redpanda_shadow_link_total_bytes_fetched`] - Monitor data transfer volume from source cluster -* xref:reference:public-metrics-reference.adoc#redpanda_shadow_link_total_bytes_written[`redpanda_shadow_link_total_bytes_written`] - Track data written to shadow cluster -* xref:reference:public-metrics-reference.adoc#redpanda_shadow_link_total_records_fetched[`redpanda_shadow_link_total_records_fetched`] - Monitor total records fetched from source cluster -* xref:reference:public-metrics-reference.adoc#redpanda_shadow_link_total_records_written[`redpanda_shadow_link_total_records_written`] - Track total messages written to shadow cluster - -For monitoring guidance and alert recommendations, see xref:manage:disaster-recovery/shadowing/monitor.adoc[]. - -== New commands - -Redpanda v25.3 introduces the following xref:reference:rpk/rpk-shadow/rpk-shadow.adoc[`rpk shadow`] commands for managing Redpanda shadow links: - -* xref:reference:rpk/rpk-shadow/rpk-shadow-config-generate.adoc[`rpk shadow config generate`] - Generate configuration files for shadow links -* xref:reference:rpk/rpk-shadow/rpk-shadow-create.adoc[`rpk shadow create`] - Create new shadow links -* xref:reference:rpk/rpk-shadow/rpk-shadow-update.adoc[`rpk shadow update`] - Update existing shadow link configurations -* xref:reference:rpk/rpk-shadow/rpk-shadow-list.adoc[`rpk shadow list`] - List all shadow links -* xref:reference:rpk/rpk-shadow/rpk-shadow-describe.adoc[`rpk shadow describe`] - View shadow link configuration details -* xref:reference:rpk/rpk-shadow/rpk-shadow-status.adoc[`rpk shadow status`] - Monitor shadow link replication status -* xref:reference:rpk/rpk-shadow/rpk-shadow-failover.adoc[`rpk shadow failover`] - Perform emergency failover operations -* xref:reference:rpk/rpk-shadow/rpk-shadow-delete.adoc[`rpk shadow delete`] - Delete shadow links - -In addition, the following commands have been added: - -* xref:reference:rpk/rpk-cluster/rpk-cluster-connections.adoc[`rpk cluster connections`] - Monitor cluster connections and client statistics. -* xref:reference:rpk/rpk-redpanda/rpk-redpanda-config-print.adoc[`rpk redpanda config print`] - Display node configuration. - -== New configuration properties - -Redpanda 25.3 introduces the following configuration properties: - -**Shadowing:** - -* xref:reference:properties/cluster-properties.adoc#enable_shadow_linking[`enable_shadow_linking`]: Enable shadow links (Enterprise license required) - -**Timestamp validation:** - -* xref:reference:properties/cluster-properties.adoc#log_message_timestamp_after_max_ms[`log_message_timestamp_after_max_ms`]: Maximum timestamp difference for future records -* xref:reference:properties/cluster-properties.adoc#log_message_timestamp_before_max_ms[`log_message_timestamp_before_max_ms`]: Maximum timestamp difference for past records -* xref:reference:properties/topic-properties.adoc#messagetimestampaftermaxms[`message.timestamp.after.max.ms`]: Topic-level timestamp validation (future) -* xref:reference:properties/topic-properties.adoc#messagetimestampbeforemaxms[`message.timestamp.before.max.ms`]: Topic-level timestamp validation (past) - -**Audit logging:** - -* xref:reference:properties/cluster-properties.adoc#audit_use_rpc[`audit_use_rpc`]: Use internal RPCs for audit logging - -**Object storage:** - -* xref:reference:properties/object-storage-properties.adoc#cloud_storage_client_lease_timeout_ms[`cloud_storage_client_lease_timeout_ms`]: Object storage connection timeout - -**Iceberg:** - -* xref:reference:properties/cluster-properties.adoc#iceberg_default_catalog_namespace[`iceberg_default_catalog_namespace`]: Default Iceberg catalog namespace for tables -* xref:reference:properties/cluster-properties.adoc#iceberg_dlq_table_suffix[`iceberg_dlq_table_suffix`]: Iceberg DLQ table name suffix -* xref:reference:properties/cluster-properties.adoc#iceberg_rest_catalog_gcp_user_project[`iceberg_rest_catalog_gcp_user_project`]: GCP project for Iceberg REST catalog billing -* xref:reference:properties/cluster-properties.adoc#iceberg_topic_name_dot_replacement[`iceberg_topic_name_dot_replacement`]: Dot replacement in Iceberg table names - -**TLS:** - -* xref:reference:properties/cluster-properties.adoc#tls_v1_2_cipher_suites[`tls_v1_2_cipher_suites`]: TLS 1.2 cipher suites for client connections -* xref:reference:properties/cluster-properties.adoc#tls_v1_3_cipher_suites[`tls_v1_3_cipher_suites`]: TLS 1.3 cipher suites for client connections - -**Tiered Storage:** - -* xref:reference:properties/cluster-properties.adoc#cloud_topics_epoch_service_epoch_increment_interval[`cloud_topics_epoch_service_epoch_increment_interval`]: Cluster epoch increment interval -* xref:reference:properties/cluster-properties.adoc#cloud_topics_epoch_service_local_epoch_cache_duration[`cloud_topics_epoch_service_local_epoch_cache_duration`]: Local epoch cache duration -* xref:reference:properties/cluster-properties.adoc#cloud_topics_short_term_gc_backoff_interval[`cloud_topics_short_term_gc_backoff_interval`]: Short-term garbage collection backoff interval -* xref:reference:properties/cluster-properties.adoc#cloud_topics_short_term_gc_interval[`cloud_topics_short_term_gc_interval`]: Short-term garbage collection interval -* xref:reference:properties/cluster-properties.adoc#cloud_topics_short_term_gc_minimum_object_age[`cloud_topics_short_term_gc_minimum_object_age`]: Minimum object age for garbage collection - -**Other configuration:** - -* xref:reference:properties/cluster-properties.adoc#controller_backend_reconciliation_concurrency[`controller_backend_reconciliation_concurrency`]: Maximum concurrent controller reconciliation operations -* xref:reference:properties/cluster-properties.adoc#fetch_max_read_concurrency[`fetch_max_read_concurrency`]: Maximum concurrent partition reads per fetch request -* xref:reference:properties/cluster-properties.adoc#kafka_max_message_size_upper_limit_bytes[`kafka_max_message_size_upper_limit_bytes`]: Maximum allowed `max.message.size` topic property value -* xref:reference:properties/cluster-properties.adoc#kafka_produce_batch_validation[`kafka_produce_batch_validation`]: Validation level for produced batches -* xref:reference:properties/cluster-properties.adoc#log_compaction_tx_batch_removal_enabled[`log_compaction_tx_batch_removal_enabled`]: Enable transactional batch removal during compaction -* xref:reference:properties/cluster-properties.adoc#sasl_mechanisms_overrides[`sasl_mechanisms_overrides`]: SASL authentication mechanisms per listener - -=== Changes to default values - -The following configuration properties have new default values in v25.3: - -* xref:reference:properties/cluster-properties.adoc#core_balancing_continuous[`core_balancing_continuous`]: Changed from `false` to `true` (Enterprise license required). -* xref:reference:properties/cluster-properties.adoc#partition_autobalancing_mode[`partition_autobalancing_mode`]: Changed from `node_add` to `continuous` (Enterprise license required). -* xref:reference:properties/cluster-properties.adoc#iceberg_throttle_backlog_size_ratio[`iceberg_throttle_backlog_size_ratio`]: Changed from `0.3` to `null`. - -[[behavior-changes]] -=== Behavior changes - -The following topic properties now support enhanced tristate behavior: - -* xref:reference:properties/topic-properties.adoc#segment-ms[`segment.ms`] -* xref:reference:properties/topic-properties.adoc#retention-bytes[`retention.bytes`] -* xref:reference:properties/topic-properties.adoc#retention-ms[`retention.ms`] -* xref:reference:properties/topic-properties.adoc#retention-local-target-bytes[`retention.local.target.bytes`] -* xref:reference:properties/topic-properties.adoc#retention-local-target-ms[`retention.local.target.ms`] -* xref:reference:properties/topic-properties.adoc#initial-retention-local-target-bytes[`initial.retention.local.target.bytes`] -* xref:reference:properties/topic-properties.adoc#initial-retention-local-target-ms[`initial.retention.local.target.ms`] -* xref:reference:properties/topic-properties.adoc#delete-retention-ms[`delete.retention.ms`] -* xref:reference:properties/topic-properties.adoc#min-cleanable-dirty-ratio[`min.cleanable.dirty.ratio`] - -Previously, these properties treated zero and negative values the same way. Now they support three distinct states: positive values set specific limits, zero provides immediate eligibility for cleanup/compaction, and negative values disable the feature entirely. Review your topic configurations if you currently use zero values for these properties. - -=== Deprecations - -The following configuration properties have been deprecated in v25.3 and will be removed in a future release: - -* `kafka_memory_batch_size_estimate_for_fetch`: No replacement. Remove from configuration. -* `log_compaction_disable_tx_batch_removal`: Use xref:reference:properties/cluster-properties.adoc#log_compaction_tx_batch_removal_enabled[`log_compaction_tx_batch_removal_enabled`] instead. Note the inverted logic: the new property enables the behavior when set to `true`. -* `log_message_timestamp_alert_after_ms`: Use xref:reference:properties/cluster-properties.adoc#log_message_timestamp_after_max_ms[`log_message_timestamp_after_max_ms`] instead. -* `log_message_timestamp_alert_before_ms`: Use xref:reference:properties/cluster-properties.adoc#log_message_timestamp_before_max_ms[`log_message_timestamp_before_max_ms`] instead. -* `raft_recovery_default_read_size`: No replacement. Remove from configuration. - -== Deprecated features - -Redpanda has deprecated support for specific TLSv1.2 and TLSv1.3 cipher suites and now uses more secure defaults. See xref:upgrade:deprecated/index.adoc[Deprecated Features] for the complete list. +You can set quotas for individual users, default users, or fine-grained user/client combinations. See xref:manage:cluster-maintenance/about-throughput-quotas.adoc[] for conceptual details, and xref:manage:cluster-maintenance/manage-throughput.adoc#set-user-based-quotas[Set user-based quotas] to get started. From 8c41a46d2cb3543b6a88ea5b69f0d80ba5fcc87f Mon Sep 17 00:00:00 2001 From: Kat Batuigas <36839689+kbatuigas@users.noreply.github.com> Date: Thu, 5 Mar 2026 12:34:39 -0800 Subject: [PATCH 09/23] Apply suggestions from code review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Gellért Peresztegi-Nagy --- .../pages/cluster-maintenance/about-throughput-quotas.adoc | 6 +++--- .../manage/pages/cluster-maintenance/manage-throughput.adoc | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc index b615fadb67..c3c0106062 100644 --- a/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc +++ b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc @@ -75,14 +75,14 @@ Use client-based quotas when: IMPORTANT: User-based quotas require xref:manage:security/authentication.adoc[authentication] to be enabled on your cluster. -User-based quotas apply to authenticated user principals (from SASL, mTLS, or OIDC authentication). Each user has a separate quota, providing true multi-tenant isolation. +User-based quotas apply to authenticated user principals (from SASL, mTLS, or OIDC authentication). Each user has a separate quota, providing a way to limit the impact of individual users on the cluster. -User-based quotas rely on Redpanda's authentication system to verify user identity. The user principal is extracted from SASL credentials, mTLS certificates, or OIDC tokens and cannot be forged by clients. This provides true security isolation between tenants. +User-based quotas rely on Redpanda's authentication system to verify user identity. The user principal is extracted from SASL credentials, mTLS certificates, or OIDC tokens and cannot be forged by clients. Use user-based quotas when: * You operate a multi-tenant environment, such as SaaS platforms or enterprises with departments. -* You require security isolation between users or tenants, including for compliance requirements. +* You require isolation between users or tenants, including for compliance requirements. * You need per-user billing or metering. === Combined user and client quotas diff --git a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc index 5c50993e22..6316290a5d 100644 --- a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc +++ b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc @@ -660,7 +660,7 @@ Alternatively, you can view or configure throughput quotas for a group of client rpk cluster quotas alter --add consumer_byte_rate=200000 --name client-id-prefix=consumer- ---- -NOTE: A client group specified with `client-id-prefix` is not the equivalent of a Kafka consumer group. It is used only to match requests based on the `client_id` prefix. +NOTE: A `client-id-prefix` quota group is not related to Kafka consumer groups. The `client_id` is an application-defined identifier sent with every request—client libraries typically default to their own name (such as kgo, rdkafka, sarama, or perf-producer-client), but applications can set it via the (https://kafka.apache.org/41/configuration/consumer-configs/#consumerconfigs_client.id)[`client.id`] configuration property. This makes prefix-based quotas useful for grouping related applications (for example, inventory-service- to match inventory-service-1, inventory-service-2, etc.). ==== Default client throughput limit From 99ef1aaa0adeec2d48969245c2e25bdd7a480909 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Thu, 5 Mar 2026 13:19:14 -0800 Subject: [PATCH 10/23] Apply edits per review --- .../about-throughput-quotas.adoc | 122 +++++++++++++++--- .../manage-throughput.adoc | 77 +++++------ 2 files changed, 145 insertions(+), 54 deletions(-) diff --git a/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc index c3c0106062..ee2d260020 100644 --- a/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc +++ b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc @@ -2,9 +2,10 @@ :description: Understand how Redpanda's user-based and client-based throughput quotas work, including entity hierarchy, precedence rules, and quota tracking behavior. :page-topic-type: concepts :page-aliases: -:learning-objective-1: Understand the difference between user-based and client-based quotas +:personas: platform_admin, developer +:learning-objective-1: Describe the difference between user-based and client-based quotas :learning-objective-2: Determine which quota type to use for your use case -:learning-objective-3: Understand quota precedence and tracking behavior +:learning-objective-3: Explain quota precedence rules and how Redpanda tracks quota usage // tag::single-source[] ifdef::env-cloud[] @@ -14,7 +15,7 @@ ifndef::env-cloud[] :authentication-doc: manage:security/authentication.adoc endif::[] -Redpanda provides throughput quotas to limit the rate of produce and consume requests from clients. Understanding how quotas work helps you design effective multi-tenant isolation and prevent resource exhaustion. +Redpanda uses throughput quotas to limit the rate of produce and consume requests from clients. Understanding how quotas work helps you design effective multi-tenant isolation and prevent resource exhaustion. After reading this page, you will be able to: @@ -113,6 +114,8 @@ When a request arrives, Redpanda resolves which quota to apply by matching the r IMPORTANT: The `` entity matches any user or client that doesn't have a more specific quota configured. This is different from an empty/unauthenticated user (`user=""`), which is treated as a specific entity. +The precedence level that matches also determines how quota usage is tracked. See <>. + === Example: Precedence resolution Given these configured quotas: @@ -125,25 +128,23 @@ rpk cluster quotas alter --add consumer_byte_rate=20000000 --name client-id=app- ---- |=== -| User + Client ID | Precedence match | Resulting quota +| User + Client ID | Precedence match | `user="alice"`, `client_id="app-1"` | Level 1: Exact user + exact client -| 5 MB/s | `user="alice"`, `client_id="app-2"` | Level 4: Exact user only -| 10 MB/s | `user="bob"`, `client_id="app-1"` | Level 9: Exact client only -| 20 MB/s | `user="bob"`, `client_id="app-2"` | Level 12: No quota configured -| Unlimited |=== +When no quota matches (level 12), the connection is not throttled. + === Unauthenticated connections Unauthenticated connections have an empty user principal (`user=""`), which is treated as a specific entity, not as `user=`. @@ -155,25 +156,112 @@ Unauthenticated connections: == Quota tracking and isolation -Each unique (user principal + `client_id`) combination is tracked independently for quota enforcement, even if multiple combinations have the same quota limit configured. This means: +Redpanda tracks quota usage using a tracker key that determines which connections share the same quota bucket. The tracker key—and therefore the isolation level—depends on which quota rule matches the request. The Level column below corresponds to the precedence levels in <>. + +=== Tracking granularity by quota type + +[cols="1,3,2,4", options="header"] +|=== +| Level | Config path | Tracker key | Isolation behavior + +| 1 +| `/config/users//clients/` +| `(user, client_id)` +| Each unique (user, client_id) pair tracked independently + +| 2 +| `/config/users//client-id-prefix/` +| `(user, prefix)` +| Clients matching the prefix share tracking within that user + +| 3 +| `/config/users//clients/` +| `(user, client_id)` +| Each unique (user, client_id) pair tracked independently + +| 4 +| `/config/users/` +| `user` +| All clients for that user share a single tracking bucket + +| 5 +| `/config/users//clients/` +| `(user, client_id)` +| Each unique (user, client_id) pair tracked independently + +| 6 +| `/config/users//client-id-prefix/` +| `(user, prefix)` +| Clients matching the prefix share tracking within each user + +| 7 +| `/config/users//clients/` +| `(user, client_id)` +| Each unique (user, client_id) pair tracked independently + +| 8 +| `/config/users/` +| `user` +| All clients for each user share a single tracking bucket (per user) + +| 9 +| `/config/clients/` +| `client_id` +| All users with that client_id share a single tracking bucket + +| 10 +| `/config/client-id-prefix/` +| `prefix` +| All clients matching the prefix share a single bucket (across all users) + +| 11 +| `/config/clients/` +| `client_id` +| Each unique client_id tracked independently + +| 12 +| No quota configured +| N/A +| No tracking / unlimited throughput +|=== + +=== Key behaviors + +* *User-only quotas share across client_ids*: When you configure a quota at the user level (for example, `/config/users/alice`, level 4) without specifying a `client_id` dimension, all connections from that user share a single quota bucket regardless of their `client_id`. +* *Client-id-only quotas share across users*: When you configure a quota at the `client_id` level (for example, `/config/clients/app-1`, level 9) without specifying a user dimension, all users connecting with that `client_id` share a single quota bucket. +* *Combined quotas for per-(user, client_id) tracking*: To get independent quota tracking per (user, client_id) combination, configure quotas that include both dimensions, such as `/config/users//clients/` (level 1) or `/config/users//clients/` (level 7). +* *Prefix-based quotas group clients*: Client ID prefix quotas (for example, `/config/client-id-prefix/app-`, level 10) cause all clients whose `client_id` starts with that prefix to share a quota bucket. + +=== Example: User-only quota + +If you configure a 10 MB/s produce quota for user `alice`: -* Different users using the same `client_id` each get their own quota tracking. -* The same user with different `client_id` values also gets separate quota tracking with each client. -* Quota usage is not shared between different (user, client) pairs. +[,bash] +---- +rpk cluster quotas alter --add producer_byte_rate=10000000 --name user=alice +---- + +Then `alice` connecting with `client_id=app-1` and `alice` connecting with `client_id=app-2` share the same 10 MB/s limit. Their combined throughput cannot exceed 10 MB/s. + +To give each of `alice`'s clients an independent 10 MB/s limit, configure: + +[,bash] +---- +rpk cluster quotas alter --add producer_byte_rate=10000000 --name user=alice,client-id=default +---- -=== Example: Separate quota tracking per user +=== Example: User default quota -The following command sets a default 10 MB/s quota for all users: +If you configure a default 10 MB/s produce quota for all users: [,bash] ---- rpk cluster quotas alter --add producer_byte_rate=10000000 --name user=default ---- -* If user `alice` authenticates and uses `client_id="app1"`, Redpanda enforces a 10 MB/s quota. -* If user `bob` authenticates and uses `client_id="app1"`, Redpanda also enforces a 10 MB/s quota, but this is tracked separately from user `alice`. +This quota applies to all users who don't have a more specific quota configured. Each user is tracked independently—`alice` gets her own 10 MB/s bucket, `bob` gets his own 10 MB/s bucket, and so on. -The combined throughput is 20 MB/s, and each user has their own independent 10 MB/s limit. +Within each user, all `client_id` values share that user's bucket. So `alice` connecting with `client_id=app-1` and `alice` connecting with `client_id=app-2` share the same 10 MB/s limit, while `bob`'s connections have a separate 10 MB/s limit. [[throttling-enforcement]] == Throughput throttling enforcement diff --git a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc index 6316290a5d..f3923be1fc 100644 --- a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc +++ b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc @@ -2,6 +2,7 @@ :description: Configure broker-wide and client-specific throughput quotas to prevent resource exhaustion and enable multi-tenant isolation. :page-categories: Management, Networking :page-topic-type: how-to +:personas: platform_admin, developer :learning-objective-1: Configure user-based throughput quotas with authentication :learning-objective-2: Set client-based quotas for backward compatibility :learning-objective-3: Monitor quota usage and throttling behavior @@ -18,7 +19,7 @@ ifndef::env-cloud[] :authentication-doc: manage:security/authentication.adoc endif::[] -Redpanda supports throughput throttling on both ingress and egress independently, and allows configuration at the broker and client levels. This helps prevent clients from causing unbounded network and disk usage on brokers. +Redpanda throttles throughput on ingress and egress independently, and you can configure limits at the broker and client levels. This prevents clients from causing unbounded network and disk usage on brokers. After reading this page, you will be able to: @@ -51,7 +52,7 @@ For conceptual information about quota types, entity hierarchy, precedence rules == View connected client details -You may find it helpful to check the xref:{monitor-doc}[current produce and consume throughput] of a client before you configure throughput quotas. +Before configuring throughput quotas, check the xref:{monitor-doc}[current produce and consume throughput] of a client. ifndef::env-cloud[] Use the xref:reference:rpk/rpk-cluster/rpk-cluster-connections-list.adoc[`rpk cluster connections list`] command or the {connected-clients-api-doc-ref}[ListKafkaConnections] Admin API endpoint to view detailed information about active Kafka client connections. @@ -660,7 +661,7 @@ Alternatively, you can view or configure throughput quotas for a group of client rpk cluster quotas alter --add consumer_byte_rate=200000 --name client-id-prefix=consumer- ---- -NOTE: A `client-id-prefix` quota group is not related to Kafka consumer groups. The `client_id` is an application-defined identifier sent with every request—client libraries typically default to their own name (such as kgo, rdkafka, sarama, or perf-producer-client), but applications can set it via the (https://kafka.apache.org/41/configuration/consumer-configs/#consumerconfigs_client.id)[`client.id`] configuration property. This makes prefix-based quotas useful for grouping related applications (for example, inventory-service- to match inventory-service-1, inventory-service-2, etc.). +NOTE: A `client-id-prefix` quota group is not related to Kafka consumer groups. The `client_id` is an application-defined identifier sent with every request—client libraries typically default to their own name (such as `kgo`, `rdkafka`, `sarama`, or `perf-producer-client`), but applications can set it via the https://kafka.apache.org/documentation/#consumerconfigs_client.id[`client.id`] configuration property. This makes prefix-based quotas useful for grouping related applications (for example, `inventory-service-` to match `inventory-service-1`, `inventory-service-2`, etc.). ==== Default client throughput limit @@ -688,11 +689,46 @@ rpk cluster quotas describe --format json [,bash] ---- -rpk cluster quotas import --from '{"quotas":[{"entity":[{"name":"foo","type":"client-id"}],"values":[{"key":"consumer_byte_rate","values":"12123123"}]},{"entity":[{"name":"foo-","type":"client-id-prefix"}],"values":[{"key":"producer_byte_rate","values":"12123123"},{"key":"consumer_byte_rate","values":"4444444"}]}]}' +rpk cluster quotas import --from '{"quotas":[{"entity":[{"name":"analytics-consumer","type":"client-id"}],"values":[{"key":"consumer_byte_rate","values":"10000000"}]},{"entity":[{"name":"analytics-","type":"client-id-prefix"}],"values":[{"key":"producer_byte_rate","values":"10000000"},{"key":"consumer_byte_rate","values":"5000000"}]}]}' ---- You can also save the JSON or YAML output to a file and pass the file path in the `--from` flag. +=== View throughput limits in {ui} + +You can also use {ui} to view enforced limits. In the side menu, go to **Quotas**. + +=== Monitor client throughput + +The following metrics provide insights into client throughput quota usage: + +* Client quota throughput per rule and quota type: +** `/public_metrics` - xref:reference:public-metrics-reference.adoc#redpanda_kafka_quotas_client_quota_throughput[`redpanda_kafka_quotas_client_quota_throughput`] +ifndef::env-cloud[] +** `/metrics` - xref:reference:internal-metrics-reference.adoc#vectorized_kafka_quotas_client_quota_throughput[`vectorized_kafka_quotas_client_quota_throughput`] +endif::[] +* Client quota throttling delay per rule and quota type, in seconds: +** `/public_metrics` - xref:reference:public-metrics-reference.adoc#redpanda_kafka_quotas_client_quota_throttle_time[`redpanda_kafka_quotas_client_quota_throttle_time`] +ifndef::env-cloud[] +** `/metrics` - xref:reference:internal-metrics-reference.adoc#vectorized_kafka_quotas_client_quota_throttle_time[`vectorized_kafka_quotas_client_quota_throttle_time`] +endif::[] + +To identify which clients are actively connected and generating traffic, see <>. + +// TODO: Verify the exact quota_rule label values for user-based quotas in v26.1. +Quota metrics use the `quota_rule` label to identify which quota was applied to a request. The label distinguishes between different entity types (user, client, or combinations). See the `quota_rule` label values in xref:reference:public-metrics-reference.adoc#redpanda_kafka_quotas_client_quota_throughput[]. + +The `kafka_quotas` logger provides details at the trace level on client quota throttling: + +[,bash] +---- +TRACE 2024-06-14 15:36:05,240 [shard 2:main] kafka_quotas - quota_manager.cc:361 - request: ctx:{quota_type: produce_quota, client_id: {rpk}}, key:k_client_id{rpk}, value:{limit: {1111}, rule: kafka_client_default}, bytes: 1316, delay:184518451ns, capped_delay:184518451ns +TRACE 2024-06-14 15:36:05,240 [shard 2:main] kafka_quotas - connection_context.cc:605 - [127.0.0.1:51256] throttle request:{snc:0, client:184}, enforce:{snc:-365123762, client:-365123762}, key:0, request_size:1316 +TRACE 2024-06-14 15:37:44,835 [shard 2:main] kafka_quotas - quota_manager.cc:361 - request: ctx:{quota_type: produce_quota, client_id: {rpk}}, key:k_client_id{rpk}, value:{limit: {1111}, rule: kafka_client_default}, bytes: 119, delay:0ns, capped_delay:0ns +TRACE 2024-06-14 15:37:59,195 [shard 2:main] kafka_quotas - quota_manager.cc:361 - request: ctx:{quota_type: produce_quota, client_id: {rpk}}, key:k_client_id{rpk}, value:{limit: {1111}, rule: kafka_client_default}, bytes: 1316, delay:184518451ns, capped_delay:184518451ns +TRACE 2024-06-14 15:37:59,195 [shard 2:main] kafka_quotas - connection_context.cc:605 - [127.0.0.1:58636] throttle request:{snc:0, client:184}, enforce:{snc:-14359, client:-14359}, key:0, request_size:1316 +---- + [[migrate]] === Migrate cluster configuration quotas to Kafka API-based quotas @@ -764,39 +800,6 @@ Replace the placeholder values with the new quota values, accounting for the con rpk cluster config set kafka_client_group_byte_rate_quota= ---- -=== View throughput limits in {ui} - -You can also use {ui} to view enforced limits. In the side menu, go to **Quotas**. - -=== Monitor client throughput - -The following metrics provide insights into client throughput quota usage: - -* Client quota throughput per rule and quota type: -** `/public_metrics` - xref:reference:public-metrics-reference.adoc#redpanda_kafka_quotas_client_quota_throughput[`redpanda_kafka_quotas_client_quota_throughput`] -ifndef::env-cloud[] -** `/metrics` - xref:reference:internal-metrics-reference.adoc#vectorized_kafka_quotas_client_quota_throughput[`vectorized_kafka_quotas_client_quota_throughput`] -endif::[] -* Client quota throttling delay per rule and quota type, in seconds: -** `/public_metrics` - xref:reference:public-metrics-reference.adoc#redpanda_kafka_quotas_client_quota_throttle_time[`redpanda_kafka_quotas_client_quota_throttle_time`] -ifndef::env-cloud[] -** `/metrics` - xref:reference:internal-metrics-reference.adoc#vectorized_kafka_quotas_client_quota_throttle_time[`vectorized_kafka_quotas_client_quota_throttle_time`] -endif::[] - -// TODO: Verify the exact quota_rule label values for user-based quotas in v26.1. -Quota metrics use the `quota_rule` label to identify which quota was applied to a request. The label distinguishes between different entity types (user, client, or combinations). See the `quota_rule` label values in xref:reference:public-metrics-reference.adoc#redpanda_kafka_quotas_client_quota_throughput[]. - -The `kafka_quotas` logger provides details at the trace level on client quota throttling: - -[,bash] ----- -TRACE 2024-06-14 15:36:05,240 [shard 2:main] kafka_quotas - quota_manager.cc:361 - request: ctx:{quota_type: produce_quota, client_id: {rpk}}, key:k_client_id{rpk}, value:{limit: {1111}, rule: kafka_client_default}, bytes: 1316, delay:184518451ns, capped_delay:184518451ns -TRACE 2024-06-14 15:36:05,240 [shard 2:main] kafka_quotas - connection_context.cc:605 - [127.0.0.1:51256] throttle request:{snc:0, client:184}, enforce:{snc:-365123762, client:-365123762}, key:0, request_size:1316 -TRACE 2024-06-14 15:37:44,835 [shard 2:main] kafka_quotas - quota_manager.cc:361 - request: ctx:{quota_type: produce_quota, client_id: {rpk}}, key:k_client_id{rpk}, value:{limit: {1111}, rule: kafka_client_default}, bytes: 119, delay:0ns, capped_delay:0ns -TRACE 2024-06-14 15:37:59,195 [shard 2:main] kafka_quotas - quota_manager.cc:361 - request: ctx:{quota_type: produce_quota, client_id: {rpk}}, key:k_client_id{rpk}, value:{limit: {1111}, rule: kafka_client_default}, bytes: 1316, delay:184518451ns, capped_delay:184518451ns -TRACE 2024-06-14 15:37:59,195 [shard 2:main] kafka_quotas - connection_context.cc:605 - [127.0.0.1:58636] throttle request:{snc:0, client:184}, enforce:{snc:-14359, client:-14359}, key:0, request_size:1316 ----- - == See also - xref:manage:cluster-maintenance/about-throughput-quotas.adoc[] From 98b835a985cedf8e83334fb758f14ad61e87d52e Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Thu, 5 Mar 2026 13:44:24 -0800 Subject: [PATCH 11/23] Update rpk examples --- .../cluster-maintenance/about-throughput-quotas.adoc | 6 +++--- .../pages/cluster-maintenance/manage-throughput.adoc | 10 +++++----- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc index ee2d260020..10c6a69aa9 100644 --- a/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc +++ b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc @@ -122,7 +122,7 @@ Given these configured quotas: [,bash] ---- -rpk cluster quotas alter --add consumer_byte_rate=5000000 --name user=alice,client-id=app-1 +rpk cluster quotas alter --add consumer_byte_rate=5000000 --name user=alice --name client-id=app-1 rpk cluster quotas alter --add consumer_byte_rate=10000000 --name user=alice rpk cluster quotas alter --add consumer_byte_rate=20000000 --name client-id=app-1 ---- @@ -247,7 +247,7 @@ To give each of `alice`'s clients an independent 10 MB/s limit, configure: [,bash] ---- -rpk cluster quotas alter --add producer_byte_rate=10000000 --name user=alice,client-id=default +rpk cluster quotas alter --add producer_byte_rate=10000000 --name user=alice --default client-id ---- === Example: User default quota @@ -256,7 +256,7 @@ If you configure a default 10 MB/s produce quota for all users: [,bash] ---- -rpk cluster quotas alter --add producer_byte_rate=10000000 --name user=default +rpk cluster quotas alter --add producer_byte_rate=10000000 --default user ---- This quota applies to all users who don't have a more specific quota configured. Each user is tracked independently—`alice` gets her own 10 MB/s bucket, `bob` gets his own 10 MB/s bucket, and so on. diff --git a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc index f3923be1fc..5726e5f43f 100644 --- a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc +++ b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc @@ -546,7 +546,7 @@ To set a fallback quota for any user without a more specific quota: [,bash] ---- -rpk cluster quotas alter --add consumer_byte_rate=5000000 --name user=default +rpk cluster quotas alter --add consumer_byte_rate=5000000 --default user ---- This applies a 5 MB/s fetch quota to all authenticated users who don't have a more specific quota configured. @@ -577,7 +577,7 @@ To limit a specific user when using a specific client: [,bash] ---- -rpk cluster quotas alter --add consumer_byte_rate=1000000 --name user=alice,client-id=consumer-1 +rpk cluster quotas alter --add consumer_byte_rate=1000000 --name user=alice --name client-id=consumer-1 ---- User `alice` using `client_id="consumer-1"` is limited to 1 MB/s. The same user with a different `client_id` would use a different quota (or fall back to less specific matches). @@ -586,7 +586,7 @@ To view combined quotas: [,bash] ---- -rpk cluster quotas describe --name user=alice,client-id=consumer-1 +rpk cluster quotas describe --name user=alice --name client-id=consumer-1 ---- ==== User with client prefix @@ -595,7 +595,7 @@ To set a shared quota for a user across multiple clients matching a prefix: [,bash] ---- -rpk cluster quotas alter --add producer_byte_rate=3000000 --name user=bob,client-id-prefix=producer- +rpk cluster quotas alter --add producer_byte_rate=3000000 --name user=bob --name client-id-prefix=producer- ---- All clients used by user `bob` with `client_id` starting with "producer-" share a combined 3 MB/s quota. @@ -606,7 +606,7 @@ To set a quota for a specific client across all users: [,bash] ---- -rpk cluster quotas alter --add producer_byte_rate=500000 --name user=default,client-id=payment-processor +rpk cluster quotas alter --add producer_byte_rate=500000 --default user --name client-id=payment-processor ---- Any user using `client_id="payment-processor"` is limited to 500 KB/s, unless they have a more specific quota configured. From 54d4a60af4e689c1cd9738e52b5e4ac333026ba0 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Thu, 5 Mar 2026 15:11:00 -0800 Subject: [PATCH 12/23] Minor style edits --- .../about-throughput-quotas.adoc | 26 +++++++++---------- .../manage-throughput.adoc | 4 +-- 2 files changed, 14 insertions(+), 16 deletions(-) diff --git a/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc index 10c6a69aa9..862f390d2f 100644 --- a/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc +++ b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc @@ -27,7 +27,7 @@ To configure and manage throughput quotas, see xref:manage:cluster-maintenance/m == Throughput control overview -Redpanda provides two types of throughput limits: broker-wide limits (configured using cluster properties) and client quotas (configured using Kafka API). +Redpanda provides two types of throughput limits: broker-wide limits configured using cluster properties and client quotas configured using Kafka API. This page covers client quotas, which enable per-user and per-client rate limiting with fine-grained control through entity hierarchy and precedence rules. For information about broker-wide limits, see xref:manage:cluster-maintenance/manage-throughput.adoc#broker-wide-throughput-limits[Broker-wide throughput limits]. @@ -55,7 +55,7 @@ All quota types can be applied to user-based entities, client-based entities, or Redpanda supports two primary entity types for client quotas: * Client-based quotas (`client_id` or `client_id` prefix): Limit throughput based on the `client_id` field that clients self-declare. Multiple clients using the same `client_id` share the quota. This quota does not require authentication. -* User-based quotas: Limit throughput based on authenticated user principal (from SASL, mTLS, or OIDC). +* User-based quotas: Limit throughput based on user principal authenticated through SASL, mTLS, or OIDC. You can also combine both types for fine-grained control, such as limiting a specific user when using a specific client application. @@ -76,7 +76,7 @@ Use client-based quotas when: IMPORTANT: User-based quotas require xref:manage:security/authentication.adoc[authentication] to be enabled on your cluster. -User-based quotas apply to authenticated user principals (from SASL, mTLS, or OIDC authentication). Each user has a separate quota, providing a way to limit the impact of individual users on the cluster. +User-based quotas apply to authenticated user principals. Each user has a separate quota, providing a way to limit the impact of individual users on the cluster. User-based quotas rely on Redpanda's authentication system to verify user identity. The user principal is extracted from SASL credentials, mTLS certificates, or OIDC tokens and cannot be forged by clients. @@ -93,7 +93,7 @@ You can combine user and client identities for fine-grained control over specifi Use combined quotas when: * You need fine-grained control, for example: "user `alice` using specific application". -* Different rate limits apply to different apps used by the same user, for example: "`alice`'s payment-processor gets 10 MB/s, but `alice`'s analytics-consumer gets 50 MB/s". (See <> for examples.) +* Different rate limits apply to different apps used by the same user, for example: "`alice`'s payment-processor gets 10 MB/s, but `alice`'s analytics-consumer gets 50 MB/s". See <> for examples. == Quota entity hierarchy @@ -151,12 +151,12 @@ Unauthenticated connections have an empty user principal (`user=""`), which is t Unauthenticated connections: -* Fall back to client-only quotas (precedence levels 9-11). +* Fall back to client-only quotas. * Have unlimited throughput only if no client-only quota matches. == Quota tracking and isolation -Redpanda tracks quota usage using a tracker key that determines which connections share the same quota bucket. The tracker key—and therefore the isolation level—depends on which quota rule matches the request. The Level column below corresponds to the precedence levels in <>. +Redpanda tracks quota usage using a tracker key that determines which connections share the same quota bucket. The tracker key, and therefore the isolation level, depends on which quota rule matches the request. The Level column below corresponds to the precedence levels in <>. === Tracking granularity by quota type @@ -225,12 +225,10 @@ Redpanda tracks quota usage using a tracker key that determines which connection | No tracking / unlimited throughput |=== -=== Key behaviors - -* *User-only quotas share across client_ids*: When you configure a quota at the user level (for example, `/config/users/alice`, level 4) without specifying a `client_id` dimension, all connections from that user share a single quota bucket regardless of their `client_id`. -* *Client-id-only quotas share across users*: When you configure a quota at the `client_id` level (for example, `/config/clients/app-1`, level 9) without specifying a user dimension, all users connecting with that `client_id` share a single quota bucket. -* *Combined quotas for per-(user, client_id) tracking*: To get independent quota tracking per (user, client_id) combination, configure quotas that include both dimensions, such as `/config/users//clients/` (level 1) or `/config/users//clients/` (level 7). -* *Prefix-based quotas group clients*: Client ID prefix quotas (for example, `/config/client-id-prefix/app-`, level 10) cause all clients whose `client_id` starts with that prefix to share a quota bucket. +* User-only quotas share across client_ids: When you configure a quota at the user level (for example, `/config/users/alice`) without specifying a `client_id` dimension, all connections from that user share a single quota bucket regardless of their `client_id`. +* Client-id-only quotas share across users: When you configure a quota at the `client_id` level (for example, `/config/clients/app-1`) without specifying a user dimension, all users connecting with that `client_id` share a single quota bucket. +* Combined quotas for per-(user, client_id) tracking: To get independent quota tracking per (user, client_id) combination, configure quotas that include both dimensions, such as `/config/users//clients/` or `/config/users//clients/`. +* Prefix-based quotas group clients: Client ID prefix quotas (for example, `/config/client-id-prefix/app-`) cause all clients whose `client_id` starts with that prefix to share a quota bucket. === Example: User-only quota @@ -259,9 +257,9 @@ If you configure a default 10 MB/s produce quota for all users: rpk cluster quotas alter --add producer_byte_rate=10000000 --default user ---- -This quota applies to all users who don't have a more specific quota configured. Each user is tracked independently—`alice` gets her own 10 MB/s bucket, `bob` gets his own 10 MB/s bucket, and so on. +This quota applies to all users who don't have a more specific quota configured. Each user is tracked independently: `alice` gets their own 10 MB/s bucket, `bob` gets their own 10 MB/s bucket, and so on. -Within each user, all `client_id` values share that user's bucket. So `alice` connecting with `client_id=app-1` and `alice` connecting with `client_id=app-2` share the same 10 MB/s limit, while `bob`'s connections have a separate 10 MB/s limit. +Within each user, all `client_id` values share that user's bucket. `alice` connecting with `client_id=app-1` and `alice` connecting with `client_id=app-2` share the same 10 MB/s limit, while `bob`'s connections have a separate 10 MB/s limit. [[throttling-enforcement]] == Throughput throttling enforcement diff --git a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc index 5726e5f43f..761aa43f79 100644 --- a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc +++ b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc @@ -508,7 +508,7 @@ endif::[] == Client throughput limits -Redpanda provides configurable throughput quotas for individual clients or authenticated users. Quotas are managed through the Kafka-compatible AlterClientQuotas and DescribeClientQuotas APIs, accessible via `rpk`, Redpanda Console, or Kafka client libraries. +Redpanda provides configurable throughput quotas for individual clients or authenticated users. Quotas are managed through the Kafka-compatible AlterClientQuotas and DescribeClientQuotas APIs, accessible with `rpk`, Redpanda Console, or Kafka client libraries. === Set user-based quotas @@ -661,7 +661,7 @@ Alternatively, you can view or configure throughput quotas for a group of client rpk cluster quotas alter --add consumer_byte_rate=200000 --name client-id-prefix=consumer- ---- -NOTE: A `client-id-prefix` quota group is not related to Kafka consumer groups. The `client_id` is an application-defined identifier sent with every request—client libraries typically default to their own name (such as `kgo`, `rdkafka`, `sarama`, or `perf-producer-client`), but applications can set it via the https://kafka.apache.org/documentation/#consumerconfigs_client.id[`client.id`] configuration property. This makes prefix-based quotas useful for grouping related applications (for example, `inventory-service-` to match `inventory-service-1`, `inventory-service-2`, etc.). +NOTE: A `client-id-prefix` quota group is not related to Kafka consumer groups. The `client_id` is an application-defined identifier sent with every request. Client libraries typically default to their own name (such as `kgo`, `rdkafka`, `sarama`, or `perf-producer-client`), but applications can set it using the https://kafka.apache.org/documentation/#consumerconfigs_client.id[`client.id`] configuration property. This makes prefix-based quotas useful for grouping related applications (for example, `inventory-service-` to match `inventory-service-1`, `inventory-service-2`, etc.). ==== Default client throughput limit From 9c1ed85883cc4e0d80b49a6ef25daff999e36682 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Thu, 5 Mar 2026 15:25:48 -0800 Subject: [PATCH 13/23] Add glossterms --- .../pages/cluster-maintenance/about-throughput-quotas.adoc | 2 +- .../manage/pages/cluster-maintenance/manage-throughput.adoc | 5 ++--- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc index 862f390d2f..0c1e934561 100644 --- a/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc +++ b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc @@ -55,7 +55,7 @@ All quota types can be applied to user-based entities, client-based entities, or Redpanda supports two primary entity types for client quotas: * Client-based quotas (`client_id` or `client_id` prefix): Limit throughput based on the `client_id` field that clients self-declare. Multiple clients using the same `client_id` share the quota. This quota does not require authentication. -* User-based quotas: Limit throughput based on user principal authenticated through SASL, mTLS, or OIDC. +* User-based quotas: Limit throughput based on user glossterm:principal[] authenticated through SASL, mTLS, or OIDC. You can also combine both types for fine-grained control, such as limiting a specific user when using a specific client application. diff --git a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc index 761aa43f79..161ac3e68e 100644 --- a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc +++ b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc @@ -44,7 +44,7 @@ endif::[] Redpanda supports two types of throughput quotas: * Client-based quotas: Limit throughput based on the self-declared `client_id` field. No authentication required. -* User-based quotas: Limit throughput based on authenticated user principal. Requires xref:{authentication-doc}[authentication]. +* User-based quotas: Limit throughput based on authenticated user glossterm:principal[]. Requires xref:{authentication-doc}[authentication]. You can also combine both types for fine-grained control (for example, limiting a specific user when using a specific client application). @@ -452,7 +452,6 @@ endif::[] To view connections for a specific authenticated user: -// TODO: Confirm syntax for user/principal flag [,bash] ---- rpk cluster connections list --user alice @@ -780,7 +779,7 @@ ifndef::env-cloud[] |=== + -The client throughput quotas set through the Kafka API apply per broker, so you must convert the cluster configuration values that were applied on a per-shard (logical CPU core) basis. For example, if you set `target_fetch_quota_byte_rate` to 100 MBps/shard, and you run Redpanda on 16-core brokers, you can set the new consumer_byte_rate quota to 100 * 16 = 1600 MBps. +The client throughput quotas set through the Kafka API apply per broker, so you must convert the cluster configuration values that were applied on a per glossterm:shard[] basis. For example, if you set `target_fetch_quota_byte_rate` to 100 MBps/shard, and you run Redpanda on 16-core brokers, you can set the new consumer_byte_rate quota to 100 * 16 = 1600 MBps. endif::[] . Use `rpk cluster quotas alter` to set the corresponding client throughput quotas based on the Kafka API: From c5d47f90dd44292a908edec4f757300f778a2df6 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Fri, 6 Mar 2026 16:03:29 -0800 Subject: [PATCH 14/23] Address comments from SME review --- .../about-throughput-quotas.adoc | 8 +- .../manage-throughput.adoc | 92 +++---------------- 2 files changed, 15 insertions(+), 85 deletions(-) diff --git a/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc index 0c1e934561..9e00dcefd1 100644 --- a/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc +++ b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc @@ -15,7 +15,7 @@ ifndef::env-cloud[] :authentication-doc: manage:security/authentication.adoc endif::[] -Redpanda uses throughput quotas to limit the rate of produce and consume requests from clients. Understanding how quotas work helps you design effective multi-tenant isolation and prevent resource exhaustion. +Redpanda uses throughput quotas to limit the rate of produce and consume requests from clients. Understanding how quotas work helps you prevent individual clients from disproportionately using resources that cause latency and throughput issues (also known as the "noisy-neighbor" problem) and ensure fair resource sharing across users and applications. After reading this page, you will be able to: @@ -54,7 +54,7 @@ All quota types can be applied to user-based entities, client-based entities, or Redpanda supports two primary entity types for client quotas: -* Client-based quotas (`client_id` or `client_id` prefix): Limit throughput based on the `client_id` field that clients self-declare. Multiple clients using the same `client_id` share the quota. This quota does not require authentication. +* Client-based quotas (`client_id` or `client_id` prefix): Limit throughput based on the `client_id` field that clients self-declare. Multiple clients using the same `client_id` share the quota. This quota does not require users to authenticate. * User-based quotas: Limit throughput based on user glossterm:principal[] authenticated through SASL, mTLS, or OIDC. You can also combine both types for fine-grained control, such as limiting a specific user when using a specific client application. @@ -63,7 +63,7 @@ You can also combine both types for fine-grained control, such as limiting a spe Client-based quotas apply to clients identified by their `client_id` field, which is set by the client application. The `client_id` field is typically a configurable property when you create a client with Kafka libraries. Multiple clients using the same `client_id` share the same quota tracking. -Client-based quotas rely on clients honestly reporting their identity. Because `client_id` is self-declared, any client can claim any identity. This makes client-based quotas unsuitable for security isolation between tenants. +Client-based quotas rely on clients honestly reporting their identity and correctly setting the `client_id` property. This makes client-based quotas unsuitable for guaranteeing isolation between tenants. Use client-based quotas when: @@ -83,7 +83,7 @@ User-based quotas rely on Redpanda's authentication system to verify user identi Use user-based quotas when: * You operate a multi-tenant environment, such as SaaS platforms or enterprises with departments. -* You require isolation between users or tenants, including for compliance requirements. +* You require isolation between users or tenants, to prevent a single user from overwhelming a broker or cluster. * You need per-user billing or metering. === Combined user and client quotas diff --git a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc index 161ac3e68e..3bba0c3647 100644 --- a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc +++ b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc @@ -1,10 +1,10 @@ = Manage Throughput -:description: Configure broker-wide and client-specific throughput quotas to prevent resource exhaustion and enable multi-tenant isolation. +:description: Configure broker-wide and client-specific throughput quotas to prevent resource exhaustion and noisy-neighbor issues. :page-categories: Management, Networking :page-topic-type: how-to :personas: platform_admin, developer -:learning-objective-1: Configure user-based throughput quotas with authentication -:learning-objective-2: Set client-based quotas for backward compatibility +:learning-objective-1: Configure user-based throughput quotas +:learning-objective-2: Set client-based quotas :learning-objective-3: Monitor quota usage and throttling behavior // tag::single-source[] @@ -43,7 +43,7 @@ endif::[] Redpanda supports two types of throughput quotas: -* Client-based quotas: Limit throughput based on the self-declared `client_id` field. No authentication required. +* Client-based quotas: Limit throughput based on the self-declared `client_id` field. * User-based quotas: Limit throughput based on authenticated user glossterm:principal[]. Requires xref:{authentication-doc}[authentication]. You can also combine both types for fine-grained control (for example, limiting a specific user when using a specific client application). @@ -579,7 +579,7 @@ To limit a specific user when using a specific client: rpk cluster quotas alter --add consumer_byte_rate=1000000 --name user=alice --name client-id=consumer-1 ---- -User `alice` using `client_id="consumer-1"` is limited to 1 MB/s. The same user with a different `client_id` would use a different quota (or fall back to less specific matches). +User `alice` using `client_id="consumer-1"` is limited to a 1 MB/s fetch rate. The same user with a different `client_id` would use a different quota (or fall back to less specific matches). To view combined quotas: @@ -594,10 +594,10 @@ To set a shared quota for a user across multiple clients matching a prefix: [,bash] ---- -rpk cluster quotas alter --add producer_byte_rate=3000000 --name user=bob --name client-id-prefix=producer- +rpk cluster quotas alter --add producer_byte_rate=3000000 --name user=bob --name client-id-prefix=app1- ---- -All clients used by user `bob` with `client_id` starting with "producer-" share a combined 3 MB/s quota. +All clients used by user `bob` with `client_id` starting with "app1-" share a combined 3 MB/s quota. ==== Default user with specific client @@ -612,9 +612,9 @@ Any user using `client_id="payment-processor"` is limited to 500 KB/s, unless th === Set client-based quotas -Client-based quotas apply to all users using a specific `client_id`. These quotas do not require authentication and provide no security isolation between users or tenants. +Client-based quotas apply to all users using a specific `client_id`. These quotas do not require authentication. Because `client_id` is self-declared, client-based quotas are not suitable for guaranteeing isolation between tenants. -For multi-tenant environments, Redpanda recommends user-based quotas to provide true security isolation. +For multi-tenant environments, Redpanda recommends user-based quotas for per-tenant isolation. ==== Individual client throughput limit @@ -714,9 +714,9 @@ endif::[] To identify which clients are actively connected and generating traffic, see <>. -// TODO: Verify the exact quota_rule label values for user-based quotas in v26.1. -Quota metrics use the `quota_rule` label to identify which quota was applied to a request. The label distinguishes between different entity types (user, client, or combinations). See the `quota_rule` label values in xref:reference:public-metrics-reference.adoc#redpanda_kafka_quotas_client_quota_throughput[]. +Quota metrics use the `redpanda_quota_rule` label to identify which quota was applied to a request. The label distinguishes between different entity types (user, client, or combinations). See the label values in xref:reference:public-metrics-reference.adoc#redpanda-kafka-quotas-client-quota-throughput[]. +ifndef::env-cloud[] The `kafka_quotas` logger provides details at the trace level on client quota throttling: [,bash] @@ -727,78 +727,8 @@ TRACE 2024-06-14 15:37:44,835 [shard 2:main] kafka_quotas - quota_manager.cc:36 TRACE 2024-06-14 15:37:59,195 [shard 2:main] kafka_quotas - quota_manager.cc:361 - request: ctx:{quota_type: produce_quota, client_id: {rpk}}, key:k_client_id{rpk}, value:{limit: {1111}, rule: kafka_client_default}, bytes: 1316, delay:184518451ns, capped_delay:184518451ns TRACE 2024-06-14 15:37:59,195 [shard 2:main] kafka_quotas - connection_context.cc:605 - [127.0.0.1:58636] throttle request:{snc:0, client:184}, enforce:{snc:-14359, client:-14359}, key:0, request_size:1316 ---- - -[[migrate]] -=== Migrate cluster configuration quotas to Kafka API-based quotas - -Prior to v24.2, quotas were configured through cluster configuration properties. These properties were deprecated in v24.2 in favor of the Kafka API-based approach. This section shows how to migrate from cluster configuration-based quotas to Kafka API-based quotas. - -. Use xref:reference:rpk/rpk-cluster/rpk-cluster-config-get.adoc[`rpk cluster config get`] to view current client quotas managed with cluster configuration. The following example shows how to retrieve the `kafka_client_group_byte_rate_quota` for two groups of producers: -+ -[,bash] ----- -rpk cluster config get kafka_client_group_byte_rate_quota ----- -+ -[,bash,role=no-copy] ----- -"kafka_client_group_byte_rate_quota": [ - { - "group_name": "group_1", - "clients_prefix": "producer_group_alone_producer", - "quota": 10240 - }, - { - "group_name": "group_2", - "clients_prefix": "producer_group_multiple", - "quota": 20480 - } -] ----- - -ifndef::env-cloud[] -. Each client quota cluster property (xref:upgrade:deprecated/index.adoc[deprecated in v24.2]) corresponds to a quota type in Kafka. Check the corresponding `rpk` arguments to use when setting the new quota values: -+ -|=== -| Cluster configuration property | `rpk cluster quotas` arguments - -| `target_quota_byte_rate` -| `--default client-id --add producer_byte_rate=` - -| `target_fetch_quota_byte_rate` -| `--default client-id --add consumer_byte_rate=` - -| `kafka_admin_topic_api_rate` -| `--default client-id --add controller_mutation_rate=` - -| `kafka_client_group_byte_rate_quota` -| `--name client-id-prefix= --add producer_byte_rate=` - -| `kafka_client_group_fetch_byte_rate_quota` -| `--name client-id-prefix= --add consumer_byte_rate=` - -|=== -+ -The client throughput quotas set through the Kafka API apply per broker, so you must convert the cluster configuration values that were applied on a per glossterm:shard[] basis. For example, if you set `target_fetch_quota_byte_rate` to 100 MBps/shard, and you run Redpanda on 16-core brokers, you can set the new consumer_byte_rate quota to 100 * 16 = 1600 MBps. endif::[] -. Use `rpk cluster quotas alter` to set the corresponding client throughput quotas based on the Kafka API: -+ -[,bash] ----- -rpk cluster quotas alter --name client-id-prefix=producer_group_alone_producer --add producer_byte_rate= -rpk cluster quotas alter --name client-id-prefix=producer_group_multiple --add producer_byte_rate= ----- -+ -Replace the placeholder values with the new quota values, accounting for the conversion to per-broker limits. For example, 10240 * broker core count = new quota. - -. Use xref:reference:rpk/rpk-cluster/rpk-cluster-config-set.adoc[`rpk cluster config set`] to remove the configuration-based quotas: -+ -[,bash] ----- -rpk cluster config set kafka_client_group_byte_rate_quota= ----- - == See also - xref:manage:cluster-maintenance/about-throughput-quotas.adoc[] From 228e201b496ee064e4d9a8beeb0283554a50f49c Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Fri, 6 Mar 2026 16:15:18 -0800 Subject: [PATCH 15/23] Minor edit --- modules/manage/pages/cluster-maintenance/manage-throughput.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc index 3bba0c3647..960cacd130 100644 --- a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc +++ b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc @@ -3,7 +3,7 @@ :page-categories: Management, Networking :page-topic-type: how-to :personas: platform_admin, developer -:learning-objective-1: Configure user-based throughput quotas +:learning-objective-1: Set user-based throughput quotas :learning-objective-2: Set client-based quotas :learning-objective-3: Monitor quota usage and throttling behavior From 4bcbaed79217d4f10a12b9eb7ef0520ba3e4a18e Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Fri, 6 Mar 2026 16:24:06 -0800 Subject: [PATCH 16/23] Minor edit --- .../pages/cluster-maintenance/about-throughput-quotas.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc index 9e00dcefd1..4a54d9e127 100644 --- a/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc +++ b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc @@ -15,7 +15,7 @@ ifndef::env-cloud[] :authentication-doc: manage:security/authentication.adoc endif::[] -Redpanda uses throughput quotas to limit the rate of produce and consume requests from clients. Understanding how quotas work helps you prevent individual clients from disproportionately using resources that cause latency and throughput issues (also known as the "noisy-neighbor" problem) and ensure fair resource sharing across users and applications. +Redpanda uses throughput quotas to limit the rate of produce and consume requests from clients. Understanding how quotas work helps you prevent individual clients from disproportionately consuming resources and causing performance degradation for other clients (also known as the "noisy-neighbor" problem), and ensure fair resource sharing across users and applications. After reading this page, you will be able to: @@ -83,7 +83,7 @@ User-based quotas rely on Redpanda's authentication system to verify user identi Use user-based quotas when: * You operate a multi-tenant environment, such as SaaS platforms or enterprises with departments. -* You require isolation between users or tenants, to prevent a single user from overwhelming a broker or cluster. +* You require isolation between users or tenants, to avoid noisy-neighbor issues. * You need per-user billing or metering. === Combined user and client quotas From 162fa82abf9e4b5f23ffe797fdc811d7673e6808 Mon Sep 17 00:00:00 2001 From: Kat Batuigas <36839689+kbatuigas@users.noreply.github.com> Date: Tue, 10 Mar 2026 19:04:40 -0700 Subject: [PATCH 17/23] Apply suggestions from code review Co-authored-by: Joyce Fee <102751339+Feediver1@users.noreply.github.com> --- .../pages/cluster-maintenance/about-throughput-quotas.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc index 4a54d9e127..5327026009 100644 --- a/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc +++ b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc @@ -93,7 +93,7 @@ You can combine user and client identities for fine-grained control over specifi Use combined quotas when: * You need fine-grained control, for example: "user `alice` using specific application". -* Different rate limits apply to different apps used by the same user, for example: "`alice`'s payment-processor gets 10 MB/s, but `alice`'s analytics-consumer gets 50 MB/s". See <> for examples. +* Different rate limits apply to different apps used by the same user. For example: "`alice`'s payment-processor gets 10 MB/s, but `alice`'s analytics-consumer gets 50 MB/s". See <> for examples. == Quota entity hierarchy @@ -257,7 +257,7 @@ If you configure a default 10 MB/s produce quota for all users: rpk cluster quotas alter --add producer_byte_rate=10000000 --default user ---- -This quota applies to all users who don't have a more specific quota configured. Each user is tracked independently: `alice` gets their own 10 MB/s bucket, `bob` gets their own 10 MB/s bucket, and so on. +This quota applies to all users who don't have a more specific quota configured. Each user is tracked independently: `alice` gets her own 10 MB/s bucket, `bob` gets his own 10 MB/s bucket, and so on. Within each user, all `client_id` values share that user's bucket. `alice` connecting with `client_id=app-1` and `alice` connecting with `client_id=app-2` share the same 10 MB/s limit, while `bob`'s connections have a separate 10 MB/s limit. From 65e953f61bc5786759454825e9de299bf6ef3088 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Tue, 10 Mar 2026 22:44:21 -0700 Subject: [PATCH 18/23] Apply suggestions from doc review --- .../about-throughput-quotas.adoc | 40 +++++++++---------- .../manage-throughput.adoc | 24 +++++------ 2 files changed, 31 insertions(+), 33 deletions(-) diff --git a/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc index 5327026009..19f264da8b 100644 --- a/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc +++ b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc @@ -27,9 +27,10 @@ To configure and manage throughput quotas, see xref:manage:cluster-maintenance/m == Throughput control overview -Redpanda provides two types of throughput limits: broker-wide limits configured using cluster properties and client quotas configured using Kafka API. +Redpanda provides two ways to control throughput: -This page covers client quotas, which enable per-user and per-client rate limiting with fine-grained control through entity hierarchy and precedence rules. For information about broker-wide limits, see xref:manage:cluster-maintenance/manage-throughput.adoc#broker-wide-throughput-limits[Broker-wide throughput limits]. +Broker-wide limits:: Configured using cluster properties. For details, see xref:manage:cluster-maintenance/manage-throughput.adoc#broker-wide-throughput-limits[Broker-wide throughput limits]. +Client throughput quotas:: Configured using the Kafka API. Client quotas enable per-user and per-client rate limiting with fine-grained control through entity hierarchy and precedence rules. This page focuses on client quotas. == Supported quota types @@ -52,16 +53,16 @@ All quota types can be applied to user-based entities, client-based entities, or == Quota entities -Redpanda supports two primary entity types for client quotas: +Redpanda supports two primary entity types for client throughput quotas: -* Client-based quotas (`client_id` or `client_id` prefix): Limit throughput based on the `client_id` field that clients self-declare. Multiple clients using the same `client_id` share the quota. This quota does not require users to authenticate. +* Client-based quotas (`client-id` or `client-id-prefix`): Limit throughput based on the `client_id` field that clients self-declare. Multiple clients using the same client ID share the quota. This quota does not require users to authenticate. * User-based quotas: Limit throughput based on user glossterm:principal[] authenticated through SASL, mTLS, or OIDC. You can also combine both types for fine-grained control, such as limiting a specific user when using a specific client application. === Client-based quotas -Client-based quotas apply to clients identified by their `client_id` field, which is set by the client application. The `client_id` field is typically a configurable property when you create a client with Kafka libraries. Multiple clients using the same `client_id` share the same quota tracking. +Client-based quotas apply to clients identified by their `client_id` field, which is set by the client application. The `client_id` field is typically a configurable property when you create a client with Kafka libraries. Multiple clients using the same client ID share the same quota tracking. Client-based quotas rely on clients honestly reporting their identity and correctly setting the `client_id` property. This makes client-based quotas unsuitable for guaranteeing isolation between tenants. @@ -92,12 +93,12 @@ You can combine user and client identities for fine-grained control over specifi Use combined quotas when: -* You need fine-grained control, for example: "user `alice` using specific application". -* Different rate limits apply to different apps used by the same user. For example: "`alice`'s payment-processor gets 10 MB/s, but `alice`'s analytics-consumer gets 50 MB/s". See <> for examples. +* You need fine-grained control, for example: user `alice` using a specific application. +* Different rate limits apply to different apps used by the same user. For example, `alice`pass:['s] `payment-processor` gets 10 MB/s, but `alice`pass:['s] `analytics-consumer` gets 50 MB/s. See <> for examples. == Quota entity hierarchy -When a request arrives, Redpanda resolves which quota to apply by matching the request's authenticated user principal and `client_id` against configured quotas. Redpanda applies the most specific match, using the following precedence order (highest priority first): +When a request arrives, Redpanda resolves which quota to apply by matching the request's authenticated user principal and client ID against configured quotas. Redpanda applies the most specific match, using the following precedence order (highest priority first): . Exact user + exact client: `/config/users//clients/` . Exact user + client prefix: `/config/users//client-id-prefix/` @@ -130,16 +131,16 @@ rpk cluster quotas alter --add consumer_byte_rate=20000000 --name client-id=app- |=== | User + Client ID | Precedence match -| `user="alice"`, `client_id="app-1"` +| `user=alice`, `client-id=app-1` | Level 1: Exact user + exact client -| `user="alice"`, `client_id="app-2"` +| `user=alice`, `client-id=app-2` | Level 4: Exact user only -| `user="bob"`, `client_id="app-1"` +| `user=bob`, `client-id=app-1` | Level 9: Exact client only -| `user="bob"`, `client_id="app-2"` +| `user=bob`, `client-id=app-2` | Level 12: No quota configured |=== @@ -160,6 +161,8 @@ Redpanda tracks quota usage using a tracker key that determines which connection === Tracking granularity by quota type +To get independent quota tracking per (user, client ID) combination, configure quotas that include both dimensions, such as `/config/users//clients/` or `/config/users//clients/`. + [cols="1,3,2,4", options="header"] |=== | Level | Config path | Tracker key | Isolation behavior @@ -224,12 +227,7 @@ Redpanda tracks quota usage using a tracker key that determines which connection | N/A | No tracking / unlimited throughput |=== - -* User-only quotas share across client_ids: When you configure a quota at the user level (for example, `/config/users/alice`) without specifying a `client_id` dimension, all connections from that user share a single quota bucket regardless of their `client_id`. -* Client-id-only quotas share across users: When you configure a quota at the `client_id` level (for example, `/config/clients/app-1`) without specifying a user dimension, all users connecting with that `client_id` share a single quota bucket. -* Combined quotas for per-(user, client_id) tracking: To get independent quota tracking per (user, client_id) combination, configure quotas that include both dimensions, such as `/config/users//clients/` or `/config/users//clients/`. -* Prefix-based quotas group clients: Client ID prefix quotas (for example, `/config/client-id-prefix/app-`) cause all clients whose `client_id` starts with that prefix to share a quota bucket. - + === Example: User-only quota If you configure a 10 MB/s produce quota for user `alice`: @@ -239,9 +237,9 @@ If you configure a 10 MB/s produce quota for user `alice`: rpk cluster quotas alter --add producer_byte_rate=10000000 --name user=alice ---- -Then `alice` connecting with `client_id=app-1` and `alice` connecting with `client_id=app-2` share the same 10 MB/s limit. Their combined throughput cannot exceed 10 MB/s. +Then `alice` connecting with client ID `app-1` and `alice` connecting with client ID `app-2` share the same 10 MB/s limit. Their combined throughput cannot exceed 10 MB/s. -To give each of `alice`'s clients an independent 10 MB/s limit, configure: +To give each of `alice`pass:['s] clients an independent 10 MB/s limit, configure: [,bash] ---- @@ -259,7 +257,7 @@ rpk cluster quotas alter --add producer_byte_rate=10000000 --default user This quota applies to all users who don't have a more specific quota configured. Each user is tracked independently: `alice` gets her own 10 MB/s bucket, `bob` gets his own 10 MB/s bucket, and so on. -Within each user, all `client_id` values share that user's bucket. `alice` connecting with `client_id=app-1` and `alice` connecting with `client_id=app-2` share the same 10 MB/s limit, while `bob`'s connections have a separate 10 MB/s limit. +Within each user, all client ID values share that user's bucket. `alice` connecting with client ID `app-1` and `alice` connecting with client ID `app-2` share the same 10 MB/s limit, while `bob`pass:['s] connections have a separate 10 MB/s limit. [[throttling-enforcement]] == Throughput throttling enforcement diff --git a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc index 960cacd130..11bc0e022e 100644 --- a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc +++ b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc @@ -31,10 +31,10 @@ You can configure limits at two levels: * Broker limits: These apply to all clients connected to the broker and restrict total traffic on the broker. See <>. ifndef::env-cloud[] -* Client limits: These apply to authenticated users or clients defined by their `client_id`. You can manage client quotas with xref:reference:rpk/rpk-cluster/rpk-cluster-quotas.adoc[`rpk cluster quotas`], with {ui}, or with the Kafka API. When no quotas apply, the client has unlimited throughput. +* Client limits: These apply to authenticated users or clients defined by their client ID. You can manage client quotas with xref:reference:rpk/rpk-cluster/rpk-cluster-quotas.adoc[`rpk cluster quotas`], with {ui}, or with the Kafka API. When no quotas apply, the client has unlimited throughput. endif::[] ifdef::env-cloud[] -* Client limits: These apply to authenticated users or clients defined by their `client_id`. You can manage client quotas with xref:reference:rpk/rpk-cluster/rpk-cluster-quotas.adoc[`rpk cluster quotas`], with the {ui} UI, with the link:https://docs.redpanda.com/api/doc/cloud-dataplane/operation/operation-quotaservice_listquotas[Redpanda Cloud Data Plane API], or with the Kafka API. When no quotas apply, the client has unlimited throughput. +* Client limits: These apply to authenticated users or clients defined by their client ID. You can manage client quotas with xref:reference:rpk/rpk-cluster/rpk-cluster-quotas.adoc[`rpk cluster quotas`], with the {ui} UI, with the link:https://docs.redpanda.com/api/doc/cloud-dataplane/operation/operation-quotaservice_listquotas[Redpanda Cloud Data Plane API], or with the Kafka API. When no quotas apply, the client has unlimited throughput. NOTE: Throughput throttling is supported for BYOC and Dedicated clusters only. endif::[] @@ -522,7 +522,7 @@ To limit throughput for a specific authenticated user across all clients: rpk cluster quotas alter --add producer_byte_rate=2000000 --name user=alice ---- -This limits user `alice` to 2 MB/s for produce requests, regardless of which `client_id` they use. +This limits user `alice` to 2 MB/s for produce requests regardless of the client ID used. To view quotas for a user: @@ -568,7 +568,7 @@ rpk cluster quotas delete --name user=alice === Set combined user and client quotas -You can set quotas for specific (user, client_id) combinations for fine-grained control. +You can set quotas for specific (user, client ID) combinations for fine-grained control. ==== User with specific client @@ -579,7 +579,7 @@ To limit a specific user when using a specific client: rpk cluster quotas alter --add consumer_byte_rate=1000000 --name user=alice --name client-id=consumer-1 ---- -User `alice` using `client_id="consumer-1"` is limited to a 1 MB/s fetch rate. The same user with a different `client_id` would use a different quota (or fall back to less specific matches). +User `alice` using `client-id=consumer-1` is limited to a 1 MB/s fetch rate. The same user with a different client ID would use a different quota (or fall back to less specific matches). To view combined quotas: @@ -594,10 +594,10 @@ To set a shared quota for a user across multiple clients matching a prefix: [,bash] ---- -rpk cluster quotas alter --add producer_byte_rate=3000000 --name user=bob --name client-id-prefix=app1- +rpk cluster quotas alter --add producer_byte_rate=3000000 --name user=bob --name client-id-prefix=app- ---- -All clients used by user `bob` with `client_id` starting with "app1-" share a combined 3 MB/s quota. +All clients used by user `bob` with a client ID starting with `app-` share a combined 3 MB/s quota. ==== Default user with specific client @@ -608,11 +608,11 @@ To set a quota for a specific client across all users: rpk cluster quotas alter --add producer_byte_rate=500000 --default user --name client-id=payment-processor ---- -Any user using `client_id="payment-processor"` is limited to 500 KB/s, unless they have a more specific quota configured. +Any user using `client-id=payment-processor` is limited to 500 KB/s, unless they have a more specific quota configured. === Set client-based quotas -Client-based quotas apply to all users using a specific `client_id`. These quotas do not require authentication. Because `client_id` is self-declared, client-based quotas are not suitable for guaranteeing isolation between tenants. +Client-based quotas apply to all users using a specific client ID. These quotas do not require authentication. Because the client ID is self-declared, client-based quotas are not suitable for guaranteeing isolation between tenants. For multi-tenant environments, Redpanda recommends user-based quotas for per-tenant isolation. @@ -660,11 +660,11 @@ Alternatively, you can view or configure throughput quotas for a group of client rpk cluster quotas alter --add consumer_byte_rate=200000 --name client-id-prefix=consumer- ---- -NOTE: A `client-id-prefix` quota group is not related to Kafka consumer groups. The `client_id` is an application-defined identifier sent with every request. Client libraries typically default to their own name (such as `kgo`, `rdkafka`, `sarama`, or `perf-producer-client`), but applications can set it using the https://kafka.apache.org/documentation/#consumerconfigs_client.id[`client.id`] configuration property. This makes prefix-based quotas useful for grouping related applications (for example, `inventory-service-` to match `inventory-service-1`, `inventory-service-2`, etc.). +NOTE: A `client-id-prefix` quota group is not related to Kafka consumer groups. The client ID is an application-defined identifier sent with every request. Client libraries typically default to their own name (such as `kgo`, `rdkafka`, `sarama`, or `perf-producer-client`), but applications can set it using the https://kafka.apache.org/documentation/#consumerconfigs_client.id[`client.id`] configuration property. This makes prefix-based quotas useful for grouping related applications (for example, `inventory-service-` to match `inventory-service-1`, `inventory-service-2`, etc.). ==== Default client throughput limit -You can apply default throughput limits to clients. Redpanda applies the default limits if no quotas are configured for a specific `client_id` or prefix. +You can apply default throughput limits to clients. Redpanda applies the default limits if no quotas are configured for a specific client ID or prefix. To specify a produce quota of 1 GB/s through the Kafka API (applies across all produce requests to a single broker), run: @@ -714,7 +714,7 @@ endif::[] To identify which clients are actively connected and generating traffic, see <>. -Quota metrics use the `redpanda_quota_rule` label to identify which quota was applied to a request. The label distinguishes between different entity types (user, client, or combinations). See the label values in xref:reference:public-metrics-reference.adoc#redpanda-kafka-quotas-client-quota-throughput[]. +Quota metrics use the `redpanda_quota_rule` label to identify which quota was applied to a request. The label distinguishes between different entity types (user, client, or combinations). See the label values in xref:reference:public-metrics-reference.adoc#redpanda_kafka_quotas_client_quota_throughput[`redpanda_kafka_quotas_client_quota_throughput`]. ifndef::env-cloud[] The `kafka_quotas` logger provides details at the trace level on client quota throttling: From bbb806d114b863226ccf2e211febd23f1d3056b6 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Wed, 11 Mar 2026 16:18:17 -0700 Subject: [PATCH 19/23] Reorganize for clarity --- .../about-throughput-quotas.adoc | 190 +++++++++--------- .../manage-throughput.adoc | 128 ++++++------ 2 files changed, 155 insertions(+), 163 deletions(-) diff --git a/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc index 19f264da8b..ef34bf3abf 100644 --- a/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc +++ b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc @@ -1,9 +1,9 @@ -= About Throughput Quotas -:description: Understand how Redpanda's user-based and client-based throughput quotas work, including entity hierarchy, precedence rules, and quota tracking behavior. += About Client Throughput Quotas +:description: Understand how Redpanda's user-based and client ID-based throughput quotas work, including entity hierarchy, precedence rules, and quota tracking behavior. :page-topic-type: concepts :page-aliases: :personas: platform_admin, developer -:learning-objective-1: Describe the difference between user-based and client-based quotas +:learning-objective-1: Describe the difference between user-based and client ID-based quotas :learning-objective-2: Determine which quota type to use for your use case :learning-objective-3: Explain quota precedence rules and how Redpanda tracks quota usage @@ -49,24 +49,24 @@ Redpanda supports three Kafka API-based quota types: | Limit rate of topic mutation requests (partitions created or deleted per second) |=== -All quota types can be applied to user-based entities, client-based entities, or combinations of both. +All quota types can be applied to groups of client connections based on user principals, client IDs, or combinations of both. == Quota entities -Redpanda supports two primary entity types for client throughput quotas: +Redpanda uses two pieces of identifying information from each client connection to determine which quota applies: -* Client-based quotas (`client-id` or `client-id-prefix`): Limit throughput based on the `client_id` field that clients self-declare. Multiple clients using the same client ID share the quota. This quota does not require users to authenticate. -* User-based quotas: Limit throughput based on user glossterm:principal[] authenticated through SASL, mTLS, or OIDC. +* Client ID: An ID that clients self-declare. Quotas can target an exact client ID (`client-id`) or a prefix (`client-id-prefix`). Multiple client connections that share a client ID or ID prefix are grouped into a single quota entity. +* User glossterm:principal[]: An authenticated identity verified through SASL, mTLS, or OIDC. Connections that share the same user are considered one entity. -You can also combine both types for fine-grained control, such as limiting a specific user when using a specific client application. +You can configure quotas that target either entity type, or combine both for fine-grained control. -=== Client-based quotas +=== Client ID-based quotas -Client-based quotas apply to clients identified by their `client_id` field, which is set by the client application. The `client_id` field is typically a configurable property when you create a client with Kafka libraries. Multiple clients using the same client ID share the same quota tracking. +Client ID-based quotas apply to clients identified by their `client-id` field, which is set by the client application. The client ID is typically a configurable property when you create a client with Kafka libraries. When using client ID-based quotas, multiple clients using the same client ID share the same quota tracking. -Client-based quotas rely on clients honestly reporting their identity and correctly setting the `client_id` property. This makes client-based quotas unsuitable for guaranteeing isolation between tenants. +Client ID-based quotas rely on clients honestly reporting their identity and correctly setting the `client-id` property. This makes client ID-based quotas unsuitable for guaranteeing isolation between tenants. -Use client-based quotas when: +Use client ID-based quotas when: * Authentication is not enabled. * Grouping by application or service name is sufficient. @@ -94,139 +94,133 @@ You can combine user and client identities for fine-grained control over specifi Use combined quotas when: * You need fine-grained control, for example: user `alice` using a specific application. -* Different rate limits apply to different apps used by the same user. For example, `alice`pass:['s] `payment-processor` gets 10 MB/s, but `alice`pass:['s] `analytics-consumer` gets 50 MB/s. See <> for examples. +* Different rate limits apply to different apps used by the same user. For example, `alice`pass:['s] `payment-processor` gets 10 MB/s, but `alice`pass:['s] `analytics-consumer` gets 50 MB/s. See <> for examples. -== Quota entity hierarchy +== Quota precedence and tracking -When a request arrives, Redpanda resolves which quota to apply by matching the request's authenticated user principal and client ID against configured quotas. Redpanda applies the most specific match, using the following precedence order (highest priority first): +When a request arrives, Redpanda resolves which quota to apply by matching the request's authenticated user principal and client ID against configured quotas. Redpanda applies the most specific match, using the precedence order in the following table (highest priority first). -. Exact user + exact client: `/config/users//clients/` -. Exact user + client prefix: `/config/users//client-id-prefix/` -. Exact user + default client: `/config/users//clients/` -. Exact user only: `/config/users/` (applies to all clients for that user) -. Default user + exact client: `/config/users//clients/` -. Default user + client prefix: `/config/users//client-id-prefix/` -. Default user + default client: `/config/users//clients/` -. Default user only: `/config/users/` (applies to all users and clients) -. Exact client only: `/config/clients/` (applies to all users) -. Client prefix only: `/config/client-id-prefix/` (applies to all users) -. Default client only: `/config/clients/` (applies to all users and clients) -. No quota configured: Unlimited throughput (no throttling) +The precedence level that matches also determines how quota usage is tracked. Redpanda tracks quota usage using a tracker key that determines which connections share the same quota bucket. How connections are grouped into buckets depends on the type of entity the quota targets. -IMPORTANT: The `` entity matches any user or client that doesn't have a more specific quota configured. This is different from an empty/unauthenticated user (`user=""`), which is treated as a specific entity. - -The precedence level that matches also determines how quota usage is tracked. See <>. - -=== Example: Precedence resolution - -Given these configured quotas: - -[,bash] ----- -rpk cluster quotas alter --add consumer_byte_rate=5000000 --name user=alice --name client-id=app-1 -rpk cluster quotas alter --add consumer_byte_rate=10000000 --name user=alice -rpk cluster quotas alter --add consumer_byte_rate=20000000 --name client-id=app-1 ----- +To get independent quota tracking per user and client ID combination, configure quotas that include both dimensions, such as `/config/users//clients/` or `/config/users//clients/`. +.Quota precedence, tracking, and isolation by configuration level +[cols="1,2,3,2,3", options="header"] |=== -| User + Client ID | Precedence match - -| `user=alice`, `client-id=app-1` -| Level 1: Exact user + exact client - -| `user=alice`, `client-id=app-2` -| Level 4: Exact user only - -| `user=bob`, `client-id=app-1` -| Level 9: Exact client only - -| `user=bob`, `client-id=app-2` -| Level 12: No quota configured -|=== - -When no quota matches (level 12), the connection is not throttled. - -=== Unauthenticated connections - -Unauthenticated connections have an empty user principal (`user=""`), which is treated as a specific entity, not as `user=`. - -Unauthenticated connections: - -* Fall back to client-only quotas. -* Have unlimited throughput only if no client-only quota matches. - -== Quota tracking and isolation - -Redpanda tracks quota usage using a tracker key that determines which connections share the same quota bucket. The tracker key, and therefore the isolation level, depends on which quota rule matches the request. The Level column below corresponds to the precedence levels in <>. - -=== Tracking granularity by quota type - -To get independent quota tracking per (user, client ID) combination, configure quotas that include both dimensions, such as `/config/users//clients/` or `/config/users//clients/`. - -[cols="1,3,2,4", options="header"] -|=== -| Level | Config path | Tracker key | Isolation behavior +| Level | Match type | Config path | Tracker key | Isolation behavior | 1 +| Exact user + exact client | `/config/users//clients/` -| `(user, client_id)` -| Each unique (user, client_id) pair tracked independently +| `(user, client-id)` +| Each unique (user, client-id) pair tracked independently | 2 +| Exact user + client prefix | `/config/users//client-id-prefix/` -| `(user, prefix)` +| `(user, client-id-prefix)` | Clients matching the prefix share tracking within that user | 3 +| Exact user + default client | `/config/users//clients/` -| `(user, client_id)` -| Each unique (user, client_id) pair tracked independently +| `(user, client-id)` +| Each unique (user, client-id) pair tracked independently | 4 +| Exact user only | `/config/users/` | `user` | All clients for that user share a single tracking bucket | 5 +| Default user + exact client | `/config/users//clients/` -| `(user, client_id)` -| Each unique (user, client_id) pair tracked independently +| `(user, client-id)` +| Each unique (user, client-id) pair tracked independently | 6 +| Default user + client prefix | `/config/users//client-id-prefix/` -| `(user, prefix)` +| `(user, client-id-prefix)` | Clients matching the prefix share tracking within each user | 7 +| Default user + default client | `/config/users//clients/` -| `(user, client_id)` -| Each unique (user, client_id) pair tracked independently +| `(user, client-id)` +| Each unique (user, client-id) pair tracked independently | 8 +| Default user only | `/config/users/` | `user` | All clients for each user share a single tracking bucket (per user) | 9 +| Exact client only | `/config/clients/` -| `client_id` -| All users with that client_id share a single tracking bucket +| `client-id` +| All users with that client ID share a single tracking bucket | 10 +| Client prefix only | `/config/client-id-prefix/` -| `prefix` -| All clients matching the prefix share a single bucket (across all users) +| `client-id-prefix` +| All clients matching the prefix share a single bucket across all users | 11 +| Default client only | `/config/clients/` -| `client_id` -| Each unique client_id tracked independently +| `client-id` +| Each unique client ID tracked independently | 12 | No quota configured | N/A +| N/A | No tracking / unlimited throughput |=== + +IMPORTANT: The `` entity matches any user or client that doesn't have a more specific quota configured. This is different from an empty/unauthenticated user (`user=""`), or undeclared client ID (`client-id=""`), which are treated as specific entities. + +=== Unauthenticated connections + +Unauthenticated connections have an empty user principal (`user=""`) and is not treated as `user=`. + +Unauthenticated connections: + +* Fall back to client-only quotas. +* Have unlimited throughput only if no client-only quota matches. + +=== Example: Precedence resolution + +Given these configured quotas: + +[,bash] +---- +rpk cluster quotas alter --add consumer_byte_rate=5000000 --name user=alice --name client-id=app-1 +rpk cluster quotas alter --add consumer_byte_rate=10000000 --name user=alice +rpk cluster quotas alter --add consumer_byte_rate=20000000 --name client-id=app-1 +---- + +|=== +| User + Client ID | Precedence match + +| `user=alice`, `client-id=app-1` +| Level 1: Exact user + exact client + +| `user=alice`, `client-id=app-2` +| Level 4: Exact user only + +| `user=bob`, `client-id=app-1` +| Level 9: Exact client only + +| `user=bob`, `client-id=app-2` +| Level 12: No quota configured +|=== + +When no quota matches (level 12), the connection is not throttled. === Example: User-only quota @@ -237,7 +231,7 @@ If you configure a 10 MB/s produce quota for user `alice`: rpk cluster quotas alter --add producer_byte_rate=10000000 --name user=alice ---- -Then `alice` connecting with client ID `app-1` and `alice` connecting with client ID `app-2` share the same 10 MB/s limit. Their combined throughput cannot exceed 10 MB/s. +Then `alice` connecting with client ID `app-1` and `alice` connecting with client ID `app-2` share the same 10 MB/s produce limit. To give each of `alice`pass:['s] clients an independent 10 MB/s limit, configure: @@ -257,11 +251,13 @@ rpk cluster quotas alter --add producer_byte_rate=10000000 --default user This quota applies to all users who don't have a more specific quota configured. Each user is tracked independently: `alice` gets her own 10 MB/s bucket, `bob` gets his own 10 MB/s bucket, and so on. -Within each user, all client ID values share that user's bucket. `alice` connecting with client ID `app-1` and `alice` connecting with client ID `app-2` share the same 10 MB/s limit, while `bob`pass:['s] connections have a separate 10 MB/s limit. +Within each user, all client ID values share that user's bucket. `alice` connecting with client ID `app-1` and `alice` connecting with client ID `app-2` share the same 10 MB/s produce limit, while `bob`pass:['s] connections have a separate 10 MB/s limit. [[throttling-enforcement]] == Throughput throttling enforcement +NOTE: As of v24.2, Redpanda enforces all throughput limits per broker, including client throughput. + Redpanda enforces throughput limits by applying backpressure to clients. When a connection exceeds its throughput limit, Redpanda throttles the connection to bring the rate back within the allowed level: . Redpanda adds a `throttle_time_ms` field to responses, indicating how long the client should wait. @@ -275,8 +271,6 @@ ifdef::env-cloud[] In Redpanda Cloud, the throttling delay is set to 30 seconds. endif::[] -NOTE: As of v24.2, Redpanda enforces all throughput limits per broker, including client throughput. - == Default behavior Quotas are opt-in restrictions and not enforced by default. When no quotas are configured, clients have unlimited throughput. diff --git a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc index 11bc0e022e..ce1e6da001 100644 --- a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc +++ b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc @@ -4,7 +4,7 @@ :page-topic-type: how-to :personas: platform_admin, developer :learning-objective-1: Set user-based throughput quotas -:learning-objective-2: Set client-based quotas +:learning-objective-2: Set client ID-based quotas :learning-objective-3: Monitor quota usage and throttling behavior // tag::single-source[] @@ -21,12 +21,6 @@ endif::[] Redpanda throttles throughput on ingress and egress independently, and you can configure limits at the broker and client levels. This prevents clients from causing unbounded network and disk usage on brokers. -After reading this page, you will be able to: - -* [ ] {learning-objective-1} -* [ ] {learning-objective-2} -* [ ] {learning-objective-3} - You can configure limits at two levels: * Broker limits: These apply to all clients connected to the broker and restrict total traffic on the broker. See <>. @@ -39,16 +33,11 @@ ifdef::env-cloud[] NOTE: Throughput throttling is supported for BYOC and Dedicated clusters only. endif::[] -== About throughput quotas - -Redpanda supports two types of throughput quotas: - -* Client-based quotas: Limit throughput based on the self-declared `client_id` field. -* User-based quotas: Limit throughput based on authenticated user glossterm:principal[]. Requires xref:{authentication-doc}[authentication]. - -You can also combine both types for fine-grained control (for example, limiting a specific user when using a specific client application). +After reading this page, you will be able to: -For conceptual information about quota types, entity hierarchy, precedence rules, and how Redpanda tracks and enforces quotas through throttling, see xref:manage:cluster-maintenance/about-throughput-quotas.adoc[]. +* [ ] {learning-objective-1} +* [ ] {learning-objective-2} +* [ ] {learning-objective-3} == View connected client details @@ -509,6 +498,15 @@ endif::[] Redpanda provides configurable throughput quotas for individual clients or authenticated users. Quotas are managed through the Kafka-compatible AlterClientQuotas and DescribeClientQuotas APIs, accessible with `rpk`, Redpanda Console, or Kafka client libraries. +Redpanda supports two types of client throughput quotas: + +* Client ID-based quotas: Limit throughput based on the self-declared `client-id` field. +* User-based quotas: Limit throughput based on authenticated user glossterm:principal[]. Requires xref:{authentication-doc}[authentication]. + +You can also combine both types for fine-grained control (for example, limiting a specific user when using a specific client application). + +For conceptual information about quota types, entity hierarchy, precedence rules, and how Redpanda tracks and enforces quotas through throttling, see xref:manage:cluster-maintenance/about-throughput-quotas.adoc[]. + === Set user-based quotas IMPORTANT: User-based quotas require authentication to be enabled. To set up authentication, see xref:{authentication-doc}[]. @@ -550,7 +548,7 @@ rpk cluster quotas alter --add consumer_byte_rate=5000000 --default user This applies a 5 MB/s fetch quota to all authenticated users who don't have a more specific quota configured. -==== Remove a user quota +=== Remove a user quota To remove a quota for a specific user: @@ -566,57 +564,13 @@ To remove all quotas for a user: rpk cluster quotas delete --name user=alice ---- -=== Set combined user and client quotas - -You can set quotas for specific (user, client ID) combinations for fine-grained control. - -==== User with specific client - -To limit a specific user when using a specific client: - -[,bash] ----- -rpk cluster quotas alter --add consumer_byte_rate=1000000 --name user=alice --name client-id=consumer-1 ----- - -User `alice` using `client-id=consumer-1` is limited to a 1 MB/s fetch rate. The same user with a different client ID would use a different quota (or fall back to less specific matches). +=== Set client ID-based quotas -To view combined quotas: - -[,bash] ----- -rpk cluster quotas describe --name user=alice --name client-id=consumer-1 ----- - -==== User with client prefix - -To set a shared quota for a user across multiple clients matching a prefix: - -[,bash] ----- -rpk cluster quotas alter --add producer_byte_rate=3000000 --name user=bob --name client-id-prefix=app- ----- - -All clients used by user `bob` with a client ID starting with `app-` share a combined 3 MB/s quota. - -==== Default user with specific client - -To set a quota for a specific client across all users: - -[,bash] ----- -rpk cluster quotas alter --add producer_byte_rate=500000 --default user --name client-id=payment-processor ----- - -Any user using `client-id=payment-processor` is limited to 500 KB/s, unless they have a more specific quota configured. - -=== Set client-based quotas - -Client-based quotas apply to all users using a specific client ID. These quotas do not require authentication. Because the client ID is self-declared, client-based quotas are not suitable for guaranteeing isolation between tenants. +Client ID-based quotas apply to all users using a specific client ID. These quotas do not require authentication. Because the client ID is self-declared, client ID-based quotas are not suitable for guaranteeing isolation between tenants. For multi-tenant environments, Redpanda recommends user-based quotas for per-tenant isolation. -==== Individual client throughput limit +==== Individual client ID throughput limit ifdef::env-cloud[] NOTE: The following sections show how to manage throughput with `rpk`. You can also manage throughput with the link:https://docs.redpanda.com/api/doc/cloud-dataplane/operation/operation-quotaservice_listquotas[Redpanda Cloud Data Plane API]. @@ -660,7 +614,7 @@ Alternatively, you can view or configure throughput quotas for a group of client rpk cluster quotas alter --add consumer_byte_rate=200000 --name client-id-prefix=consumer- ---- -NOTE: A `client-id-prefix` quota group is not related to Kafka consumer groups. The client ID is an application-defined identifier sent with every request. Client libraries typically default to their own name (such as `kgo`, `rdkafka`, `sarama`, or `perf-producer-client`), but applications can set it using the https://kafka.apache.org/documentation/#consumerconfigs_client.id[`client.id`] configuration property. This makes prefix-based quotas useful for grouping related applications (for example, `inventory-service-` to match `inventory-service-1`, `inventory-service-2`, etc.). +NOTE: A `client-id-prefix` quota group is not related to Kafka consumer groups. The client ID is an application-defined identifier sent with every request. Client libraries typically default to their own name (such as `kgo`, `rdkafka`, `sarama`, or `perf-producer-client`), but applications can set it using the https://kafka.apache.org/documentation/#consumerconfigs_client.id[`client.id`^] configuration property. This makes prefix-based quotas useful for grouping related applications (for example, `inventory-service-` to match `inventory-service-1`, `inventory-service-2`, etc.). ==== Default client throughput limit @@ -673,6 +627,50 @@ To specify a produce quota of 1 GB/s through the Kafka API (applies across all p rpk cluster quotas alter --default client-id --add producer_byte_rate=1000000000 ---- +=== Set combined user and client quotas + +You can set quotas for specific (user, client ID) combinations for fine-grained control. + +==== User with specific client + +To limit a specific user when using a specific client: + +[,bash] +---- +rpk cluster quotas alter --add consumer_byte_rate=1000000 --name user=alice --name client-id=consumer-1 +---- + +User `alice` using `client-id=consumer-1` is limited to a 1 MB/s fetch rate. The same user with a different client ID would use a different quota (or fall back to less specific matches). + +To view combined quotas: + +[,bash] +---- +rpk cluster quotas describe --name user=alice --name client-id=consumer-1 +---- + +==== User with client prefix + +To set a shared quota for a user across multiple clients matching a prefix: + +[,bash] +---- +rpk cluster quotas alter --add producer_byte_rate=3000000 --name user=bob --name client-id-prefix=app- +---- + +All clients used by user `bob` with a client ID starting with `app-` share a combined 3 MB/s produce quota. + +==== Default user with specific client + +To set a quota for a specific client across all users: + +[,bash] +---- +rpk cluster quotas alter --add producer_byte_rate=500000 --default user --name client-id=payment-processor +---- + +Any user using `client-id=payment-processor` is limited to a 500 KB/s produce rate, unless they have a more specific quota configured. + === Bulk manage client throughput limits To more easily manage multiple quotas, you can use the `cluster quotas describe` and xref:reference:rpk/rpk-cluster/rpk-cluster-quotas-import.adoc[`cluster quotas import`] commands to do a bulk export and update. From 52454d510ffe2252a2e47daafdf4b0d81eb173a3 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Wed, 11 Mar 2026 16:22:19 -0700 Subject: [PATCH 20/23] Minor edit --- modules/get-started/pages/release-notes/redpanda.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/get-started/pages/release-notes/redpanda.adoc b/modules/get-started/pages/release-notes/redpanda.adoc index 9dafbe0ee1..d2f378a46d 100644 --- a/modules/get-started/pages/release-notes/redpanda.adoc +++ b/modules/get-started/pages/release-notes/redpanda.adoc @@ -9,6 +9,6 @@ This topic includes new content added in version {page-component-version}. For a == User-based throughput quotas -Redpanda now supports throughput quotas based on authenticated user principals. Unlike client-based quotas (which rely on self-declared `client_id` values), user-based quotas enforce limits using verified identities from SASL, mTLS, or OIDC authentication. +Redpanda now supports throughput quotas based on authenticated user principals. Unlike client-based quotas (which rely on self-declared `client-id` values), user-based quotas enforce limits using verified identities from SASL, mTLS, or OIDC authentication. You can set quotas for individual users, default users, or fine-grained user/client combinations. See xref:manage:cluster-maintenance/about-throughput-quotas.adoc[] for conceptual details, and xref:manage:cluster-maintenance/manage-throughput.adoc#set-user-based-quotas[Set user-based quotas] to get started. From 6c510aedf081735129cf1878569b40c856ad8e79 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Wed, 11 Mar 2026 16:26:47 -0700 Subject: [PATCH 21/23] Minor grammar edit --- .../pages/cluster-maintenance/about-throughput-quotas.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc index ef34bf3abf..f687f4fe77 100644 --- a/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc +++ b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc @@ -186,7 +186,7 @@ IMPORTANT: The `` entity matches any user or client that doesn't have a === Unauthenticated connections -Unauthenticated connections have an empty user principal (`user=""`) and is not treated as `user=`. +Unauthenticated connections have an empty user principal (`user=""`) and are not treated as `user=`. Unauthenticated connections: From 67e29bb4cd4193e696e5027c6623134df3e1d4f7 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Wed, 11 Mar 2026 16:32:21 -0700 Subject: [PATCH 22/23] Change to normal unordered list with no extra formatting --- .../pages/cluster-maintenance/about-throughput-quotas.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc index f687f4fe77..ad1d468ddc 100644 --- a/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc +++ b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc @@ -29,8 +29,8 @@ To configure and manage throughput quotas, see xref:manage:cluster-maintenance/m Redpanda provides two ways to control throughput: -Broker-wide limits:: Configured using cluster properties. For details, see xref:manage:cluster-maintenance/manage-throughput.adoc#broker-wide-throughput-limits[Broker-wide throughput limits]. -Client throughput quotas:: Configured using the Kafka API. Client quotas enable per-user and per-client rate limiting with fine-grained control through entity hierarchy and precedence rules. This page focuses on client quotas. +* Broker-wide limits: Configured using cluster properties. For details, see xref:manage:cluster-maintenance/manage-throughput.adoc#broker-wide-throughput-limits[Broker-wide throughput limits]. +* Client throughput quotas: Configured using the Kafka API. Client quotas enable per-user and per-client rate limiting with fine-grained control through entity hierarchy and precedence rules. This page focuses on client quotas. == Supported quota types From 5113b70a09900ae53a8f47145e87719f9f0bfd88 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Wed, 11 Mar 2026 17:06:51 -0700 Subject: [PATCH 23/23] Use simpler passthrough --- .../pages/cluster-maintenance/about-throughput-quotas.adoc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc index ad1d468ddc..ebb13c63c7 100644 --- a/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc +++ b/modules/manage/pages/cluster-maintenance/about-throughput-quotas.adoc @@ -94,7 +94,7 @@ You can combine user and client identities for fine-grained control over specifi Use combined quotas when: * You need fine-grained control, for example: user `alice` using a specific application. -* Different rate limits apply to different apps used by the same user. For example, `alice`pass:['s] `payment-processor` gets 10 MB/s, but `alice`pass:['s] `analytics-consumer` gets 50 MB/s. See <> for examples. +* Different rate limits apply to different apps used by the same user. For example, `alice`+'s+ `payment-processor` gets 10 MB/s, but `alice`+'s+ `analytics-consumer` gets 50 MB/s. See <> for examples. == Quota precedence and tracking @@ -233,7 +233,7 @@ rpk cluster quotas alter --add producer_byte_rate=10000000 --name user=alice Then `alice` connecting with client ID `app-1` and `alice` connecting with client ID `app-2` share the same 10 MB/s produce limit. -To give each of `alice`pass:['s] clients an independent 10 MB/s limit, configure: +To give each of `alice`+'s+ clients an independent 10 MB/s limit, configure: [,bash] ---- @@ -251,7 +251,7 @@ rpk cluster quotas alter --add producer_byte_rate=10000000 --default user This quota applies to all users who don't have a more specific quota configured. Each user is tracked independently: `alice` gets her own 10 MB/s bucket, `bob` gets his own 10 MB/s bucket, and so on. -Within each user, all client ID values share that user's bucket. `alice` connecting with client ID `app-1` and `alice` connecting with client ID `app-2` share the same 10 MB/s produce limit, while `bob`pass:['s] connections have a separate 10 MB/s limit. +Within each user, all client ID values share that user's bucket. `alice` connecting with client ID `app-1` and `alice` connecting with client ID `app-2` share the same 10 MB/s produce limit, while `bob`+'s+ connections have a separate 10 MB/s limit. [[throttling-enforcement]] == Throughput throttling enforcement