Doc 1602: Cloud Topics GA

modules/ROOT/nav.adoc

@@ -36,7 +36,9 @@
 ** xref:develop:kafka-clients.adoc[Kafka Compatibility]
 ** xref:develop:benchmark.adoc[]
 ** xref:develop:http-proxy.adoc[]
-** xref:develop:config-topics.adoc[]
+** xref:develop:manage-topics/index.adoc[]
+*** xref:develop:manage-topics/config-topics.adoc[]
+*** xref:develop:manage-topics/cloud-topics.adoc[]
 ** xref:console:ui/edit-topic-configuration.adoc[Edit Topic Configuration]
 ** xref:develop:produce-data/index.adoc[Produce Data]
 *** xref:develop:produce-data/configure-producers.adoc[]

modules/console/pages/ui/edit-topic-configuration.adoc

@@ -18,4 +18,4 @@
 == Suggested reading
 
 - xref:reference:properties/topic-properties.adoc[]
-- xref:develop:config-topics.adoc[]
+- xref:develop:manage-topics/config-topics.adoc[]

modules/deploy/pages/redpanda/manual/production/dev-deployment.adoc

@@ -8,7 +8,7 @@ You can deploy Redpanda using well-known configuration properties optimized for
 
 [NOTE]
 ====
-* Development mode enables write caching by default. This is a relaxed mode of xref:develop:produce-data/configure-producers.adoc#acksall[`acks=all`] that acknowledges a message as soon as it is received and acknowledged on a majority of brokers, without waiting for it to fsync to disk. Write caching provides lower latency while still ensuring that a majority of brokers acknowledge the write. For more information, or to disable this, see xref:develop:config-topics.adoc#configure-write-caching[write caching].
+* Development mode enables write caching by default. This is a relaxed mode of xref:develop:produce-data/configure-producers.adoc#acksall[`acks=all`] that acknowledges a message as soon as it is received and acknowledged on a majority of brokers, without waiting for it to fsync to disk. Write caching provides lower latency while still ensuring that a majority of brokers acknowledge the write. For more information, or to disable this, see xref:develop:manage-topics/config-topics.adoc#configure-write-caching[write caching].
 * Development mode also bypasses `fsync`, acknowledging messages before they're stored to disk. This reduces the durability of messages, could cause potential data loss, and could give unrealistic performance characteristics for a production environment.
 ====
 

modules/develop/pages/kafka-clients.adoc

@@ -60,7 +60,7 @@ endif::[]
 * HTTP Proxy (pandaproxy): Unlike other REST proxy implementations in the Kafka ecosystem, Redpanda HTTP Proxy does not support topic and ACLs CRUD through the HTTP Proxy. HTTP Proxy is designed for clients producing and consuming data that do not perform administrative functions.
 ifdef::env-cloud[]
 +
-* The `delete.retention.ms` topic configuration in Kafka is not supported. Tombstone markers are not removed for topics with a `compact` xref:get-started:config-topics.adoc#change-the-cleanup-policy[cleanup policy]. Redpanda only deletes tombstone markers when topics with a cleanup policy of `compact,delete` have reached their xref:get-started:create-topic.adoc[retention limits].
+* The `delete.retention.ms` topic configuration in Kafka is not supported. Tombstone markers are not removed for topics with a `compact` cleanup policy. Redpanda only deletes tombstone markers when topics with a cleanup policy of `compact,delete` have reached their retention limits.
 endif::[]
 ifndef::env-cloud[]
 +

modules/develop/pages/manage-topics/cloud-topics.adoc

@@ -0,0 +1,92 @@
+= Manage Cloud Topics
+:description: Cloud Topics are Redpanda topics that enable users to trade off latency for lower costs.
+:learning-objective-1: Describe the latency and cost trade-offs of Cloud Topics compared to standard Redpanda topics
+:learning-objective-2: Create a Cloud Topic using rpk after enabling Cloud Topics on your cluster
+:learning-objective-3: Identify Cloud Topics limitations and configurations that reduce cross-AZ networking costs
+// tag::single-source[]
+
+ifndef::env-cloud[]
+[NOTE]
+====
+include::shared:partial$enterprise-license.adoc[]
+====
+endif::[]
+
+Starting in v26.1, Redpanda provides Cloud Topics to support multi-modal streaming workloads in the most cost-effective way possible: as a configurable option on a single cluster. While standard Redpanda xref:config-topics.adoc[topics] that use local storage or Tiered Storage are ideal for latency-tolerant  workloads (for example, for audit logs or analytics), Cloud Topics are optimized for latency-tolerant, high-throughput workloads where cross-AZ networking charges are a major consideration that can become the dominant cost driver at high throughput. These workloads can include observability streams, offline analytics, AI/ML model training data feeds, or development environments that have flexible latency requirements.
+
+Instead of replicating every byte across expensive network links, Cloud Topics leverage durable, inexpensive Cloud storage (S3, ADLS, GCS, MinIO) as the primary mechanism to back up data. This eliminates over 90% of the cost of replicating data over network links in multi-AZ clusters. The difference in end-to-end latency is typically between 500 ms - 1 s, which is often acceptable for many streaming workloads, and can unlock new streaming use cases that previously were not cost effective.
+
+With Cloud Topics, data from the client is not acknowledged until it is uploaded to cloud storage. This maintains durability in the face of infrastructure failures, but results in an increase in both produce latency and end-to-end latency, driven by both batching of produced data and the inherent latency of the underlying object store. You should generally expect latencies in the range of 250 ms - 1 s.
+
+== Prerequisites
+
+- xref:get-started:rpk-install.adoc[] v26.1 or later.
+ifndef::env-cloud[]
+- xref:manage:tiered-storage.adoc#set-up-tiered-storage[Enable Tiered Storage] on your Redpanda cluster.
++
+[NOTE]
+====
+When you enable Tiered Storage at the cluster level, by default, all new topics are Tiered Storage (or standard) topics. However, if you plan to use Cloud Topics for all new topics in a Redpanda cluster, be sure to set the following cluster-level property:
++
+[,bash]
+----
+default_redpanda_storage_mode=cloud
+----
++
+This ensures that newly-created Redpanda topics are Cloud Topics by default.
++
+For details, see xref:manage:tiered-storage.adoc#enable-tiered-storage-for-a-cluster[Enable Tiered Storage for a cluster].
+====
+
+- xref:manage:tiered-storage.adoc#configure-object-storage[Configure object storage].
+- Ensure that you have an Enterprise license.
++
+To check your license status, run:
++
+[,bash]
+----
+rpk cluster license info
+----
++
+- If you intend to use Docker Compose while working with Cloud Topics, see the https://docs.docker.com/compose/install/[Docker Compose installation documentation^]. To verify you have it installed, run:
++
+[source,bash]
+----
+docker compose version
+----
++
+You should see the installed Docker Compose version.
++
+For more details about using Docker Compose with Redpanda, see the xref:get-started:quick-start.adoc[Redpanda Self-Managed Quickstart].
+
+endif::[]
+
+== Limitations
+
+The Remote Read Replicas feature is not supported for Cloud Topics.
+
+IMPORTANT: Any topic specified to be a Cloud Topic cannot be subsequently converted back to a standard Redpanda topic that uses local or Tiered Storage.
+
+== Enable Cloud Topics for your cluster
+
+To enable Cloud Topics for a cluster:
+
+[,bash]
+----
+rpk cluster config set cloud_topics_enabled=true
+----
+
+NOTE: This configuration update requires a restart.
+
+[,bash]
+----
+rpk topic create -c redpanda.storage.mode=cloud <new-cloud-topic-name>
+TOPIC                    STATUS
+audit.analytics.may2025  OK
+----
+
+You can make a topic a Cloud Topic only at topic creation time.
+
+In addition to replication, cross-AZ ingress (producer) and egress (consumer) traffic can also contribute substantially to cloud networking cost. When running multi-AZ clusters in general, Redpanda strongly recommends using xref:../consume-data/follower-fetching.adoc[Follower Fetching], which allows consumers to avoid crossing network zones. When possible, you can use xref:../produce-data/leader-pinning.adoc[leader pinning], which positions a topic's partition leader close to the producers, providing a similar benefit for ingress traffic. These features can add additional savings to the replication cost savings of Cloud Topics.
+
+// end::single-source[]