diff --git a/src/current/_includes/v20.2/misc/tooling.md b/src/current/_includes/v20.2/misc/tooling.md
index a6d0a64b30b..3cb34b7d2e2 100644
--- a/src/current/_includes/v20.2/misc/tooling.md
+++ b/src/current/_includes/v20.2/misc/tooling.md
@@ -9,7 +9,7 @@ Cockroach Labs has partnered with open-source projects, vendors, and individuals
Unless explicitly stated, support for a [driver](#drivers) or [data access framework](#data-access-frameworks-e-g-orms) does not include [automatic, client-side transaction retry handling](transactions.html#client-side-intervention). For client-side transaction retry handling samples, see [Example Apps](example-apps.html).
{{site.data.alerts.end}}
-If you encounter problems using CockroachDB with any of the tools listed on this page, please [open an issue](https://github.com/cockroachdb/cockroach/issues/new) with details to help us make progress toward better support.
+If you encounter problems using CockroachDB with any of the tools listed on this page, please open an issue with details to help us make progress toward better support.
For a list of tools supported by the CockroachDB community, see [Third-Party Tools Supported by the Community](community-tooling.html).
@@ -19,21 +19,21 @@ For a list of tools supported by the CockroachDB community, see [Third-Party Too
|----------+--------+-----------------------+---------------------+---------------------+----------|
| C | [libpq](http://www.postgresql.org/docs/13/static/libpq.html)| PostgreSQL 13 | Beta | N/A | N/A |
| C# (.NET) | [Npgsql](https://www.nuget.org/packages/Npgsql/) | 4.1.3.1 | Beta | N/A | [Build a C# App with CockroachDB (Npgsql)](build-a-csharp-app-with-cockroachdb.html) |
-| Go | [pgx](https://github.com/jackc/pgx/releases)
[pq](https://github.com/lib/pq) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/pgx.go ||var supportedPGXTag = "||"\n\n %}
(use latest version of CockroachDB adapter)
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/libpq.go ||var libPQSupportedTag = "||"\n\n %} | Full
Full | [`crdbpgx`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbpgx)
(includes client-side transaction retry handling)
N/A | [Build a Go App with CockroachDB (pgx)](build-a-go-app-with-cockroachdb.html)
[Build a Go App with CockroachDB (pq)](build-a-go-app-with-cockroachdb-pq.html) ||
-|| Java | [JDBC](https://jdbc.postgresql.org/download/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/pgjdbc.go ||var supportedPGJDBCTag = "||"\n\n %} | Full | N/A | [Build a Java App with CockroachDB (JDBC)](build-a-java-app-with-cockroachdb.html) |
+| Go | [pgx](https://github.com/jackc/pgx/releases)
[pq](https://github.com/lib/pq) | v5.3.1
(use latest version of CockroachDB adapter)
v1.10.5 | Full
Full | [`crdbpgx`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbpgx)
(includes client-side transaction retry handling)
N/A | [Build a Go App with CockroachDB (pgx)](build-a-go-app-with-cockroachdb.html)
[Build a Go App with CockroachDB (pq)](build-a-go-app-with-cockroachdb-pq.html) ||
+|| Java | [JDBC](https://jdbc.postgresql.org/download/) | REL42.7.3 | Full | N/A | [Build a Java App with CockroachDB (JDBC)](build-a-java-app-with-cockroachdb.html) |
| JavaScript | [pg](https://www.npmjs.com/package/pg) | 8.2.1 | Full | N/A | [Build a Node.js App with CockroachDB (pg)](build-a-nodejs-app-with-cockroachdb.html) |
| Python | [psycopg2](https://www.psycopg.org/docs/install.html) | 2.8.6 | Full | N/A | [Build a Python App with CockroachDB (psycopg2)](build-a-python-app-with-cockroachdb.html) |
-| Ruby | [pg](https://rubygems.org/gems/pg) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/ruby_pg.go ||var rubyPGVersion = "||"\n\n %} | Full | N/A | [Build a Ruby App with CockroachDB (pg)](build-a-ruby-app-with-cockroachdb.html) |
+| Ruby | [pg](https://rubygems.org/gems/pg) | v1.4.6 | Full | N/A | [Build a Ruby App with CockroachDB (pg)](build-a-ruby-app-with-cockroachdb.html) |
## Data access frameworks (e.g., ORMs)
| Language | Framework | Latest tested version | Support level | CockroachDB adapter | Tutorial |
|----------+-----------+-----------------------+---------------+---------------------+----------|
-| Go | [GORM](https://github.com/jinzhu/gorm/releases)
[go-pg](https://github.com/go-pg/pg)
[upper/db](https://github.com/upper/db) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/gorm.go ||var gormSupportedTag = "||"\n\n %}
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/gopg.go ||var gopgSupportedTag = "||"\n\n %}
v4 | Full
Full
Full | [`crdbgorm`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbgorm)
(includes client-side transaction retry handling)
N/A
N/A | [Build a Go App with CockroachDB (GORM)](build-a-go-app-with-cockroachdb-gorm.html)
N/A
[Build a Go App with CockroachDB (upper/db)](build-a-go-app-with-cockroachdb-upperdb.html) ||
-|| Java | [Hibernate](https://hibernate.org/orm/)
(including [Hibernate Spatial](https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#spatial))
[jOOQ](https://www.jooq.org/)
[MyBatis](https://mybatis.org/mybatis-3/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/hibernate.go ||var supportedHibernateTag = "||"\n\n %} (must be 5.4.19)
3.13.2 (must be 3.13.0)
3.5.5| Full
Full
Full | N/A
N/A
N/A | [Build a Java App with CockroachDB (Hibernate)](build-a-java-app-with-cockroachdb-hibernate.html)
[Build a Java App with CockroachDB (jOOQ)](build-a-java-app-with-cockroachdb-jooq.html)
[Build a Spring App with CockroachDB (MyBatis)](build-a-spring-app-with-cockroachdb-mybatis.html) ||
-|| JavaScript/TypeScript | [Sequelize](https://www.npmjs.com/package/sequelize)
[TypeORM](https://www.npmjs.com/package/typeorm) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/sequelize.go ||var supportedSequelizeCockroachDBRelease = "||"\n\n %}
(use latest version of CockroachDB adapter)
0.3.17 {% comment %}{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/typeorm.go ||const supportedTypeORMRelease = "||"\n %}{% endcomment %} | Full
Full | [`sequelize-cockroachdb`](https://www.npmjs.com/package/sequelize-cockroachdb)
N/A | [Build a Node.js App with CockroachDB (Sequelize)](build-a-nodejs-app-with-cockroachdb-sequelize.html)
[Build a TypeScript App with CockroachDB (TypeORM)](build-a-typescript-app-with-cockroachdb.html) ||
-|| Ruby | [ActiveRecord](https://rubygems.org/gems/activerecord)
[RGeo/RGeo-ActiveRecord](https://github.com/cockroachdb/activerecord-cockroachdb-adapter#working-with-spatial-data) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/activerecord.go ||var supportedRailsVersion = "||"\nvar %}
(use latest version of CockroachDB adapter) | Full | [`activerecord-cockroachdb-adapter`](https://rubygems.org/gems/activerecord-cockroachdb-adapter)
(includes client-side transaction retry handling) | [Build a Ruby App with CockroachDB (ActiveRecord)](build-a-ruby-app-with-cockroachdb-activerecord.html) ||
-|| Python | [Django](https://pypi.org/project/Django/)
(including [GeoDjango](https://docs.djangoproject.com/en/3.1/ref/contrib/gis/))
[peewee](https://github.com/coleifer/peewee/)
[SQLAlchemy](https://www.sqlalchemy.org/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/django.go ||var djangoSupportedTag = "cockroach-||"\nvar %}
(use latest version of CockroachDB adapter)
3.13.3
0.7.13
1.4.17
(use latest version of CockroachDB adapter) | Full
Full
Full
Full | [`django-cockroachdb`](https://pypi.org/project/django-cockroachdb/)
N/A
N/A
[`sqlalchemy-cockroachdb`](https://pypi.org/project/sqlalchemy-cockroachdb)
(includes client-side transaction retry handling) | [Build a Python App with CockroachDB (Django)](build-a-python-app-with-cockroachdb-django.html)
N/A (See [peewee docs](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#cockroach-database).)
[Build a Python App with CockroachDB (SQLAlchemy)](build-a-python-app-with-cockroachdb-sqlalchemy.html) |
+| Go | [GORM](https://github.com/jinzhu/gorm/releases)
[go-pg](https://github.com/go-pg/pg)
[upper/db](https://github.com/upper/db) | v1.24.1
v10.9.0
v4 | Full
Full
Full | [`crdbgorm`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbgorm)
(includes client-side transaction retry handling)
N/A
N/A | [Build a Go App with CockroachDB (GORM)](build-a-go-app-with-cockroachdb-gorm.html)
N/A
[Build a Go App with CockroachDB (upper/db)](build-a-go-app-with-cockroachdb-upperdb.html) ||
+|| Java | [Hibernate](https://hibernate.org/orm/)
(including [Hibernate Spatial](https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#spatial))
[jOOQ](https://www.jooq.org/)
[MyBatis](https://mybatis.org/mybatis-3/) | 6.6.20 (must be 5.4.19)
3.13.2 (must be 3.13.0)
3.5.5| Full
Full
Full | N/A
N/A
N/A | [Build a Java App with CockroachDB (Hibernate)](build-a-java-app-with-cockroachdb-hibernate.html)
[Build a Java App with CockroachDB (jOOQ)](build-a-java-app-with-cockroachdb-jooq.html)
[Build a Spring App with CockroachDB (MyBatis)](build-a-spring-app-with-cockroachdb-mybatis.html) ||
+|| JavaScript/TypeScript | [Sequelize](https://www.npmjs.com/package/sequelize)
[TypeORM](https://www.npmjs.com/package/typeorm) | v6.0.5
(use latest version of CockroachDB adapter)
0.3.17 {% comment %}remove-unsafe-crdb-setting{% endcomment %} | Full
Full | [`sequelize-cockroachdb`](https://www.npmjs.com/package/sequelize-cockroachdb)
N/A | [Build a Node.js App with CockroachDB (Sequelize)](build-a-nodejs-app-with-cockroachdb-sequelize.html)
[Build a TypeScript App with CockroachDB (TypeORM)](build-a-typescript-app-with-cockroachdb.html) ||
+|| Ruby | [ActiveRecord](https://rubygems.org/gems/activerecord)
[RGeo/RGeo-ActiveRecord](https://github.com/cockroachdb/activerecord-cockroachdb-adapter#working-with-spatial-data) | 8.1.0
(use latest version of CockroachDB adapter) | Full | [`activerecord-cockroachdb-adapter`](https://rubygems.org/gems/activerecord-cockroachdb-adapter)
(includes client-side transaction retry handling) | [Build a Ruby App with CockroachDB (ActiveRecord)](build-a-ruby-app-with-cockroachdb-activerecord.html) ||
+|| Python | [Django](https://pypi.org/project/Django/)
(including [GeoDjango](https://docs.djangoproject.com/en/3.1/ref/contrib/gis/))
[peewee](https://github.com/coleifer/peewee/)
[SQLAlchemy](https://www.sqlalchemy.org/) | 4.1.x
(use latest version of CockroachDB adapter)
3.13.3
0.7.13
1.4.17
(use latest version of CockroachDB adapter) | Full
Full
Full
Full | [`django-cockroachdb`](https://pypi.org/project/django-cockroachdb/)
N/A
N/A
[`sqlalchemy-cockroachdb`](https://pypi.org/project/sqlalchemy-cockroachdb)
(includes client-side transaction retry handling) | [Build a Python App with CockroachDB (Django)](build-a-python-app-with-cockroachdb-django.html)
N/A (See [peewee docs](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#cockroach-database).)
[Build a Python App with CockroachDB (SQLAlchemy)](build-a-python-app-with-cockroachdb-sqlalchemy.html) |
## Application frameworks
diff --git a/src/current/_includes/v20.2/orchestration/kubernetes-prometheus-alertmanager.md b/src/current/_includes/v20.2/orchestration/kubernetes-prometheus-alertmanager.md
index 9f7fbd4136e..707dfdb52c6 100644
--- a/src/current/_includes/v20.2/orchestration/kubernetes-prometheus-alertmanager.md
+++ b/src/current/_includes/v20.2/orchestration/kubernetes-prometheus-alertmanager.md
@@ -90,7 +90,7 @@ If you're on Hosted GKE, before starting, make sure the email address associated
prometheus-operator 1/1 1 1 27s
~~~
-4. Use our [`prometheus.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/prometheus.yaml) file to create the various objects necessary to run a Prometheus instance:
+4. Use our [`prometheus.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/prometheus.yaml) file to create the various objects necessary to run a Prometheus instance:
{{site.data.alerts.callout_info}}
By default, this manifest uses the secret name generated by the CockroachDB Kubernetes Operator. If you generated your own certificates and keys when starting CockroachDB, be sure that `ca.secret.name` matches the name of the node secret you created.
@@ -99,7 +99,7 @@ If you're on Hosted GKE, before starting, make sure the email address associated
{% include copy-clipboard.html %}
~~~ shell
$ kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/prometheus.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/prometheus.yaml
~~~
~~~
@@ -137,14 +137,14 @@ If you're on Hosted GKE, before starting, make sure the email address associated
### Configure Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
-1. Download our alertmanager-config.yaml configuration file:
+1. Download our alertmanager-config.yaml configuration file:
{% include copy-clipboard.html %}
~~~ shell
$ curl -OOOOOOOOO \
- https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager-config.yaml
+ https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager-config.yaml
~~~
2. Edit the `alertmanager-config.yaml` file to [specify the desired receivers for notifications](https://prometheus.io/docs/alerting/configuration/#receiver). Initially, the file contains a placeholder web hook.
@@ -174,12 +174,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
The name of the secret, `alertmanager-cockroachdb`, must match the name used in the `alertmanager.yaml` file. If they differ, the Alertmanager instance will start without configuration, and nothing will happen.
{{site.data.alerts.end}}
-4. Use our [`alertmanager.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
+4. Use our [`alertmanager.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
{% include copy-clipboard.html %}
~~~ shell
$ kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml
~~~
~~~
@@ -204,12 +204,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
-7. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml):
+7. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml):
{% include copy-clipboard.html %}
~~~ shell
$ kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alert-rules.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml
~~~
~~~
diff --git a/src/current/_includes/v20.2/orchestration/start-cockroachdb-insecure.md b/src/current/_includes/v20.2/orchestration/start-cockroachdb-insecure.md
index 695d0832fce..ba581071373 100644
--- a/src/current/_includes/v20.2/orchestration/start-cockroachdb-insecure.md
+++ b/src/current/_includes/v20.2/orchestration/start-cockroachdb-insecure.md
@@ -1,10 +1,10 @@
-1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it.
+1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it.
- Download [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml):
+ Download [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml):
{% include copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml
~~~
{{site.data.alerts.callout_danger}}
@@ -40,11 +40,11 @@
Alternatively, if you'd rather start with a configuration file that has been customized for performance:
- 1. Download our [performance version of `cockroachdb-statefulset-insecure.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml):
+ 1. Download our [performance version of `cockroachdb-statefulset-insecure.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml):
{% include copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml
~~~
2. Modify the file wherever there is a `TODO` comment.
@@ -85,12 +85,12 @@
pvc-5315efda-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-2 27s
~~~
-4. Use our [`cluster-init.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
+4. Use our [`cluster-init.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
{% include copy-clipboard.html %}
~~~ shell
$ kubectl create \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml
~~~
~~~
diff --git a/src/current/_includes/v20.2/orchestration/start-cockroachdb-local-insecure.md b/src/current/_includes/v20.2/orchestration/start-cockroachdb-local-insecure.md
index bebb6eb3062..c22f405666e 100644
--- a/src/current/_includes/v20.2/orchestration/start-cockroachdb-local-insecure.md
+++ b/src/current/_includes/v20.2/orchestration/start-cockroachdb-local-insecure.md
@@ -1,8 +1,8 @@
-1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it:
+1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it:
{% include copy-clipboard.html %}
~~~ shell
- $ kubectl create -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml
+ $ kubectl create -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml
~~~
~~~
@@ -41,12 +41,12 @@
pvc-5315efda-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-2 27s
~~~
-4. Use our [`cluster-init.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
+4. Use our [`cluster-init.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
{% include copy-clipboard.html %}
~~~ shell
$ kubectl create \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml
~~~
~~~
diff --git a/src/current/_includes/v20.2/orchestration/start-cockroachdb-secure.md b/src/current/_includes/v20.2/orchestration/start-cockroachdb-secure.md
index 415e16323fc..00f4f929d25 100644
--- a/src/current/_includes/v20.2/orchestration/start-cockroachdb-secure.md
+++ b/src/current/_includes/v20.2/orchestration/start-cockroachdb-secure.md
@@ -1,10 +1,10 @@
#### Set up configuration file
-1. Download and modify our [StatefulSet configuration](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml):
+1. Download and modify our [StatefulSet configuration](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml
~~~
1. Allocate CPU and memory resources to CockroachDB on each pod. These settings should be appropriate for your workload. For more context on provisioning CPU and memory, see the [Production Checklist](recommended-production-settings.html#hardware).
diff --git a/src/current/_includes/v20.2/orchestration/test-cluster-secure.md b/src/current/_includes/v20.2/orchestration/test-cluster-secure.md
index b162949a7b5..b745ff3b529 100644
--- a/src/current/_includes/v20.2/orchestration/test-cluster-secure.md
+++ b/src/current/_includes/v20.2/orchestration/test-cluster-secure.md
@@ -30,7 +30,7 @@ To use the built-in SQL client, you need to launch a pod that runs indefinitely
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/client.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/client.yaml
~~~
~~~
@@ -72,14 +72,14 @@ To use the built-in SQL client, you need to launch a pod that runs indefinitely
To use the built-in SQL client, you need to launch a pod that runs indefinitely with the `cockroach` binary inside it, get a shell into the pod, and then start the built-in SQL client.
-1. From your local workstation, use our [`client-secure.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/client-secure.yaml) file to launch a pod and keep it running indefinitely.
+1. From your local workstation, use our [`client-secure.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/client-secure.yaml) file to launch a pod and keep it running indefinitely.
1. Download the file:
{% include_cached copy-clipboard.html %}
~~~ shell
$ curl -OOOOOOOOO \
- https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/client-secure.yaml
+ https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/client-secure.yaml
~~~
1. In the file, change `serviceAccountName: cockroachdb` to `serviceAccountName: my-release-cockroachdb`.
diff --git a/src/current/_includes/v23.1/misc/tooling.md b/src/current/_includes/v23.1/misc/tooling.md
index 4ade5aaeb60..a8e414e92f0 100644
--- a/src/current/_includes/v23.1/misc/tooling.md
+++ b/src/current/_includes/v23.1/misc/tooling.md
@@ -23,7 +23,7 @@ Customers should contact their account team before moving production workloads t
Unless explicitly stated, support for a [driver](#drivers) or [data access framework](#data-access-frameworks-e-g-orms) does not include [automatic, client-side transaction retry handling]({% link {{ page.version.version }}/transaction-retry-error-reference.md %}#client-side-retry-handling). For client-side transaction retry handling samples, see [Example Apps]({% link {{ page.version.version }}/example-apps.md %}).
{{site.data.alerts.end}}
-If you encounter problems using CockroachDB with any of the tools listed on this page, please [open an issue](https://github.com/cockroachdb/cockroach/issues/new) with details to help us make progress toward better support.
+If you encounter problems using CockroachDB with any of the tools listed on this page, please open an issue with details to help us make progress toward better support.
For a list of tools supported by the CockroachDB community, see [Third-Party Tools Supported by the Community]({% link {{ page.version.version }}/community-tooling.md %}).
@@ -32,23 +32,23 @@ For a list of tools supported by the CockroachDB community, see [Third-Party Too
| Language | Driver | Latest tested version | Support level | CockroachDB adapter | Tutorial |
|----------+--------+-----------------------+---------------------+---------------------+----------|
| C | [libpq](http://www.postgresql.org/docs/13/static/libpq.html)| PostgreSQL 13 | Partial | N/A | N/A |
-| C# (.NET) | [Npgsql](https://www.nuget.org/packages/Npgsql/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/npgsql.go ||var npgsqlSupportedTag = "v||"\n\n %} | Full | N/A | [Build a C# App with CockroachDB (Npgsql)](build-a-csharp-app-with-cockroachdb.html) |
-| Go | [pgx](https://github.com/jackc/pgx/releases)
[pq](https://github.com/lib/pq) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/pgx.go ||var supportedPGXTag = "||"\n\n %}
(use latest version of CockroachDB adapter)
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/libpq.go ||var libPQSupportedTag = "||"\n\n %} | Full
Full | [`crdbpgx`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbpgx)
(includes client-side transaction retry handling)
N/A | [Build a Go App with CockroachDB (pgx)](build-a-go-app-with-cockroachdb.html)
[Build a Go App with CockroachDB (pq)](build-a-go-app-with-cockroachdb-pq.html) |
-| Java | [JDBC](https://jdbc.postgresql.org/download/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/pgjdbc.go ||var supportedPGJDBCTag = "||"\n\n %} | Full | N/A | [Build a Java App with CockroachDB (JDBC)](build-a-java-app-with-cockroachdb.html) |
+| C# (.NET) | [Npgsql](https://www.nuget.org/packages/Npgsql/) | 7.0.2 | Full | N/A | [Build a C# App with CockroachDB (Npgsql)](build-a-csharp-app-with-cockroachdb.html) |
+| Go | [pgx](https://github.com/jackc/pgx/releases)
[pq](https://github.com/lib/pq) | v5.3.1
(use latest version of CockroachDB adapter)
v1.10.5 | Full
Full | [`crdbpgx`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbpgx)
(includes client-side transaction retry handling)
N/A | [Build a Go App with CockroachDB (pgx)](build-a-go-app-with-cockroachdb.html)
[Build a Go App with CockroachDB (pq)](build-a-go-app-with-cockroachdb-pq.html) |
+| Java | [JDBC](https://jdbc.postgresql.org/download/) | REL42.7.3 | Full | N/A | [Build a Java App with CockroachDB (JDBC)](build-a-java-app-with-cockroachdb.html) |
| JavaScript | [pg](https://www.npmjs.com/package/pg) | 8.2.1 | Full | N/A | [Build a Node.js App with CockroachDB (pg)](build-a-nodejs-app-with-cockroachdb.html) |
-| Python | [psycopg3](https://www.psycopg.org/psycopg3/docs/)
[psycopg2](https://www.psycopg.org/docs/install.html)
[asyncpg](https://magicstack.github.io/asyncpg/current/index.html) | 3.0.16
2.8.6
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/asyncpg.go || var asyncpgSupportedTag = "||"\n\n %} | Full
Full
Partial | N/A
N/A
N/A | [Build a Python App with CockroachDB (psycopg3)](build-a-python-app-with-cockroachdb-psycopg3.html)
[Build a Python App with CockroachDB (psycopg2)](build-a-python-app-with-cockroachdb.html)
[Build a Python App with CockroachDB (asyncpg)](build-a-python-app-with-cockroachdb-asyncpg.html) |
-| Ruby | [pg](https://rubygems.org/gems/pg) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/ruby_pg.go ||var rubyPGVersion = "||"\n\n %} | Full | N/A | [Build a Ruby App with CockroachDB (pg)](build-a-ruby-app-with-cockroachdb.html) |
+| Python | [psycopg3](https://www.psycopg.org/psycopg3/docs/)
[psycopg2](https://www.psycopg.org/docs/install.html)
[asyncpg](https://magicstack.github.io/asyncpg/current/index.html) | 3.0.16
2.8.6
v0.24.0 | Full
Full
Partial | N/A
N/A
N/A | [Build a Python App with CockroachDB (psycopg3)](build-a-python-app-with-cockroachdb-psycopg3.html)
[Build a Python App with CockroachDB (psycopg2)](build-a-python-app-with-cockroachdb.html)
[Build a Python App with CockroachDB (asyncpg)](build-a-python-app-with-cockroachdb-asyncpg.html) |
+| Ruby | [pg](https://rubygems.org/gems/pg) | v1.4.6 | Full | N/A | [Build a Ruby App with CockroachDB (pg)](build-a-ruby-app-with-cockroachdb.html) |
| Rust | [rust-postgres](https://github.com/sfackler/rust-postgres) | 0.19.2 | Partial | N/A | [Build a Rust App with CockroachDB]({% link {{ page.version.version }}/build-a-rust-app-with-cockroachdb.md %}) |
## Data access frameworks (e.g., ORMs)
| Language | Framework | Latest tested version | Support level | CockroachDB adapter | Tutorial |
|----------+-----------+-----------------------+---------------+---------------------+----------|
-| Go | [GORM](https://github.com/jinzhu/gorm/releases)
[go-pg](https://github.com/go-pg/pg)
[upper/db](https://github.com/upper/db) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/gorm.go ||var gormSupportedTag = "||"\n\n %}
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/gopg.go ||var gopgSupportedTag = "||"\n\n %}
v4 | Full
Full
Full | [`crdbgorm`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbgorm)
(includes client-side transaction retry handling)
N/A
N/A | [Build a Go App with CockroachDB (GORM)](build-a-go-app-with-cockroachdb-gorm.html)
N/A
[Build a Go App with CockroachDB (upper/db)](build-a-go-app-with-cockroachdb-upperdb.html) |
-| Java | [Hibernate](https://hibernate.org/orm/)
(including [Hibernate Spatial](https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#spatial))
[jOOQ](https://www.jooq.org/)
[MyBatis](https://mybatis.org/mybatis-3/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/release-23.1/pkg/cmd/roachtest/tests/hibernate.go ||var supportedHibernateTag = "||"\n\n %} (must be at least 5.4.19)
3.13.2 (must be at least 3.13.0)
3.5.5| Full
Full
Full | N/A
N/A
N/A | [Build a Java App with CockroachDB (Hibernate)](build-a-java-app-with-cockroachdb-hibernate.html)
[Build a Java App with CockroachDB (jOOQ)](build-a-java-app-with-cockroachdb-jooq.html)
[Build a Spring App with CockroachDB (MyBatis)]({% link {{ page.version.version }}/build-a-spring-app-with-cockroachdb-mybatis.md %}) |
-| JavaScript/TypeScript | [Sequelize](https://www.npmjs.com/package/sequelize)
[Knex.js](https://knexjs.org/)
[Prisma](https://prisma.io)
[TypeORM](https://www.npmjs.com/package/typeorm) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/sequelize.go ||var supportedSequelizeCockroachDBRelease = "||"\n\n %}
(use latest version of CockroachDB adapter)
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/knex.go ||const supportedKnexTag = "||"\n\n %}
3.14.0
0.3.17 {% comment %}{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/typeorm.go ||const supportedTypeORMRelease = "||"\n %}{% endcomment %} | Full
Full
Full
Full | [`sequelize-cockroachdb`](https://www.npmjs.com/package/sequelize-cockroachdb)
N/A
N/A
N/A | [Build a Node.js App with CockroachDB (Sequelize)](build-a-nodejs-app-with-cockroachdb-sequelize.html)
[Build a Node.js App with CockroachDB (Knex.js)](build-a-nodejs-app-with-cockroachdb-knexjs.html)
[Build a Node.js App with CockroachDB (Prisma)](build-a-nodejs-app-with-cockroachdb-prisma.html)
[Build a TypeScript App with CockroachDB (TypeORM)](build-a-typescript-app-with-cockroachdb.html) |
-| Ruby | [ActiveRecord](https://rubygems.org/gems/activerecord)
[RGeo/RGeo-ActiveRecord](https://github.com/cockroachdb/activerecord-cockroachdb-adapter#working-with-spatial-data) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/activerecord.go ||var supportedRailsVersion = "||"\nvar %}
(use latest version of CockroachDB adapter) | Full | [`activerecord-cockroachdb-adapter`](https://rubygems.org/gems/activerecord-cockroachdb-adapter)
(includes client-side transaction retry handling) | [Build a Ruby App with CockroachDB (ActiveRecord)](build-a-ruby-app-with-cockroachdb-activerecord.html) |
-| Python | [Django](https://pypi.org/project/Django/)
(including [GeoDjango](https://docs.djangoproject.com/en/3.1/ref/contrib/gis/))
[peewee](https://github.com/coleifer/peewee/)
[SQLAlchemy](https://www.sqlalchemy.org/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/django.go ||var djangoSupportedTag = "cockroach-||"\nvar %}
(use latest version of CockroachDB adapter)
3.13.3
0.7.13
1.4.17
(use latest version of CockroachDB adapter) | Full
Full
Full
Full | [`django-cockroachdb`](https://pypi.org/project/django-cockroachdb/)
N/A
N/A
[`sqlalchemy-cockroachdb`](https://pypi.org/project/sqlalchemy-cockroachdb)
(includes client-side transaction retry handling) | [Build a Python App with CockroachDB (Django)](build-a-python-app-with-cockroachdb-django.html)
N/A (See [peewee docs](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#cockroach-database).)
[Build a Python App with CockroachDB (SQLAlchemy)](build-a-python-app-with-cockroachdb-sqlalchemy.html) |
+| Go | [GORM](https://github.com/jinzhu/gorm/releases)
[go-pg](https://github.com/go-pg/pg)
[upper/db](https://github.com/upper/db) | v1.24.1
v10.9.0
v4 | Full
Full
Full | [`crdbgorm`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbgorm)
(includes client-side transaction retry handling)
N/A
N/A | [Build a Go App with CockroachDB (GORM)](build-a-go-app-with-cockroachdb-gorm.html)
N/A
[Build a Go App with CockroachDB (upper/db)](build-a-go-app-with-cockroachdb-upperdb.html) |
+| Java | [Hibernate](https://hibernate.org/orm/)
(including [Hibernate Spatial](https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#spatial))
[jOOQ](https://www.jooq.org/)
[MyBatis](https://mybatis.org/mybatis-3/) | 6.3.1 (must be at least 5.4.19)
3.13.2 (must be at least 3.13.0)
3.5.5| Full
Full
Full | N/A
N/A
N/A | [Build a Java App with CockroachDB (Hibernate)](build-a-java-app-with-cockroachdb-hibernate.html)
[Build a Java App with CockroachDB (jOOQ)](build-a-java-app-with-cockroachdb-jooq.html)
[Build a Spring App with CockroachDB (MyBatis)]({% link {{ page.version.version }}/build-a-spring-app-with-cockroachdb-mybatis.md %}) |
+| JavaScript/TypeScript | [Sequelize](https://www.npmjs.com/package/sequelize)
[Knex.js](https://knexjs.org/)
[Prisma](https://prisma.io)
[TypeORM](https://www.npmjs.com/package/typeorm) | v6.0.5
(use latest version of CockroachDB adapter)
2.5.1
3.14.0
0.3.17 {% comment %}remove-unsafe-crdb-setting{% endcomment %} | Full
Full
Full
Full | [`sequelize-cockroachdb`](https://www.npmjs.com/package/sequelize-cockroachdb)
N/A
N/A
N/A | [Build a Node.js App with CockroachDB (Sequelize)](build-a-nodejs-app-with-cockroachdb-sequelize.html)
[Build a Node.js App with CockroachDB (Knex.js)](build-a-nodejs-app-with-cockroachdb-knexjs.html)
[Build a Node.js App with CockroachDB (Prisma)](build-a-nodejs-app-with-cockroachdb-prisma.html)
[Build a TypeScript App with CockroachDB (TypeORM)](build-a-typescript-app-with-cockroachdb.html) |
+| Ruby | [ActiveRecord](https://rubygems.org/gems/activerecord)
[RGeo/RGeo-ActiveRecord](https://github.com/cockroachdb/activerecord-cockroachdb-adapter#working-with-spatial-data) | 8.1.0
(use latest version of CockroachDB adapter) | Full | [`activerecord-cockroachdb-adapter`](https://rubygems.org/gems/activerecord-cockroachdb-adapter)
(includes client-side transaction retry handling) | [Build a Ruby App with CockroachDB (ActiveRecord)](build-a-ruby-app-with-cockroachdb-activerecord.html) |
+| Python | [Django](https://pypi.org/project/Django/)
(including [GeoDjango](https://docs.djangoproject.com/en/3.1/ref/contrib/gis/))
[peewee](https://github.com/coleifer/peewee/)
[SQLAlchemy](https://www.sqlalchemy.org/) | 4.1.x
(use latest version of CockroachDB adapter)
3.13.3
0.7.13
1.4.17
(use latest version of CockroachDB adapter) | Full
Full
Full
Full | [`django-cockroachdb`](https://pypi.org/project/django-cockroachdb/)
N/A
N/A
[`sqlalchemy-cockroachdb`](https://pypi.org/project/sqlalchemy-cockroachdb)
(includes client-side transaction retry handling) | [Build a Python App with CockroachDB (Django)](build-a-python-app-with-cockroachdb-django.html)
N/A (See [peewee docs](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#cockroach-database).)
[Build a Python App with CockroachDB (SQLAlchemy)](build-a-python-app-with-cockroachdb-sqlalchemy.html) |
## Application frameworks
diff --git a/src/current/_includes/v23.1/orchestration/start-cockroachdb-insecure.md b/src/current/_includes/v23.1/orchestration/start-cockroachdb-insecure.md
index 3406d48edbb..c9af5cc0267 100644
--- a/src/current/_includes/v23.1/orchestration/start-cockroachdb-insecure.md
+++ b/src/current/_includes/v23.1/orchestration/start-cockroachdb-insecure.md
@@ -1,10 +1,10 @@
-1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it.
+1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it.
- Download [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml):
+ Download [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml
~~~
{{site.data.alerts.callout_info}}
@@ -27,11 +27,11 @@
Alternatively, if you'd rather start with a configuration file that has been customized for performance:
- 1. Download our [performance version of `cockroachdb-statefulset-insecure.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml):
+ 1. Download our [performance version of `cockroachdb-statefulset-insecure.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml
~~~
1. Modify the file wherever there is a `TODO` comment.
@@ -72,12 +72,12 @@
pvc-5315efda-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-2 27s
~~~
-1. Use our [`cluster-init.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
+1. Use our [`cluster-init.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml
~~~
~~~
diff --git a/src/current/_includes/v23.1/orchestration/start-cockroachdb-local-insecure.md b/src/current/_includes/v23.1/orchestration/start-cockroachdb-local-insecure.md
index 552cb3cd25f..196a53bce4c 100644
--- a/src/current/_includes/v23.1/orchestration/start-cockroachdb-local-insecure.md
+++ b/src/current/_includes/v23.1/orchestration/start-cockroachdb-local-insecure.md
@@ -1,8 +1,8 @@
-1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it:
+1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it:
{% include_cached copy-clipboard.html %}
~~~ shell
- $ kubectl create -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml
+ $ kubectl create -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml
~~~
~~~
@@ -41,12 +41,12 @@
pvc-5315efda-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-2 27s
~~~
-1. Use our [`cluster-init.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
+1. Use our [`cluster-init.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml
~~~
~~~
diff --git a/src/current/_includes/v23.1/orchestration/start-cockroachdb-secure.md b/src/current/_includes/v23.1/orchestration/start-cockroachdb-secure.md
index 972cabc2d8e..a7299e1aa25 100644
--- a/src/current/_includes/v23.1/orchestration/start-cockroachdb-secure.md
+++ b/src/current/_includes/v23.1/orchestration/start-cockroachdb-secure.md
@@ -1,10 +1,10 @@
### Configure the cluster
-1. Download and modify our [StatefulSet configuration](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml):
+1. Download and modify our [StatefulSet configuration](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml
~~~
1. Update `secretName` with the name of the corresponding node secret.
diff --git a/src/current/_includes/v23.1/orchestration/test-cluster-secure.md b/src/current/_includes/v23.1/orchestration/test-cluster-secure.md
index f96d779fd0a..68542cd3e3f 100644
--- a/src/current/_includes/v23.1/orchestration/test-cluster-secure.md
+++ b/src/current/_includes/v23.1/orchestration/test-cluster-secure.md
@@ -44,7 +44,7 @@
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/client.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/client.yaml
~~~
~~~
diff --git a/src/current/_includes/v23.2/misc/tooling.md b/src/current/_includes/v23.2/misc/tooling.md
index dcd24363435..01dc63c840f 100644
--- a/src/current/_includes/v23.2/misc/tooling.md
+++ b/src/current/_includes/v23.2/misc/tooling.md
@@ -23,7 +23,7 @@ Customers should contact their account team before moving production workloads t
Unless explicitly stated, support for a [driver](#drivers) or [data access framework](#data-access-frameworks-e-g-orms) does not include [automatic, client-side transaction retry handling]({% link {{ page.version.version }}/transaction-retry-error-reference.md %}#client-side-retry-handling). For client-side transaction retry handling samples, see [Example Apps]({% link {{ page.version.version }}/example-apps.md %}).
{{site.data.alerts.end}}
-If you encounter problems using CockroachDB with any of the tools listed on this page, please [open an issue](https://github.com/cockroachdb/cockroach/issues/new) with details to help us make progress toward better support.
+If you encounter problems using CockroachDB with any of the tools listed on this page, please open an issue with details to help us make progress toward better support.
For a list of tools supported by the CockroachDB community, see [Third-Party Tools Supported by the Community]({% link {{ page.version.version }}/community-tooling.md %}).
@@ -32,23 +32,23 @@ For a list of tools supported by the CockroachDB community, see [Third-Party Too
| Language | Driver | Latest tested version | Support level | CockroachDB adapter | Tutorial |
|----------+--------+-----------------------+---------------------+---------------------+----------|
| C | [libpq](http://www.postgresql.org/docs/13/static/libpq.html)| PostgreSQL 13 | Partial | N/A | N/A |
-| C# (.NET) | [Npgsql](https://www.nuget.org/packages/Npgsql/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/npgsql.go ||var npgsqlSupportedTag = "v||"\n\n %} | Full | N/A | [Build a C# App with CockroachDB (Npgsql)](build-a-csharp-app-with-cockroachdb.html) |
-| Go | [pgx](https://github.com/jackc/pgx/releases)
[pq](https://github.com/lib/pq) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/pgx.go ||var supportedPGXTag = "||"\n\n %}
(use latest version of CockroachDB adapter)
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/libpq.go ||var libPQSupportedTag = "||"\n\n %} | Full
Full | [`crdbpgx`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbpgx)
(includes client-side transaction retry handling)
N/A | [Build a Go App with CockroachDB (pgx)](build-a-go-app-with-cockroachdb.html)
[Build a Go App with CockroachDB (pq)](build-a-go-app-with-cockroachdb-pq.html) |
-| Java | [JDBC](https://jdbc.postgresql.org/download/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/pgjdbc.go ||var supportedPGJDBCTag = "||"\n\n %} | Full | N/A | [Build a Java App with CockroachDB (JDBC)](build-a-java-app-with-cockroachdb.html) |
+| C# (.NET) | [Npgsql](https://www.nuget.org/packages/Npgsql/) | 7.0.2 | Full | N/A | [Build a C# App with CockroachDB (Npgsql)](build-a-csharp-app-with-cockroachdb.html) |
+| Go | [pgx](https://github.com/jackc/pgx/releases)
[pq](https://github.com/lib/pq) | v5.3.1
(use latest version of CockroachDB adapter)
v1.10.5 | Full
Full | [`crdbpgx`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbpgx)
(includes client-side transaction retry handling)
N/A | [Build a Go App with CockroachDB (pgx)](build-a-go-app-with-cockroachdb.html)
[Build a Go App with CockroachDB (pq)](build-a-go-app-with-cockroachdb-pq.html) |
+| Java | [JDBC](https://jdbc.postgresql.org/download/) | REL42.7.3 | Full | N/A | [Build a Java App with CockroachDB (JDBC)](build-a-java-app-with-cockroachdb.html) |
| JavaScript | [pg](https://www.npmjs.com/package/pg) | 8.2.1 | Full | N/A | [Build a Node.js App with CockroachDB (pg)](build-a-nodejs-app-with-cockroachdb.html) |
-| Python | [psycopg3](https://www.psycopg.org/psycopg3/docs/)
[psycopg2](https://www.psycopg.org/docs/install.html)
[asyncpg](https://magicstack.github.io/asyncpg/current/index.html) | 3.0.16
2.8.6
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/asyncpg.go || var asyncpgSupportedTag = "||"\n\n %} | Full
Full
Partial | N/A
N/A
N/A | [Build a Python App with CockroachDB (psycopg3)](build-a-python-app-with-cockroachdb-psycopg3.html)
[Build a Python App with CockroachDB (psycopg2)](build-a-python-app-with-cockroachdb.html)
[Build a Python App with CockroachDB (asyncpg)](build-a-python-app-with-cockroachdb-asyncpg.html) |
-| Ruby | [pg](https://rubygems.org/gems/pg) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/ruby_pg.go ||var rubyPGVersion = "||"\n\n %} | Full | N/A | [Build a Ruby App with CockroachDB (pg)](build-a-ruby-app-with-cockroachdb.html) |
+| Python | [psycopg3](https://www.psycopg.org/psycopg3/docs/)
[psycopg2](https://www.psycopg.org/docs/install.html)
[asyncpg](https://magicstack.github.io/asyncpg/current/index.html) | 3.0.16
2.8.6
v0.24.0 | Full
Full
Partial | N/A
N/A
N/A | [Build a Python App with CockroachDB (psycopg3)](build-a-python-app-with-cockroachdb-psycopg3.html)
[Build a Python App with CockroachDB (psycopg2)](build-a-python-app-with-cockroachdb.html)
[Build a Python App with CockroachDB (asyncpg)](build-a-python-app-with-cockroachdb-asyncpg.html) |
+| Ruby | [pg](https://rubygems.org/gems/pg) | v1.4.6 | Full | N/A | [Build a Ruby App with CockroachDB (pg)](build-a-ruby-app-with-cockroachdb.html) |
| Rust | [rust-postgres](https://github.com/sfackler/rust-postgres) | 0.19.2 | Partial | N/A | [Build a Rust App with CockroachDB]({% link {{ page.version.version }}/build-a-rust-app-with-cockroachdb.md %}) |
## Data access frameworks (e.g., ORMs)
| Language | Framework | Latest tested version | Support level | CockroachDB adapter | Tutorial |
|----------+-----------+-----------------------+---------------+---------------------+----------|
-| Go | [GORM](https://github.com/jinzhu/gorm/releases)
[go-pg](https://github.com/go-pg/pg)
[upper/db](https://github.com/upper/db) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/gorm.go ||var gormSupportedTag = "||"\n\n %}
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/gopg.go ||var gopgSupportedTag = "||"\n\n %}
v4 | Full
Full
Full | [`crdbgorm`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbgorm)
(includes client-side transaction retry handling)
N/A
N/A | [Build a Go App with CockroachDB (GORM)](build-a-go-app-with-cockroachdb-gorm.html)
N/A
[Build a Go App with CockroachDB (upper/db)](build-a-go-app-with-cockroachdb-upperdb.html) |
-| Java | [Hibernate](https://hibernate.org/orm/)
(including [Hibernate Spatial](https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#spatial))
[jOOQ](https://www.jooq.org/)
[MyBatis](https://mybatis.org/mybatis-3/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/hibernate.go ||var supportedHibernateTag = "||"\n\n %} (must be at least 5.4.19)
3.13.2 (must be at least 3.13.0)
3.5.5| Full
Full
Full | N/A
N/A
N/A | [Build a Java App with CockroachDB (Hibernate)](build-a-java-app-with-cockroachdb-hibernate.html)
[Build a Java App with CockroachDB (jOOQ)](build-a-java-app-with-cockroachdb-jooq.html)
[Build a Spring App with CockroachDB (MyBatis)]({% link {{ page.version.version }}/build-a-spring-app-with-cockroachdb-mybatis.md %}) |
-| JavaScript/TypeScript | [Sequelize](https://www.npmjs.com/package/sequelize)
[Knex.js](https://knexjs.org/)
[Prisma](https://prisma.io)
[TypeORM](https://www.npmjs.com/package/typeorm) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/sequelize.go ||var supportedSequelizeCockroachDBRelease = "||"\n\n %}
(use latest version of CockroachDB adapter)
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/knex.go ||const supportedKnexTag = "||"\n\n %}
3.14.0
0.3.17 {% comment %}{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/typeorm.go ||const supportedTypeORMRelease = "||"\n %}{% endcomment %} | Full
Full
Full
Full | [`sequelize-cockroachdb`](https://www.npmjs.com/package/sequelize-cockroachdb)
N/A
N/A
N/A | [Build a Node.js App with CockroachDB (Sequelize)](build-a-nodejs-app-with-cockroachdb-sequelize.html)
[Build a Node.js App with CockroachDB (Knex.js)](build-a-nodejs-app-with-cockroachdb-knexjs.html)
[Build a Node.js App with CockroachDB (Prisma)](build-a-nodejs-app-with-cockroachdb-prisma.html)
[Build a TypeScript App with CockroachDB (TypeORM)](build-a-typescript-app-with-cockroachdb.html) |
-| Ruby | [ActiveRecord](https://rubygems.org/gems/activerecord)
[RGeo/RGeo-ActiveRecord](https://github.com/cockroachdb/activerecord-cockroachdb-adapter#working-with-spatial-data) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/activerecord.go ||var supportedRailsVersion = "||"\nvar %}
(use latest version of CockroachDB adapter) | Full | [`activerecord-cockroachdb-adapter`](https://rubygems.org/gems/activerecord-cockroachdb-adapter)
(includes client-side transaction retry handling) | [Build a Ruby App with CockroachDB (ActiveRecord)](build-a-ruby-app-with-cockroachdb-activerecord.html) |
-| Python | [Django](https://pypi.org/project/Django/)
(including [GeoDjango](https://docs.djangoproject.com/en/3.1/ref/contrib/gis/))
[peewee](https://github.com/coleifer/peewee/)
[SQLAlchemy](https://www.sqlalchemy.org/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/django.go ||var djangoSupportedTag = "cockroach-||"\nvar %}
(use latest version of CockroachDB adapter)
3.13.3
0.7.13
1.4.17
(use latest version of CockroachDB adapter) | Full
Full
Full
Full | [`django-cockroachdb`](https://pypi.org/project/django-cockroachdb/)
N/A
N/A
[`sqlalchemy-cockroachdb`](https://pypi.org/project/sqlalchemy-cockroachdb)
(includes client-side transaction retry handling) | [Build a Python App with CockroachDB (Django)](build-a-python-app-with-cockroachdb-django.html)
N/A (See [peewee docs](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#cockroach-database).)
[Build a Python App with CockroachDB (SQLAlchemy)](build-a-python-app-with-cockroachdb-sqlalchemy.html) |
+| Go | [GORM](https://github.com/jinzhu/gorm/releases)
[go-pg](https://github.com/go-pg/pg)
[upper/db](https://github.com/upper/db) | v1.24.1
v10.9.0
v4 | Full
Full
Full | [`crdbgorm`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbgorm)
(includes client-side transaction retry handling)
N/A
N/A | [Build a Go App with CockroachDB (GORM)](build-a-go-app-with-cockroachdb-gorm.html)
N/A
[Build a Go App with CockroachDB (upper/db)](build-a-go-app-with-cockroachdb-upperdb.html) |
+| Java | [Hibernate](https://hibernate.org/orm/)
(including [Hibernate Spatial](https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#spatial))
[jOOQ](https://www.jooq.org/)
[MyBatis](https://mybatis.org/mybatis-3/) | 6.6.20 (must be at least 5.4.19)
3.13.2 (must be at least 3.13.0)
3.5.5| Full
Full
Full | N/A
N/A
N/A | [Build a Java App with CockroachDB (Hibernate)](build-a-java-app-with-cockroachdb-hibernate.html)
[Build a Java App with CockroachDB (jOOQ)](build-a-java-app-with-cockroachdb-jooq.html)
[Build a Spring App with CockroachDB (MyBatis)]({% link {{ page.version.version }}/build-a-spring-app-with-cockroachdb-mybatis.md %}) |
+| JavaScript/TypeScript | [Sequelize](https://www.npmjs.com/package/sequelize)
[Knex.js](https://knexjs.org/)
[Prisma](https://prisma.io)
[TypeORM](https://www.npmjs.com/package/typeorm) | v6.0.5
(use latest version of CockroachDB adapter)
2.5.1
3.14.0
0.3.17 {% comment %}remove-unsafe-crdb-setting{% endcomment %} | Full
Full
Full
Full | [`sequelize-cockroachdb`](https://www.npmjs.com/package/sequelize-cockroachdb)
N/A
N/A
N/A | [Build a Node.js App with CockroachDB (Sequelize)](build-a-nodejs-app-with-cockroachdb-sequelize.html)
[Build a Node.js App with CockroachDB (Knex.js)](build-a-nodejs-app-with-cockroachdb-knexjs.html)
[Build a Node.js App with CockroachDB (Prisma)](build-a-nodejs-app-with-cockroachdb-prisma.html)
[Build a TypeScript App with CockroachDB (TypeORM)](build-a-typescript-app-with-cockroachdb.html) |
+| Ruby | [ActiveRecord](https://rubygems.org/gems/activerecord)
[RGeo/RGeo-ActiveRecord](https://github.com/cockroachdb/activerecord-cockroachdb-adapter#working-with-spatial-data) | 8.1.0
(use latest version of CockroachDB adapter) | Full | [`activerecord-cockroachdb-adapter`](https://rubygems.org/gems/activerecord-cockroachdb-adapter)
(includes client-side transaction retry handling) | [Build a Ruby App with CockroachDB (ActiveRecord)](build-a-ruby-app-with-cockroachdb-activerecord.html) |
+| Python | [Django](https://pypi.org/project/Django/)
(including [GeoDjango](https://docs.djangoproject.com/en/3.1/ref/contrib/gis/))
[peewee](https://github.com/coleifer/peewee/)
[SQLAlchemy](https://www.sqlalchemy.org/) | 4.1.x
(use latest version of CockroachDB adapter)
3.13.3
0.7.13
1.4.17
(use latest version of CockroachDB adapter) | Full
Full
Full
Full | [`django-cockroachdb`](https://pypi.org/project/django-cockroachdb/)
N/A
N/A
[`sqlalchemy-cockroachdb`](https://pypi.org/project/sqlalchemy-cockroachdb)
(includes client-side transaction retry handling) | [Build a Python App with CockroachDB (Django)](build-a-python-app-with-cockroachdb-django.html)
N/A (See [peewee docs](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#cockroach-database).)
[Build a Python App with CockroachDB (SQLAlchemy)](build-a-python-app-with-cockroachdb-sqlalchemy.html) |
## Application frameworks
diff --git a/src/current/_includes/v23.2/orchestration/start-cockroachdb-insecure.md b/src/current/_includes/v23.2/orchestration/start-cockroachdb-insecure.md
index 3406d48edbb..c9af5cc0267 100644
--- a/src/current/_includes/v23.2/orchestration/start-cockroachdb-insecure.md
+++ b/src/current/_includes/v23.2/orchestration/start-cockroachdb-insecure.md
@@ -1,10 +1,10 @@
-1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it.
+1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it.
- Download [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml):
+ Download [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml
~~~
{{site.data.alerts.callout_info}}
@@ -27,11 +27,11 @@
Alternatively, if you'd rather start with a configuration file that has been customized for performance:
- 1. Download our [performance version of `cockroachdb-statefulset-insecure.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml):
+ 1. Download our [performance version of `cockroachdb-statefulset-insecure.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml
~~~
1. Modify the file wherever there is a `TODO` comment.
@@ -72,12 +72,12 @@
pvc-5315efda-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-2 27s
~~~
-1. Use our [`cluster-init.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
+1. Use our [`cluster-init.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml
~~~
~~~
diff --git a/src/current/_includes/v23.2/orchestration/start-cockroachdb-local-insecure.md b/src/current/_includes/v23.2/orchestration/start-cockroachdb-local-insecure.md
index 552cb3cd25f..196a53bce4c 100644
--- a/src/current/_includes/v23.2/orchestration/start-cockroachdb-local-insecure.md
+++ b/src/current/_includes/v23.2/orchestration/start-cockroachdb-local-insecure.md
@@ -1,8 +1,8 @@
-1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it:
+1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it:
{% include_cached copy-clipboard.html %}
~~~ shell
- $ kubectl create -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml
+ $ kubectl create -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml
~~~
~~~
@@ -41,12 +41,12 @@
pvc-5315efda-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-2 27s
~~~
-1. Use our [`cluster-init.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
+1. Use our [`cluster-init.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml
~~~
~~~
diff --git a/src/current/_includes/v23.2/orchestration/start-cockroachdb-secure.md b/src/current/_includes/v23.2/orchestration/start-cockroachdb-secure.md
index 972cabc2d8e..a7299e1aa25 100644
--- a/src/current/_includes/v23.2/orchestration/start-cockroachdb-secure.md
+++ b/src/current/_includes/v23.2/orchestration/start-cockroachdb-secure.md
@@ -1,10 +1,10 @@
### Configure the cluster
-1. Download and modify our [StatefulSet configuration](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml):
+1. Download and modify our [StatefulSet configuration](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml
~~~
1. Update `secretName` with the name of the corresponding node secret.
diff --git a/src/current/_includes/v23.2/orchestration/test-cluster-secure.md b/src/current/_includes/v23.2/orchestration/test-cluster-secure.md
index f255d8d62fc..c146d634626 100644
--- a/src/current/_includes/v23.2/orchestration/test-cluster-secure.md
+++ b/src/current/_includes/v23.2/orchestration/test-cluster-secure.md
@@ -42,7 +42,7 @@ $ kubectl create \
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
--f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/client.yaml
+-f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/client.yaml
~~~
~~~
diff --git a/src/current/_includes/v24.1/misc/tooling.md b/src/current/_includes/v24.1/misc/tooling.md
index dcd24363435..01dc63c840f 100644
--- a/src/current/_includes/v24.1/misc/tooling.md
+++ b/src/current/_includes/v24.1/misc/tooling.md
@@ -23,7 +23,7 @@ Customers should contact their account team before moving production workloads t
Unless explicitly stated, support for a [driver](#drivers) or [data access framework](#data-access-frameworks-e-g-orms) does not include [automatic, client-side transaction retry handling]({% link {{ page.version.version }}/transaction-retry-error-reference.md %}#client-side-retry-handling). For client-side transaction retry handling samples, see [Example Apps]({% link {{ page.version.version }}/example-apps.md %}).
{{site.data.alerts.end}}
-If you encounter problems using CockroachDB with any of the tools listed on this page, please [open an issue](https://github.com/cockroachdb/cockroach/issues/new) with details to help us make progress toward better support.
+If you encounter problems using CockroachDB with any of the tools listed on this page, please open an issue with details to help us make progress toward better support.
For a list of tools supported by the CockroachDB community, see [Third-Party Tools Supported by the Community]({% link {{ page.version.version }}/community-tooling.md %}).
@@ -32,23 +32,23 @@ For a list of tools supported by the CockroachDB community, see [Third-Party Too
| Language | Driver | Latest tested version | Support level | CockroachDB adapter | Tutorial |
|----------+--------+-----------------------+---------------------+---------------------+----------|
| C | [libpq](http://www.postgresql.org/docs/13/static/libpq.html)| PostgreSQL 13 | Partial | N/A | N/A |
-| C# (.NET) | [Npgsql](https://www.nuget.org/packages/Npgsql/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/npgsql.go ||var npgsqlSupportedTag = "v||"\n\n %} | Full | N/A | [Build a C# App with CockroachDB (Npgsql)](build-a-csharp-app-with-cockroachdb.html) |
-| Go | [pgx](https://github.com/jackc/pgx/releases)
[pq](https://github.com/lib/pq) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/pgx.go ||var supportedPGXTag = "||"\n\n %}
(use latest version of CockroachDB adapter)
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/libpq.go ||var libPQSupportedTag = "||"\n\n %} | Full
Full | [`crdbpgx`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbpgx)
(includes client-side transaction retry handling)
N/A | [Build a Go App with CockroachDB (pgx)](build-a-go-app-with-cockroachdb.html)
[Build a Go App with CockroachDB (pq)](build-a-go-app-with-cockroachdb-pq.html) |
-| Java | [JDBC](https://jdbc.postgresql.org/download/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/pgjdbc.go ||var supportedPGJDBCTag = "||"\n\n %} | Full | N/A | [Build a Java App with CockroachDB (JDBC)](build-a-java-app-with-cockroachdb.html) |
+| C# (.NET) | [Npgsql](https://www.nuget.org/packages/Npgsql/) | 7.0.2 | Full | N/A | [Build a C# App with CockroachDB (Npgsql)](build-a-csharp-app-with-cockroachdb.html) |
+| Go | [pgx](https://github.com/jackc/pgx/releases)
[pq](https://github.com/lib/pq) | v5.3.1
(use latest version of CockroachDB adapter)
v1.10.5 | Full
Full | [`crdbpgx`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbpgx)
(includes client-side transaction retry handling)
N/A | [Build a Go App with CockroachDB (pgx)](build-a-go-app-with-cockroachdb.html)
[Build a Go App with CockroachDB (pq)](build-a-go-app-with-cockroachdb-pq.html) |
+| Java | [JDBC](https://jdbc.postgresql.org/download/) | REL42.7.3 | Full | N/A | [Build a Java App with CockroachDB (JDBC)](build-a-java-app-with-cockroachdb.html) |
| JavaScript | [pg](https://www.npmjs.com/package/pg) | 8.2.1 | Full | N/A | [Build a Node.js App with CockroachDB (pg)](build-a-nodejs-app-with-cockroachdb.html) |
-| Python | [psycopg3](https://www.psycopg.org/psycopg3/docs/)
[psycopg2](https://www.psycopg.org/docs/install.html)
[asyncpg](https://magicstack.github.io/asyncpg/current/index.html) | 3.0.16
2.8.6
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/asyncpg.go || var asyncpgSupportedTag = "||"\n\n %} | Full
Full
Partial | N/A
N/A
N/A | [Build a Python App with CockroachDB (psycopg3)](build-a-python-app-with-cockroachdb-psycopg3.html)
[Build a Python App with CockroachDB (psycopg2)](build-a-python-app-with-cockroachdb.html)
[Build a Python App with CockroachDB (asyncpg)](build-a-python-app-with-cockroachdb-asyncpg.html) |
-| Ruby | [pg](https://rubygems.org/gems/pg) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/ruby_pg.go ||var rubyPGVersion = "||"\n\n %} | Full | N/A | [Build a Ruby App with CockroachDB (pg)](build-a-ruby-app-with-cockroachdb.html) |
+| Python | [psycopg3](https://www.psycopg.org/psycopg3/docs/)
[psycopg2](https://www.psycopg.org/docs/install.html)
[asyncpg](https://magicstack.github.io/asyncpg/current/index.html) | 3.0.16
2.8.6
v0.24.0 | Full
Full
Partial | N/A
N/A
N/A | [Build a Python App with CockroachDB (psycopg3)](build-a-python-app-with-cockroachdb-psycopg3.html)
[Build a Python App with CockroachDB (psycopg2)](build-a-python-app-with-cockroachdb.html)
[Build a Python App with CockroachDB (asyncpg)](build-a-python-app-with-cockroachdb-asyncpg.html) |
+| Ruby | [pg](https://rubygems.org/gems/pg) | v1.4.6 | Full | N/A | [Build a Ruby App with CockroachDB (pg)](build-a-ruby-app-with-cockroachdb.html) |
| Rust | [rust-postgres](https://github.com/sfackler/rust-postgres) | 0.19.2 | Partial | N/A | [Build a Rust App with CockroachDB]({% link {{ page.version.version }}/build-a-rust-app-with-cockroachdb.md %}) |
## Data access frameworks (e.g., ORMs)
| Language | Framework | Latest tested version | Support level | CockroachDB adapter | Tutorial |
|----------+-----------+-----------------------+---------------+---------------------+----------|
-| Go | [GORM](https://github.com/jinzhu/gorm/releases)
[go-pg](https://github.com/go-pg/pg)
[upper/db](https://github.com/upper/db) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/gorm.go ||var gormSupportedTag = "||"\n\n %}
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/gopg.go ||var gopgSupportedTag = "||"\n\n %}
v4 | Full
Full
Full | [`crdbgorm`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbgorm)
(includes client-side transaction retry handling)
N/A
N/A | [Build a Go App with CockroachDB (GORM)](build-a-go-app-with-cockroachdb-gorm.html)
N/A
[Build a Go App with CockroachDB (upper/db)](build-a-go-app-with-cockroachdb-upperdb.html) |
-| Java | [Hibernate](https://hibernate.org/orm/)
(including [Hibernate Spatial](https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#spatial))
[jOOQ](https://www.jooq.org/)
[MyBatis](https://mybatis.org/mybatis-3/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/hibernate.go ||var supportedHibernateTag = "||"\n\n %} (must be at least 5.4.19)
3.13.2 (must be at least 3.13.0)
3.5.5| Full
Full
Full | N/A
N/A
N/A | [Build a Java App with CockroachDB (Hibernate)](build-a-java-app-with-cockroachdb-hibernate.html)
[Build a Java App with CockroachDB (jOOQ)](build-a-java-app-with-cockroachdb-jooq.html)
[Build a Spring App with CockroachDB (MyBatis)]({% link {{ page.version.version }}/build-a-spring-app-with-cockroachdb-mybatis.md %}) |
-| JavaScript/TypeScript | [Sequelize](https://www.npmjs.com/package/sequelize)
[Knex.js](https://knexjs.org/)
[Prisma](https://prisma.io)
[TypeORM](https://www.npmjs.com/package/typeorm) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/sequelize.go ||var supportedSequelizeCockroachDBRelease = "||"\n\n %}
(use latest version of CockroachDB adapter)
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/knex.go ||const supportedKnexTag = "||"\n\n %}
3.14.0
0.3.17 {% comment %}{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/typeorm.go ||const supportedTypeORMRelease = "||"\n %}{% endcomment %} | Full
Full
Full
Full | [`sequelize-cockroachdb`](https://www.npmjs.com/package/sequelize-cockroachdb)
N/A
N/A
N/A | [Build a Node.js App with CockroachDB (Sequelize)](build-a-nodejs-app-with-cockroachdb-sequelize.html)
[Build a Node.js App with CockroachDB (Knex.js)](build-a-nodejs-app-with-cockroachdb-knexjs.html)
[Build a Node.js App with CockroachDB (Prisma)](build-a-nodejs-app-with-cockroachdb-prisma.html)
[Build a TypeScript App with CockroachDB (TypeORM)](build-a-typescript-app-with-cockroachdb.html) |
-| Ruby | [ActiveRecord](https://rubygems.org/gems/activerecord)
[RGeo/RGeo-ActiveRecord](https://github.com/cockroachdb/activerecord-cockroachdb-adapter#working-with-spatial-data) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/activerecord.go ||var supportedRailsVersion = "||"\nvar %}
(use latest version of CockroachDB adapter) | Full | [`activerecord-cockroachdb-adapter`](https://rubygems.org/gems/activerecord-cockroachdb-adapter)
(includes client-side transaction retry handling) | [Build a Ruby App with CockroachDB (ActiveRecord)](build-a-ruby-app-with-cockroachdb-activerecord.html) |
-| Python | [Django](https://pypi.org/project/Django/)
(including [GeoDjango](https://docs.djangoproject.com/en/3.1/ref/contrib/gis/))
[peewee](https://github.com/coleifer/peewee/)
[SQLAlchemy](https://www.sqlalchemy.org/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/django.go ||var djangoSupportedTag = "cockroach-||"\nvar %}
(use latest version of CockroachDB adapter)
3.13.3
0.7.13
1.4.17
(use latest version of CockroachDB adapter) | Full
Full
Full
Full | [`django-cockroachdb`](https://pypi.org/project/django-cockroachdb/)
N/A
N/A
[`sqlalchemy-cockroachdb`](https://pypi.org/project/sqlalchemy-cockroachdb)
(includes client-side transaction retry handling) | [Build a Python App with CockroachDB (Django)](build-a-python-app-with-cockroachdb-django.html)
N/A (See [peewee docs](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#cockroach-database).)
[Build a Python App with CockroachDB (SQLAlchemy)](build-a-python-app-with-cockroachdb-sqlalchemy.html) |
+| Go | [GORM](https://github.com/jinzhu/gorm/releases)
[go-pg](https://github.com/go-pg/pg)
[upper/db](https://github.com/upper/db) | v1.24.1
v10.9.0
v4 | Full
Full
Full | [`crdbgorm`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbgorm)
(includes client-side transaction retry handling)
N/A
N/A | [Build a Go App with CockroachDB (GORM)](build-a-go-app-with-cockroachdb-gorm.html)
N/A
[Build a Go App with CockroachDB (upper/db)](build-a-go-app-with-cockroachdb-upperdb.html) |
+| Java | [Hibernate](https://hibernate.org/orm/)
(including [Hibernate Spatial](https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#spatial))
[jOOQ](https://www.jooq.org/)
[MyBatis](https://mybatis.org/mybatis-3/) | 6.6.20 (must be at least 5.4.19)
3.13.2 (must be at least 3.13.0)
3.5.5| Full
Full
Full | N/A
N/A
N/A | [Build a Java App with CockroachDB (Hibernate)](build-a-java-app-with-cockroachdb-hibernate.html)
[Build a Java App with CockroachDB (jOOQ)](build-a-java-app-with-cockroachdb-jooq.html)
[Build a Spring App with CockroachDB (MyBatis)]({% link {{ page.version.version }}/build-a-spring-app-with-cockroachdb-mybatis.md %}) |
+| JavaScript/TypeScript | [Sequelize](https://www.npmjs.com/package/sequelize)
[Knex.js](https://knexjs.org/)
[Prisma](https://prisma.io)
[TypeORM](https://www.npmjs.com/package/typeorm) | v6.0.5
(use latest version of CockroachDB adapter)
2.5.1
3.14.0
0.3.17 {% comment %}remove-unsafe-crdb-setting{% endcomment %} | Full
Full
Full
Full | [`sequelize-cockroachdb`](https://www.npmjs.com/package/sequelize-cockroachdb)
N/A
N/A
N/A | [Build a Node.js App with CockroachDB (Sequelize)](build-a-nodejs-app-with-cockroachdb-sequelize.html)
[Build a Node.js App with CockroachDB (Knex.js)](build-a-nodejs-app-with-cockroachdb-knexjs.html)
[Build a Node.js App with CockroachDB (Prisma)](build-a-nodejs-app-with-cockroachdb-prisma.html)
[Build a TypeScript App with CockroachDB (TypeORM)](build-a-typescript-app-with-cockroachdb.html) |
+| Ruby | [ActiveRecord](https://rubygems.org/gems/activerecord)
[RGeo/RGeo-ActiveRecord](https://github.com/cockroachdb/activerecord-cockroachdb-adapter#working-with-spatial-data) | 8.1.0
(use latest version of CockroachDB adapter) | Full | [`activerecord-cockroachdb-adapter`](https://rubygems.org/gems/activerecord-cockroachdb-adapter)
(includes client-side transaction retry handling) | [Build a Ruby App with CockroachDB (ActiveRecord)](build-a-ruby-app-with-cockroachdb-activerecord.html) |
+| Python | [Django](https://pypi.org/project/Django/)
(including [GeoDjango](https://docs.djangoproject.com/en/3.1/ref/contrib/gis/))
[peewee](https://github.com/coleifer/peewee/)
[SQLAlchemy](https://www.sqlalchemy.org/) | 4.1.x
(use latest version of CockroachDB adapter)
3.13.3
0.7.13
1.4.17
(use latest version of CockroachDB adapter) | Full
Full
Full
Full | [`django-cockroachdb`](https://pypi.org/project/django-cockroachdb/)
N/A
N/A
[`sqlalchemy-cockroachdb`](https://pypi.org/project/sqlalchemy-cockroachdb)
(includes client-side transaction retry handling) | [Build a Python App with CockroachDB (Django)](build-a-python-app-with-cockroachdb-django.html)
N/A (See [peewee docs](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#cockroach-database).)
[Build a Python App with CockroachDB (SQLAlchemy)](build-a-python-app-with-cockroachdb-sqlalchemy.html) |
## Application frameworks
diff --git a/src/current/_includes/v24.1/orchestration/start-cockroachdb-insecure.md b/src/current/_includes/v24.1/orchestration/start-cockroachdb-insecure.md
index 3406d48edbb..c9af5cc0267 100644
--- a/src/current/_includes/v24.1/orchestration/start-cockroachdb-insecure.md
+++ b/src/current/_includes/v24.1/orchestration/start-cockroachdb-insecure.md
@@ -1,10 +1,10 @@
-1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it.
+1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it.
- Download [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml):
+ Download [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml
~~~
{{site.data.alerts.callout_info}}
@@ -27,11 +27,11 @@
Alternatively, if you'd rather start with a configuration file that has been customized for performance:
- 1. Download our [performance version of `cockroachdb-statefulset-insecure.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml):
+ 1. Download our [performance version of `cockroachdb-statefulset-insecure.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml
~~~
1. Modify the file wherever there is a `TODO` comment.
@@ -72,12 +72,12 @@
pvc-5315efda-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-2 27s
~~~
-1. Use our [`cluster-init.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
+1. Use our [`cluster-init.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml
~~~
~~~
diff --git a/src/current/_includes/v24.1/orchestration/start-cockroachdb-local-insecure.md b/src/current/_includes/v24.1/orchestration/start-cockroachdb-local-insecure.md
index 552cb3cd25f..196a53bce4c 100644
--- a/src/current/_includes/v24.1/orchestration/start-cockroachdb-local-insecure.md
+++ b/src/current/_includes/v24.1/orchestration/start-cockroachdb-local-insecure.md
@@ -1,8 +1,8 @@
-1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it:
+1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it:
{% include_cached copy-clipboard.html %}
~~~ shell
- $ kubectl create -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml
+ $ kubectl create -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml
~~~
~~~
@@ -41,12 +41,12 @@
pvc-5315efda-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-2 27s
~~~
-1. Use our [`cluster-init.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
+1. Use our [`cluster-init.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml
~~~
~~~
diff --git a/src/current/_includes/v24.1/orchestration/start-cockroachdb-secure.md b/src/current/_includes/v24.1/orchestration/start-cockroachdb-secure.md
index 972cabc2d8e..a7299e1aa25 100644
--- a/src/current/_includes/v24.1/orchestration/start-cockroachdb-secure.md
+++ b/src/current/_includes/v24.1/orchestration/start-cockroachdb-secure.md
@@ -1,10 +1,10 @@
### Configure the cluster
-1. Download and modify our [StatefulSet configuration](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml):
+1. Download and modify our [StatefulSet configuration](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml
~~~
1. Update `secretName` with the name of the corresponding node secret.
diff --git a/src/current/_includes/v24.1/orchestration/test-cluster-secure.md b/src/current/_includes/v24.1/orchestration/test-cluster-secure.md
index f255d8d62fc..c146d634626 100644
--- a/src/current/_includes/v24.1/orchestration/test-cluster-secure.md
+++ b/src/current/_includes/v24.1/orchestration/test-cluster-secure.md
@@ -42,7 +42,7 @@ $ kubectl create \
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
--f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/client.yaml
+-f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/client.yaml
~~~
~~~
diff --git a/src/current/_includes/v24.2/misc/tooling.md b/src/current/_includes/v24.2/misc/tooling.md
index dcd24363435..01dc63c840f 100644
--- a/src/current/_includes/v24.2/misc/tooling.md
+++ b/src/current/_includes/v24.2/misc/tooling.md
@@ -23,7 +23,7 @@ Customers should contact their account team before moving production workloads t
Unless explicitly stated, support for a [driver](#drivers) or [data access framework](#data-access-frameworks-e-g-orms) does not include [automatic, client-side transaction retry handling]({% link {{ page.version.version }}/transaction-retry-error-reference.md %}#client-side-retry-handling). For client-side transaction retry handling samples, see [Example Apps]({% link {{ page.version.version }}/example-apps.md %}).
{{site.data.alerts.end}}
-If you encounter problems using CockroachDB with any of the tools listed on this page, please [open an issue](https://github.com/cockroachdb/cockroach/issues/new) with details to help us make progress toward better support.
+If you encounter problems using CockroachDB with any of the tools listed on this page, please open an issue with details to help us make progress toward better support.
For a list of tools supported by the CockroachDB community, see [Third-Party Tools Supported by the Community]({% link {{ page.version.version }}/community-tooling.md %}).
@@ -32,23 +32,23 @@ For a list of tools supported by the CockroachDB community, see [Third-Party Too
| Language | Driver | Latest tested version | Support level | CockroachDB adapter | Tutorial |
|----------+--------+-----------------------+---------------------+---------------------+----------|
| C | [libpq](http://www.postgresql.org/docs/13/static/libpq.html)| PostgreSQL 13 | Partial | N/A | N/A |
-| C# (.NET) | [Npgsql](https://www.nuget.org/packages/Npgsql/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/npgsql.go ||var npgsqlSupportedTag = "v||"\n\n %} | Full | N/A | [Build a C# App with CockroachDB (Npgsql)](build-a-csharp-app-with-cockroachdb.html) |
-| Go | [pgx](https://github.com/jackc/pgx/releases)
[pq](https://github.com/lib/pq) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/pgx.go ||var supportedPGXTag = "||"\n\n %}
(use latest version of CockroachDB adapter)
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/libpq.go ||var libPQSupportedTag = "||"\n\n %} | Full
Full | [`crdbpgx`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbpgx)
(includes client-side transaction retry handling)
N/A | [Build a Go App with CockroachDB (pgx)](build-a-go-app-with-cockroachdb.html)
[Build a Go App with CockroachDB (pq)](build-a-go-app-with-cockroachdb-pq.html) |
-| Java | [JDBC](https://jdbc.postgresql.org/download/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/pgjdbc.go ||var supportedPGJDBCTag = "||"\n\n %} | Full | N/A | [Build a Java App with CockroachDB (JDBC)](build-a-java-app-with-cockroachdb.html) |
+| C# (.NET) | [Npgsql](https://www.nuget.org/packages/Npgsql/) | 7.0.2 | Full | N/A | [Build a C# App with CockroachDB (Npgsql)](build-a-csharp-app-with-cockroachdb.html) |
+| Go | [pgx](https://github.com/jackc/pgx/releases)
[pq](https://github.com/lib/pq) | v5.3.1
(use latest version of CockroachDB adapter)
v1.10.5 | Full
Full | [`crdbpgx`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbpgx)
(includes client-side transaction retry handling)
N/A | [Build a Go App with CockroachDB (pgx)](build-a-go-app-with-cockroachdb.html)
[Build a Go App with CockroachDB (pq)](build-a-go-app-with-cockroachdb-pq.html) |
+| Java | [JDBC](https://jdbc.postgresql.org/download/) | REL42.7.3 | Full | N/A | [Build a Java App with CockroachDB (JDBC)](build-a-java-app-with-cockroachdb.html) |
| JavaScript | [pg](https://www.npmjs.com/package/pg) | 8.2.1 | Full | N/A | [Build a Node.js App with CockroachDB (pg)](build-a-nodejs-app-with-cockroachdb.html) |
-| Python | [psycopg3](https://www.psycopg.org/psycopg3/docs/)
[psycopg2](https://www.psycopg.org/docs/install.html)
[asyncpg](https://magicstack.github.io/asyncpg/current/index.html) | 3.0.16
2.8.6
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/asyncpg.go || var asyncpgSupportedTag = "||"\n\n %} | Full
Full
Partial | N/A
N/A
N/A | [Build a Python App with CockroachDB (psycopg3)](build-a-python-app-with-cockroachdb-psycopg3.html)
[Build a Python App with CockroachDB (psycopg2)](build-a-python-app-with-cockroachdb.html)
[Build a Python App with CockroachDB (asyncpg)](build-a-python-app-with-cockroachdb-asyncpg.html) |
-| Ruby | [pg](https://rubygems.org/gems/pg) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/ruby_pg.go ||var rubyPGVersion = "||"\n\n %} | Full | N/A | [Build a Ruby App with CockroachDB (pg)](build-a-ruby-app-with-cockroachdb.html) |
+| Python | [psycopg3](https://www.psycopg.org/psycopg3/docs/)
[psycopg2](https://www.psycopg.org/docs/install.html)
[asyncpg](https://magicstack.github.io/asyncpg/current/index.html) | 3.0.16
2.8.6
v0.24.0 | Full
Full
Partial | N/A
N/A
N/A | [Build a Python App with CockroachDB (psycopg3)](build-a-python-app-with-cockroachdb-psycopg3.html)
[Build a Python App with CockroachDB (psycopg2)](build-a-python-app-with-cockroachdb.html)
[Build a Python App with CockroachDB (asyncpg)](build-a-python-app-with-cockroachdb-asyncpg.html) |
+| Ruby | [pg](https://rubygems.org/gems/pg) | v1.4.6 | Full | N/A | [Build a Ruby App with CockroachDB (pg)](build-a-ruby-app-with-cockroachdb.html) |
| Rust | [rust-postgres](https://github.com/sfackler/rust-postgres) | 0.19.2 | Partial | N/A | [Build a Rust App with CockroachDB]({% link {{ page.version.version }}/build-a-rust-app-with-cockroachdb.md %}) |
## Data access frameworks (e.g., ORMs)
| Language | Framework | Latest tested version | Support level | CockroachDB adapter | Tutorial |
|----------+-----------+-----------------------+---------------+---------------------+----------|
-| Go | [GORM](https://github.com/jinzhu/gorm/releases)
[go-pg](https://github.com/go-pg/pg)
[upper/db](https://github.com/upper/db) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/gorm.go ||var gormSupportedTag = "||"\n\n %}
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/gopg.go ||var gopgSupportedTag = "||"\n\n %}
v4 | Full
Full
Full | [`crdbgorm`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbgorm)
(includes client-side transaction retry handling)
N/A
N/A | [Build a Go App with CockroachDB (GORM)](build-a-go-app-with-cockroachdb-gorm.html)
N/A
[Build a Go App with CockroachDB (upper/db)](build-a-go-app-with-cockroachdb-upperdb.html) |
-| Java | [Hibernate](https://hibernate.org/orm/)
(including [Hibernate Spatial](https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#spatial))
[jOOQ](https://www.jooq.org/)
[MyBatis](https://mybatis.org/mybatis-3/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/hibernate.go ||var supportedHibernateTag = "||"\n\n %} (must be at least 5.4.19)
3.13.2 (must be at least 3.13.0)
3.5.5| Full
Full
Full | N/A
N/A
N/A | [Build a Java App with CockroachDB (Hibernate)](build-a-java-app-with-cockroachdb-hibernate.html)
[Build a Java App with CockroachDB (jOOQ)](build-a-java-app-with-cockroachdb-jooq.html)
[Build a Spring App with CockroachDB (MyBatis)]({% link {{ page.version.version }}/build-a-spring-app-with-cockroachdb-mybatis.md %}) |
-| JavaScript/TypeScript | [Sequelize](https://www.npmjs.com/package/sequelize)
[Knex.js](https://knexjs.org/)
[Prisma](https://prisma.io)
[TypeORM](https://www.npmjs.com/package/typeorm) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/sequelize.go ||var supportedSequelizeCockroachDBRelease = "||"\n\n %}
(use latest version of CockroachDB adapter)
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/knex.go ||const supportedKnexTag = "||"\n\n %}
3.14.0
0.3.17 {% comment %}{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/typeorm.go ||const supportedTypeORMRelease = "||"\n %}{% endcomment %} | Full
Full
Full
Full | [`sequelize-cockroachdb`](https://www.npmjs.com/package/sequelize-cockroachdb)
N/A
N/A
N/A | [Build a Node.js App with CockroachDB (Sequelize)](build-a-nodejs-app-with-cockroachdb-sequelize.html)
[Build a Node.js App with CockroachDB (Knex.js)](build-a-nodejs-app-with-cockroachdb-knexjs.html)
[Build a Node.js App with CockroachDB (Prisma)](build-a-nodejs-app-with-cockroachdb-prisma.html)
[Build a TypeScript App with CockroachDB (TypeORM)](build-a-typescript-app-with-cockroachdb.html) |
-| Ruby | [ActiveRecord](https://rubygems.org/gems/activerecord)
[RGeo/RGeo-ActiveRecord](https://github.com/cockroachdb/activerecord-cockroachdb-adapter#working-with-spatial-data) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/activerecord.go ||var supportedRailsVersion = "||"\nvar %}
(use latest version of CockroachDB adapter) | Full | [`activerecord-cockroachdb-adapter`](https://rubygems.org/gems/activerecord-cockroachdb-adapter)
(includes client-side transaction retry handling) | [Build a Ruby App with CockroachDB (ActiveRecord)](build-a-ruby-app-with-cockroachdb-activerecord.html) |
-| Python | [Django](https://pypi.org/project/Django/)
(including [GeoDjango](https://docs.djangoproject.com/en/3.1/ref/contrib/gis/))
[peewee](https://github.com/coleifer/peewee/)
[SQLAlchemy](https://www.sqlalchemy.org/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/django.go ||var djangoSupportedTag = "cockroach-||"\nvar %}
(use latest version of CockroachDB adapter)
3.13.3
0.7.13
1.4.17
(use latest version of CockroachDB adapter) | Full
Full
Full
Full | [`django-cockroachdb`](https://pypi.org/project/django-cockroachdb/)
N/A
N/A
[`sqlalchemy-cockroachdb`](https://pypi.org/project/sqlalchemy-cockroachdb)
(includes client-side transaction retry handling) | [Build a Python App with CockroachDB (Django)](build-a-python-app-with-cockroachdb-django.html)
N/A (See [peewee docs](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#cockroach-database).)
[Build a Python App with CockroachDB (SQLAlchemy)](build-a-python-app-with-cockroachdb-sqlalchemy.html) |
+| Go | [GORM](https://github.com/jinzhu/gorm/releases)
[go-pg](https://github.com/go-pg/pg)
[upper/db](https://github.com/upper/db) | v1.24.1
v10.9.0
v4 | Full
Full
Full | [`crdbgorm`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbgorm)
(includes client-side transaction retry handling)
N/A
N/A | [Build a Go App with CockroachDB (GORM)](build-a-go-app-with-cockroachdb-gorm.html)
N/A
[Build a Go App with CockroachDB (upper/db)](build-a-go-app-with-cockroachdb-upperdb.html) |
+| Java | [Hibernate](https://hibernate.org/orm/)
(including [Hibernate Spatial](https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#spatial))
[jOOQ](https://www.jooq.org/)
[MyBatis](https://mybatis.org/mybatis-3/) | 6.6.20 (must be at least 5.4.19)
3.13.2 (must be at least 3.13.0)
3.5.5| Full
Full
Full | N/A
N/A
N/A | [Build a Java App with CockroachDB (Hibernate)](build-a-java-app-with-cockroachdb-hibernate.html)
[Build a Java App with CockroachDB (jOOQ)](build-a-java-app-with-cockroachdb-jooq.html)
[Build a Spring App with CockroachDB (MyBatis)]({% link {{ page.version.version }}/build-a-spring-app-with-cockroachdb-mybatis.md %}) |
+| JavaScript/TypeScript | [Sequelize](https://www.npmjs.com/package/sequelize)
[Knex.js](https://knexjs.org/)
[Prisma](https://prisma.io)
[TypeORM](https://www.npmjs.com/package/typeorm) | v6.0.5
(use latest version of CockroachDB adapter)
2.5.1
3.14.0
0.3.17 {% comment %}remove-unsafe-crdb-setting{% endcomment %} | Full
Full
Full
Full | [`sequelize-cockroachdb`](https://www.npmjs.com/package/sequelize-cockroachdb)
N/A
N/A
N/A | [Build a Node.js App with CockroachDB (Sequelize)](build-a-nodejs-app-with-cockroachdb-sequelize.html)
[Build a Node.js App with CockroachDB (Knex.js)](build-a-nodejs-app-with-cockroachdb-knexjs.html)
[Build a Node.js App with CockroachDB (Prisma)](build-a-nodejs-app-with-cockroachdb-prisma.html)
[Build a TypeScript App with CockroachDB (TypeORM)](build-a-typescript-app-with-cockroachdb.html) |
+| Ruby | [ActiveRecord](https://rubygems.org/gems/activerecord)
[RGeo/RGeo-ActiveRecord](https://github.com/cockroachdb/activerecord-cockroachdb-adapter#working-with-spatial-data) | 8.1.0
(use latest version of CockroachDB adapter) | Full | [`activerecord-cockroachdb-adapter`](https://rubygems.org/gems/activerecord-cockroachdb-adapter)
(includes client-side transaction retry handling) | [Build a Ruby App with CockroachDB (ActiveRecord)](build-a-ruby-app-with-cockroachdb-activerecord.html) |
+| Python | [Django](https://pypi.org/project/Django/)
(including [GeoDjango](https://docs.djangoproject.com/en/3.1/ref/contrib/gis/))
[peewee](https://github.com/coleifer/peewee/)
[SQLAlchemy](https://www.sqlalchemy.org/) | 4.1.x
(use latest version of CockroachDB adapter)
3.13.3
0.7.13
1.4.17
(use latest version of CockroachDB adapter) | Full
Full
Full
Full | [`django-cockroachdb`](https://pypi.org/project/django-cockroachdb/)
N/A
N/A
[`sqlalchemy-cockroachdb`](https://pypi.org/project/sqlalchemy-cockroachdb)
(includes client-side transaction retry handling) | [Build a Python App with CockroachDB (Django)](build-a-python-app-with-cockroachdb-django.html)
N/A (See [peewee docs](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#cockroach-database).)
[Build a Python App with CockroachDB (SQLAlchemy)](build-a-python-app-with-cockroachdb-sqlalchemy.html) |
## Application frameworks
diff --git a/src/current/_includes/v24.2/orchestration/start-cockroachdb-insecure.md b/src/current/_includes/v24.2/orchestration/start-cockroachdb-insecure.md
index 3406d48edbb..c9af5cc0267 100644
--- a/src/current/_includes/v24.2/orchestration/start-cockroachdb-insecure.md
+++ b/src/current/_includes/v24.2/orchestration/start-cockroachdb-insecure.md
@@ -1,10 +1,10 @@
-1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it.
+1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it.
- Download [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml):
+ Download [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml
~~~
{{site.data.alerts.callout_info}}
@@ -27,11 +27,11 @@
Alternatively, if you'd rather start with a configuration file that has been customized for performance:
- 1. Download our [performance version of `cockroachdb-statefulset-insecure.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml):
+ 1. Download our [performance version of `cockroachdb-statefulset-insecure.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml
~~~
1. Modify the file wherever there is a `TODO` comment.
@@ -72,12 +72,12 @@
pvc-5315efda-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-2 27s
~~~
-1. Use our [`cluster-init.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
+1. Use our [`cluster-init.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml
~~~
~~~
diff --git a/src/current/_includes/v24.2/orchestration/start-cockroachdb-local-insecure.md b/src/current/_includes/v24.2/orchestration/start-cockroachdb-local-insecure.md
index 552cb3cd25f..196a53bce4c 100644
--- a/src/current/_includes/v24.2/orchestration/start-cockroachdb-local-insecure.md
+++ b/src/current/_includes/v24.2/orchestration/start-cockroachdb-local-insecure.md
@@ -1,8 +1,8 @@
-1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it:
+1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it:
{% include_cached copy-clipboard.html %}
~~~ shell
- $ kubectl create -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml
+ $ kubectl create -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml
~~~
~~~
@@ -41,12 +41,12 @@
pvc-5315efda-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-2 27s
~~~
-1. Use our [`cluster-init.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
+1. Use our [`cluster-init.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml
~~~
~~~
diff --git a/src/current/_includes/v24.2/orchestration/start-cockroachdb-secure.md b/src/current/_includes/v24.2/orchestration/start-cockroachdb-secure.md
index 972cabc2d8e..a7299e1aa25 100644
--- a/src/current/_includes/v24.2/orchestration/start-cockroachdb-secure.md
+++ b/src/current/_includes/v24.2/orchestration/start-cockroachdb-secure.md
@@ -1,10 +1,10 @@
### Configure the cluster
-1. Download and modify our [StatefulSet configuration](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml):
+1. Download and modify our [StatefulSet configuration](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml
~~~
1. Update `secretName` with the name of the corresponding node secret.
diff --git a/src/current/_includes/v24.2/orchestration/test-cluster-secure.md b/src/current/_includes/v24.2/orchestration/test-cluster-secure.md
index f255d8d62fc..c146d634626 100644
--- a/src/current/_includes/v24.2/orchestration/test-cluster-secure.md
+++ b/src/current/_includes/v24.2/orchestration/test-cluster-secure.md
@@ -42,7 +42,7 @@ $ kubectl create \
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
--f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/client.yaml
+-f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/client.yaml
~~~
~~~
diff --git a/src/current/_includes/v24.3/misc/tooling.md b/src/current/_includes/v24.3/misc/tooling.md
index dcd24363435..01dc63c840f 100644
--- a/src/current/_includes/v24.3/misc/tooling.md
+++ b/src/current/_includes/v24.3/misc/tooling.md
@@ -23,7 +23,7 @@ Customers should contact their account team before moving production workloads t
Unless explicitly stated, support for a [driver](#drivers) or [data access framework](#data-access-frameworks-e-g-orms) does not include [automatic, client-side transaction retry handling]({% link {{ page.version.version }}/transaction-retry-error-reference.md %}#client-side-retry-handling). For client-side transaction retry handling samples, see [Example Apps]({% link {{ page.version.version }}/example-apps.md %}).
{{site.data.alerts.end}}
-If you encounter problems using CockroachDB with any of the tools listed on this page, please [open an issue](https://github.com/cockroachdb/cockroach/issues/new) with details to help us make progress toward better support.
+If you encounter problems using CockroachDB with any of the tools listed on this page, please open an issue with details to help us make progress toward better support.
For a list of tools supported by the CockroachDB community, see [Third-Party Tools Supported by the Community]({% link {{ page.version.version }}/community-tooling.md %}).
@@ -32,23 +32,23 @@ For a list of tools supported by the CockroachDB community, see [Third-Party Too
| Language | Driver | Latest tested version | Support level | CockroachDB adapter | Tutorial |
|----------+--------+-----------------------+---------------------+---------------------+----------|
| C | [libpq](http://www.postgresql.org/docs/13/static/libpq.html)| PostgreSQL 13 | Partial | N/A | N/A |
-| C# (.NET) | [Npgsql](https://www.nuget.org/packages/Npgsql/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/npgsql.go ||var npgsqlSupportedTag = "v||"\n\n %} | Full | N/A | [Build a C# App with CockroachDB (Npgsql)](build-a-csharp-app-with-cockroachdb.html) |
-| Go | [pgx](https://github.com/jackc/pgx/releases)
[pq](https://github.com/lib/pq) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/pgx.go ||var supportedPGXTag = "||"\n\n %}
(use latest version of CockroachDB adapter)
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/libpq.go ||var libPQSupportedTag = "||"\n\n %} | Full
Full | [`crdbpgx`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbpgx)
(includes client-side transaction retry handling)
N/A | [Build a Go App with CockroachDB (pgx)](build-a-go-app-with-cockroachdb.html)
[Build a Go App with CockroachDB (pq)](build-a-go-app-with-cockroachdb-pq.html) |
-| Java | [JDBC](https://jdbc.postgresql.org/download/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/pgjdbc.go ||var supportedPGJDBCTag = "||"\n\n %} | Full | N/A | [Build a Java App with CockroachDB (JDBC)](build-a-java-app-with-cockroachdb.html) |
+| C# (.NET) | [Npgsql](https://www.nuget.org/packages/Npgsql/) | 7.0.2 | Full | N/A | [Build a C# App with CockroachDB (Npgsql)](build-a-csharp-app-with-cockroachdb.html) |
+| Go | [pgx](https://github.com/jackc/pgx/releases)
[pq](https://github.com/lib/pq) | v5.3.1
(use latest version of CockroachDB adapter)
v1.10.5 | Full
Full | [`crdbpgx`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbpgx)
(includes client-side transaction retry handling)
N/A | [Build a Go App with CockroachDB (pgx)](build-a-go-app-with-cockroachdb.html)
[Build a Go App with CockroachDB (pq)](build-a-go-app-with-cockroachdb-pq.html) |
+| Java | [JDBC](https://jdbc.postgresql.org/download/) | REL42.7.3 | Full | N/A | [Build a Java App with CockroachDB (JDBC)](build-a-java-app-with-cockroachdb.html) |
| JavaScript | [pg](https://www.npmjs.com/package/pg) | 8.2.1 | Full | N/A | [Build a Node.js App with CockroachDB (pg)](build-a-nodejs-app-with-cockroachdb.html) |
-| Python | [psycopg3](https://www.psycopg.org/psycopg3/docs/)
[psycopg2](https://www.psycopg.org/docs/install.html)
[asyncpg](https://magicstack.github.io/asyncpg/current/index.html) | 3.0.16
2.8.6
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/asyncpg.go || var asyncpgSupportedTag = "||"\n\n %} | Full
Full
Partial | N/A
N/A
N/A | [Build a Python App with CockroachDB (psycopg3)](build-a-python-app-with-cockroachdb-psycopg3.html)
[Build a Python App with CockroachDB (psycopg2)](build-a-python-app-with-cockroachdb.html)
[Build a Python App with CockroachDB (asyncpg)](build-a-python-app-with-cockroachdb-asyncpg.html) |
-| Ruby | [pg](https://rubygems.org/gems/pg) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/ruby_pg.go ||var rubyPGVersion = "||"\n\n %} | Full | N/A | [Build a Ruby App with CockroachDB (pg)](build-a-ruby-app-with-cockroachdb.html) |
+| Python | [psycopg3](https://www.psycopg.org/psycopg3/docs/)
[psycopg2](https://www.psycopg.org/docs/install.html)
[asyncpg](https://magicstack.github.io/asyncpg/current/index.html) | 3.0.16
2.8.6
v0.24.0 | Full
Full
Partial | N/A
N/A
N/A | [Build a Python App with CockroachDB (psycopg3)](build-a-python-app-with-cockroachdb-psycopg3.html)
[Build a Python App with CockroachDB (psycopg2)](build-a-python-app-with-cockroachdb.html)
[Build a Python App with CockroachDB (asyncpg)](build-a-python-app-with-cockroachdb-asyncpg.html) |
+| Ruby | [pg](https://rubygems.org/gems/pg) | v1.4.6 | Full | N/A | [Build a Ruby App with CockroachDB (pg)](build-a-ruby-app-with-cockroachdb.html) |
| Rust | [rust-postgres](https://github.com/sfackler/rust-postgres) | 0.19.2 | Partial | N/A | [Build a Rust App with CockroachDB]({% link {{ page.version.version }}/build-a-rust-app-with-cockroachdb.md %}) |
## Data access frameworks (e.g., ORMs)
| Language | Framework | Latest tested version | Support level | CockroachDB adapter | Tutorial |
|----------+-----------+-----------------------+---------------+---------------------+----------|
-| Go | [GORM](https://github.com/jinzhu/gorm/releases)
[go-pg](https://github.com/go-pg/pg)
[upper/db](https://github.com/upper/db) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/gorm.go ||var gormSupportedTag = "||"\n\n %}
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/gopg.go ||var gopgSupportedTag = "||"\n\n %}
v4 | Full
Full
Full | [`crdbgorm`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbgorm)
(includes client-side transaction retry handling)
N/A
N/A | [Build a Go App with CockroachDB (GORM)](build-a-go-app-with-cockroachdb-gorm.html)
N/A
[Build a Go App with CockroachDB (upper/db)](build-a-go-app-with-cockroachdb-upperdb.html) |
-| Java | [Hibernate](https://hibernate.org/orm/)
(including [Hibernate Spatial](https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#spatial))
[jOOQ](https://www.jooq.org/)
[MyBatis](https://mybatis.org/mybatis-3/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/hibernate.go ||var supportedHibernateTag = "||"\n\n %} (must be at least 5.4.19)
3.13.2 (must be at least 3.13.0)
3.5.5| Full
Full
Full | N/A
N/A
N/A | [Build a Java App with CockroachDB (Hibernate)](build-a-java-app-with-cockroachdb-hibernate.html)
[Build a Java App with CockroachDB (jOOQ)](build-a-java-app-with-cockroachdb-jooq.html)
[Build a Spring App with CockroachDB (MyBatis)]({% link {{ page.version.version }}/build-a-spring-app-with-cockroachdb-mybatis.md %}) |
-| JavaScript/TypeScript | [Sequelize](https://www.npmjs.com/package/sequelize)
[Knex.js](https://knexjs.org/)
[Prisma](https://prisma.io)
[TypeORM](https://www.npmjs.com/package/typeorm) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/sequelize.go ||var supportedSequelizeCockroachDBRelease = "||"\n\n %}
(use latest version of CockroachDB adapter)
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/knex.go ||const supportedKnexTag = "||"\n\n %}
3.14.0
0.3.17 {% comment %}{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/typeorm.go ||const supportedTypeORMRelease = "||"\n %}{% endcomment %} | Full
Full
Full
Full | [`sequelize-cockroachdb`](https://www.npmjs.com/package/sequelize-cockroachdb)
N/A
N/A
N/A | [Build a Node.js App with CockroachDB (Sequelize)](build-a-nodejs-app-with-cockroachdb-sequelize.html)
[Build a Node.js App with CockroachDB (Knex.js)](build-a-nodejs-app-with-cockroachdb-knexjs.html)
[Build a Node.js App with CockroachDB (Prisma)](build-a-nodejs-app-with-cockroachdb-prisma.html)
[Build a TypeScript App with CockroachDB (TypeORM)](build-a-typescript-app-with-cockroachdb.html) |
-| Ruby | [ActiveRecord](https://rubygems.org/gems/activerecord)
[RGeo/RGeo-ActiveRecord](https://github.com/cockroachdb/activerecord-cockroachdb-adapter#working-with-spatial-data) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/activerecord.go ||var supportedRailsVersion = "||"\nvar %}
(use latest version of CockroachDB adapter) | Full | [`activerecord-cockroachdb-adapter`](https://rubygems.org/gems/activerecord-cockroachdb-adapter)
(includes client-side transaction retry handling) | [Build a Ruby App with CockroachDB (ActiveRecord)](build-a-ruby-app-with-cockroachdb-activerecord.html) |
-| Python | [Django](https://pypi.org/project/Django/)
(including [GeoDjango](https://docs.djangoproject.com/en/3.1/ref/contrib/gis/))
[peewee](https://github.com/coleifer/peewee/)
[SQLAlchemy](https://www.sqlalchemy.org/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/django.go ||var djangoSupportedTag = "cockroach-||"\nvar %}
(use latest version of CockroachDB adapter)
3.13.3
0.7.13
1.4.17
(use latest version of CockroachDB adapter) | Full
Full
Full
Full | [`django-cockroachdb`](https://pypi.org/project/django-cockroachdb/)
N/A
N/A
[`sqlalchemy-cockroachdb`](https://pypi.org/project/sqlalchemy-cockroachdb)
(includes client-side transaction retry handling) | [Build a Python App with CockroachDB (Django)](build-a-python-app-with-cockroachdb-django.html)
N/A (See [peewee docs](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#cockroach-database).)
[Build a Python App with CockroachDB (SQLAlchemy)](build-a-python-app-with-cockroachdb-sqlalchemy.html) |
+| Go | [GORM](https://github.com/jinzhu/gorm/releases)
[go-pg](https://github.com/go-pg/pg)
[upper/db](https://github.com/upper/db) | v1.24.1
v10.9.0
v4 | Full
Full
Full | [`crdbgorm`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbgorm)
(includes client-side transaction retry handling)
N/A
N/A | [Build a Go App with CockroachDB (GORM)](build-a-go-app-with-cockroachdb-gorm.html)
N/A
[Build a Go App with CockroachDB (upper/db)](build-a-go-app-with-cockroachdb-upperdb.html) |
+| Java | [Hibernate](https://hibernate.org/orm/)
(including [Hibernate Spatial](https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#spatial))
[jOOQ](https://www.jooq.org/)
[MyBatis](https://mybatis.org/mybatis-3/) | 6.6.20 (must be at least 5.4.19)
3.13.2 (must be at least 3.13.0)
3.5.5| Full
Full
Full | N/A
N/A
N/A | [Build a Java App with CockroachDB (Hibernate)](build-a-java-app-with-cockroachdb-hibernate.html)
[Build a Java App with CockroachDB (jOOQ)](build-a-java-app-with-cockroachdb-jooq.html)
[Build a Spring App with CockroachDB (MyBatis)]({% link {{ page.version.version }}/build-a-spring-app-with-cockroachdb-mybatis.md %}) |
+| JavaScript/TypeScript | [Sequelize](https://www.npmjs.com/package/sequelize)
[Knex.js](https://knexjs.org/)
[Prisma](https://prisma.io)
[TypeORM](https://www.npmjs.com/package/typeorm) | v6.0.5
(use latest version of CockroachDB adapter)
2.5.1
3.14.0
0.3.17 {% comment %}remove-unsafe-crdb-setting{% endcomment %} | Full
Full
Full
Full | [`sequelize-cockroachdb`](https://www.npmjs.com/package/sequelize-cockroachdb)
N/A
N/A
N/A | [Build a Node.js App with CockroachDB (Sequelize)](build-a-nodejs-app-with-cockroachdb-sequelize.html)
[Build a Node.js App with CockroachDB (Knex.js)](build-a-nodejs-app-with-cockroachdb-knexjs.html)
[Build a Node.js App with CockroachDB (Prisma)](build-a-nodejs-app-with-cockroachdb-prisma.html)
[Build a TypeScript App with CockroachDB (TypeORM)](build-a-typescript-app-with-cockroachdb.html) |
+| Ruby | [ActiveRecord](https://rubygems.org/gems/activerecord)
[RGeo/RGeo-ActiveRecord](https://github.com/cockroachdb/activerecord-cockroachdb-adapter#working-with-spatial-data) | 8.1.0
(use latest version of CockroachDB adapter) | Full | [`activerecord-cockroachdb-adapter`](https://rubygems.org/gems/activerecord-cockroachdb-adapter)
(includes client-side transaction retry handling) | [Build a Ruby App with CockroachDB (ActiveRecord)](build-a-ruby-app-with-cockroachdb-activerecord.html) |
+| Python | [Django](https://pypi.org/project/Django/)
(including [GeoDjango](https://docs.djangoproject.com/en/3.1/ref/contrib/gis/))
[peewee](https://github.com/coleifer/peewee/)
[SQLAlchemy](https://www.sqlalchemy.org/) | 4.1.x
(use latest version of CockroachDB adapter)
3.13.3
0.7.13
1.4.17
(use latest version of CockroachDB adapter) | Full
Full
Full
Full | [`django-cockroachdb`](https://pypi.org/project/django-cockroachdb/)
N/A
N/A
[`sqlalchemy-cockroachdb`](https://pypi.org/project/sqlalchemy-cockroachdb)
(includes client-side transaction retry handling) | [Build a Python App with CockroachDB (Django)](build-a-python-app-with-cockroachdb-django.html)
N/A (See [peewee docs](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#cockroach-database).)
[Build a Python App with CockroachDB (SQLAlchemy)](build-a-python-app-with-cockroachdb-sqlalchemy.html) |
## Application frameworks
diff --git a/src/current/_includes/v24.3/orchestration/start-cockroachdb-insecure.md b/src/current/_includes/v24.3/orchestration/start-cockroachdb-insecure.md
index 3406d48edbb..c9af5cc0267 100644
--- a/src/current/_includes/v24.3/orchestration/start-cockroachdb-insecure.md
+++ b/src/current/_includes/v24.3/orchestration/start-cockroachdb-insecure.md
@@ -1,10 +1,10 @@
-1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it.
+1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it.
- Download [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml):
+ Download [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml
~~~
{{site.data.alerts.callout_info}}
@@ -27,11 +27,11 @@
Alternatively, if you'd rather start with a configuration file that has been customized for performance:
- 1. Download our [performance version of `cockroachdb-statefulset-insecure.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml):
+ 1. Download our [performance version of `cockroachdb-statefulset-insecure.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml
~~~
1. Modify the file wherever there is a `TODO` comment.
@@ -72,12 +72,12 @@
pvc-5315efda-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-2 27s
~~~
-1. Use our [`cluster-init.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
+1. Use our [`cluster-init.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml
~~~
~~~
diff --git a/src/current/_includes/v24.3/orchestration/start-cockroachdb-local-insecure.md b/src/current/_includes/v24.3/orchestration/start-cockroachdb-local-insecure.md
index 552cb3cd25f..196a53bce4c 100644
--- a/src/current/_includes/v24.3/orchestration/start-cockroachdb-local-insecure.md
+++ b/src/current/_includes/v24.3/orchestration/start-cockroachdb-local-insecure.md
@@ -1,8 +1,8 @@
-1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it:
+1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it:
{% include_cached copy-clipboard.html %}
~~~ shell
- $ kubectl create -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml
+ $ kubectl create -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml
~~~
~~~
@@ -41,12 +41,12 @@
pvc-5315efda-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-2 27s
~~~
-1. Use our [`cluster-init.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
+1. Use our [`cluster-init.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml
~~~
~~~
diff --git a/src/current/_includes/v24.3/orchestration/start-cockroachdb-secure.md b/src/current/_includes/v24.3/orchestration/start-cockroachdb-secure.md
index 972cabc2d8e..a7299e1aa25 100644
--- a/src/current/_includes/v24.3/orchestration/start-cockroachdb-secure.md
+++ b/src/current/_includes/v24.3/orchestration/start-cockroachdb-secure.md
@@ -1,10 +1,10 @@
### Configure the cluster
-1. Download and modify our [StatefulSet configuration](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml):
+1. Download and modify our [StatefulSet configuration](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml
~~~
1. Update `secretName` with the name of the corresponding node secret.
diff --git a/src/current/_includes/v24.3/orchestration/test-cluster-secure.md b/src/current/_includes/v24.3/orchestration/test-cluster-secure.md
index f255d8d62fc..c146d634626 100644
--- a/src/current/_includes/v24.3/orchestration/test-cluster-secure.md
+++ b/src/current/_includes/v24.3/orchestration/test-cluster-secure.md
@@ -42,7 +42,7 @@ $ kubectl create \
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
--f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/client.yaml
+-f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/client.yaml
~~~
~~~
diff --git a/src/current/_includes/v25.1/misc/tooling.md b/src/current/_includes/v25.1/misc/tooling.md
index dcd24363435..01dc63c840f 100644
--- a/src/current/_includes/v25.1/misc/tooling.md
+++ b/src/current/_includes/v25.1/misc/tooling.md
@@ -23,7 +23,7 @@ Customers should contact their account team before moving production workloads t
Unless explicitly stated, support for a [driver](#drivers) or [data access framework](#data-access-frameworks-e-g-orms) does not include [automatic, client-side transaction retry handling]({% link {{ page.version.version }}/transaction-retry-error-reference.md %}#client-side-retry-handling). For client-side transaction retry handling samples, see [Example Apps]({% link {{ page.version.version }}/example-apps.md %}).
{{site.data.alerts.end}}
-If you encounter problems using CockroachDB with any of the tools listed on this page, please [open an issue](https://github.com/cockroachdb/cockroach/issues/new) with details to help us make progress toward better support.
+If you encounter problems using CockroachDB with any of the tools listed on this page, please open an issue with details to help us make progress toward better support.
For a list of tools supported by the CockroachDB community, see [Third-Party Tools Supported by the Community]({% link {{ page.version.version }}/community-tooling.md %}).
@@ -32,23 +32,23 @@ For a list of tools supported by the CockroachDB community, see [Third-Party Too
| Language | Driver | Latest tested version | Support level | CockroachDB adapter | Tutorial |
|----------+--------+-----------------------+---------------------+---------------------+----------|
| C | [libpq](http://www.postgresql.org/docs/13/static/libpq.html)| PostgreSQL 13 | Partial | N/A | N/A |
-| C# (.NET) | [Npgsql](https://www.nuget.org/packages/Npgsql/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/npgsql.go ||var npgsqlSupportedTag = "v||"\n\n %} | Full | N/A | [Build a C# App with CockroachDB (Npgsql)](build-a-csharp-app-with-cockroachdb.html) |
-| Go | [pgx](https://github.com/jackc/pgx/releases)
[pq](https://github.com/lib/pq) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/pgx.go ||var supportedPGXTag = "||"\n\n %}
(use latest version of CockroachDB adapter)
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/libpq.go ||var libPQSupportedTag = "||"\n\n %} | Full
Full | [`crdbpgx`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbpgx)
(includes client-side transaction retry handling)
N/A | [Build a Go App with CockroachDB (pgx)](build-a-go-app-with-cockroachdb.html)
[Build a Go App with CockroachDB (pq)](build-a-go-app-with-cockroachdb-pq.html) |
-| Java | [JDBC](https://jdbc.postgresql.org/download/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/pgjdbc.go ||var supportedPGJDBCTag = "||"\n\n %} | Full | N/A | [Build a Java App with CockroachDB (JDBC)](build-a-java-app-with-cockroachdb.html) |
+| C# (.NET) | [Npgsql](https://www.nuget.org/packages/Npgsql/) | 7.0.2 | Full | N/A | [Build a C# App with CockroachDB (Npgsql)](build-a-csharp-app-with-cockroachdb.html) |
+| Go | [pgx](https://github.com/jackc/pgx/releases)
[pq](https://github.com/lib/pq) | v5.3.1
(use latest version of CockroachDB adapter)
v1.10.5 | Full
Full | [`crdbpgx`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbpgx)
(includes client-side transaction retry handling)
N/A | [Build a Go App with CockroachDB (pgx)](build-a-go-app-with-cockroachdb.html)
[Build a Go App with CockroachDB (pq)](build-a-go-app-with-cockroachdb-pq.html) |
+| Java | [JDBC](https://jdbc.postgresql.org/download/) | REL42.7.3 | Full | N/A | [Build a Java App with CockroachDB (JDBC)](build-a-java-app-with-cockroachdb.html) |
| JavaScript | [pg](https://www.npmjs.com/package/pg) | 8.2.1 | Full | N/A | [Build a Node.js App with CockroachDB (pg)](build-a-nodejs-app-with-cockroachdb.html) |
-| Python | [psycopg3](https://www.psycopg.org/psycopg3/docs/)
[psycopg2](https://www.psycopg.org/docs/install.html)
[asyncpg](https://magicstack.github.io/asyncpg/current/index.html) | 3.0.16
2.8.6
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/asyncpg.go || var asyncpgSupportedTag = "||"\n\n %} | Full
Full
Partial | N/A
N/A
N/A | [Build a Python App with CockroachDB (psycopg3)](build-a-python-app-with-cockroachdb-psycopg3.html)
[Build a Python App with CockroachDB (psycopg2)](build-a-python-app-with-cockroachdb.html)
[Build a Python App with CockroachDB (asyncpg)](build-a-python-app-with-cockroachdb-asyncpg.html) |
-| Ruby | [pg](https://rubygems.org/gems/pg) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/ruby_pg.go ||var rubyPGVersion = "||"\n\n %} | Full | N/A | [Build a Ruby App with CockroachDB (pg)](build-a-ruby-app-with-cockroachdb.html) |
+| Python | [psycopg3](https://www.psycopg.org/psycopg3/docs/)
[psycopg2](https://www.psycopg.org/docs/install.html)
[asyncpg](https://magicstack.github.io/asyncpg/current/index.html) | 3.0.16
2.8.6
v0.24.0 | Full
Full
Partial | N/A
N/A
N/A | [Build a Python App with CockroachDB (psycopg3)](build-a-python-app-with-cockroachdb-psycopg3.html)
[Build a Python App with CockroachDB (psycopg2)](build-a-python-app-with-cockroachdb.html)
[Build a Python App with CockroachDB (asyncpg)](build-a-python-app-with-cockroachdb-asyncpg.html) |
+| Ruby | [pg](https://rubygems.org/gems/pg) | v1.4.6 | Full | N/A | [Build a Ruby App with CockroachDB (pg)](build-a-ruby-app-with-cockroachdb.html) |
| Rust | [rust-postgres](https://github.com/sfackler/rust-postgres) | 0.19.2 | Partial | N/A | [Build a Rust App with CockroachDB]({% link {{ page.version.version }}/build-a-rust-app-with-cockroachdb.md %}) |
## Data access frameworks (e.g., ORMs)
| Language | Framework | Latest tested version | Support level | CockroachDB adapter | Tutorial |
|----------+-----------+-----------------------+---------------+---------------------+----------|
-| Go | [GORM](https://github.com/jinzhu/gorm/releases)
[go-pg](https://github.com/go-pg/pg)
[upper/db](https://github.com/upper/db) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/gorm.go ||var gormSupportedTag = "||"\n\n %}
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/gopg.go ||var gopgSupportedTag = "||"\n\n %}
v4 | Full
Full
Full | [`crdbgorm`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbgorm)
(includes client-side transaction retry handling)
N/A
N/A | [Build a Go App with CockroachDB (GORM)](build-a-go-app-with-cockroachdb-gorm.html)
N/A
[Build a Go App with CockroachDB (upper/db)](build-a-go-app-with-cockroachdb-upperdb.html) |
-| Java | [Hibernate](https://hibernate.org/orm/)
(including [Hibernate Spatial](https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#spatial))
[jOOQ](https://www.jooq.org/)
[MyBatis](https://mybatis.org/mybatis-3/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/hibernate.go ||var supportedHibernateTag = "||"\n\n %} (must be at least 5.4.19)
3.13.2 (must be at least 3.13.0)
3.5.5| Full
Full
Full | N/A
N/A
N/A | [Build a Java App with CockroachDB (Hibernate)](build-a-java-app-with-cockroachdb-hibernate.html)
[Build a Java App with CockroachDB (jOOQ)](build-a-java-app-with-cockroachdb-jooq.html)
[Build a Spring App with CockroachDB (MyBatis)]({% link {{ page.version.version }}/build-a-spring-app-with-cockroachdb-mybatis.md %}) |
-| JavaScript/TypeScript | [Sequelize](https://www.npmjs.com/package/sequelize)
[Knex.js](https://knexjs.org/)
[Prisma](https://prisma.io)
[TypeORM](https://www.npmjs.com/package/typeorm) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/sequelize.go ||var supportedSequelizeCockroachDBRelease = "||"\n\n %}
(use latest version of CockroachDB adapter)
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/knex.go ||const supportedKnexTag = "||"\n\n %}
3.14.0
0.3.17 {% comment %}{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/typeorm.go ||const supportedTypeORMRelease = "||"\n %}{% endcomment %} | Full
Full
Full
Full | [`sequelize-cockroachdb`](https://www.npmjs.com/package/sequelize-cockroachdb)
N/A
N/A
N/A | [Build a Node.js App with CockroachDB (Sequelize)](build-a-nodejs-app-with-cockroachdb-sequelize.html)
[Build a Node.js App with CockroachDB (Knex.js)](build-a-nodejs-app-with-cockroachdb-knexjs.html)
[Build a Node.js App with CockroachDB (Prisma)](build-a-nodejs-app-with-cockroachdb-prisma.html)
[Build a TypeScript App with CockroachDB (TypeORM)](build-a-typescript-app-with-cockroachdb.html) |
-| Ruby | [ActiveRecord](https://rubygems.org/gems/activerecord)
[RGeo/RGeo-ActiveRecord](https://github.com/cockroachdb/activerecord-cockroachdb-adapter#working-with-spatial-data) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/activerecord.go ||var supportedRailsVersion = "||"\nvar %}
(use latest version of CockroachDB adapter) | Full | [`activerecord-cockroachdb-adapter`](https://rubygems.org/gems/activerecord-cockroachdb-adapter)
(includes client-side transaction retry handling) | [Build a Ruby App with CockroachDB (ActiveRecord)](build-a-ruby-app-with-cockroachdb-activerecord.html) |
-| Python | [Django](https://pypi.org/project/Django/)
(including [GeoDjango](https://docs.djangoproject.com/en/3.1/ref/contrib/gis/))
[peewee](https://github.com/coleifer/peewee/)
[SQLAlchemy](https://www.sqlalchemy.org/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/django.go ||var djangoSupportedTag = "cockroach-||"\nvar %}
(use latest version of CockroachDB adapter)
3.13.3
0.7.13
1.4.17
(use latest version of CockroachDB adapter) | Full
Full
Full
Full | [`django-cockroachdb`](https://pypi.org/project/django-cockroachdb/)
N/A
N/A
[`sqlalchemy-cockroachdb`](https://pypi.org/project/sqlalchemy-cockroachdb)
(includes client-side transaction retry handling) | [Build a Python App with CockroachDB (Django)](build-a-python-app-with-cockroachdb-django.html)
N/A (See [peewee docs](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#cockroach-database).)
[Build a Python App with CockroachDB (SQLAlchemy)](build-a-python-app-with-cockroachdb-sqlalchemy.html) |
+| Go | [GORM](https://github.com/jinzhu/gorm/releases)
[go-pg](https://github.com/go-pg/pg)
[upper/db](https://github.com/upper/db) | v1.24.1
v10.9.0
v4 | Full
Full
Full | [`crdbgorm`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbgorm)
(includes client-side transaction retry handling)
N/A
N/A | [Build a Go App with CockroachDB (GORM)](build-a-go-app-with-cockroachdb-gorm.html)
N/A
[Build a Go App with CockroachDB (upper/db)](build-a-go-app-with-cockroachdb-upperdb.html) |
+| Java | [Hibernate](https://hibernate.org/orm/)
(including [Hibernate Spatial](https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#spatial))
[jOOQ](https://www.jooq.org/)
[MyBatis](https://mybatis.org/mybatis-3/) | 6.6.20 (must be at least 5.4.19)
3.13.2 (must be at least 3.13.0)
3.5.5| Full
Full
Full | N/A
N/A
N/A | [Build a Java App with CockroachDB (Hibernate)](build-a-java-app-with-cockroachdb-hibernate.html)
[Build a Java App with CockroachDB (jOOQ)](build-a-java-app-with-cockroachdb-jooq.html)
[Build a Spring App with CockroachDB (MyBatis)]({% link {{ page.version.version }}/build-a-spring-app-with-cockroachdb-mybatis.md %}) |
+| JavaScript/TypeScript | [Sequelize](https://www.npmjs.com/package/sequelize)
[Knex.js](https://knexjs.org/)
[Prisma](https://prisma.io)
[TypeORM](https://www.npmjs.com/package/typeorm) | v6.0.5
(use latest version of CockroachDB adapter)
2.5.1
3.14.0
0.3.17 {% comment %}remove-unsafe-crdb-setting{% endcomment %} | Full
Full
Full
Full | [`sequelize-cockroachdb`](https://www.npmjs.com/package/sequelize-cockroachdb)
N/A
N/A
N/A | [Build a Node.js App with CockroachDB (Sequelize)](build-a-nodejs-app-with-cockroachdb-sequelize.html)
[Build a Node.js App with CockroachDB (Knex.js)](build-a-nodejs-app-with-cockroachdb-knexjs.html)
[Build a Node.js App with CockroachDB (Prisma)](build-a-nodejs-app-with-cockroachdb-prisma.html)
[Build a TypeScript App with CockroachDB (TypeORM)](build-a-typescript-app-with-cockroachdb.html) |
+| Ruby | [ActiveRecord](https://rubygems.org/gems/activerecord)
[RGeo/RGeo-ActiveRecord](https://github.com/cockroachdb/activerecord-cockroachdb-adapter#working-with-spatial-data) | 8.1.0
(use latest version of CockroachDB adapter) | Full | [`activerecord-cockroachdb-adapter`](https://rubygems.org/gems/activerecord-cockroachdb-adapter)
(includes client-side transaction retry handling) | [Build a Ruby App with CockroachDB (ActiveRecord)](build-a-ruby-app-with-cockroachdb-activerecord.html) |
+| Python | [Django](https://pypi.org/project/Django/)
(including [GeoDjango](https://docs.djangoproject.com/en/3.1/ref/contrib/gis/))
[peewee](https://github.com/coleifer/peewee/)
[SQLAlchemy](https://www.sqlalchemy.org/) | 4.1.x
(use latest version of CockroachDB adapter)
3.13.3
0.7.13
1.4.17
(use latest version of CockroachDB adapter) | Full
Full
Full
Full | [`django-cockroachdb`](https://pypi.org/project/django-cockroachdb/)
N/A
N/A
[`sqlalchemy-cockroachdb`](https://pypi.org/project/sqlalchemy-cockroachdb)
(includes client-side transaction retry handling) | [Build a Python App with CockroachDB (Django)](build-a-python-app-with-cockroachdb-django.html)
N/A (See [peewee docs](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#cockroach-database).)
[Build a Python App with CockroachDB (SQLAlchemy)](build-a-python-app-with-cockroachdb-sqlalchemy.html) |
## Application frameworks
diff --git a/src/current/_includes/v25.1/orchestration/start-cockroachdb-insecure.md b/src/current/_includes/v25.1/orchestration/start-cockroachdb-insecure.md
index 3406d48edbb..c9af5cc0267 100644
--- a/src/current/_includes/v25.1/orchestration/start-cockroachdb-insecure.md
+++ b/src/current/_includes/v25.1/orchestration/start-cockroachdb-insecure.md
@@ -1,10 +1,10 @@
-1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it.
+1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it.
- Download [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml):
+ Download [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml
~~~
{{site.data.alerts.callout_info}}
@@ -27,11 +27,11 @@
Alternatively, if you'd rather start with a configuration file that has been customized for performance:
- 1. Download our [performance version of `cockroachdb-statefulset-insecure.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml):
+ 1. Download our [performance version of `cockroachdb-statefulset-insecure.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml
~~~
1. Modify the file wherever there is a `TODO` comment.
@@ -72,12 +72,12 @@
pvc-5315efda-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-2 27s
~~~
-1. Use our [`cluster-init.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
+1. Use our [`cluster-init.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml
~~~
~~~
diff --git a/src/current/_includes/v25.1/orchestration/start-cockroachdb-local-insecure.md b/src/current/_includes/v25.1/orchestration/start-cockroachdb-local-insecure.md
index 552cb3cd25f..196a53bce4c 100644
--- a/src/current/_includes/v25.1/orchestration/start-cockroachdb-local-insecure.md
+++ b/src/current/_includes/v25.1/orchestration/start-cockroachdb-local-insecure.md
@@ -1,8 +1,8 @@
-1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it:
+1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it:
{% include_cached copy-clipboard.html %}
~~~ shell
- $ kubectl create -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml
+ $ kubectl create -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml
~~~
~~~
@@ -41,12 +41,12 @@
pvc-5315efda-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-2 27s
~~~
-1. Use our [`cluster-init.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
+1. Use our [`cluster-init.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml
~~~
~~~
diff --git a/src/current/_includes/v25.1/orchestration/start-cockroachdb-secure.md b/src/current/_includes/v25.1/orchestration/start-cockroachdb-secure.md
index 972cabc2d8e..a7299e1aa25 100644
--- a/src/current/_includes/v25.1/orchestration/start-cockroachdb-secure.md
+++ b/src/current/_includes/v25.1/orchestration/start-cockroachdb-secure.md
@@ -1,10 +1,10 @@
### Configure the cluster
-1. Download and modify our [StatefulSet configuration](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml):
+1. Download and modify our [StatefulSet configuration](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml
~~~
1. Update `secretName` with the name of the corresponding node secret.
diff --git a/src/current/_includes/v25.1/orchestration/test-cluster-secure.md b/src/current/_includes/v25.1/orchestration/test-cluster-secure.md
index f255d8d62fc..c146d634626 100644
--- a/src/current/_includes/v25.1/orchestration/test-cluster-secure.md
+++ b/src/current/_includes/v25.1/orchestration/test-cluster-secure.md
@@ -42,7 +42,7 @@ $ kubectl create \
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
--f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/client.yaml
+-f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/client.yaml
~~~
~~~
diff --git a/src/current/_includes/v25.2/misc/tooling.md b/src/current/_includes/v25.2/misc/tooling.md
index dcd24363435..01dc63c840f 100644
--- a/src/current/_includes/v25.2/misc/tooling.md
+++ b/src/current/_includes/v25.2/misc/tooling.md
@@ -23,7 +23,7 @@ Customers should contact their account team before moving production workloads t
Unless explicitly stated, support for a [driver](#drivers) or [data access framework](#data-access-frameworks-e-g-orms) does not include [automatic, client-side transaction retry handling]({% link {{ page.version.version }}/transaction-retry-error-reference.md %}#client-side-retry-handling). For client-side transaction retry handling samples, see [Example Apps]({% link {{ page.version.version }}/example-apps.md %}).
{{site.data.alerts.end}}
-If you encounter problems using CockroachDB with any of the tools listed on this page, please [open an issue](https://github.com/cockroachdb/cockroach/issues/new) with details to help us make progress toward better support.
+If you encounter problems using CockroachDB with any of the tools listed on this page, please open an issue with details to help us make progress toward better support.
For a list of tools supported by the CockroachDB community, see [Third-Party Tools Supported by the Community]({% link {{ page.version.version }}/community-tooling.md %}).
@@ -32,23 +32,23 @@ For a list of tools supported by the CockroachDB community, see [Third-Party Too
| Language | Driver | Latest tested version | Support level | CockroachDB adapter | Tutorial |
|----------+--------+-----------------------+---------------------+---------------------+----------|
| C | [libpq](http://www.postgresql.org/docs/13/static/libpq.html)| PostgreSQL 13 | Partial | N/A | N/A |
-| C# (.NET) | [Npgsql](https://www.nuget.org/packages/Npgsql/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/npgsql.go ||var npgsqlSupportedTag = "v||"\n\n %} | Full | N/A | [Build a C# App with CockroachDB (Npgsql)](build-a-csharp-app-with-cockroachdb.html) |
-| Go | [pgx](https://github.com/jackc/pgx/releases)
[pq](https://github.com/lib/pq) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/pgx.go ||var supportedPGXTag = "||"\n\n %}
(use latest version of CockroachDB adapter)
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/libpq.go ||var libPQSupportedTag = "||"\n\n %} | Full
Full | [`crdbpgx`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbpgx)
(includes client-side transaction retry handling)
N/A | [Build a Go App with CockroachDB (pgx)](build-a-go-app-with-cockroachdb.html)
[Build a Go App with CockroachDB (pq)](build-a-go-app-with-cockroachdb-pq.html) |
-| Java | [JDBC](https://jdbc.postgresql.org/download/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/pgjdbc.go ||var supportedPGJDBCTag = "||"\n\n %} | Full | N/A | [Build a Java App with CockroachDB (JDBC)](build-a-java-app-with-cockroachdb.html) |
+| C# (.NET) | [Npgsql](https://www.nuget.org/packages/Npgsql/) | 7.0.2 | Full | N/A | [Build a C# App with CockroachDB (Npgsql)](build-a-csharp-app-with-cockroachdb.html) |
+| Go | [pgx](https://github.com/jackc/pgx/releases)
[pq](https://github.com/lib/pq) | v5.3.1
(use latest version of CockroachDB adapter)
v1.10.5 | Full
Full | [`crdbpgx`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbpgx)
(includes client-side transaction retry handling)
N/A | [Build a Go App with CockroachDB (pgx)](build-a-go-app-with-cockroachdb.html)
[Build a Go App with CockroachDB (pq)](build-a-go-app-with-cockroachdb-pq.html) |
+| Java | [JDBC](https://jdbc.postgresql.org/download/) | REL42.7.3 | Full | N/A | [Build a Java App with CockroachDB (JDBC)](build-a-java-app-with-cockroachdb.html) |
| JavaScript | [pg](https://www.npmjs.com/package/pg) | 8.2.1 | Full | N/A | [Build a Node.js App with CockroachDB (pg)](build-a-nodejs-app-with-cockroachdb.html) |
-| Python | [psycopg3](https://www.psycopg.org/psycopg3/docs/)
[psycopg2](https://www.psycopg.org/docs/install.html)
[asyncpg](https://magicstack.github.io/asyncpg/current/index.html) | 3.0.16
2.8.6
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/asyncpg.go || var asyncpgSupportedTag = "||"\n\n %} | Full
Full
Partial | N/A
N/A
N/A | [Build a Python App with CockroachDB (psycopg3)](build-a-python-app-with-cockroachdb-psycopg3.html)
[Build a Python App with CockroachDB (psycopg2)](build-a-python-app-with-cockroachdb.html)
[Build a Python App with CockroachDB (asyncpg)](build-a-python-app-with-cockroachdb-asyncpg.html) |
-| Ruby | [pg](https://rubygems.org/gems/pg) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/ruby_pg.go ||var rubyPGVersion = "||"\n\n %} | Full | N/A | [Build a Ruby App with CockroachDB (pg)](build-a-ruby-app-with-cockroachdb.html) |
+| Python | [psycopg3](https://www.psycopg.org/psycopg3/docs/)
[psycopg2](https://www.psycopg.org/docs/install.html)
[asyncpg](https://magicstack.github.io/asyncpg/current/index.html) | 3.0.16
2.8.6
v0.24.0 | Full
Full
Partial | N/A
N/A
N/A | [Build a Python App with CockroachDB (psycopg3)](build-a-python-app-with-cockroachdb-psycopg3.html)
[Build a Python App with CockroachDB (psycopg2)](build-a-python-app-with-cockroachdb.html)
[Build a Python App with CockroachDB (asyncpg)](build-a-python-app-with-cockroachdb-asyncpg.html) |
+| Ruby | [pg](https://rubygems.org/gems/pg) | v1.4.6 | Full | N/A | [Build a Ruby App with CockroachDB (pg)](build-a-ruby-app-with-cockroachdb.html) |
| Rust | [rust-postgres](https://github.com/sfackler/rust-postgres) | 0.19.2 | Partial | N/A | [Build a Rust App with CockroachDB]({% link {{ page.version.version }}/build-a-rust-app-with-cockroachdb.md %}) |
## Data access frameworks (e.g., ORMs)
| Language | Framework | Latest tested version | Support level | CockroachDB adapter | Tutorial |
|----------+-----------+-----------------------+---------------+---------------------+----------|
-| Go | [GORM](https://github.com/jinzhu/gorm/releases)
[go-pg](https://github.com/go-pg/pg)
[upper/db](https://github.com/upper/db) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/gorm.go ||var gormSupportedTag = "||"\n\n %}
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/gopg.go ||var gopgSupportedTag = "||"\n\n %}
v4 | Full
Full
Full | [`crdbgorm`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbgorm)
(includes client-side transaction retry handling)
N/A
N/A | [Build a Go App with CockroachDB (GORM)](build-a-go-app-with-cockroachdb-gorm.html)
N/A
[Build a Go App with CockroachDB (upper/db)](build-a-go-app-with-cockroachdb-upperdb.html) |
-| Java | [Hibernate](https://hibernate.org/orm/)
(including [Hibernate Spatial](https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#spatial))
[jOOQ](https://www.jooq.org/)
[MyBatis](https://mybatis.org/mybatis-3/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/hibernate.go ||var supportedHibernateTag = "||"\n\n %} (must be at least 5.4.19)
3.13.2 (must be at least 3.13.0)
3.5.5| Full
Full
Full | N/A
N/A
N/A | [Build a Java App with CockroachDB (Hibernate)](build-a-java-app-with-cockroachdb-hibernate.html)
[Build a Java App with CockroachDB (jOOQ)](build-a-java-app-with-cockroachdb-jooq.html)
[Build a Spring App with CockroachDB (MyBatis)]({% link {{ page.version.version }}/build-a-spring-app-with-cockroachdb-mybatis.md %}) |
-| JavaScript/TypeScript | [Sequelize](https://www.npmjs.com/package/sequelize)
[Knex.js](https://knexjs.org/)
[Prisma](https://prisma.io)
[TypeORM](https://www.npmjs.com/package/typeorm) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/sequelize.go ||var supportedSequelizeCockroachDBRelease = "||"\n\n %}
(use latest version of CockroachDB adapter)
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/knex.go ||const supportedKnexTag = "||"\n\n %}
3.14.0
0.3.17 {% comment %}{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/typeorm.go ||const supportedTypeORMRelease = "||"\n %}{% endcomment %} | Full
Full
Full
Full | [`sequelize-cockroachdb`](https://www.npmjs.com/package/sequelize-cockroachdb)
N/A
N/A
N/A | [Build a Node.js App with CockroachDB (Sequelize)](build-a-nodejs-app-with-cockroachdb-sequelize.html)
[Build a Node.js App with CockroachDB (Knex.js)](build-a-nodejs-app-with-cockroachdb-knexjs.html)
[Build a Node.js App with CockroachDB (Prisma)](build-a-nodejs-app-with-cockroachdb-prisma.html)
[Build a TypeScript App with CockroachDB (TypeORM)](build-a-typescript-app-with-cockroachdb.html) |
-| Ruby | [ActiveRecord](https://rubygems.org/gems/activerecord)
[RGeo/RGeo-ActiveRecord](https://github.com/cockroachdb/activerecord-cockroachdb-adapter#working-with-spatial-data) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/activerecord.go ||var supportedRailsVersion = "||"\nvar %}
(use latest version of CockroachDB adapter) | Full | [`activerecord-cockroachdb-adapter`](https://rubygems.org/gems/activerecord-cockroachdb-adapter)
(includes client-side transaction retry handling) | [Build a Ruby App with CockroachDB (ActiveRecord)](build-a-ruby-app-with-cockroachdb-activerecord.html) |
-| Python | [Django](https://pypi.org/project/Django/)
(including [GeoDjango](https://docs.djangoproject.com/en/3.1/ref/contrib/gis/))
[peewee](https://github.com/coleifer/peewee/)
[SQLAlchemy](https://www.sqlalchemy.org/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/django.go ||var djangoSupportedTag = "cockroach-||"\nvar %}
(use latest version of CockroachDB adapter)
3.13.3
0.7.13
1.4.17
(use latest version of CockroachDB adapter) | Full
Full
Full
Full | [`django-cockroachdb`](https://pypi.org/project/django-cockroachdb/)
N/A
N/A
[`sqlalchemy-cockroachdb`](https://pypi.org/project/sqlalchemy-cockroachdb)
(includes client-side transaction retry handling) | [Build a Python App with CockroachDB (Django)](build-a-python-app-with-cockroachdb-django.html)
N/A (See [peewee docs](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#cockroach-database).)
[Build a Python App with CockroachDB (SQLAlchemy)](build-a-python-app-with-cockroachdb-sqlalchemy.html) |
+| Go | [GORM](https://github.com/jinzhu/gorm/releases)
[go-pg](https://github.com/go-pg/pg)
[upper/db](https://github.com/upper/db) | v1.24.1
v10.9.0
v4 | Full
Full
Full | [`crdbgorm`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbgorm)
(includes client-side transaction retry handling)
N/A
N/A | [Build a Go App with CockroachDB (GORM)](build-a-go-app-with-cockroachdb-gorm.html)
N/A
[Build a Go App with CockroachDB (upper/db)](build-a-go-app-with-cockroachdb-upperdb.html) |
+| Java | [Hibernate](https://hibernate.org/orm/)
(including [Hibernate Spatial](https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#spatial))
[jOOQ](https://www.jooq.org/)
[MyBatis](https://mybatis.org/mybatis-3/) | 6.6.20 (must be at least 5.4.19)
3.13.2 (must be at least 3.13.0)
3.5.5| Full
Full
Full | N/A
N/A
N/A | [Build a Java App with CockroachDB (Hibernate)](build-a-java-app-with-cockroachdb-hibernate.html)
[Build a Java App with CockroachDB (jOOQ)](build-a-java-app-with-cockroachdb-jooq.html)
[Build a Spring App with CockroachDB (MyBatis)]({% link {{ page.version.version }}/build-a-spring-app-with-cockroachdb-mybatis.md %}) |
+| JavaScript/TypeScript | [Sequelize](https://www.npmjs.com/package/sequelize)
[Knex.js](https://knexjs.org/)
[Prisma](https://prisma.io)
[TypeORM](https://www.npmjs.com/package/typeorm) | v6.0.5
(use latest version of CockroachDB adapter)
2.5.1
3.14.0
0.3.17 {% comment %}remove-unsafe-crdb-setting{% endcomment %} | Full
Full
Full
Full | [`sequelize-cockroachdb`](https://www.npmjs.com/package/sequelize-cockroachdb)
N/A
N/A
N/A | [Build a Node.js App with CockroachDB (Sequelize)](build-a-nodejs-app-with-cockroachdb-sequelize.html)
[Build a Node.js App with CockroachDB (Knex.js)](build-a-nodejs-app-with-cockroachdb-knexjs.html)
[Build a Node.js App with CockroachDB (Prisma)](build-a-nodejs-app-with-cockroachdb-prisma.html)
[Build a TypeScript App with CockroachDB (TypeORM)](build-a-typescript-app-with-cockroachdb.html) |
+| Ruby | [ActiveRecord](https://rubygems.org/gems/activerecord)
[RGeo/RGeo-ActiveRecord](https://github.com/cockroachdb/activerecord-cockroachdb-adapter#working-with-spatial-data) | 8.1.0
(use latest version of CockroachDB adapter) | Full | [`activerecord-cockroachdb-adapter`](https://rubygems.org/gems/activerecord-cockroachdb-adapter)
(includes client-side transaction retry handling) | [Build a Ruby App with CockroachDB (ActiveRecord)](build-a-ruby-app-with-cockroachdb-activerecord.html) |
+| Python | [Django](https://pypi.org/project/Django/)
(including [GeoDjango](https://docs.djangoproject.com/en/3.1/ref/contrib/gis/))
[peewee](https://github.com/coleifer/peewee/)
[SQLAlchemy](https://www.sqlalchemy.org/) | 4.1.x
(use latest version of CockroachDB adapter)
3.13.3
0.7.13
1.4.17
(use latest version of CockroachDB adapter) | Full
Full
Full
Full | [`django-cockroachdb`](https://pypi.org/project/django-cockroachdb/)
N/A
N/A
[`sqlalchemy-cockroachdb`](https://pypi.org/project/sqlalchemy-cockroachdb)
(includes client-side transaction retry handling) | [Build a Python App with CockroachDB (Django)](build-a-python-app-with-cockroachdb-django.html)
N/A (See [peewee docs](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#cockroach-database).)
[Build a Python App with CockroachDB (SQLAlchemy)](build-a-python-app-with-cockroachdb-sqlalchemy.html) |
## Application frameworks
diff --git a/src/current/_includes/v25.2/orchestration/start-cockroachdb-insecure.md b/src/current/_includes/v25.2/orchestration/start-cockroachdb-insecure.md
index 3406d48edbb..c9af5cc0267 100644
--- a/src/current/_includes/v25.2/orchestration/start-cockroachdb-insecure.md
+++ b/src/current/_includes/v25.2/orchestration/start-cockroachdb-insecure.md
@@ -1,10 +1,10 @@
-1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it.
+1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it.
- Download [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml):
+ Download [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml
~~~
{{site.data.alerts.callout_info}}
@@ -27,11 +27,11 @@
Alternatively, if you'd rather start with a configuration file that has been customized for performance:
- 1. Download our [performance version of `cockroachdb-statefulset-insecure.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml):
+ 1. Download our [performance version of `cockroachdb-statefulset-insecure.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml
~~~
1. Modify the file wherever there is a `TODO` comment.
@@ -72,12 +72,12 @@
pvc-5315efda-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-2 27s
~~~
-1. Use our [`cluster-init.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
+1. Use our [`cluster-init.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml
~~~
~~~
diff --git a/src/current/_includes/v25.2/orchestration/start-cockroachdb-local-insecure.md b/src/current/_includes/v25.2/orchestration/start-cockroachdb-local-insecure.md
index 552cb3cd25f..196a53bce4c 100644
--- a/src/current/_includes/v25.2/orchestration/start-cockroachdb-local-insecure.md
+++ b/src/current/_includes/v25.2/orchestration/start-cockroachdb-local-insecure.md
@@ -1,8 +1,8 @@
-1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it:
+1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it:
{% include_cached copy-clipboard.html %}
~~~ shell
- $ kubectl create -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml
+ $ kubectl create -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml
~~~
~~~
@@ -41,12 +41,12 @@
pvc-5315efda-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-2 27s
~~~
-1. Use our [`cluster-init.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
+1. Use our [`cluster-init.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml
~~~
~~~
diff --git a/src/current/_includes/v25.2/orchestration/start-cockroachdb-secure.md b/src/current/_includes/v25.2/orchestration/start-cockroachdb-secure.md
index 972cabc2d8e..a7299e1aa25 100644
--- a/src/current/_includes/v25.2/orchestration/start-cockroachdb-secure.md
+++ b/src/current/_includes/v25.2/orchestration/start-cockroachdb-secure.md
@@ -1,10 +1,10 @@
### Configure the cluster
-1. Download and modify our [StatefulSet configuration](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml):
+1. Download and modify our [StatefulSet configuration](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml
~~~
1. Update `secretName` with the name of the corresponding node secret.
diff --git a/src/current/_includes/v25.2/orchestration/test-cluster-secure.md b/src/current/_includes/v25.2/orchestration/test-cluster-secure.md
index f255d8d62fc..c146d634626 100644
--- a/src/current/_includes/v25.2/orchestration/test-cluster-secure.md
+++ b/src/current/_includes/v25.2/orchestration/test-cluster-secure.md
@@ -42,7 +42,7 @@ $ kubectl create \
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
--f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/client.yaml
+-f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/client.yaml
~~~
~~~
diff --git a/src/current/_includes/v25.3/misc/tooling.md b/src/current/_includes/v25.3/misc/tooling.md
index dcd24363435..01dc63c840f 100644
--- a/src/current/_includes/v25.3/misc/tooling.md
+++ b/src/current/_includes/v25.3/misc/tooling.md
@@ -23,7 +23,7 @@ Customers should contact their account team before moving production workloads t
Unless explicitly stated, support for a [driver](#drivers) or [data access framework](#data-access-frameworks-e-g-orms) does not include [automatic, client-side transaction retry handling]({% link {{ page.version.version }}/transaction-retry-error-reference.md %}#client-side-retry-handling). For client-side transaction retry handling samples, see [Example Apps]({% link {{ page.version.version }}/example-apps.md %}).
{{site.data.alerts.end}}
-If you encounter problems using CockroachDB with any of the tools listed on this page, please [open an issue](https://github.com/cockroachdb/cockroach/issues/new) with details to help us make progress toward better support.
+If you encounter problems using CockroachDB with any of the tools listed on this page, please open an issue with details to help us make progress toward better support.
For a list of tools supported by the CockroachDB community, see [Third-Party Tools Supported by the Community]({% link {{ page.version.version }}/community-tooling.md %}).
@@ -32,23 +32,23 @@ For a list of tools supported by the CockroachDB community, see [Third-Party Too
| Language | Driver | Latest tested version | Support level | CockroachDB adapter | Tutorial |
|----------+--------+-----------------------+---------------------+---------------------+----------|
| C | [libpq](http://www.postgresql.org/docs/13/static/libpq.html)| PostgreSQL 13 | Partial | N/A | N/A |
-| C# (.NET) | [Npgsql](https://www.nuget.org/packages/Npgsql/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/npgsql.go ||var npgsqlSupportedTag = "v||"\n\n %} | Full | N/A | [Build a C# App with CockroachDB (Npgsql)](build-a-csharp-app-with-cockroachdb.html) |
-| Go | [pgx](https://github.com/jackc/pgx/releases)
[pq](https://github.com/lib/pq) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/pgx.go ||var supportedPGXTag = "||"\n\n %}
(use latest version of CockroachDB adapter)
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/libpq.go ||var libPQSupportedTag = "||"\n\n %} | Full
Full | [`crdbpgx`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbpgx)
(includes client-side transaction retry handling)
N/A | [Build a Go App with CockroachDB (pgx)](build-a-go-app-with-cockroachdb.html)
[Build a Go App with CockroachDB (pq)](build-a-go-app-with-cockroachdb-pq.html) |
-| Java | [JDBC](https://jdbc.postgresql.org/download/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/pgjdbc.go ||var supportedPGJDBCTag = "||"\n\n %} | Full | N/A | [Build a Java App with CockroachDB (JDBC)](build-a-java-app-with-cockroachdb.html) |
+| C# (.NET) | [Npgsql](https://www.nuget.org/packages/Npgsql/) | 7.0.2 | Full | N/A | [Build a C# App with CockroachDB (Npgsql)](build-a-csharp-app-with-cockroachdb.html) |
+| Go | [pgx](https://github.com/jackc/pgx/releases)
[pq](https://github.com/lib/pq) | v5.3.1
(use latest version of CockroachDB adapter)
v1.10.5 | Full
Full | [`crdbpgx`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbpgx)
(includes client-side transaction retry handling)
N/A | [Build a Go App with CockroachDB (pgx)](build-a-go-app-with-cockroachdb.html)
[Build a Go App with CockroachDB (pq)](build-a-go-app-with-cockroachdb-pq.html) |
+| Java | [JDBC](https://jdbc.postgresql.org/download/) | REL42.7.3 | Full | N/A | [Build a Java App with CockroachDB (JDBC)](build-a-java-app-with-cockroachdb.html) |
| JavaScript | [pg](https://www.npmjs.com/package/pg) | 8.2.1 | Full | N/A | [Build a Node.js App with CockroachDB (pg)](build-a-nodejs-app-with-cockroachdb.html) |
-| Python | [psycopg3](https://www.psycopg.org/psycopg3/docs/)
[psycopg2](https://www.psycopg.org/docs/install.html)
[asyncpg](https://magicstack.github.io/asyncpg/current/index.html) | 3.0.16
2.8.6
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/asyncpg.go || var asyncpgSupportedTag = "||"\n\n %} | Full
Full
Partial | N/A
N/A
N/A | [Build a Python App with CockroachDB (psycopg3)](build-a-python-app-with-cockroachdb-psycopg3.html)
[Build a Python App with CockroachDB (psycopg2)](build-a-python-app-with-cockroachdb.html)
[Build a Python App with CockroachDB (asyncpg)](build-a-python-app-with-cockroachdb-asyncpg.html) |
-| Ruby | [pg](https://rubygems.org/gems/pg) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/ruby_pg.go ||var rubyPGVersion = "||"\n\n %} | Full | N/A | [Build a Ruby App with CockroachDB (pg)](build-a-ruby-app-with-cockroachdb.html) |
+| Python | [psycopg3](https://www.psycopg.org/psycopg3/docs/)
[psycopg2](https://www.psycopg.org/docs/install.html)
[asyncpg](https://magicstack.github.io/asyncpg/current/index.html) | 3.0.16
2.8.6
v0.24.0 | Full
Full
Partial | N/A
N/A
N/A | [Build a Python App with CockroachDB (psycopg3)](build-a-python-app-with-cockroachdb-psycopg3.html)
[Build a Python App with CockroachDB (psycopg2)](build-a-python-app-with-cockroachdb.html)
[Build a Python App with CockroachDB (asyncpg)](build-a-python-app-with-cockroachdb-asyncpg.html) |
+| Ruby | [pg](https://rubygems.org/gems/pg) | v1.4.6 | Full | N/A | [Build a Ruby App with CockroachDB (pg)](build-a-ruby-app-with-cockroachdb.html) |
| Rust | [rust-postgres](https://github.com/sfackler/rust-postgres) | 0.19.2 | Partial | N/A | [Build a Rust App with CockroachDB]({% link {{ page.version.version }}/build-a-rust-app-with-cockroachdb.md %}) |
## Data access frameworks (e.g., ORMs)
| Language | Framework | Latest tested version | Support level | CockroachDB adapter | Tutorial |
|----------+-----------+-----------------------+---------------+---------------------+----------|
-| Go | [GORM](https://github.com/jinzhu/gorm/releases)
[go-pg](https://github.com/go-pg/pg)
[upper/db](https://github.com/upper/db) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/gorm.go ||var gormSupportedTag = "||"\n\n %}
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/gopg.go ||var gopgSupportedTag = "||"\n\n %}
v4 | Full
Full
Full | [`crdbgorm`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbgorm)
(includes client-side transaction retry handling)
N/A
N/A | [Build a Go App with CockroachDB (GORM)](build-a-go-app-with-cockroachdb-gorm.html)
N/A
[Build a Go App with CockroachDB (upper/db)](build-a-go-app-with-cockroachdb-upperdb.html) |
-| Java | [Hibernate](https://hibernate.org/orm/)
(including [Hibernate Spatial](https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#spatial))
[jOOQ](https://www.jooq.org/)
[MyBatis](https://mybatis.org/mybatis-3/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/hibernate.go ||var supportedHibernateTag = "||"\n\n %} (must be at least 5.4.19)
3.13.2 (must be at least 3.13.0)
3.5.5| Full
Full
Full | N/A
N/A
N/A | [Build a Java App with CockroachDB (Hibernate)](build-a-java-app-with-cockroachdb-hibernate.html)
[Build a Java App with CockroachDB (jOOQ)](build-a-java-app-with-cockroachdb-jooq.html)
[Build a Spring App with CockroachDB (MyBatis)]({% link {{ page.version.version }}/build-a-spring-app-with-cockroachdb-mybatis.md %}) |
-| JavaScript/TypeScript | [Sequelize](https://www.npmjs.com/package/sequelize)
[Knex.js](https://knexjs.org/)
[Prisma](https://prisma.io)
[TypeORM](https://www.npmjs.com/package/typeorm) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/sequelize.go ||var supportedSequelizeCockroachDBRelease = "||"\n\n %}
(use latest version of CockroachDB adapter)
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/knex.go ||const supportedKnexTag = "||"\n\n %}
3.14.0
0.3.17 {% comment %}{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/typeorm.go ||const supportedTypeORMRelease = "||"\n %}{% endcomment %} | Full
Full
Full
Full | [`sequelize-cockroachdb`](https://www.npmjs.com/package/sequelize-cockroachdb)
N/A
N/A
N/A | [Build a Node.js App with CockroachDB (Sequelize)](build-a-nodejs-app-with-cockroachdb-sequelize.html)
[Build a Node.js App with CockroachDB (Knex.js)](build-a-nodejs-app-with-cockroachdb-knexjs.html)
[Build a Node.js App with CockroachDB (Prisma)](build-a-nodejs-app-with-cockroachdb-prisma.html)
[Build a TypeScript App with CockroachDB (TypeORM)](build-a-typescript-app-with-cockroachdb.html) |
-| Ruby | [ActiveRecord](https://rubygems.org/gems/activerecord)
[RGeo/RGeo-ActiveRecord](https://github.com/cockroachdb/activerecord-cockroachdb-adapter#working-with-spatial-data) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/activerecord.go ||var supportedRailsVersion = "||"\nvar %}
(use latest version of CockroachDB adapter) | Full | [`activerecord-cockroachdb-adapter`](https://rubygems.org/gems/activerecord-cockroachdb-adapter)
(includes client-side transaction retry handling) | [Build a Ruby App with CockroachDB (ActiveRecord)](build-a-ruby-app-with-cockroachdb-activerecord.html) |
-| Python | [Django](https://pypi.org/project/Django/)
(including [GeoDjango](https://docs.djangoproject.com/en/3.1/ref/contrib/gis/))
[peewee](https://github.com/coleifer/peewee/)
[SQLAlchemy](https://www.sqlalchemy.org/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/django.go ||var djangoSupportedTag = "cockroach-||"\nvar %}
(use latest version of CockroachDB adapter)
3.13.3
0.7.13
1.4.17
(use latest version of CockroachDB adapter) | Full
Full
Full
Full | [`django-cockroachdb`](https://pypi.org/project/django-cockroachdb/)
N/A
N/A
[`sqlalchemy-cockroachdb`](https://pypi.org/project/sqlalchemy-cockroachdb)
(includes client-side transaction retry handling) | [Build a Python App with CockroachDB (Django)](build-a-python-app-with-cockroachdb-django.html)
N/A (See [peewee docs](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#cockroach-database).)
[Build a Python App with CockroachDB (SQLAlchemy)](build-a-python-app-with-cockroachdb-sqlalchemy.html) |
+| Go | [GORM](https://github.com/jinzhu/gorm/releases)
[go-pg](https://github.com/go-pg/pg)
[upper/db](https://github.com/upper/db) | v1.24.1
v10.9.0
v4 | Full
Full
Full | [`crdbgorm`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbgorm)
(includes client-side transaction retry handling)
N/A
N/A | [Build a Go App with CockroachDB (GORM)](build-a-go-app-with-cockroachdb-gorm.html)
N/A
[Build a Go App with CockroachDB (upper/db)](build-a-go-app-with-cockroachdb-upperdb.html) |
+| Java | [Hibernate](https://hibernate.org/orm/)
(including [Hibernate Spatial](https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#spatial))
[jOOQ](https://www.jooq.org/)
[MyBatis](https://mybatis.org/mybatis-3/) | 6.6.20 (must be at least 5.4.19)
3.13.2 (must be at least 3.13.0)
3.5.5| Full
Full
Full | N/A
N/A
N/A | [Build a Java App with CockroachDB (Hibernate)](build-a-java-app-with-cockroachdb-hibernate.html)
[Build a Java App with CockroachDB (jOOQ)](build-a-java-app-with-cockroachdb-jooq.html)
[Build a Spring App with CockroachDB (MyBatis)]({% link {{ page.version.version }}/build-a-spring-app-with-cockroachdb-mybatis.md %}) |
+| JavaScript/TypeScript | [Sequelize](https://www.npmjs.com/package/sequelize)
[Knex.js](https://knexjs.org/)
[Prisma](https://prisma.io)
[TypeORM](https://www.npmjs.com/package/typeorm) | v6.0.5
(use latest version of CockroachDB adapter)
2.5.1
3.14.0
0.3.17 {% comment %}remove-unsafe-crdb-setting{% endcomment %} | Full
Full
Full
Full | [`sequelize-cockroachdb`](https://www.npmjs.com/package/sequelize-cockroachdb)
N/A
N/A
N/A | [Build a Node.js App with CockroachDB (Sequelize)](build-a-nodejs-app-with-cockroachdb-sequelize.html)
[Build a Node.js App with CockroachDB (Knex.js)](build-a-nodejs-app-with-cockroachdb-knexjs.html)
[Build a Node.js App with CockroachDB (Prisma)](build-a-nodejs-app-with-cockroachdb-prisma.html)
[Build a TypeScript App with CockroachDB (TypeORM)](build-a-typescript-app-with-cockroachdb.html) |
+| Ruby | [ActiveRecord](https://rubygems.org/gems/activerecord)
[RGeo/RGeo-ActiveRecord](https://github.com/cockroachdb/activerecord-cockroachdb-adapter#working-with-spatial-data) | 8.1.0
(use latest version of CockroachDB adapter) | Full | [`activerecord-cockroachdb-adapter`](https://rubygems.org/gems/activerecord-cockroachdb-adapter)
(includes client-side transaction retry handling) | [Build a Ruby App with CockroachDB (ActiveRecord)](build-a-ruby-app-with-cockroachdb-activerecord.html) |
+| Python | [Django](https://pypi.org/project/Django/)
(including [GeoDjango](https://docs.djangoproject.com/en/3.1/ref/contrib/gis/))
[peewee](https://github.com/coleifer/peewee/)
[SQLAlchemy](https://www.sqlalchemy.org/) | 4.1.x
(use latest version of CockroachDB adapter)
3.13.3
0.7.13
1.4.17
(use latest version of CockroachDB adapter) | Full
Full
Full
Full | [`django-cockroachdb`](https://pypi.org/project/django-cockroachdb/)
N/A
N/A
[`sqlalchemy-cockroachdb`](https://pypi.org/project/sqlalchemy-cockroachdb)
(includes client-side transaction retry handling) | [Build a Python App with CockroachDB (Django)](build-a-python-app-with-cockroachdb-django.html)
N/A (See [peewee docs](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#cockroach-database).)
[Build a Python App with CockroachDB (SQLAlchemy)](build-a-python-app-with-cockroachdb-sqlalchemy.html) |
## Application frameworks
diff --git a/src/current/_includes/v25.3/orchestration/start-cockroachdb-insecure.md b/src/current/_includes/v25.3/orchestration/start-cockroachdb-insecure.md
index 3406d48edbb..c9af5cc0267 100644
--- a/src/current/_includes/v25.3/orchestration/start-cockroachdb-insecure.md
+++ b/src/current/_includes/v25.3/orchestration/start-cockroachdb-insecure.md
@@ -1,10 +1,10 @@
-1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it.
+1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it.
- Download [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml):
+ Download [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml
~~~
{{site.data.alerts.callout_info}}
@@ -27,11 +27,11 @@
Alternatively, if you'd rather start with a configuration file that has been customized for performance:
- 1. Download our [performance version of `cockroachdb-statefulset-insecure.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml):
+ 1. Download our [performance version of `cockroachdb-statefulset-insecure.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml
~~~
1. Modify the file wherever there is a `TODO` comment.
@@ -72,12 +72,12 @@
pvc-5315efda-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-2 27s
~~~
-1. Use our [`cluster-init.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
+1. Use our [`cluster-init.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml
~~~
~~~
diff --git a/src/current/_includes/v25.3/orchestration/start-cockroachdb-local-insecure.md b/src/current/_includes/v25.3/orchestration/start-cockroachdb-local-insecure.md
index 552cb3cd25f..196a53bce4c 100644
--- a/src/current/_includes/v25.3/orchestration/start-cockroachdb-local-insecure.md
+++ b/src/current/_includes/v25.3/orchestration/start-cockroachdb-local-insecure.md
@@ -1,8 +1,8 @@
-1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it:
+1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it:
{% include_cached copy-clipboard.html %}
~~~ shell
- $ kubectl create -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml
+ $ kubectl create -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml
~~~
~~~
@@ -41,12 +41,12 @@
pvc-5315efda-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-2 27s
~~~
-1. Use our [`cluster-init.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
+1. Use our [`cluster-init.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml
~~~
~~~
diff --git a/src/current/_includes/v25.3/orchestration/start-cockroachdb-secure.md b/src/current/_includes/v25.3/orchestration/start-cockroachdb-secure.md
index 972cabc2d8e..a7299e1aa25 100644
--- a/src/current/_includes/v25.3/orchestration/start-cockroachdb-secure.md
+++ b/src/current/_includes/v25.3/orchestration/start-cockroachdb-secure.md
@@ -1,10 +1,10 @@
### Configure the cluster
-1. Download and modify our [StatefulSet configuration](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml):
+1. Download and modify our [StatefulSet configuration](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml
~~~
1. Update `secretName` with the name of the corresponding node secret.
diff --git a/src/current/_includes/v25.3/orchestration/test-cluster-secure.md b/src/current/_includes/v25.3/orchestration/test-cluster-secure.md
index f255d8d62fc..c146d634626 100644
--- a/src/current/_includes/v25.3/orchestration/test-cluster-secure.md
+++ b/src/current/_includes/v25.3/orchestration/test-cluster-secure.md
@@ -42,7 +42,7 @@ $ kubectl create \
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
--f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/client.yaml
+-f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/client.yaml
~~~
~~~
diff --git a/src/current/_includes/v25.4/misc/tooling.md b/src/current/_includes/v25.4/misc/tooling.md
index dcd24363435..01dc63c840f 100644
--- a/src/current/_includes/v25.4/misc/tooling.md
+++ b/src/current/_includes/v25.4/misc/tooling.md
@@ -23,7 +23,7 @@ Customers should contact their account team before moving production workloads t
Unless explicitly stated, support for a [driver](#drivers) or [data access framework](#data-access-frameworks-e-g-orms) does not include [automatic, client-side transaction retry handling]({% link {{ page.version.version }}/transaction-retry-error-reference.md %}#client-side-retry-handling). For client-side transaction retry handling samples, see [Example Apps]({% link {{ page.version.version }}/example-apps.md %}).
{{site.data.alerts.end}}
-If you encounter problems using CockroachDB with any of the tools listed on this page, please [open an issue](https://github.com/cockroachdb/cockroach/issues/new) with details to help us make progress toward better support.
+If you encounter problems using CockroachDB with any of the tools listed on this page, please open an issue with details to help us make progress toward better support.
For a list of tools supported by the CockroachDB community, see [Third-Party Tools Supported by the Community]({% link {{ page.version.version }}/community-tooling.md %}).
@@ -32,23 +32,23 @@ For a list of tools supported by the CockroachDB community, see [Third-Party Too
| Language | Driver | Latest tested version | Support level | CockroachDB adapter | Tutorial |
|----------+--------+-----------------------+---------------------+---------------------+----------|
| C | [libpq](http://www.postgresql.org/docs/13/static/libpq.html)| PostgreSQL 13 | Partial | N/A | N/A |
-| C# (.NET) | [Npgsql](https://www.nuget.org/packages/Npgsql/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/npgsql.go ||var npgsqlSupportedTag = "v||"\n\n %} | Full | N/A | [Build a C# App with CockroachDB (Npgsql)](build-a-csharp-app-with-cockroachdb.html) |
-| Go | [pgx](https://github.com/jackc/pgx/releases)
[pq](https://github.com/lib/pq) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/pgx.go ||var supportedPGXTag = "||"\n\n %}
(use latest version of CockroachDB adapter)
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/libpq.go ||var libPQSupportedTag = "||"\n\n %} | Full
Full | [`crdbpgx`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbpgx)
(includes client-side transaction retry handling)
N/A | [Build a Go App with CockroachDB (pgx)](build-a-go-app-with-cockroachdb.html)
[Build a Go App with CockroachDB (pq)](build-a-go-app-with-cockroachdb-pq.html) |
-| Java | [JDBC](https://jdbc.postgresql.org/download/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/pgjdbc.go ||var supportedPGJDBCTag = "||"\n\n %} | Full | N/A | [Build a Java App with CockroachDB (JDBC)](build-a-java-app-with-cockroachdb.html) |
+| C# (.NET) | [Npgsql](https://www.nuget.org/packages/Npgsql/) | 7.0.2 | Full | N/A | [Build a C# App with CockroachDB (Npgsql)](build-a-csharp-app-with-cockroachdb.html) |
+| Go | [pgx](https://github.com/jackc/pgx/releases)
[pq](https://github.com/lib/pq) | v5.3.1
(use latest version of CockroachDB adapter)
v1.10.5 | Full
Full | [`crdbpgx`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbpgx)
(includes client-side transaction retry handling)
N/A | [Build a Go App with CockroachDB (pgx)](build-a-go-app-with-cockroachdb.html)
[Build a Go App with CockroachDB (pq)](build-a-go-app-with-cockroachdb-pq.html) |
+| Java | [JDBC](https://jdbc.postgresql.org/download/) | REL42.7.3 | Full | N/A | [Build a Java App with CockroachDB (JDBC)](build-a-java-app-with-cockroachdb.html) |
| JavaScript | [pg](https://www.npmjs.com/package/pg) | 8.2.1 | Full | N/A | [Build a Node.js App with CockroachDB (pg)](build-a-nodejs-app-with-cockroachdb.html) |
-| Python | [psycopg3](https://www.psycopg.org/psycopg3/docs/)
[psycopg2](https://www.psycopg.org/docs/install.html)
[asyncpg](https://magicstack.github.io/asyncpg/current/index.html) | 3.0.16
2.8.6
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/asyncpg.go || var asyncpgSupportedTag = "||"\n\n %} | Full
Full
Partial | N/A
N/A
N/A | [Build a Python App with CockroachDB (psycopg3)](build-a-python-app-with-cockroachdb-psycopg3.html)
[Build a Python App with CockroachDB (psycopg2)](build-a-python-app-with-cockroachdb.html)
[Build a Python App with CockroachDB (asyncpg)](build-a-python-app-with-cockroachdb-asyncpg.html) |
-| Ruby | [pg](https://rubygems.org/gems/pg) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/ruby_pg.go ||var rubyPGVersion = "||"\n\n %} | Full | N/A | [Build a Ruby App with CockroachDB (pg)](build-a-ruby-app-with-cockroachdb.html) |
+| Python | [psycopg3](https://www.psycopg.org/psycopg3/docs/)
[psycopg2](https://www.psycopg.org/docs/install.html)
[asyncpg](https://magicstack.github.io/asyncpg/current/index.html) | 3.0.16
2.8.6
v0.24.0 | Full
Full
Partial | N/A
N/A
N/A | [Build a Python App with CockroachDB (psycopg3)](build-a-python-app-with-cockroachdb-psycopg3.html)
[Build a Python App with CockroachDB (psycopg2)](build-a-python-app-with-cockroachdb.html)
[Build a Python App with CockroachDB (asyncpg)](build-a-python-app-with-cockroachdb-asyncpg.html) |
+| Ruby | [pg](https://rubygems.org/gems/pg) | v1.4.6 | Full | N/A | [Build a Ruby App with CockroachDB (pg)](build-a-ruby-app-with-cockroachdb.html) |
| Rust | [rust-postgres](https://github.com/sfackler/rust-postgres) | 0.19.2 | Partial | N/A | [Build a Rust App with CockroachDB]({% link {{ page.version.version }}/build-a-rust-app-with-cockroachdb.md %}) |
## Data access frameworks (e.g., ORMs)
| Language | Framework | Latest tested version | Support level | CockroachDB adapter | Tutorial |
|----------+-----------+-----------------------+---------------+---------------------+----------|
-| Go | [GORM](https://github.com/jinzhu/gorm/releases)
[go-pg](https://github.com/go-pg/pg)
[upper/db](https://github.com/upper/db) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/gorm.go ||var gormSupportedTag = "||"\n\n %}
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/gopg.go ||var gopgSupportedTag = "||"\n\n %}
v4 | Full
Full
Full | [`crdbgorm`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbgorm)
(includes client-side transaction retry handling)
N/A
N/A | [Build a Go App with CockroachDB (GORM)](build-a-go-app-with-cockroachdb-gorm.html)
N/A
[Build a Go App with CockroachDB (upper/db)](build-a-go-app-with-cockroachdb-upperdb.html) |
-| Java | [Hibernate](https://hibernate.org/orm/)
(including [Hibernate Spatial](https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#spatial))
[jOOQ](https://www.jooq.org/)
[MyBatis](https://mybatis.org/mybatis-3/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/hibernate.go ||var supportedHibernateTag = "||"\n\n %} (must be at least 5.4.19)
3.13.2 (must be at least 3.13.0)
3.5.5| Full
Full
Full | N/A
N/A
N/A | [Build a Java App with CockroachDB (Hibernate)](build-a-java-app-with-cockroachdb-hibernate.html)
[Build a Java App with CockroachDB (jOOQ)](build-a-java-app-with-cockroachdb-jooq.html)
[Build a Spring App with CockroachDB (MyBatis)]({% link {{ page.version.version }}/build-a-spring-app-with-cockroachdb-mybatis.md %}) |
-| JavaScript/TypeScript | [Sequelize](https://www.npmjs.com/package/sequelize)
[Knex.js](https://knexjs.org/)
[Prisma](https://prisma.io)
[TypeORM](https://www.npmjs.com/package/typeorm) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/sequelize.go ||var supportedSequelizeCockroachDBRelease = "||"\n\n %}
(use latest version of CockroachDB adapter)
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/knex.go ||const supportedKnexTag = "||"\n\n %}
3.14.0
0.3.17 {% comment %}{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/typeorm.go ||const supportedTypeORMRelease = "||"\n %}{% endcomment %} | Full
Full
Full
Full | [`sequelize-cockroachdb`](https://www.npmjs.com/package/sequelize-cockroachdb)
N/A
N/A
N/A | [Build a Node.js App with CockroachDB (Sequelize)](build-a-nodejs-app-with-cockroachdb-sequelize.html)
[Build a Node.js App with CockroachDB (Knex.js)](build-a-nodejs-app-with-cockroachdb-knexjs.html)
[Build a Node.js App with CockroachDB (Prisma)](build-a-nodejs-app-with-cockroachdb-prisma.html)
[Build a TypeScript App with CockroachDB (TypeORM)](build-a-typescript-app-with-cockroachdb.html) |
-| Ruby | [ActiveRecord](https://rubygems.org/gems/activerecord)
[RGeo/RGeo-ActiveRecord](https://github.com/cockroachdb/activerecord-cockroachdb-adapter#working-with-spatial-data) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/activerecord.go ||var supportedRailsVersion = "||"\nvar %}
(use latest version of CockroachDB adapter) | Full | [`activerecord-cockroachdb-adapter`](https://rubygems.org/gems/activerecord-cockroachdb-adapter)
(includes client-side transaction retry handling) | [Build a Ruby App with CockroachDB (ActiveRecord)](build-a-ruby-app-with-cockroachdb-activerecord.html) |
-| Python | [Django](https://pypi.org/project/Django/)
(including [GeoDjango](https://docs.djangoproject.com/en/3.1/ref/contrib/gis/))
[peewee](https://github.com/coleifer/peewee/)
[SQLAlchemy](https://www.sqlalchemy.org/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/django.go ||var djangoSupportedTag = "cockroach-||"\nvar %}
(use latest version of CockroachDB adapter)
3.13.3
0.7.13
1.4.17
(use latest version of CockroachDB adapter) | Full
Full
Full
Full | [`django-cockroachdb`](https://pypi.org/project/django-cockroachdb/)
N/A
N/A
[`sqlalchemy-cockroachdb`](https://pypi.org/project/sqlalchemy-cockroachdb)
(includes client-side transaction retry handling) | [Build a Python App with CockroachDB (Django)](build-a-python-app-with-cockroachdb-django.html)
N/A (See [peewee docs](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#cockroach-database).)
[Build a Python App with CockroachDB (SQLAlchemy)](build-a-python-app-with-cockroachdb-sqlalchemy.html) |
+| Go | [GORM](https://github.com/jinzhu/gorm/releases)
[go-pg](https://github.com/go-pg/pg)
[upper/db](https://github.com/upper/db) | v1.24.1
v10.9.0
v4 | Full
Full
Full | [`crdbgorm`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbgorm)
(includes client-side transaction retry handling)
N/A
N/A | [Build a Go App with CockroachDB (GORM)](build-a-go-app-with-cockroachdb-gorm.html)
N/A
[Build a Go App with CockroachDB (upper/db)](build-a-go-app-with-cockroachdb-upperdb.html) |
+| Java | [Hibernate](https://hibernate.org/orm/)
(including [Hibernate Spatial](https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#spatial))
[jOOQ](https://www.jooq.org/)
[MyBatis](https://mybatis.org/mybatis-3/) | 6.6.20 (must be at least 5.4.19)
3.13.2 (must be at least 3.13.0)
3.5.5| Full
Full
Full | N/A
N/A
N/A | [Build a Java App with CockroachDB (Hibernate)](build-a-java-app-with-cockroachdb-hibernate.html)
[Build a Java App with CockroachDB (jOOQ)](build-a-java-app-with-cockroachdb-jooq.html)
[Build a Spring App with CockroachDB (MyBatis)]({% link {{ page.version.version }}/build-a-spring-app-with-cockroachdb-mybatis.md %}) |
+| JavaScript/TypeScript | [Sequelize](https://www.npmjs.com/package/sequelize)
[Knex.js](https://knexjs.org/)
[Prisma](https://prisma.io)
[TypeORM](https://www.npmjs.com/package/typeorm) | v6.0.5
(use latest version of CockroachDB adapter)
2.5.1
3.14.0
0.3.17 {% comment %}remove-unsafe-crdb-setting{% endcomment %} | Full
Full
Full
Full | [`sequelize-cockroachdb`](https://www.npmjs.com/package/sequelize-cockroachdb)
N/A
N/A
N/A | [Build a Node.js App with CockroachDB (Sequelize)](build-a-nodejs-app-with-cockroachdb-sequelize.html)
[Build a Node.js App with CockroachDB (Knex.js)](build-a-nodejs-app-with-cockroachdb-knexjs.html)
[Build a Node.js App with CockroachDB (Prisma)](build-a-nodejs-app-with-cockroachdb-prisma.html)
[Build a TypeScript App with CockroachDB (TypeORM)](build-a-typescript-app-with-cockroachdb.html) |
+| Ruby | [ActiveRecord](https://rubygems.org/gems/activerecord)
[RGeo/RGeo-ActiveRecord](https://github.com/cockroachdb/activerecord-cockroachdb-adapter#working-with-spatial-data) | 8.1.0
(use latest version of CockroachDB adapter) | Full | [`activerecord-cockroachdb-adapter`](https://rubygems.org/gems/activerecord-cockroachdb-adapter)
(includes client-side transaction retry handling) | [Build a Ruby App with CockroachDB (ActiveRecord)](build-a-ruby-app-with-cockroachdb-activerecord.html) |
+| Python | [Django](https://pypi.org/project/Django/)
(including [GeoDjango](https://docs.djangoproject.com/en/3.1/ref/contrib/gis/))
[peewee](https://github.com/coleifer/peewee/)
[SQLAlchemy](https://www.sqlalchemy.org/) | 4.1.x
(use latest version of CockroachDB adapter)
3.13.3
0.7.13
1.4.17
(use latest version of CockroachDB adapter) | Full
Full
Full
Full | [`django-cockroachdb`](https://pypi.org/project/django-cockroachdb/)
N/A
N/A
[`sqlalchemy-cockroachdb`](https://pypi.org/project/sqlalchemy-cockroachdb)
(includes client-side transaction retry handling) | [Build a Python App with CockroachDB (Django)](build-a-python-app-with-cockroachdb-django.html)
N/A (See [peewee docs](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#cockroach-database).)
[Build a Python App with CockroachDB (SQLAlchemy)](build-a-python-app-with-cockroachdb-sqlalchemy.html) |
## Application frameworks
diff --git a/src/current/_includes/v25.4/orchestration/start-cockroachdb-insecure.md b/src/current/_includes/v25.4/orchestration/start-cockroachdb-insecure.md
index 3406d48edbb..c9af5cc0267 100644
--- a/src/current/_includes/v25.4/orchestration/start-cockroachdb-insecure.md
+++ b/src/current/_includes/v25.4/orchestration/start-cockroachdb-insecure.md
@@ -1,10 +1,10 @@
-1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it.
+1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it.
- Download [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml):
+ Download [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml
~~~
{{site.data.alerts.callout_info}}
@@ -27,11 +27,11 @@
Alternatively, if you'd rather start with a configuration file that has been customized for performance:
- 1. Download our [performance version of `cockroachdb-statefulset-insecure.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml):
+ 1. Download our [performance version of `cockroachdb-statefulset-insecure.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml
~~~
1. Modify the file wherever there is a `TODO` comment.
@@ -72,12 +72,12 @@
pvc-5315efda-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-2 27s
~~~
-1. Use our [`cluster-init.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
+1. Use our [`cluster-init.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml
~~~
~~~
diff --git a/src/current/_includes/v25.4/orchestration/start-cockroachdb-local-insecure.md b/src/current/_includes/v25.4/orchestration/start-cockroachdb-local-insecure.md
index 552cb3cd25f..196a53bce4c 100644
--- a/src/current/_includes/v25.4/orchestration/start-cockroachdb-local-insecure.md
+++ b/src/current/_includes/v25.4/orchestration/start-cockroachdb-local-insecure.md
@@ -1,8 +1,8 @@
-1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it:
+1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it:
{% include_cached copy-clipboard.html %}
~~~ shell
- $ kubectl create -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml
+ $ kubectl create -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml
~~~
~~~
@@ -41,12 +41,12 @@
pvc-5315efda-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-2 27s
~~~
-1. Use our [`cluster-init.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
+1. Use our [`cluster-init.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml
~~~
~~~
diff --git a/src/current/_includes/v25.4/orchestration/start-cockroachdb-secure.md b/src/current/_includes/v25.4/orchestration/start-cockroachdb-secure.md
index 972cabc2d8e..a7299e1aa25 100644
--- a/src/current/_includes/v25.4/orchestration/start-cockroachdb-secure.md
+++ b/src/current/_includes/v25.4/orchestration/start-cockroachdb-secure.md
@@ -1,10 +1,10 @@
### Configure the cluster
-1. Download and modify our [StatefulSet configuration](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml):
+1. Download and modify our [StatefulSet configuration](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml
~~~
1. Update `secretName` with the name of the corresponding node secret.
diff --git a/src/current/_includes/v25.4/orchestration/test-cluster-secure.md b/src/current/_includes/v25.4/orchestration/test-cluster-secure.md
index f255d8d62fc..c146d634626 100644
--- a/src/current/_includes/v25.4/orchestration/test-cluster-secure.md
+++ b/src/current/_includes/v25.4/orchestration/test-cluster-secure.md
@@ -42,7 +42,7 @@ $ kubectl create \
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
--f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/client.yaml
+-f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/client.yaml
~~~
~~~
diff --git a/src/current/_includes/v26.1/misc/tooling.md b/src/current/_includes/v26.1/misc/tooling.md
index d5f69bf12a9..072d2cd6a73 100644
--- a/src/current/_includes/v26.1/misc/tooling.md
+++ b/src/current/_includes/v26.1/misc/tooling.md
@@ -23,7 +23,7 @@ Customers should contact their account team before moving production workloads t
Unless explicitly stated, support for a [driver](#drivers) or [data access framework](#data-access-frameworks-e-g-orms) does not include [automatic, client-side transaction retry handling]({% link {{ page.version.version }}/transaction-retry-error-reference.md %}#client-side-retry-handling). For client-side transaction retry handling samples, see [Develop with CockroachDB]({% link {{ page.version.version }}/developer-guide-overview.md %}).
{{site.data.alerts.end}}
-If you encounter problems using CockroachDB with any of the tools listed on this page, please [open an issue](https://github.com/cockroachdb/cockroach/issues/new) with details to help us make progress toward better support.
+If you encounter problems using CockroachDB with any of the tools listed on this page, please open an issue with details to help us make progress toward better support.
For a list of tools supported by the CockroachDB community, see [Third-Party Tools Supported by the Community]({% link {{ page.version.version }}/community-tooling.md %}).
@@ -32,23 +32,23 @@ For a list of tools supported by the CockroachDB community, see [Third-Party Too
| Language | Driver | Latest tested version | Support level | CockroachDB adapter | Tutorial |
|----------+--------+-----------------------+---------------------+---------------------+----------|
| C | [libpq](http://www.postgresql.org/docs/13/static/libpq.html)| PostgreSQL 13 | Partial | N/A | N/A |
-| C# (.NET) | [Npgsql](https://www.nuget.org/packages/Npgsql/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/npgsql.go ||var npgsqlSupportedTag = "v||"\n\n %} | Full | N/A | [Build a C# App with CockroachDB (Npgsql)](build-a-csharp-app-with-cockroachdb.html) |
-| Go | [pgx](https://github.com/jackc/pgx/releases)
[pq](https://github.com/lib/pq) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/pgx.go ||var supportedPGXTag = "||"\n\n %}
(use latest version of CockroachDB adapter)
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/libpq.go ||var libPQSupportedTag = "||"\n\n %} | Full
Full | [`crdbpgx`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbpgx)
(includes client-side transaction retry handling)
N/A | [Build a Go App with CockroachDB (pgx)](build-a-go-app-with-cockroachdb.html)
[Build a Go App with CockroachDB (pq)](build-a-go-app-with-cockroachdb-pq.html) |
-| Java | [JDBC](https://jdbc.postgresql.org/download/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/pgjdbc.go ||var supportedPGJDBCTag = "||"\n\n %} | Full | N/A | [Build a Java App with CockroachDB (JDBC)](build-a-java-app-with-cockroachdb.html) |
+| C# (.NET) | [Npgsql](https://www.nuget.org/packages/Npgsql/) | 7.0.2 | Full | N/A | [Build a C# App with CockroachDB (Npgsql)](build-a-csharp-app-with-cockroachdb.html) |
+| Go | [pgx](https://github.com/jackc/pgx/releases)
[pq](https://github.com/lib/pq) | v5.3.1
(use latest version of CockroachDB adapter)
v1.10.5 | Full
Full | [`crdbpgx`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbpgx)
(includes client-side transaction retry handling)
N/A | [Build a Go App with CockroachDB (pgx)](build-a-go-app-with-cockroachdb.html)
[Build a Go App with CockroachDB (pq)](build-a-go-app-with-cockroachdb-pq.html) |
+| Java | [JDBC](https://jdbc.postgresql.org/download/) | REL42.7.3 | Full | N/A | [Build a Java App with CockroachDB (JDBC)](build-a-java-app-with-cockroachdb.html) |
| JavaScript | [pg](https://www.npmjs.com/package/pg) | 8.2.1 | Full | N/A | [Build a Node.js App with CockroachDB (pg)](build-a-nodejs-app-with-cockroachdb.html) |
-| Python | [psycopg3](https://www.psycopg.org/psycopg3/docs/)
[psycopg2](https://www.psycopg.org/docs/install.html)
[asyncpg](https://magicstack.github.io/asyncpg/current/index.html) | 3.0.16
2.8.6
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/asyncpg.go || var asyncpgSupportedTag = "||"\n\n %} | Full
Full
Partial | N/A
N/A
N/A | [Build a Python App with CockroachDB (psycopg3)](build-a-python-app-with-cockroachdb-psycopg3.html)
[Build a Python App with CockroachDB (psycopg2)](build-a-python-app-with-cockroachdb.html)
[Build a Python App with CockroachDB (asyncpg)](build-a-python-app-with-cockroachdb-asyncpg.html) |
-| Ruby | [pg](https://rubygems.org/gems/pg) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/ruby_pg.go ||var rubyPGVersion = "||"\n\n %} | Full | N/A | [Build a Ruby App with CockroachDB (pg)](build-a-ruby-app-with-cockroachdb.html) |
+| Python | [psycopg3](https://www.psycopg.org/psycopg3/docs/)
[psycopg2](https://www.psycopg.org/docs/install.html)
[asyncpg](https://magicstack.github.io/asyncpg/current/index.html) | 3.0.16
2.8.6
v0.24.0 | Full
Full
Partial | N/A
N/A
N/A | [Build a Python App with CockroachDB (psycopg3)](build-a-python-app-with-cockroachdb-psycopg3.html)
[Build a Python App with CockroachDB (psycopg2)](build-a-python-app-with-cockroachdb.html)
[Build a Python App with CockroachDB (asyncpg)](build-a-python-app-with-cockroachdb-asyncpg.html) |
+| Ruby | [pg](https://rubygems.org/gems/pg) | v1.4.6 | Full | N/A | [Build a Ruby App with CockroachDB (pg)](build-a-ruby-app-with-cockroachdb.html) |
| Rust | [rust-postgres](https://github.com/sfackler/rust-postgres) | 0.19.2 | Partial | N/A | [Build a Rust App with CockroachDB]({% link {{ page.version.version }}/build-a-rust-app-with-cockroachdb.md %}) |
## Data access frameworks (e.g., ORMs)
| Language | Framework | Latest tested version | Support level | CockroachDB adapter | Tutorial |
|----------+-----------+-----------------------+---------------+---------------------+----------|
-| Go | [GORM](https://github.com/jinzhu/gorm/releases)
[go-pg](https://github.com/go-pg/pg)
[upper/db](https://github.com/upper/db) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/gorm.go ||var gormSupportedTag = "||"\n\n %}
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/gopg.go ||var gopgSupportedTag = "||"\n\n %}
v4 | Full
Full
Full | [`crdbgorm`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbgorm)
(includes client-side transaction retry handling)
N/A
N/A | [Build a Go App with CockroachDB (GORM)](build-a-go-app-with-cockroachdb-gorm.html)
N/A
[Build a Go App with CockroachDB (upper/db)](build-a-go-app-with-cockroachdb-upperdb.html) |
-| Java | [Hibernate](https://hibernate.org/orm/)
(including [Hibernate Spatial](https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#spatial))
[jOOQ](https://www.jooq.org/)
[MyBatis](https://mybatis.org/mybatis-3/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/hibernate.go ||var supportedHibernateTag = "||"\n\n %} (must be at least 5.4.19)
3.13.2 (must be at least 3.13.0)
3.5.5| Full
Full
Full | N/A
N/A
N/A | [Build a Java App with CockroachDB (Hibernate)](build-a-java-app-with-cockroachdb-hibernate.html)
[Build a Java App with CockroachDB (jOOQ)](build-a-java-app-with-cockroachdb-jooq.html)
[Build a Spring App with CockroachDB (MyBatis)]({% link {{ page.version.version }}/build-a-spring-app-with-cockroachdb-mybatis.md %}) |
-| JavaScript/TypeScript | [Sequelize](https://www.npmjs.com/package/sequelize)
[Knex.js](https://knexjs.org/)
[Prisma](https://prisma.io)
[TypeORM](https://www.npmjs.com/package/typeorm) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/sequelize.go ||var supportedSequelizeCockroachDBRelease = "||"\n\n %}
(use latest version of CockroachDB adapter)
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/knex.go ||const supportedKnexTag = "||"\n\n %}
3.14.0
0.3.17 {% comment %}{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/typeorm.go ||const supportedTypeORMRelease = "||"\n %}{% endcomment %} | Full
Full
Full
Full | [`sequelize-cockroachdb`](https://www.npmjs.com/package/sequelize-cockroachdb)
N/A
N/A
N/A | [Build a Node.js App with CockroachDB (Sequelize)](build-a-nodejs-app-with-cockroachdb-sequelize.html)
[Build a Node.js App with CockroachDB (Knex.js)](build-a-nodejs-app-with-cockroachdb-knexjs.html)
[Build a Node.js App with CockroachDB (Prisma)](build-a-nodejs-app-with-cockroachdb-prisma.html)
[Build a TypeScript App with CockroachDB (TypeORM)](build-a-typescript-app-with-cockroachdb.html) |
-| Ruby | [ActiveRecord](https://rubygems.org/gems/activerecord)
[RGeo/RGeo-ActiveRecord](https://github.com/cockroachdb/activerecord-cockroachdb-adapter#working-with-spatial-data) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/activerecord.go ||var supportedRailsVersion = "||"\nvar %}
(use latest version of CockroachDB adapter) | Full | [`activerecord-cockroachdb-adapter`](https://rubygems.org/gems/activerecord-cockroachdb-adapter)
(includes client-side transaction retry handling) | [Build a Ruby App with CockroachDB (ActiveRecord)](build-a-ruby-app-with-cockroachdb-activerecord.html) |
-| Python | [Django](https://pypi.org/project/Django/)
(including [GeoDjango](https://docs.djangoproject.com/en/3.1/ref/contrib/gis/))
[peewee](https://github.com/coleifer/peewee/)
[SQLAlchemy](https://www.sqlalchemy.org/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/django.go ||var djangoSupportedTag = "cockroach-||"\nvar %}
(use latest version of CockroachDB adapter)
3.13.3
0.7.13
1.4.17
(use latest version of CockroachDB adapter) | Full
Full
Full
Full | [`django-cockroachdb`](https://pypi.org/project/django-cockroachdb/)
N/A
N/A
[`sqlalchemy-cockroachdb`](https://pypi.org/project/sqlalchemy-cockroachdb)
(includes client-side transaction retry handling) | [Build a Python App with CockroachDB (Django)](build-a-python-app-with-cockroachdb-django.html)
N/A (See [peewee docs](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#cockroach-database).)
[Build a Python App with CockroachDB (SQLAlchemy)](build-a-python-app-with-cockroachdb-sqlalchemy.html) |
+| Go | [GORM](https://github.com/jinzhu/gorm/releases)
[go-pg](https://github.com/go-pg/pg)
[upper/db](https://github.com/upper/db) | v1.24.1
v10.9.0
v4 | Full
Full
Full | [`crdbgorm`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbgorm)
(includes client-side transaction retry handling)
N/A
N/A | [Build a Go App with CockroachDB (GORM)](build-a-go-app-with-cockroachdb-gorm.html)
N/A
[Build a Go App with CockroachDB (upper/db)](build-a-go-app-with-cockroachdb-upperdb.html) |
+| Java | [Hibernate](https://hibernate.org/orm/)
(including [Hibernate Spatial](https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#spatial))
[jOOQ](https://www.jooq.org/)
[MyBatis](https://mybatis.org/mybatis-3/) | 6.6.20 (must be at least 5.4.19)
3.13.2 (must be at least 3.13.0)
3.5.5| Full
Full
Full | N/A
N/A
N/A | [Build a Java App with CockroachDB (Hibernate)](build-a-java-app-with-cockroachdb-hibernate.html)
[Build a Java App with CockroachDB (jOOQ)](build-a-java-app-with-cockroachdb-jooq.html)
[Build a Spring App with CockroachDB (MyBatis)]({% link {{ page.version.version }}/build-a-spring-app-with-cockroachdb-mybatis.md %}) |
+| JavaScript/TypeScript | [Sequelize](https://www.npmjs.com/package/sequelize)
[Knex.js](https://knexjs.org/)
[Prisma](https://prisma.io)
[TypeORM](https://www.npmjs.com/package/typeorm) | v6.0.5
(use latest version of CockroachDB adapter)
2.5.1
3.14.0
0.3.17 {% comment %}remove-unsafe-crdb-setting{% endcomment %} | Full
Full
Full
Full | [`sequelize-cockroachdb`](https://www.npmjs.com/package/sequelize-cockroachdb)
N/A
N/A
N/A | [Build a Node.js App with CockroachDB (Sequelize)](build-a-nodejs-app-with-cockroachdb-sequelize.html)
[Build a Node.js App with CockroachDB (Knex.js)](build-a-nodejs-app-with-cockroachdb-knexjs.html)
[Build a Node.js App with CockroachDB (Prisma)](build-a-nodejs-app-with-cockroachdb-prisma.html)
[Build a TypeScript App with CockroachDB (TypeORM)](build-a-typescript-app-with-cockroachdb.html) |
+| Ruby | [ActiveRecord](https://rubygems.org/gems/activerecord)
[RGeo/RGeo-ActiveRecord](https://github.com/cockroachdb/activerecord-cockroachdb-adapter#working-with-spatial-data) | 8.1.0
(use latest version of CockroachDB adapter) | Full | [`activerecord-cockroachdb-adapter`](https://rubygems.org/gems/activerecord-cockroachdb-adapter)
(includes client-side transaction retry handling) | [Build a Ruby App with CockroachDB (ActiveRecord)](build-a-ruby-app-with-cockroachdb-activerecord.html) |
+| Python | [Django](https://pypi.org/project/Django/)
(including [GeoDjango](https://docs.djangoproject.com/en/3.1/ref/contrib/gis/))
[peewee](https://github.com/coleifer/peewee/)
[SQLAlchemy](https://www.sqlalchemy.org/) | 4.1.x
(use latest version of CockroachDB adapter)
3.13.3
0.7.13
1.4.17
(use latest version of CockroachDB adapter) | Full
Full
Full
Full | [`django-cockroachdb`](https://pypi.org/project/django-cockroachdb/)
N/A
N/A
[`sqlalchemy-cockroachdb`](https://pypi.org/project/sqlalchemy-cockroachdb)
(includes client-side transaction retry handling) | [Build a Python App with CockroachDB (Django)](build-a-python-app-with-cockroachdb-django.html)
N/A (See [peewee docs](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#cockroach-database).)
[Build a Python App with CockroachDB (SQLAlchemy)](build-a-python-app-with-cockroachdb-sqlalchemy.html) |
## Graphical user interfaces (GUIs)
diff --git a/src/current/_includes/v26.1/orchestration/start-cockroachdb-insecure.md b/src/current/_includes/v26.1/orchestration/start-cockroachdb-insecure.md
index 3406d48edbb..c9af5cc0267 100644
--- a/src/current/_includes/v26.1/orchestration/start-cockroachdb-insecure.md
+++ b/src/current/_includes/v26.1/orchestration/start-cockroachdb-insecure.md
@@ -1,10 +1,10 @@
-1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it.
+1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it.
- Download [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml):
+ Download [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml
~~~
{{site.data.alerts.callout_info}}
@@ -27,11 +27,11 @@
Alternatively, if you'd rather start with a configuration file that has been customized for performance:
- 1. Download our [performance version of `cockroachdb-statefulset-insecure.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml):
+ 1. Download our [performance version of `cockroachdb-statefulset-insecure.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml
~~~
1. Modify the file wherever there is a `TODO` comment.
@@ -72,12 +72,12 @@
pvc-5315efda-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-2 27s
~~~
-1. Use our [`cluster-init.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
+1. Use our [`cluster-init.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml
~~~
~~~
diff --git a/src/current/_includes/v26.1/orchestration/start-cockroachdb-local-insecure.md b/src/current/_includes/v26.1/orchestration/start-cockroachdb-local-insecure.md
index 552cb3cd25f..196a53bce4c 100644
--- a/src/current/_includes/v26.1/orchestration/start-cockroachdb-local-insecure.md
+++ b/src/current/_includes/v26.1/orchestration/start-cockroachdb-local-insecure.md
@@ -1,8 +1,8 @@
-1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it:
+1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it:
{% include_cached copy-clipboard.html %}
~~~ shell
- $ kubectl create -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml
+ $ kubectl create -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml
~~~
~~~
@@ -41,12 +41,12 @@
pvc-5315efda-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-2 27s
~~~
-1. Use our [`cluster-init.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
+1. Use our [`cluster-init.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml
~~~
~~~
diff --git a/src/current/_includes/v26.1/orchestration/start-cockroachdb-secure.md b/src/current/_includes/v26.1/orchestration/start-cockroachdb-secure.md
index 972cabc2d8e..a7299e1aa25 100644
--- a/src/current/_includes/v26.1/orchestration/start-cockroachdb-secure.md
+++ b/src/current/_includes/v26.1/orchestration/start-cockroachdb-secure.md
@@ -1,10 +1,10 @@
### Configure the cluster
-1. Download and modify our [StatefulSet configuration](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml):
+1. Download and modify our [StatefulSet configuration](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml
~~~
1. Update `secretName` with the name of the corresponding node secret.
diff --git a/src/current/_includes/v26.1/orchestration/test-cluster-secure.md b/src/current/_includes/v26.1/orchestration/test-cluster-secure.md
index f255d8d62fc..c146d634626 100644
--- a/src/current/_includes/v26.1/orchestration/test-cluster-secure.md
+++ b/src/current/_includes/v26.1/orchestration/test-cluster-secure.md
@@ -42,7 +42,7 @@ $ kubectl create \
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
--f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/client.yaml
+-f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/client.yaml
~~~
~~~
diff --git a/src/current/_includes/v26.2/misc/tooling.md b/src/current/_includes/v26.2/misc/tooling.md
index 254a6425840..6981a90f68f 100644
--- a/src/current/_includes/v26.2/misc/tooling.md
+++ b/src/current/_includes/v26.2/misc/tooling.md
@@ -23,7 +23,7 @@ Customers should contact their account team before moving production workloads t
Unless explicitly stated, support for a [driver](#drivers) or [data access framework](#data-access-frameworks-e-g-orms) does not include [automatic, client-side transaction retry handling]({% link {{ page.version.version }}/transaction-retry-error-reference.md %}#client-side-retry-handling). For client-side transaction retry handling samples, see [Develop with CockroachDB]({% link {{ page.version.version }}/developer-guide-overview.md %}).
{{site.data.alerts.end}}
-If you encounter problems using CockroachDB with any of the tools listed on this page, please [open an issue](https://github.com/cockroachdb/cockroach/issues/new) with details to help us make progress toward better support.
+If you encounter problems using CockroachDB with any of the tools listed on this page, please open an issue with details to help us make progress toward better support.
For a list of tools supported by the CockroachDB community, see [Third-Party Tools Supported by the Community]({% link {{ page.version.version }}/community-tooling.md %}).
@@ -32,23 +32,23 @@ For a list of tools supported by the CockroachDB community, see [Third-Party Too
| Language | Driver | Latest tested version | Support level | CockroachDB adapter | Tutorial |
|----------+--------+-----------------------+---------------------+---------------------+----------|
| C | [libpq](http://www.postgresql.org/docs/13/static/libpq.html)| PostgreSQL 13 | Partial | N/A | N/A |
-| C# (.NET) | [Npgsql](https://www.nuget.org/packages/Npgsql/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/npgsql.go ||var npgsqlSupportedTag = "v||"\n\n %} | Full | N/A | [Build a C# App with CockroachDB (Npgsql)](build-a-csharp-app-with-cockroachdb.html) |
-| Go | [pgx](https://github.com/jackc/pgx/releases)
[pq](https://github.com/lib/pq) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/pgx.go ||var supportedPGXTag = "||"\n\n %}
(use latest version of CockroachDB adapter)
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/libpq.go ||var libPQSupportedTag = "||"\n\n %} | Full
Full | [`crdbpgx`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbpgx)
(includes client-side transaction retry handling)
N/A | [Build a Go App with CockroachDB (pgx)](build-a-go-app-with-cockroachdb.html)
[Build a Go App with CockroachDB (pq)](build-a-go-app-with-cockroachdb-pq.html) |
-| Java | [JDBC](https://jdbc.postgresql.org/download/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/pgjdbc.go ||var supportedPGJDBCTag = "||"\n\n %} | Full | N/A | [Build a Java App with CockroachDB (JDBC)](build-a-java-app-with-cockroachdb.html) |
+| C# (.NET) | [Npgsql](https://www.nuget.org/packages/Npgsql/) | 7.0.2 | Full | N/A | [Build a C# App with CockroachDB (Npgsql)](build-a-csharp-app-with-cockroachdb.html) |
+| Go | [pgx](https://github.com/jackc/pgx/releases)
[pq](https://github.com/lib/pq) | v5.3.1
(use latest version of CockroachDB adapter)
v1.10.5 | Full
Full | [`crdbpgx`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbpgx)
(includes client-side transaction retry handling)
N/A | [Build a Go App with CockroachDB (pgx)](build-a-go-app-with-cockroachdb.html)
[Build a Go App with CockroachDB (pq)](build-a-go-app-with-cockroachdb-pq.html) |
+| Java | [JDBC](https://jdbc.postgresql.org/download/) | REL42.7.3 | Full | N/A | [Build a Java App with CockroachDB (JDBC)](build-a-java-app-with-cockroachdb.html) |
| JavaScript | [pg](https://www.npmjs.com/package/pg) | 8.2.1 | Full | N/A | [Build a Node.js App with CockroachDB (pg)](build-a-nodejs-app-with-cockroachdb.html) |
-| Python | [psycopg3](https://www.psycopg.org/psycopg3/docs/)
[psycopg2](https://www.psycopg.org/docs/install.html)
[asyncpg](https://magicstack.github.io/asyncpg/current/index.html) | 3.0.16
2.8.6
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/asyncpg.go || var asyncpgSupportedTag = "||"\n\n %} | Full
Full
Partial | N/A
N/A
N/A | [Build a Python App with CockroachDB (psycopg3)](build-a-python-app-with-cockroachdb-psycopg3.html)
[Build a Python App with CockroachDB (psycopg2)](build-a-python-app-with-cockroachdb.html)
[Build a Python App with CockroachDB (asyncpg)](build-a-python-app-with-cockroachdb-asyncpg.html) |
-| Ruby | [pg](https://rubygems.org/gems/pg) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/ruby_pg.go ||var rubyPGVersion = "||"\n\n %} | Full | N/A | [Build a Ruby App with CockroachDB (pg)](build-a-ruby-app-with-cockroachdb.html) |
+| Python | [psycopg3](https://www.psycopg.org/psycopg3/docs/)
[psycopg2](https://www.psycopg.org/docs/install.html)
[asyncpg](https://magicstack.github.io/asyncpg/current/index.html) | 3.0.16
2.8.6
v0.24.0 | Full
Full
Partial | N/A
N/A
N/A | [Build a Python App with CockroachDB (psycopg3)](build-a-python-app-with-cockroachdb-psycopg3.html)
[Build a Python App with CockroachDB (psycopg2)](build-a-python-app-with-cockroachdb.html)
[Build a Python App with CockroachDB (asyncpg)](build-a-python-app-with-cockroachdb-asyncpg.html) |
+| Ruby | [pg](https://rubygems.org/gems/pg) | v1.4.6 | Full | N/A | [Build a Ruby App with CockroachDB (pg)](build-a-ruby-app-with-cockroachdb.html) |
| Rust | [rust-postgres](https://github.com/sfackler/rust-postgres)
[sqlx](https://github.com/launchbadge/sqlx) | 0.19.2
0.8.6 | Partial
Partial | N/A
N/A | [Build a Rust App with CockroachDB]({% link {{ page.version.version }}/build-a-rust-app-with-cockroachdb.md %})
N/A |
## Data access frameworks (e.g., ORMs)
| Language | Framework | Latest tested version | Support level | CockroachDB adapter | Tutorial |
|----------+-----------+-----------------------+---------------+---------------------+----------|
-| Go | [GORM](https://github.com/jinzhu/gorm/releases)
[go-pg](https://github.com/go-pg/pg)
[upper/db](https://github.com/upper/db) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/gorm.go ||var gormSupportedTag = "||"\n\n %}
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/gopg.go ||var gopgSupportedTag = "||"\n\n %}
v4 | Full
Full
Full | [`crdbgorm`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbgorm)
(includes client-side transaction retry handling)
N/A
N/A | [Build a Go App with CockroachDB (GORM)](build-a-go-app-with-cockroachdb-gorm.html)
N/A
[Build a Go App with CockroachDB (upper/db)](build-a-go-app-with-cockroachdb-upperdb.html) |
-| Java | [Hibernate](https://hibernate.org/orm/)
(including [Hibernate Spatial](https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#spatial))
[jOOQ](https://www.jooq.org/)
[MyBatis](https://mybatis.org/mybatis-3/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/hibernate.go ||var supportedHibernateTag = "||"\n\n %} (must be at least 5.4.19)
3.13.2 (must be at least 3.13.0)
3.5.5| Full
Full
Full | N/A
N/A
N/A | [Build a Java App with CockroachDB (Hibernate)](build-a-java-app-with-cockroachdb-hibernate.html)
[Build a Java App with CockroachDB (jOOQ)](build-a-java-app-with-cockroachdb-jooq.html)
[Build a Spring App with CockroachDB (MyBatis)]({% link {{ page.version.version }}/build-a-spring-app-with-cockroachdb-mybatis.md %}) |
-| JavaScript/TypeScript | [Sequelize](https://www.npmjs.com/package/sequelize)
[Knex.js](https://knexjs.org/)
[Prisma](https://prisma.io)
[TypeORM](https://www.npmjs.com/package/typeorm) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/sequelize.go ||var supportedSequelizeCockroachDBRelease = "||"\n\n %}
(use latest version of CockroachDB adapter)
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/knex.go ||const supportedKnexTag = "||"\n\n %}
3.14.0
0.3.17 {% comment %}{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/typeorm.go ||const supportedTypeORMRelease = "||"\n %}{% endcomment %} | Full
Full
Full
Full | [`sequelize-cockroachdb`](https://www.npmjs.com/package/sequelize-cockroachdb)
N/A
N/A
N/A | [Build a Node.js App with CockroachDB (Sequelize)](build-a-nodejs-app-with-cockroachdb-sequelize.html)
[Build a Node.js App with CockroachDB (Knex.js)](build-a-nodejs-app-with-cockroachdb-knexjs.html)
[Build a Node.js App with CockroachDB (Prisma)](build-a-nodejs-app-with-cockroachdb-prisma.html)
[Build a TypeScript App with CockroachDB (TypeORM)](build-a-typescript-app-with-cockroachdb.html) |
-| Ruby | [ActiveRecord](https://rubygems.org/gems/activerecord)
[RGeo/RGeo-ActiveRecord](https://github.com/cockroachdb/activerecord-cockroachdb-adapter#working-with-spatial-data) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/activerecord.go ||var supportedRailsVersion = "||"\nvar %}
(use latest version of CockroachDB adapter) | Full | [`activerecord-cockroachdb-adapter`](https://rubygems.org/gems/activerecord-cockroachdb-adapter)
(includes client-side transaction retry handling) | [Build a Ruby App with CockroachDB (ActiveRecord)](build-a-ruby-app-with-cockroachdb-activerecord.html) |
-| Python | [Django](https://pypi.org/project/Django/)
(including [GeoDjango](https://docs.djangoproject.com/en/3.1/ref/contrib/gis/))
[peewee](https://github.com/coleifer/peewee/)
[SQLAlchemy](https://www.sqlalchemy.org/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/django.go ||var djangoSupportedTag = "cockroach-||"\nvar %}
(use latest version of CockroachDB adapter)
3.13.3
0.7.13
1.4.17
(use latest version of CockroachDB adapter) | Full
Full
Full
Full | [`django-cockroachdb`](https://pypi.org/project/django-cockroachdb/)
N/A
N/A
[`sqlalchemy-cockroachdb`](https://pypi.org/project/sqlalchemy-cockroachdb)
(includes client-side transaction retry handling) | [Build a Python App with CockroachDB (Django)](build-a-python-app-with-cockroachdb-django.html)
N/A (See [peewee docs](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#cockroach-database).)
[Build a Python App with CockroachDB (SQLAlchemy)](build-a-python-app-with-cockroachdb-sqlalchemy.html) |
+| Go | [GORM](https://github.com/jinzhu/gorm/releases)
[go-pg](https://github.com/go-pg/pg)
[upper/db](https://github.com/upper/db) | v1.24.1
v10.9.0
v4 | Full
Full
Full | [`crdbgorm`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbgorm)
(includes client-side transaction retry handling)
N/A
N/A | [Build a Go App with CockroachDB (GORM)](build-a-go-app-with-cockroachdb-gorm.html)
N/A
[Build a Go App with CockroachDB (upper/db)](build-a-go-app-with-cockroachdb-upperdb.html) |
+| Java | [Hibernate](https://hibernate.org/orm/)
(including [Hibernate Spatial](https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#spatial))
[jOOQ](https://www.jooq.org/)
[MyBatis](https://mybatis.org/mybatis-3/) | 6.6.20 (must be at least 5.4.19)
3.13.2 (must be at least 3.13.0)
3.5.5| Full
Full
Full | N/A
N/A
N/A | [Build a Java App with CockroachDB (Hibernate)](build-a-java-app-with-cockroachdb-hibernate.html)
[Build a Java App with CockroachDB (jOOQ)](build-a-java-app-with-cockroachdb-jooq.html)
[Build a Spring App with CockroachDB (MyBatis)]({% link {{ page.version.version }}/build-a-spring-app-with-cockroachdb-mybatis.md %}) |
+| JavaScript/TypeScript | [Sequelize](https://www.npmjs.com/package/sequelize)
[Knex.js](https://knexjs.org/)
[Prisma](https://prisma.io)
[TypeORM](https://www.npmjs.com/package/typeorm) | v6.0.5
(use latest version of CockroachDB adapter)
2.5.1
3.14.0
0.3.17 {% comment %}remove-unsafe-crdb-setting{% endcomment %} | Full
Full
Full
Full | [`sequelize-cockroachdb`](https://www.npmjs.com/package/sequelize-cockroachdb)
N/A
N/A
N/A | [Build a Node.js App with CockroachDB (Sequelize)](build-a-nodejs-app-with-cockroachdb-sequelize.html)
[Build a Node.js App with CockroachDB (Knex.js)](build-a-nodejs-app-with-cockroachdb-knexjs.html)
[Build a Node.js App with CockroachDB (Prisma)](build-a-nodejs-app-with-cockroachdb-prisma.html)
[Build a TypeScript App with CockroachDB (TypeORM)](build-a-typescript-app-with-cockroachdb.html) |
+| Ruby | [ActiveRecord](https://rubygems.org/gems/activerecord)
[RGeo/RGeo-ActiveRecord](https://github.com/cockroachdb/activerecord-cockroachdb-adapter#working-with-spatial-data) | 8.1.0
(use latest version of CockroachDB adapter) | Full | [`activerecord-cockroachdb-adapter`](https://rubygems.org/gems/activerecord-cockroachdb-adapter)
(includes client-side transaction retry handling) | [Build a Ruby App with CockroachDB (ActiveRecord)](build-a-ruby-app-with-cockroachdb-activerecord.html) |
+| Python | [Django](https://pypi.org/project/Django/)
(including [GeoDjango](https://docs.djangoproject.com/en/3.1/ref/contrib/gis/))
[peewee](https://github.com/coleifer/peewee/)
[SQLAlchemy](https://www.sqlalchemy.org/) | 4.1.x
(use latest version of CockroachDB adapter)
3.13.3
0.7.13
1.4.17
(use latest version of CockroachDB adapter) | Full
Full
Full
Full | [`django-cockroachdb`](https://pypi.org/project/django-cockroachdb/)
N/A
N/A
[`sqlalchemy-cockroachdb`](https://pypi.org/project/sqlalchemy-cockroachdb)
(includes client-side transaction retry handling) | [Build a Python App with CockroachDB (Django)](build-a-python-app-with-cockroachdb-django.html)
N/A (See [peewee docs](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#cockroach-database).)
[Build a Python App with CockroachDB (SQLAlchemy)](build-a-python-app-with-cockroachdb-sqlalchemy.html) |
## Graphical user interfaces (GUIs)
diff --git a/src/current/_includes/v26.2/orchestration/start-cockroachdb-insecure.md b/src/current/_includes/v26.2/orchestration/start-cockroachdb-insecure.md
index 3406d48edbb..c9af5cc0267 100644
--- a/src/current/_includes/v26.2/orchestration/start-cockroachdb-insecure.md
+++ b/src/current/_includes/v26.2/orchestration/start-cockroachdb-insecure.md
@@ -1,10 +1,10 @@
-1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it.
+1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it.
- Download [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml):
+ Download [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml
~~~
{{site.data.alerts.callout_info}}
@@ -27,11 +27,11 @@
Alternatively, if you'd rather start with a configuration file that has been customized for performance:
- 1. Download our [performance version of `cockroachdb-statefulset-insecure.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml):
+ 1. Download our [performance version of `cockroachdb-statefulset-insecure.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml
~~~
1. Modify the file wherever there is a `TODO` comment.
@@ -72,12 +72,12 @@
pvc-5315efda-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-2 27s
~~~
-1. Use our [`cluster-init.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
+1. Use our [`cluster-init.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml
~~~
~~~
diff --git a/src/current/_includes/v26.2/orchestration/start-cockroachdb-local-insecure.md b/src/current/_includes/v26.2/orchestration/start-cockroachdb-local-insecure.md
index 552cb3cd25f..196a53bce4c 100644
--- a/src/current/_includes/v26.2/orchestration/start-cockroachdb-local-insecure.md
+++ b/src/current/_includes/v26.2/orchestration/start-cockroachdb-local-insecure.md
@@ -1,8 +1,8 @@
-1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it:
+1. From your local workstation, use our [`cockroachdb-statefulset.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml) file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it:
{% include_cached copy-clipboard.html %}
~~~ shell
- $ kubectl create -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cockroachdb-statefulset.yaml
+ $ kubectl create -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml
~~~
~~~
@@ -41,12 +41,12 @@
pvc-5315efda-8bd5-11e6-a4f4-42010a800002 1Gi RWO Delete Bound default/datadir-cockroachdb-2 27s
~~~
-1. Use our [`cluster-init.yaml`](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
+1. Use our [`cluster-init.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml) file to perform a one-time initialization that joins the CockroachDB nodes into a single cluster:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/cluster-init.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml
~~~
~~~
diff --git a/src/current/_includes/v26.2/orchestration/start-cockroachdb-secure.md b/src/current/_includes/v26.2/orchestration/start-cockroachdb-secure.md
index 972cabc2d8e..a7299e1aa25 100644
--- a/src/current/_includes/v26.2/orchestration/start-cockroachdb-secure.md
+++ b/src/current/_includes/v26.2/orchestration/start-cockroachdb-secure.md
@@ -1,10 +1,10 @@
### Configure the cluster
-1. Download and modify our [StatefulSet configuration](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml):
+1. Download and modify our [StatefulSet configuration](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml
~~~
1. Update `secretName` with the name of the corresponding node secret.
diff --git a/src/current/_includes/v26.2/orchestration/test-cluster-secure.md b/src/current/_includes/v26.2/orchestration/test-cluster-secure.md
index f255d8d62fc..c146d634626 100644
--- a/src/current/_includes/v26.2/orchestration/test-cluster-secure.md
+++ b/src/current/_includes/v26.2/orchestration/test-cluster-secure.md
@@ -42,7 +42,7 @@ $ kubectl create \
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl create \
--f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/bring-your-own-certs/client.yaml
+-f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/client.yaml
~~~
~~~
diff --git a/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/client.yaml b/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/client.yaml
new file mode 100644
index 00000000000..ade0eb4af06
--- /dev/null
+++ b/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/client.yaml
@@ -0,0 +1,35 @@
+# This config file demonstrates how to connect to the CockroachDB StatefulSet
+# defined in bring-your-own-certs-statefulset.yaml that uses certificates
+# created outside of Kubernetes. See that file for why you may want to use it.
+# You should be able to adapt the core ideas to deploy your own custom
+# applications and connect them to the database similarly.
+#
+# The pod that this file defines will sleep in the cluster not using any
+# resources. After creating the pod, you can use it to open up a SQL shell to
+# the database by running:
+#
+# kubectl exec -it cockroachdb-client-secure -- ./cockroach sql --url="postgres://root@cockroachdb-public:26257/?sslmode=verify-full&sslcert=/cockroach-certs/client.root.crt&sslkey=/cockroach-certs/client.root.key&sslrootcert=/cockroach-certs/ca.crt"
+apiVersion: v1
+kind: Pod
+metadata:
+ name: cockroachdb-client-secure
+ labels:
+ app: cockroachdb-client
+spec:
+ serviceAccountName: cockroachdb
+ containers:
+ - name: cockroachdb-client
+ image: cockroachdb/cockroach:latest
+ # Keep a pod open indefinitely so kubectl exec can be used to get a shell to it
+ # and run cockroach client commands, such as cockroach sql, cockroach node status, etc.
+ command:
+ - sleep
+ - "2147483648" # 2^31
+ volumeMounts:
+ - name: client-certs
+ mountPath: /cockroach-certs
+ volumes:
+ - name: client-certs
+ secret:
+ secretName: cockroachdb.client.root
+ defaultMode: 256
diff --git a/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml b/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml
new file mode 100644
index 00000000000..96fe724d25c
--- /dev/null
+++ b/src/current/files/cockroach/cloud/kubernetes/bring-your-own-certs/cockroachdb-statefulset.yaml
@@ -0,0 +1,244 @@
+# This config file defines a CockroachDB StatefulSet that uses certificates
+# created outside of Kubernetes. You may want to use it if you want to use a
+# different certificate authority from the one being used by Kubernetes or if
+# your Kubernetes cluster doesn't fully support certificate-signing requests
+# (e.g. as of July 2018, EKS doesn't work properly).
+#
+# To use this config file, first set up your certificates and load them into
+# your Kubernetes cluster as Secrets using the commands below:
+#
+# mkdir certs
+# mkdir my-safe-directory
+# cockroach cert create-ca --certs-dir=certs --ca-key=my-safe-directory/ca.key
+# cockroach cert create-client root --certs-dir=certs --ca-key=my-safe-directory/ca.key
+# kubectl create secret generic cockroachdb.client.root --from-file=certs
+# cockroach cert create-node --certs-dir=certs --ca-key=my-safe-directory/ca.key localhost 127.0.0.1 cockroachdb-public cockroachdb-public.default cockroachdb-public.default.svc.cluster.local *.cockroachdb *.cockroachdb.default *.cockroachdb.default.svc.cluster.local
+# kubectl create secret generic cockroachdb.node --from-file=certs
+# kubectl create -f bring-your-own-certs-statefulset.yaml
+# kubectl exec -it cockroachdb-0 -- /cockroach/cockroach init --certs-dir=/cockroach/cockroach-certs
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+rules:
+- apiGroups:
+ - ""
+ resources:
+ - secrets
+ verbs:
+ - get
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+roleRef:
+ apiGroup: rbac.authorization.k8s.io
+ kind: Role
+ name: cockroachdb
+subjects:
+- kind: ServiceAccount
+ name: cockroachdb
+ namespace: default
+---
+apiVersion: v1
+kind: Service
+metadata:
+ # This service is meant to be used by clients of the database. It exposes a ClusterIP that will
+ # automatically load balance connections to the different database pods.
+ name: cockroachdb-public
+ labels:
+ app: cockroachdb
+spec:
+ ports:
+ # The main port, served by gRPC, serves Postgres-flavor SQL, internode
+ # traffic and the cli.
+ - port: 26257
+ targetPort: 26257
+ name: grpc
+ # The secondary port serves the UI as well as health and debug endpoints.
+ - port: 8080
+ targetPort: 8080
+ name: http
+ selector:
+ app: cockroachdb
+---
+apiVersion: v1
+kind: Service
+metadata:
+ # This service only exists to create DNS entries for each pod in the stateful
+ # set such that they can resolve each other's IP addresses. It does not
+ # create a load-balanced ClusterIP and should not be used directly by clients
+ # in most circumstances.
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+ annotations:
+ # Use this annotation in addition to the actual publishNotReadyAddresses
+ # field below because the annotation will stop being respected soon but the
+ # field is broken in some versions of Kubernetes:
+ # https://github.com/kubernetes/kubernetes/issues/58662
+ service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
+ # Enable automatic monitoring of all instances when Prometheus is running in the cluster.
+ prometheus.io/scrape: "true"
+ prometheus.io/path: "_status/vars"
+ prometheus.io/port: "8080"
+spec:
+ ports:
+ - port: 26257
+ targetPort: 26257
+ name: grpc
+ - port: 8080
+ targetPort: 8080
+ name: http
+ # We want all pods in the StatefulSet to have their addresses published for
+ # the sake of the other CockroachDB pods even before they're ready, since they
+ # have to be able to talk to each other in order to become ready.
+ publishNotReadyAddresses: true
+ clusterIP: None
+ selector:
+ app: cockroachdb
+---
+apiVersion: policy/v1
+kind: PodDisruptionBudget
+metadata:
+ name: cockroachdb-budget
+ labels:
+ app: cockroachdb
+spec:
+ selector:
+ matchLabels:
+ app: cockroachdb
+ maxUnavailable: 1
+---
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: cockroachdb
+spec:
+ serviceName: "cockroachdb"
+ replicas: 3
+ selector:
+ matchLabels:
+ app: cockroachdb
+ template:
+ metadata:
+ labels:
+ app: cockroachdb
+ spec:
+ serviceAccountName: cockroachdb
+ affinity:
+ podAntiAffinity:
+ preferredDuringSchedulingIgnoredDuringExecution:
+ - weight: 100
+ podAffinityTerm:
+ labelSelector:
+ matchExpressions:
+ - key: app
+ operator: In
+ values:
+ - cockroachdb
+ topologyKey: kubernetes.io/hostname
+ containers:
+ - name: cockroachdb
+ image: cockroachdb/cockroach:latest
+ imagePullPolicy: IfNotPresent
+ # TODO: Change these to appropriate values for the hardware that you're running. You can see
+ # the resources that can be allocated on each of your Kubernetes nodes by running:
+ # kubectl describe nodes
+ # Note that requests and limits should have identical values.
+ resources:
+ requests:
+ cpu: "2"
+ memory: "8Gi"
+ limits:
+ cpu: "2"
+ memory: "8Gi"
+ ports:
+ - containerPort: 26257
+ name: grpc
+ - containerPort: 8080
+ name: http
+# We recommend that you do not configure a liveness probe on a production environment, as this can impact the availability of production databases.
+# livenessProbe:
+# httpGet:
+# path: "/health"
+# port: http
+# scheme: HTTPS
+# initialDelaySeconds: 30
+# periodSeconds: 5
+ readinessProbe:
+ httpGet:
+ path: "/health?ready=1"
+ port: http
+ scheme: HTTPS
+ initialDelaySeconds: 10
+ periodSeconds: 5
+ failureThreshold: 2
+ volumeMounts:
+ - name: datadir
+ mountPath: /cockroach/cockroach-data
+ - name: certs
+ mountPath: /cockroach/cockroach-certs
+ env:
+ - name: COCKROACH_CHANNEL
+ value: kubernetes-secure
+ - name: GOMAXPROCS
+ valueFrom:
+ resourceFieldRef:
+ resource: limits.cpu
+ divisor: "1"
+ - name: MEMORY_LIMIT_MIB
+ valueFrom:
+ resourceFieldRef:
+ resource: limits.memory
+ divisor: "1Mi"
+ command:
+ - "/bin/bash"
+ - "-ecx"
+ # The use of qualified `hostname -f` is crucial:
+ # Other nodes aren't able to look up the unqualified hostname.
+ - exec
+ /cockroach/cockroach
+ start
+ --logtostderr
+ --certs-dir /cockroach/cockroach-certs
+ --advertise-host $(hostname -f)
+ --http-addr 0.0.0.0
+ --join cockroachdb-0.cockroachdb,cockroachdb-1.cockroachdb,cockroachdb-2.cockroachdb
+ --cache $(expr $MEMORY_LIMIT_MIB / 4)MiB
+ --max-sql-memory $(expr $MEMORY_LIMIT_MIB / 4)MiB
+ # No pre-stop hook is required, a SIGTERM plus some time is all that's
+ # needed for graceful shutdown of a node.
+ terminationGracePeriodSeconds: 60
+ volumes:
+ - name: datadir
+ persistentVolumeClaim:
+ claimName: datadir
+ - name: certs
+ secret:
+ secretName: cockroachdb.node
+ defaultMode: 256
+ podManagementPolicy: Parallel
+ updateStrategy:
+ type: RollingUpdate
+ volumeClaimTemplates:
+ - metadata:
+ name: datadir
+ spec:
+ accessModes:
+ - "ReadWriteOnce"
+ resources:
+ requests:
+ storage: 100Gi
diff --git a/src/current/files/cockroach/cloud/kubernetes/client-secure.yaml b/src/current/files/cockroach/cloud/kubernetes/client-secure.yaml
new file mode 100644
index 00000000000..87e51bca336
--- /dev/null
+++ b/src/current/files/cockroach/cloud/kubernetes/client-secure.yaml
@@ -0,0 +1,48 @@
+apiVersion: v1
+kind: Pod
+metadata:
+ name: cockroachdb-client-secure
+ labels:
+ app: cockroachdb-client
+spec:
+ serviceAccountName: cockroachdb
+ initContainers:
+ # The init-certs container sends a certificate signing request to the
+ # kubernetes cluster.
+ # You can see pending requests using: kubectl get csr
+ # CSRs can be approved using: kubectl certificate approve
+ #
+ # In addition to the client certificate and key, the init-certs entrypoint will symlink
+ # the cluster CA to the certs directory.
+ - name: init-certs
+ image: cockroachdb/cockroach-k8s-request-cert:0.4
+ imagePullPolicy: IfNotPresent
+ command:
+ - "/bin/ash"
+ - "-ecx"
+ - "/request-cert -namespace=${POD_NAMESPACE} -certs-dir=/cockroach-certs -type=client -user=root -symlink-ca-from=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
+ env:
+ - name: POD_NAMESPACE
+ valueFrom:
+ fieldRef:
+ fieldPath: metadata.namespace
+ volumeMounts:
+ - name: client-certs
+ mountPath: /cockroach-certs
+ containers:
+ - name: cockroachdb-client
+ image: cockroachdb/cockroach:latest
+ imagePullPolicy: IfNotPresent
+ volumeMounts:
+ - name: client-certs
+ mountPath: /cockroach-certs
+ # Keep a pod open indefinitely so kubectl exec can be used to get a shell to it
+ # and run cockroach client commands, such as cockroach sql, cockroach node status, etc.
+ command:
+ - sleep
+ - "2147483648" # 2^31
+ # This pod isn't doing anything important, so don't bother waiting to terminate it.
+ terminationGracePeriodSeconds: 0
+ volumes:
+ - name: client-certs
+ emptyDir: {}
diff --git a/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml b/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml
new file mode 100644
index 00000000000..779fd77b899
--- /dev/null
+++ b/src/current/files/cockroach/cloud/kubernetes/cluster-init.yaml
@@ -0,0 +1,19 @@
+apiVersion: batch/v1
+kind: Job
+metadata:
+ name: cluster-init
+ labels:
+ app: cockroachdb
+spec:
+ template:
+ spec:
+ containers:
+ - name: cluster-init
+ image: cockroachdb/cockroach:latest
+ imagePullPolicy: IfNotPresent
+ command:
+ - "/cockroach/cockroach"
+ - "init"
+ - "--insecure"
+ - "--host=cockroachdb-0.cockroachdb"
+ restartPolicy: OnFailure
diff --git a/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset-secure.yaml b/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset-secure.yaml
new file mode 100644
index 00000000000..891a43aba62
--- /dev/null
+++ b/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset-secure.yaml
@@ -0,0 +1,285 @@
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+rules:
+- apiGroups:
+ - ""
+ resources:
+ - secrets
+ verbs:
+ - create
+ - get
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+rules:
+- apiGroups:
+ - certificates.k8s.io
+ resources:
+ - certificatesigningrequests
+ verbs:
+ - create
+ - get
+ - watch
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+roleRef:
+ apiGroup: rbac.authorization.k8s.io
+ kind: Role
+ name: cockroachdb
+subjects:
+- kind: ServiceAccount
+ name: cockroachdb
+ namespace: default
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+roleRef:
+ apiGroup: rbac.authorization.k8s.io
+ kind: ClusterRole
+ name: cockroachdb
+subjects:
+- kind: ServiceAccount
+ name: cockroachdb
+ namespace: default
+---
+apiVersion: v1
+kind: Service
+metadata:
+ # This service is meant to be used by clients of the database. It exposes a ClusterIP that will
+ # automatically load balance connections to the different database pods.
+ name: cockroachdb-public
+ labels:
+ app: cockroachdb
+spec:
+ ports:
+ # The main port, served by gRPC, serves Postgres-flavor SQL, internode
+ # traffic and the cli.
+ - port: 26257
+ targetPort: 26257
+ name: grpc
+ # The secondary port serves the UI as well as health and debug endpoints.
+ - port: 8080
+ targetPort: 8080
+ name: http
+ selector:
+ app: cockroachdb
+---
+apiVersion: v1
+kind: Service
+metadata:
+ # This service only exists to create DNS entries for each pod in the stateful
+ # set such that they can resolve each other's IP addresses. It does not
+ # create a load-balanced ClusterIP and should not be used directly by clients
+ # in most circumstances.
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+ annotations:
+ # Use this annotation in addition to the actual publishNotReadyAddresses
+ # field below because the annotation will stop being respected soon but the
+ # field is broken in some versions of Kubernetes:
+ # https://github.com/kubernetes/kubernetes/issues/58662
+ service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
+ # Enable automatic monitoring of all instances when Prometheus is running in the cluster.
+ prometheus.io/scrape: "true"
+ prometheus.io/path: "_status/vars"
+ prometheus.io/port: "8080"
+spec:
+ ports:
+ - port: 26257
+ targetPort: 26257
+ name: grpc
+ - port: 8080
+ targetPort: 8080
+ name: http
+ # We want all pods in the StatefulSet to have their addresses published for
+ # the sake of the other CockroachDB pods even before they're ready, since they
+ # have to be able to talk to each other in order to become ready.
+ publishNotReadyAddresses: true
+ clusterIP: None
+ selector:
+ app: cockroachdb
+---
+apiVersion: policy/v1
+kind: PodDisruptionBudget
+metadata:
+ name: cockroachdb-budget
+ labels:
+ app: cockroachdb
+spec:
+ selector:
+ matchLabels:
+ app: cockroachdb
+ maxUnavailable: 1
+---
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: cockroachdb
+spec:
+ serviceName: "cockroachdb"
+ replicas: 3
+ selector:
+ matchLabels:
+ app: cockroachdb
+ template:
+ metadata:
+ labels:
+ app: cockroachdb
+ spec:
+ serviceAccountName: cockroachdb
+ # Init containers are run only once in the lifetime of a pod, before
+ # it's started up for the first time. It has to exit successfully
+ # before the pod's main containers are allowed to start.
+ initContainers:
+ # The init-certs container sends a certificate signing request to the
+ # kubernetes cluster.
+ # You can see pending requests using: kubectl get csr
+ # CSRs can be approved using: kubectl certificate approve
+ #
+ # All addresses used to contact a node must be specified in the --addresses arg.
+ #
+ # In addition to the node certificate and key, the init-certs entrypoint will symlink
+ # the cluster CA to the certs directory.
+ - name: init-certs
+ image: cockroachdb/cockroach-k8s-request-cert:0.4
+ imagePullPolicy: IfNotPresent
+ command:
+ - "/bin/ash"
+ - "-ecx"
+ - "/request-cert -namespace=${POD_NAMESPACE} -certs-dir=/cockroach-certs -type=node -addresses=localhost,127.0.0.1,$(hostname -f),$(hostname -f|cut -f 1-2 -d '.'),cockroachdb-public,cockroachdb-public.$(hostname -f|cut -f 3- -d '.'),cockroachdb-public.$(hostname -f|cut -f 3-4 -d '.'),cockroachdb-public.$(hostname -f|cut -f 3 -d '.') -symlink-ca-from=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
+ env:
+ - name: POD_NAMESPACE
+ valueFrom:
+ fieldRef:
+ fieldPath: metadata.namespace
+ volumeMounts:
+ - name: certs
+ mountPath: /cockroach-certs
+ affinity:
+ podAntiAffinity:
+ preferredDuringSchedulingIgnoredDuringExecution:
+ - weight: 100
+ podAffinityTerm:
+ labelSelector:
+ matchExpressions:
+ - key: app
+ operator: In
+ values:
+ - cockroachdb
+ topologyKey: kubernetes.io/hostname
+ containers:
+ - name: cockroachdb
+ image: cockroachdb/cockroach:latest
+ imagePullPolicy: IfNotPresent
+ # TODO: Change these to appropriate values for the hardware that you're running. You can see
+ # the resources that can be allocated on each of your Kubernetes nodes by running:
+ # kubectl describe nodes
+ # Note that requests and limits should have identical values.
+ resources:
+ requests:
+ cpu: "2"
+ memory: "8Gi"
+ limits:
+ cpu: "2"
+ memory: "8Gi"
+ ports:
+ - containerPort: 26257
+ name: grpc
+ - containerPort: 8080
+ name: http
+# We recommend that you do not configure a liveness probe on a production environment, as this can impact the availability of production databases.
+# livenessProbe:
+# httpGet:
+# path: "/health"
+# port: http
+# scheme: HTTPS
+# initialDelaySeconds: 30
+# periodSeconds: 5
+ readinessProbe:
+ httpGet:
+ path: "/health?ready=1"
+ port: http
+ scheme: HTTPS
+ initialDelaySeconds: 10
+ periodSeconds: 5
+ failureThreshold: 2
+ volumeMounts:
+ - name: datadir
+ mountPath: /cockroach/cockroach-data
+ - name: certs
+ mountPath: /cockroach/cockroach-certs
+ env:
+ - name: COCKROACH_CHANNEL
+ value: kubernetes-secure
+ - name: GOMAXPROCS
+ valueFrom:
+ resourceFieldRef:
+ resource: limits.cpu
+ divisor: "1"
+ - name: MEMORY_LIMIT_MIB
+ valueFrom:
+ resourceFieldRef:
+ resource: limits.memory
+ divisor: "1Mi"
+ command:
+ - "/bin/bash"
+ - "-ecx"
+ # The use of qualified `hostname -f` is crucial:
+ # Other nodes aren't able to look up the unqualified hostname.
+ # Memory caches are set as a fraction of the pod's memory limit.
+ - exec
+ /cockroach/cockroach
+ start
+ --logtostderr
+ --certs-dir /cockroach/cockroach-certs
+ --advertise-host $(hostname -f)
+ --http-addr 0.0.0.0
+ --join cockroachdb-0.cockroachdb,cockroachdb-1.cockroachdb,cockroachdb-2.cockroachdb
+ --cache $(expr $MEMORY_LIMIT_MIB / 4)MiB
+ --max-sql-memory $(expr $MEMORY_LIMIT_MIB / 4)MiB
+ # No pre-stop hook is required, a SIGTERM plus some time is all that's
+ # needed for graceful shutdown of a node.
+ terminationGracePeriodSeconds: 60
+ volumes:
+ - name: datadir
+ persistentVolumeClaim:
+ claimName: datadir
+ - name: certs
+ emptyDir: {}
+ podManagementPolicy: Parallel
+ updateStrategy:
+ type: RollingUpdate
+ volumeClaimTemplates:
+ - metadata:
+ name: datadir
+ spec:
+ accessModes:
+ - "ReadWriteOnce"
+ resources:
+ requests:
+ storage: 100Gi
diff --git a/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml b/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml
new file mode 100644
index 00000000000..f5623e9f88a
--- /dev/null
+++ b/src/current/files/cockroach/cloud/kubernetes/cockroachdb-statefulset.yaml
@@ -0,0 +1,181 @@
+apiVersion: v1
+kind: Service
+metadata:
+ # This service is meant to be used by clients of the database. It exposes a ClusterIP that will
+ # automatically load balance connections to the different database pods.
+ name: cockroachdb-public
+ labels:
+ app: cockroachdb
+spec:
+ ports:
+ # The main port, served by gRPC, serves Postgres-flavor SQL, internode
+ # traffic and the cli.
+ - port: 26257
+ targetPort: 26257
+ name: grpc
+ # The secondary port serves the UI as well as health and debug endpoints.
+ - port: 8080
+ targetPort: 8080
+ name: http
+ selector:
+ app: cockroachdb
+---
+apiVersion: v1
+kind: Service
+metadata:
+ # This service only exists to create DNS entries for each pod in the stateful
+ # set such that they can resolve each other's IP addresses. It does not
+ # create a load-balanced ClusterIP and should not be used directly by clients
+ # in most circumstances.
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+ annotations:
+ # Use this annotation in addition to the actual publishNotReadyAddresses
+ # field below because the annotation will stop being respected soon but the
+ # field is broken in some versions of Kubernetes:
+ # https://github.com/kubernetes/kubernetes/issues/58662
+ service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
+ # Enable automatic monitoring of all instances when Prometheus is running in the cluster.
+ prometheus.io/scrape: "true"
+ prometheus.io/path: "_status/vars"
+ prometheus.io/port: "8080"
+spec:
+ ports:
+ - port: 26257
+ targetPort: 26257
+ name: grpc
+ - port: 8080
+ targetPort: 8080
+ name: http
+ # We want all pods in the StatefulSet to have their addresses published for
+ # the sake of the other CockroachDB pods even before they're ready, since they
+ # have to be able to talk to each other in order to become ready.
+ publishNotReadyAddresses: true
+ clusterIP: None
+ selector:
+ app: cockroachdb
+---
+apiVersion: policy/v1
+kind: PodDisruptionBudget
+metadata:
+ name: cockroachdb-budget
+ labels:
+ app: cockroachdb
+spec:
+ selector:
+ matchLabels:
+ app: cockroachdb
+ maxUnavailable: 1
+---
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: cockroachdb
+spec:
+ serviceName: "cockroachdb"
+ replicas: 3
+ selector:
+ matchLabels:
+ app: cockroachdb
+ template:
+ metadata:
+ labels:
+ app: cockroachdb
+ spec:
+ affinity:
+ podAntiAffinity:
+ preferredDuringSchedulingIgnoredDuringExecution:
+ - weight: 100
+ podAffinityTerm:
+ labelSelector:
+ matchExpressions:
+ - key: app
+ operator: In
+ values:
+ - cockroachdb
+ topologyKey: kubernetes.io/hostname
+ containers:
+ - name: cockroachdb
+ image: cockroachdb/cockroach:latest
+ imagePullPolicy: IfNotPresent
+ # TODO: Change these to appropriate values for the hardware that you're running. You can see
+ # the resources that can be allocated on each of your Kubernetes nodes by running:
+ # kubectl describe nodes
+ # Note that requests and limits should have identical values.
+ resources:
+ requests:
+ cpu: "2"
+ memory: "8Gi"
+ limits:
+ cpu: "2"
+ memory: "8Gi"
+ ports:
+ - containerPort: 26257
+ name: grpc
+ - containerPort: 8080
+ name: http
+# We recommend that you do not configure a liveness probe on a production environment, as this can impact the availability of production databases.
+# livenessProbe:
+# httpGet:
+# path: "/health"
+# port: http
+# initialDelaySeconds: 30
+# periodSeconds: 5
+ readinessProbe:
+ httpGet:
+ path: "/health?ready=1"
+ port: http
+ initialDelaySeconds: 10
+ periodSeconds: 5
+ failureThreshold: 2
+ volumeMounts:
+ - name: datadir
+ mountPath: /cockroach/cockroach-data
+ env:
+ - name: COCKROACH_CHANNEL
+ value: kubernetes-insecure
+ - name: GOMAXPROCS
+ valueFrom:
+ resourceFieldRef:
+ resource: limits.cpu
+ divisor: "1"
+ - name: MEMORY_LIMIT_MIB
+ valueFrom:
+ resourceFieldRef:
+ resource: limits.memory
+ divisor: "1Mi"
+ command:
+ - "/bin/bash"
+ - "-ecx"
+ # The use of qualified `hostname -f` is crucial:
+ # Other nodes aren't able to look up the unqualified hostname.
+ - exec
+ /cockroach/cockroach
+ start
+ --logtostderr
+ --insecure
+ --advertise-host $(hostname -f)
+ --http-addr 0.0.0.0
+ --join cockroachdb-0.cockroachdb,cockroachdb-1.cockroachdb,cockroachdb-2.cockroachdb
+ --cache $(expr $MEMORY_LIMIT_MIB / 4)MiB
+ --max-sql-memory $(expr $MEMORY_LIMIT_MIB / 4)MiB
+ # No pre-stop hook is required, a SIGTERM plus some time is all that's
+ # needed for graceful shutdown of a node.
+ terminationGracePeriodSeconds: 60
+ volumes:
+ - name: datadir
+ persistentVolumeClaim:
+ claimName: datadir
+ podManagementPolicy: Parallel
+ updateStrategy:
+ type: RollingUpdate
+ volumeClaimTemplates:
+ - metadata:
+ name: datadir
+ spec:
+ accessModes:
+ - "ReadWriteOnce"
+ resources:
+ requests:
+ storage: 100Gi
diff --git a/src/current/files/cockroach/cloud/kubernetes/example-app.yaml b/src/current/files/cockroach/cloud/kubernetes/example-app.yaml
new file mode 100644
index 00000000000..1c358d5eded
--- /dev/null
+++ b/src/current/files/cockroach/cloud/kubernetes/example-app.yaml
@@ -0,0 +1,21 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: example
+spec:
+ replicas: 1
+ selector:
+ matchLabels:
+ app: loadgen
+ template:
+ metadata:
+ labels:
+ app: loadgen
+ spec:
+ containers:
+ - name: loadgen
+ image: cockroachdb/loadgen-kv:0.1
+ imagePullPolicy: IfNotPresent
+ command:
+ - "/kv"
+ - "postgres://root@cockroachdb-public:26257/kv?sslmode=disable"
diff --git a/src/current/files/cockroach/cloud/kubernetes/multiregion/README.md b/src/current/files/cockroach/cloud/kubernetes/multiregion/README.md
new file mode 100644
index 00000000000..57b1fd0b9b9
--- /dev/null
+++ b/src/current/files/cockroach/cloud/kubernetes/multiregion/README.md
@@ -0,0 +1,86 @@
+# Running CockroachDB across multiple Kubernetes clusters (GKE)
+
+The script and configuration files in this directory enable deploying
+CockroachDB across multiple Kubernetes clusters that are spread across different
+geographic regions and hosted on [GKE](https://cloud.google.com/kubernetes-engine). It deploys a CockroachDB
+[StatefulSet](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/)
+into each separate cluster, and links them together using DNS.
+
+To use the configuration provided here, check out this repository (or otherwise
+download a copy of this directory), fill in the constants at the top of
+[setup.py](setup.py) with the relevant information about your Kubernetes
+clusters, optionally make any desired modifications to
+[cockroachdb-statefulset-secure.yaml](cockroachdb-statefulset-secure.yaml) as
+explained in [our Kubernetes performance tuning
+guide](https://www.cockroachlabs.com/docs/stable/kubernetes-performance.html),
+then finally run [setup.py](setup.py).
+
+You should see a lot of output as it does its thing, hopefully ending after
+printing out `job "cluster-init-secure" created`. This implies that everything
+was created successfully, and you should soon see the CockroachDB cluster
+initialized with 3 pods in the "READY" state in each Kubernetes cluster. At this
+point you can manage the StatefulSet in each cluster independently if you so
+desire, scaling up the number of replicas, changing their resource requests, or
+making other modifications as you please.
+
+If anything goes wrong along the way, please let us know via any of the [normal
+troubleshooting
+channels](https://www.cockroachlabs.com/docs/stable/support-resources.html).
+While we believe this creates a highly available, maintainable multi-region
+deployment, it is still pushing the boundaries of how Kubernetes is typically
+used, so feedback and issue reports are very appreciated.
+
+## Limitations
+
+### Pod-to-pod connectivity
+
+The deployment outlined in this directory relies on pod IP addresses being
+routable even across Kubernetes clusters and regions. This achieves optimal
+performance, particularly when compared to alternative solutions that route all packets between clusters through load balancers, but means that it won't work in certain environments.
+
+This requirement is satisfied by clusters deployed in cloud environments such as Google Kubernetes Engine, and
+can also be satisfied by on-prem environments depending on the [Kubernetes networking setup](https://kubernetes.io/docs/concepts/cluster-administration/networking/) used. If you want to test whether your cluster will work, you can run this basic network test:
+
+```shell
+$ kubectl run network-test --image=alpine --restart=Never -- sleep 999999
+pod "network-test" created
+$ kubectl describe pod network-test | grep IP
+IP: THAT-PODS-IP-ADDRESS
+$ kubectl config use-context YOUR-OTHER-CLUSTERS-CONTEXT-HERE
+$ kubectl run -it network-test --image=alpine --restart=Never -- ping THAT-PODS-IP-ADDRESS
+If you don't see a command prompt, try pressing enter.
+64 bytes from 10.12.14.10: seq=1 ttl=62 time=0.570 ms
+64 bytes from 10.12.14.10: seq=2 ttl=62 time=0.449 ms
+64 bytes from 10.12.14.10: seq=3 ttl=62 time=0.635 ms
+64 bytes from 10.12.14.10: seq=4 ttl=62 time=0.722 ms
+64 bytes from 10.12.14.10: seq=5 ttl=62 time=0.504 ms
+...
+```
+
+If the pods can directly connect, you should see successful ping output like the
+above. If they can't, you won't see any successful ping responses. Make sure to
+delete the `network-test` pod in each cluster when you're done!
+
+### Exposing DNS servers to the Internet
+
+As currently configured, the way that the DNS servers from each Kubernetes
+cluster are hooked together is by exposing them via a load balanced IP address
+that's visible to the public Internet. This is because [Google Cloud Platform's Internal Load Balancers do not currently support clients in one region using a load balancer in another region](https://cloud.google.com/compute/docs/load-balancing/internal/#deploying_internal_load_balancing_with_clients_across_vpn_or_interconnect).
+
+None of the services in your Kubernetes cluster will be made accessible, but
+their names could leak out to a motivated attacker. If this is unacceptable,
+please let us know and we can demonstrate other options. [Your voice could also
+help convince Google to allow clients from one region to use an Internal Load
+Balancer in another](https://issuetracker.google.com/issues/111021512),
+eliminating the problem.
+
+## Cleaning up
+
+To remove all the resources created in your clusters by [setup.py](setup.py),
+copy the parameters you provided at the top of [setup.py](setup.py) to the top
+of [teardown.py](teardown.py) and run [teardown.py](teardown.py).
+
+## More information
+
+For more information on running CockroachDB in Kubernetes, please see the [README
+in the parent directory](../README.md).
diff --git a/src/current/files/cockroach/cloud/kubernetes/multiregion/client-secure.yaml b/src/current/files/cockroach/cloud/kubernetes/multiregion/client-secure.yaml
new file mode 100644
index 00000000000..965b5dfcc4e
--- /dev/null
+++ b/src/current/files/cockroach/cloud/kubernetes/multiregion/client-secure.yaml
@@ -0,0 +1,27 @@
+apiVersion: v1
+kind: Pod
+metadata:
+ name: cockroachdb-client-secure
+ labels:
+ app: cockroachdb-client
+spec:
+ serviceAccountName: cockroachdb
+ containers:
+ - name: cockroachdb-client
+ image: cockroachdb/cockroach:latest
+ imagePullPolicy: IfNotPresent
+ volumeMounts:
+ - name: client-certs
+ mountPath: /cockroach-certs
+ # Keep a pod open indefinitely so kubectl exec can be used to get a shell to it
+ # and run cockroach client commands, such as cockroach sql, cockroach node status, etc.
+ command:
+ - sleep
+ - "2147483648" # 2^31
+ # This pod isn't doing anything important, so don't bother waiting to terminate it.
+ terminationGracePeriodSeconds: 0
+ volumes:
+ - name: client-certs
+ secret:
+ secretName: cockroachdb.client.root
+ defaultMode: 256
diff --git a/src/current/files/cockroach/cloud/kubernetes/multiregion/cluster-init-secure.yaml b/src/current/files/cockroach/cloud/kubernetes/multiregion/cluster-init-secure.yaml
new file mode 100644
index 00000000000..a915f2028ec
--- /dev/null
+++ b/src/current/files/cockroach/cloud/kubernetes/multiregion/cluster-init-secure.yaml
@@ -0,0 +1,28 @@
+apiVersion: batch/v1
+kind: Job
+metadata:
+ name: cluster-init-secure
+ labels:
+ app: cockroachdb
+spec:
+ template:
+ spec:
+ serviceAccountName: cockroachdb
+ containers:
+ - name: cluster-init
+ image: cockroachdb/cockroach:latest
+ imagePullPolicy: IfNotPresent
+ volumeMounts:
+ - name: client-certs
+ mountPath: /cockroach-certs
+ command:
+ - "/cockroach/cockroach"
+ - "init"
+ - "--certs-dir=/cockroach-certs"
+ - "--host=cockroachdb-0.cockroachdb"
+ restartPolicy: OnFailure
+ volumes:
+ - name: client-certs
+ secret:
+ secretName: cockroachdb.client.root
+ defaultMode: 256
diff --git a/src/current/files/cockroach/cloud/kubernetes/multiregion/cockroachdb-statefulset-secure.yaml b/src/current/files/cockroach/cloud/kubernetes/multiregion/cockroachdb-statefulset-secure.yaml
new file mode 100644
index 00000000000..c7bde61ef84
--- /dev/null
+++ b/src/current/files/cockroach/cloud/kubernetes/multiregion/cockroachdb-statefulset-secure.yaml
@@ -0,0 +1,248 @@
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+rules:
+- apiGroups:
+ - ""
+ resources:
+ - secrets
+ verbs:
+ - create
+ - get
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+rules:
+- apiGroups:
+ - certificates.k8s.io
+ resources:
+ - certificatesigningrequests
+ verbs:
+ - create
+ - get
+ - watch
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+roleRef:
+ apiGroup: rbac.authorization.k8s.io
+ kind: Role
+ name: cockroachdb
+subjects:
+- kind: ServiceAccount
+ name: cockroachdb
+ namespace: default
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+roleRef:
+ apiGroup: rbac.authorization.k8s.io
+ kind: ClusterRole
+ name: cockroachdb
+subjects:
+- kind: ServiceAccount
+ name: cockroachdb
+ namespace: default
+---
+apiVersion: v1
+kind: Service
+metadata:
+ # This service is meant to be used by clients of the database. It exposes a ClusterIP that will
+ # automatically load balance connections to the different database pods.
+ name: cockroachdb-public
+ labels:
+ app: cockroachdb
+spec:
+ ports:
+ # The main port, served by gRPC, serves Postgres-flavor SQL, internode
+ # traffic and the cli.
+ - port: 26257
+ targetPort: 26257
+ name: grpc
+ # The secondary port serves the UI as well as health and debug endpoints.
+ - port: 8080
+ targetPort: 8080
+ name: http
+ selector:
+ app: cockroachdb
+---
+apiVersion: v1
+kind: Service
+metadata:
+ # This service only exists to create DNS entries for each pod in the stateful
+ # set such that they can resolve each other's IP addresses. It does not
+ # create a load-balanced ClusterIP and should not be used directly by clients
+ # in most circumstances.
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+ annotations:
+ # Use this annotation in addition to the actual publishNotReadyAddresses
+ # field below because the annotation will stop being respected soon but the
+ # field is broken in some versions of Kubernetes:
+ # https://github.com/kubernetes/kubernetes/issues/58662
+ service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
+ # Enable automatic monitoring of all instances when Prometheus is running in the cluster.
+ prometheus.io/scrape: "true"
+ prometheus.io/path: "_status/vars"
+ prometheus.io/port: "8080"
+spec:
+ ports:
+ - port: 26257
+ targetPort: 26257
+ name: grpc
+ - port: 8080
+ targetPort: 8080
+ name: http
+ # We want all pods in the StatefulSet to have their addresses published for
+ # the sake of the other CockroachDB pods even before they're ready, since they
+ # have to be able to talk to each other in order to become ready.
+ publishNotReadyAddresses: true
+ clusterIP: None
+ selector:
+ app: cockroachdb
+---
+apiVersion: policy/v1
+kind: PodDisruptionBudget
+metadata:
+ name: cockroachdb-budget
+ labels:
+ app: cockroachdb
+spec:
+ selector:
+ matchLabels:
+ app: cockroachdb
+ maxUnavailable: 1
+---
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: cockroachdb
+spec:
+ serviceName: "cockroachdb"
+ replicas: 3
+ selector:
+ matchLabels:
+ app: cockroachdb
+ template:
+ metadata:
+ labels:
+ app: cockroachdb
+ spec:
+ serviceAccountName: cockroachdb
+ affinity:
+ podAntiAffinity:
+ preferredDuringSchedulingIgnoredDuringExecution:
+ - weight: 100
+ podAffinityTerm:
+ labelSelector:
+ matchExpressions:
+ - key: app
+ operator: In
+ values:
+ - cockroachdb
+ topologyKey: kubernetes.io/hostname
+ containers:
+ - name: cockroachdb
+ image: cockroachdb/cockroach:latest
+ imagePullPolicy: IfNotPresent
+ ports:
+ - containerPort: 26257
+ name: grpc
+ - containerPort: 8080
+ name: http
+# We recommend that you do not configure a liveness probe on a production environment, as this can impact the availability of production databases.
+# livenessProbe:
+# httpGet:
+# path: "/health"
+# port: http
+# scheme: HTTPS
+# initialDelaySeconds: 30
+# periodSeconds: 5
+ readinessProbe:
+ httpGet:
+ path: "/health?ready=1"
+ port: http
+ scheme: HTTPS
+ initialDelaySeconds: 10
+ periodSeconds: 5
+ failureThreshold: 2
+ volumeMounts:
+ - name: datadir
+ mountPath: /cockroach/cockroach-data
+ - name: certs
+ mountPath: /cockroach/cockroach-certs
+ env:
+ - name: COCKROACH_CHANNEL
+ value: kubernetes-multiregion
+ - name: GOMAXPROCS
+ valueFrom:
+ resourceFieldRef:
+ resource: limits.cpu
+ divisor: "1"
+ - name: MEMORY_LIMIT_MIB
+ valueFrom:
+ resourceFieldRef:
+ resource: limits.memory
+ divisor: "1Mi"
+ command:
+ - "/bin/bash"
+ - "-ecx"
+ # The use of qualified `hostname -f` is crucial:
+ # Other nodes aren't able to look up the unqualified hostname.
+ - exec
+ /cockroach/cockroach
+ start
+ --logtostderr
+ --certs-dir /cockroach/cockroach-certs
+ --advertise-host $(hostname -f)
+ --http-addr 0.0.0.0
+ --join JOINLIST
+ --locality LOCALITYLIST
+ --cache $(expr $MEMORY_LIMIT_MIB / 4)MiB
+ --max-sql-memory $(expr $MEMORY_LIMIT_MIB / 4)MiB
+ # No pre-stop hook is required, a SIGTERM plus some time is all that's
+ # needed for graceful shutdown of a node.
+ terminationGracePeriodSeconds: 60
+ volumes:
+ - name: datadir
+ persistentVolumeClaim:
+ claimName: datadir
+ - name: certs
+ secret:
+ secretName: cockroachdb.node
+ defaultMode: 256
+ podManagementPolicy: Parallel
+ updateStrategy:
+ type: RollingUpdate
+ volumeClaimTemplates:
+ - metadata:
+ name: datadir
+ spec:
+ accessModes:
+ - "ReadWriteOnce"
+ resources:
+ requests:
+ storage: 100Gi
diff --git a/src/current/files/cockroach/cloud/kubernetes/multiregion/dns-lb.yaml b/src/current/files/cockroach/cloud/kubernetes/multiregion/dns-lb.yaml
new file mode 100644
index 00000000000..b70474cbd8f
--- /dev/null
+++ b/src/current/files/cockroach/cloud/kubernetes/multiregion/dns-lb.yaml
@@ -0,0 +1,23 @@
+apiVersion: v1
+kind: Service
+metadata:
+ annotations:
+ # TODO: Check whether AWS/Azure can use internal load balancers. Google
+ # can't, unfortunately.
+ # service.beta.kubernetes.io/aws-load-balancer-internal: "true"
+ # service.beta.kubernetes.io/azure-load-balancer-internal: "true"
+ # cloud.google.com/load-balancer-type: "Internal"
+ labels:
+ k8s-app: kube-dns
+ name: kube-dns-lb
+ namespace: kube-system
+spec:
+ ports:
+ - name: dns
+ port: 53
+ protocol: UDP
+ targetPort: 53
+ selector:
+ k8s-app: kube-dns
+ sessionAffinity: None
+ type: LoadBalancer
diff --git a/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml b/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml
new file mode 100644
index 00000000000..2806594ba3a
--- /dev/null
+++ b/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml
@@ -0,0 +1,286 @@
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+rules:
+- apiGroups:
+ - ""
+ resources:
+ - secrets
+ verbs:
+ - create
+ - get
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+rules:
+- apiGroups:
+ - certificates.k8s.io
+ resources:
+ - certificatesigningrequests
+ verbs:
+ - create
+ - get
+ - watch
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+roleRef:
+ apiGroup: rbac.authorization.k8s.io
+ kind: Role
+ name: cockroachdb
+subjects:
+- kind: ServiceAccount
+ name: cockroachdb
+ namespace: default
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+roleRef:
+ apiGroup: rbac.authorization.k8s.io
+ kind: ClusterRole
+ name: cockroachdb
+subjects:
+- kind: ServiceAccount
+ name: cockroachdb
+ namespace: default
+---
+apiVersion: v1
+kind: Service
+metadata:
+ # This service is meant to be used by clients of the database. It exposes a ClusterIP that will
+ # automatically load balance connections to the different database pods.
+ name: cockroachdb-public
+ labels:
+ app: cockroachdb
+spec:
+ ports:
+ # The main port, served by gRPC, serves Postgres-flavor SQL, internode
+ # traffic and the cli.
+ - port: 26257
+ targetPort: 26257
+ name: grpc
+ # The secondary port serves the UI as well as health and debug endpoints.
+ - port: 8080
+ targetPort: 8080
+ name: http
+ selector:
+ app: cockroachdb
+---
+apiVersion: v1
+kind: Service
+metadata:
+ # This service only exists to create DNS entries for each pod in the stateful
+ # set such that they can resolve each other's IP addresses. It does not
+ # create a load-balanced ClusterIP and should not be used directly by clients
+ # in most circumstances.
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+ annotations:
+ # Use this annotation in addition to the actual publishNotReadyAddresses
+ # field below because the annotation will stop being respected soon but the
+ # field is broken in some versions of Kubernetes:
+ # https://github.com/kubernetes/kubernetes/issues/58662
+ service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
+ # Enable automatic monitoring of all instances when Prometheus is running in the cluster.
+ prometheus.io/scrape: "true"
+ prometheus.io/path: "_status/vars"
+ prometheus.io/port: "8080"
+spec:
+ ports:
+ - port: 26257
+ targetPort: 26257
+ name: grpc
+ - port: 8080
+ targetPort: 8080
+ name: http
+ # We want all pods in the StatefulSet to have their addresses published for
+ # the sake of the other CockroachDB pods even before they're ready, since they
+ # have to be able to talk to each other in order to become ready.
+ publishNotReadyAddresses: true
+ clusterIP: None
+ selector:
+ app: cockroachdb
+---
+apiVersion: policy/v1
+kind: PodDisruptionBudget
+metadata:
+ name: cockroachdb-budget
+ labels:
+ app: cockroachdb
+spec:
+ selector:
+ matchLabels:
+ app: cockroachdb
+ maxUnavailable: 1
+---
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: cockroachdb
+ # TODO: Use this field to specify a namespace other than "default" in which to deploy CockroachDB (e.g., us-east-1).
+ # namespace:
+spec:
+ serviceName: "cockroachdb"
+ replicas: 3
+ selector:
+ matchLabels:
+ app: cockroachdb
+ template:
+ metadata:
+ labels:
+ app: cockroachdb
+ spec:
+ serviceAccountName: cockroachdb
+ affinity:
+ podAntiAffinity:
+ preferredDuringSchedulingIgnoredDuringExecution:
+ - weight: 100
+ podAffinityTerm:
+ labelSelector:
+ matchExpressions:
+ - key: app
+ operator: In
+ values:
+ - cockroachdb
+ topologyKey: kubernetes.io/hostname
+ # This init container is used to determine the availability zones of the Cockroach pods. The AZs are used to define --locality when starting Cockroach nodes.
+ initContainers:
+ - command:
+ - sh
+ - -ecx
+ - |
+ TOKEN=$(curl -X PUT "http://169.254.169.254/latest/api/token" \
+ -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")
+ echo "aws-$(curl -H "X-aws-ec2-metadata-token: $TOKEN" \
+ http://169.254.169.254/latest/meta-data/placement/availability-zone/)" \
+ > /etc/cockroach-env/zone
+ image: byrnedo/alpine-curl:3.20
+ imagePullPolicy: IfNotPresent
+ name: locality-container
+ resources: {}
+ terminationMessagePath: /dev/termination-log
+ terminationMessagePolicy: File
+ volumeMounts:
+ - mountPath: /etc/cockroach-env
+ name: cockroach-env
+ containers:
+ - name: cockroachdb
+ image: cockroachdb/cockroach:latest
+ imagePullPolicy: IfNotPresent
+ # TODO: Change these to appropriate values for the hardware that you're running. You can see
+ # the resources that can be allocated on each of your Kubernetes nodes by running:
+ # kubectl describe nodes
+ # Note that requests and limits should have identical values.
+ resources:
+ requests:
+ cpu: "2"
+ memory: "8Gi"
+ limits:
+ cpu: "2"
+ memory: "8Gi"
+ ports:
+ - containerPort: 26257
+ name: grpc
+ - containerPort: 8080
+ name: http
+# We recommend that you do not configure a liveness probe on a production environment, as this can impact the availability of production databases.
+# livenessProbe:
+# httpGet:
+# path: "/health"
+# port: http
+# scheme: HTTPS
+# initialDelaySeconds: 30
+# periodSeconds: 5
+ readinessProbe:
+ httpGet:
+ path: "/health?ready=1"
+ port: http
+ scheme: HTTPS
+ initialDelaySeconds: 10
+ periodSeconds: 5
+ failureThreshold: 2
+ volumeMounts:
+ - name: datadir
+ mountPath: /cockroach/cockroach-data
+ - name: certs
+ mountPath: /cockroach/cockroach-certs
+ - name: cockroach-env
+ mountPath: /etc/cockroach-env
+ env:
+ - name: COCKROACH_CHANNEL
+ value: kubernetes-multiregion
+ - name: GOMAXPROCS
+ valueFrom:
+ resourceFieldRef:
+ resource: limits.cpu
+ divisor: "1"
+ - name: MEMORY_LIMIT_MIB
+ valueFrom:
+ resourceFieldRef:
+ resource: limits.memory
+ divisor: "1Mi"
+ command:
+ - "/bin/bash"
+ - "-ecx"
+ # The use of qualified `hostname -f` is crucial:
+ # Other nodes aren't able to look up the unqualified hostname.
+ - exec
+ /cockroach/cockroach
+ start
+ --logtostderr
+ --certs-dir /cockroach/cockroach-certs
+ --advertise-host $(hostname -f)
+ --http-addr 0.0.0.0
+ # TODO: Replace the placeholder values in --join and --locality with the namespace of the CockroachDB cluster in each region (e.g., us-east-1).
+ # --join cockroachdb-0.cockroachdb.,cockroachdb-1.cockroachdb.,cockroachdb-2.cockroachdb.,cockroachdb-0.cockroachdb.,cockroachdb-1.cockroachdb.,cockroachdb-2.cockroachdb.,cockroachdb-0.cockroachdb.,cockroachdb-1.cockroachdb.,cockroachdb-2.cockroachdb.
+ # --locality=region=,az=$(cat /etc/cockroach-env/zone),dns=$(hostname -f)
+ --cache $(expr $MEMORY_LIMIT_MIB / 4)MiB
+ --max-sql-memory $(expr $MEMORY_LIMIT_MIB / 4)MiB
+ # No pre-stop hook is required, a SIGTERM plus some time is all that's
+ # needed for graceful shutdown of a node.
+ terminationGracePeriodSeconds: 60
+ volumes:
+ - name: datadir
+ persistentVolumeClaim:
+ claimName: datadir
+ - name: certs
+ secret:
+ secretName: cockroachdb.node
+ defaultMode: 256
+ - name: cockroach-env
+ emptyDir: {}
+ podManagementPolicy: Parallel
+ updateStrategy:
+ type: RollingUpdate
+ volumeClaimTemplates:
+ - metadata:
+ name: datadir
+ spec:
+ accessModes:
+ - "ReadWriteOnce"
+ resources:
+ requests:
+ storage: 100Gi
diff --git a/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/configmap.yaml b/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/configmap.yaml
new file mode 100644
index 00000000000..cb276be2efa
--- /dev/null
+++ b/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/configmap.yaml
@@ -0,0 +1,41 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: coredns
+ namespace: kube-system
+data:
+ Corefile: |
+ .:53 {
+ errors
+ ready
+ health
+ kubernetes cluster.local in-addr.arpa ip6.arpa {
+ pods insecure
+ upstream
+ fallthrough in-addr.arpa ip6.arpa
+ }
+ prometheus :9153
+ forward . /etc/resolv.conf
+ cache 10
+ loop
+ reload
+ loadbalance
+ }
+ .svc.cluster.local:53 { # <---- Modify
+ log
+ errors
+ ready
+ cache 10
+ forward . { # <---- Modify
+ force_tcp # <---- Modify
+ }
+ }
+ .svc.cluster.local:53 { # <---- Modify
+ log
+ errors
+ ready
+ cache 10
+ forward . { # <---- Modify
+ force_tcp # <---- Modify
+ }
+ }
diff --git a/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml b/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml
new file mode 100644
index 00000000000..e80c63f29d3
--- /dev/null
+++ b/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml
@@ -0,0 +1,19 @@
+apiVersion: v1
+kind: Service
+metadata:
+ labels:
+ k8s-app: kube-dns
+ name: cockroachdb-dns-external
+ namespace: kube-system
+ annotations:
+ service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
+spec:
+ ports:
+ - name: dns
+ port: 53
+ protocol: TCP
+ targetPort: 53
+ selector:
+ k8s-app: kube-dns
+ type: LoadBalancer
+ loadBalancerSourceRanges: ["0.0.0.0/0"]
diff --git a/src/current/files/cockroach/cloud/kubernetes/multiregion/example-app-secure.yaml b/src/current/files/cockroach/cloud/kubernetes/multiregion/example-app-secure.yaml
new file mode 100644
index 00000000000..0925dc1b800
--- /dev/null
+++ b/src/current/files/cockroach/cloud/kubernetes/multiregion/example-app-secure.yaml
@@ -0,0 +1,30 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: example-secure
+spec:
+ replicas: 1
+ selector:
+ matchLabels:
+ app: loadgen
+ template:
+ metadata:
+ labels:
+ app: loadgen
+ spec:
+ serviceAccountName: cockroachdb
+ volumes:
+ - name: client-certs
+ secret:
+ secretName: cockroachdb.client.root
+ defaultMode: 256
+ containers:
+ - name: loadgen
+ image: cockroachdb/loadgen-kv:0.1
+ imagePullPolicy: IfNotPresent
+ volumeMounts:
+ - name: client-certs
+ mountPath: /cockroach-certs
+ command:
+ - "/kv"
+ - "postgres://root@cockroachdb-public:26257/kv?sslmode=verify-full&sslcert=/cockroach-certs/client.root.crt&sslkey=/cockroach-certs/client.root.key&sslrootcert=/cockroach-certs/ca.crt"
diff --git a/src/current/files/cockroach/cloud/kubernetes/multiregion/external-name-svc.yaml b/src/current/files/cockroach/cloud/kubernetes/multiregion/external-name-svc.yaml
new file mode 100644
index 00000000000..d45b53b8acc
--- /dev/null
+++ b/src/current/files/cockroach/cloud/kubernetes/multiregion/external-name-svc.yaml
@@ -0,0 +1,64 @@
+# This file contains the definitions needed to expose cockroachdb in a namespace
+# other than the one it's running in.
+# To use this file:
+# 1. Replace "YOUR_ZONE_HERE" in this file with the name of the namespace that
+# cockroachdb is running in in the given cluster.
+# 2. Create a secret containing the certificates in the namespace that you want
+# to expose the service in (the "default" namespace is assumed by the
+# certificate creation commands in setup.py):
+# kubectl create secret generic cockroachdb.client.root --namespace=YOUR_ZONE_HERE --from-file=certs
+# 3. Create the resources in this cluster:
+# kubectl apply -f external-name-svc.yaml
+#
+# After completing these steps, you should be able to access the cockroachdb
+# cluster at the name `cockroachdb-public` in the default Kubernetes namespace
+# (or at the name `cockroachdb-public.default` from any namespace).
+#
+# Note that the ServiceAccount and roles defined below are only needed for
+# accessing the Secret containing the root client certificate. If you are
+# managing client certificates (or passwords) some other way, you can do away
+# with everything in this file other than the Service.
+kind: Service
+apiVersion: v1
+metadata:
+ name: cockroachdb-public
+spec:
+ type: ExternalName
+ externalName: cockroachdb-public.YOUR_ZONE_HERE.svc.cluster.local
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+rules:
+- apiGroups:
+ - ""
+ resources:
+ - secrets
+ verbs:
+ - create
+ - get
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+roleRef:
+ apiGroup: rbac.authorization.k8s.io
+ kind: Role
+ name: cockroachdb
+subjects:
+- kind: ServiceAccount
+ name: cockroachdb
+ namespace: default
diff --git a/src/current/files/cockroach/cloud/kubernetes/multiregion/setup.py b/src/current/files/cockroach/cloud/kubernetes/multiregion/setup.py
new file mode 100644
index 00000000000..0531d47736b
--- /dev/null
+++ b/src/current/files/cockroach/cloud/kubernetes/multiregion/setup.py
@@ -0,0 +1,183 @@
+#!/usr/bin/env python
+
+# Copyright 2018 The Cockroach Authors.
+#
+# Use of this software is governed by the CockroachDB Software License
+# included in the /LICENSE file.
+
+
+import json
+import os
+from subprocess import check_call,check_output
+from sys import exit
+from time import sleep
+
+# Before running the script, fill in appropriate values for all the parameters
+# above the dashed line.
+
+# Fill in the `contexts` map with the zones of your clusters and their
+# corresponding kubectl context names.
+#
+# To get the names of your kubectl "contexts" for each of your clusters, run:
+# kubectl config get-contexts
+#
+# example:
+# contexts = {
+# 'us-central1-a': 'gke_cockroach-alex_us-central1-a_my-cluster',
+# 'us-central1-b': 'gke_cockroach-alex_us-central1-b_my-cluster',
+# 'us-west1-b': 'gke_cockroach-alex_us-west1-b_my-cluster',
+# }
+contexts = {
+}
+
+# Fill in the `regions` map with the zones and corresponding regions of your
+# clusters.
+#
+# Setting regions is optional, but recommended, because it improves cockroach's
+# ability to diversify data placement if you use more than one zone in the same
+# region. If you aren't specifying regions, just leave the map empty.
+#
+# example:
+# regions = {
+# 'us-central1-a': 'us-central1',
+# 'us-central1-b': 'us-central1',
+# 'us-west1-b': 'us-west1',
+# }
+regions = {
+}
+
+# Paths to directories in which to store certificates and generated YAML files.
+certs_dir = './certs'
+ca_key_dir = './my-safe-directory'
+generated_files_dir = './generated'
+
+# Path to the cockroach binary on your local machine that you want to use
+# generate certificates. Defaults to trying to find cockroach in your PATH.
+cockroach_path = 'cockroach'
+
+# ------------------------------------------------------------------------------
+
+# First, do some basic input validation.
+if len(contexts) == 0:
+ exit("must provide at least one Kubernetes cluster in the `contexts` map at the top of the script")
+
+if len(regions) != 0 and len(regions) != len(contexts):
+ exit("regions not specified for all kubectl contexts (%d regions, %d contexts)" % (len(regions), len(contexts)))
+
+try:
+ check_call(["which", cockroach_path])
+except:
+ exit("no binary found at provided path '" + cockroach_path + "'; please put a cockroach binary in your path or change the cockroach_path variable")
+
+for zone, context in contexts.items():
+ try:
+ check_call(['kubectl', 'get', 'pods', '--context', context])
+ except:
+ exit("unable to make basic API call using kubectl context '%s' for cluster in zone '%s'; please check if the context is correct and your Kubernetes cluster is working" % (context, zone))
+
+# Set up the necessary directories and certificates. Ignore errors because they may already exist.
+try:
+ os.mkdir(certs_dir)
+except OSError:
+ pass
+try:
+ os.mkdir(ca_key_dir)
+except OSError:
+ pass
+try:
+ os.mkdir(generated_files_dir)
+except OSError:
+ pass
+
+check_call([cockroach_path, 'cert', 'create-ca', '--certs-dir', certs_dir, '--ca-key', ca_key_dir+'/ca.key'])
+check_call([cockroach_path, 'cert', 'create-client', 'root', '--certs-dir', certs_dir, '--ca-key', ca_key_dir+'/ca.key'])
+
+# For each cluster, create secrets containing the node and client certificates.
+# Note that we create the root client certificate in both the zone namespace
+# and the default namespace so that it's easier for clients in the default
+# namespace to use without additional steps.
+#
+# Also create a load balancer to each cluster's DNS pods.
+for zone, context in contexts.items():
+ check_call(['kubectl', 'create', 'namespace', zone, '--context', context])
+ check_call(['kubectl', 'create', 'secret', 'generic', 'cockroachdb.client.root', '--from-file', certs_dir, '--context', context])
+ check_call(['kubectl', 'create', 'secret', 'generic', 'cockroachdb.client.root', '--namespace', zone, '--from-file', certs_dir, '--context', context])
+ check_call([cockroach_path, 'cert', 'create-node', '--certs-dir', certs_dir, '--ca-key', ca_key_dir+'/ca.key', 'localhost', '127.0.0.1', 'cockroachdb-public', 'cockroachdb-public.default', 'cockroachdb-public.'+zone, 'cockroachdb-public.%s.svc.cluster.local' % (zone), '*.cockroachdb', '*.cockroachdb.'+zone, '*.cockroachdb.%s.svc.cluster.local' % (zone)])
+ check_call(['kubectl', 'create', 'secret', 'generic', 'cockroachdb.node', '--namespace', zone, '--from-file', certs_dir, '--context', context])
+ check_call('rm %s/node.*' % (certs_dir), shell=True)
+
+ check_call(['kubectl', 'apply', '-f', 'dns-lb.yaml', '--context', context])
+
+# Set up each cluster to forward DNS requests for zone-scoped namespaces to the
+# relevant cluster's DNS server, using load balancers in order to create a
+# static IP for each cluster's DNS endpoint.
+dns_ips = dict()
+for zone, context in contexts.items():
+ external_ip = ''
+ while True:
+ external_ip = check_output(['kubectl', 'get', 'svc', 'kube-dns-lb', '--namespace', 'kube-system', '--context', context, '--template', '{{range .status.loadBalancer.ingress}}{{.ip}}{{end}}']).decode('utf-8')
+ if external_ip:
+ break
+ print('Waiting for DNS load balancer IP in %s...' % (zone))
+ sleep(10)
+ print('DNS endpoint for zone %s: %s' % (zone, external_ip))
+ dns_ips[zone] = external_ip
+
+# Update each cluster's DNS configuration with an appropriate configmap. Note
+# that we have to leave the local cluster out of its own configmap to avoid
+# infinite recursion through the load balancer IP. We then have to delete the
+# existing DNS pods in order for the new configuration to take effect.
+for zone, context in contexts.items():
+ remote_dns_ips = dict()
+ for z, ip in dns_ips.items():
+ if z == zone:
+ continue
+ remote_dns_ips[z+'.svc.cluster.local'] = [ip]
+ config_filename = '%s/dns-configmap-%s.yaml' % (generated_files_dir, zone)
+ with open(config_filename, 'w') as f:
+ f.write("""\
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: kube-dns
+ namespace: kube-system
+data:
+ stubDomains: |
+ %s
+""" % (json.dumps(remote_dns_ips)))
+ check_call(['kubectl', 'apply', '-f', config_filename, '--namespace', 'kube-system', '--context', context])
+ check_call(['kubectl', 'delete', 'pods', '-l', 'k8s-app=kube-dns', '--namespace', 'kube-system', '--context', context])
+
+# Create a cockroachdb-public service in the default namespace in each cluster.
+for zone, context in contexts.items():
+ yaml_file = '%s/external-name-svc-%s.yaml' % (generated_files_dir, zone)
+ with open(yaml_file, 'w') as f:
+ check_call(['sed', 's/YOUR_ZONE_HERE/%s/g' % (zone), 'external-name-svc.yaml'], stdout=f)
+ check_call(['kubectl', 'apply', '-f', yaml_file, '--context', context])
+
+# Generate the join string to be used.
+join_addrs = []
+for zone in contexts:
+ for i in range(3):
+ join_addrs.append('cockroachdb-%d.cockroachdb.%s' % (i, zone))
+join_str = ','.join(join_addrs)
+
+# Create the cockroach resources in each cluster.
+for zone, context in contexts.items():
+ if zone in regions:
+ locality = 'region=%s,zone=%s' % (regions[zone], zone)
+ else:
+ locality = 'zone=%s' % (zone)
+ yaml_file = '%s/cockroachdb-statefulset-%s.yaml' % (generated_files_dir, zone)
+ with open(yaml_file, 'w') as f:
+ check_call(['sed', 's/JOINLIST/%s/g;s/LOCALITYLIST/%s/g' % (join_str, locality), 'cockroachdb-statefulset-secure.yaml'], stdout=f)
+ check_call(['kubectl', 'apply', '-f', yaml_file, '--namespace', zone, '--context', context])
+
+# Finally, initialize the cluster.
+print('Sleeping 30 seconds before attempting to initialize cluster to give time for volumes to be created and pods started.')
+sleep(30)
+for zone, context in contexts.items():
+ check_call(['kubectl', 'create', '-f', 'cluster-init-secure.yaml', '--namespace', zone, '--context', context])
+ # We only need run the init command in one zone given that all the zones are
+ # joined together as one cluster.
+ break
diff --git a/src/current/files/cockroach/cloud/kubernetes/multiregion/teardown.py b/src/current/files/cockroach/cloud/kubernetes/multiregion/teardown.py
new file mode 100644
index 00000000000..765ca3e9bbb
--- /dev/null
+++ b/src/current/files/cockroach/cloud/kubernetes/multiregion/teardown.py
@@ -0,0 +1,53 @@
+#!/usr/bin/env python
+
+# Copyright 2018 The Cockroach Authors.
+#
+# Use of this software is governed by the CockroachDB Software License
+# included in the /LICENSE file.
+
+
+from shutil import rmtree
+from subprocess import call
+
+# Before running the script, fill in appropriate values for all the parameters
+# above the dashed line. You should use the same values when tearing down a
+# cluster that you used when setting it up.
+
+# To get the names of your kubectl "contexts" for each of your clusters, run:
+# kubectl config get-contexts
+contexts = {
+ 'us-central1-a': 'gke_cockroach-alex_us-central1-a_dns',
+ 'us-central1-b': 'gke_cockroach-alex_us-central1-b_dns',
+ 'us-west1-b': 'gke_cockroach-alex_us-west1-b_dns',
+}
+
+certs_dir = './certs'
+ca_key_dir = './my-safe-directory'
+generated_files_dir = './generated'
+
+# ------------------------------------------------------------------------------
+
+# Delete each cluster's special zone-scoped namespace, which transitively
+# deletes all resources that were created in the namespace, along with the few
+# other resources we created that weren't in that namespace
+for zone, context in contexts.items():
+ call(['kubectl', 'delete', 'namespace', zone, '--context', context])
+ call(['kubectl', 'delete', 'secret', 'cockroachdb.client.root', '--context', context])
+ call(['kubectl', 'delete', '-f', 'external-name-svc.yaml', '--context', context])
+ call(['kubectl', 'delete', '-f', 'dns-lb.yaml', '--context', context])
+ call(['kubectl', 'delete', 'configmap', 'kube-dns', '--namespace', 'kube-system', '--context', context])
+ # Restart the DNS pods to clear out our stub-domains configuration.
+ call(['kubectl', 'delete', 'pods', '-l', 'k8s-app=kube-dns', '--namespace', 'kube-system', '--context', context])
+
+try:
+ rmtree(certs_dir)
+except OSError:
+ pass
+try:
+ rmtree(ca_key_dir)
+except OSError:
+ pass
+try:
+ rmtree(generated_files_dir)
+except OSError:
+ pass
diff --git a/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml b/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml
new file mode 100644
index 00000000000..1a678f19daf
--- /dev/null
+++ b/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-insecure.yaml
@@ -0,0 +1,215 @@
+# This configuration file sets up an insecure StatefulSet running CockroachDB with
+# tweaks to make it more performant than our default configuration files. All
+# changes from the default insecure configuration have been marked with a comment
+# starting with "NOTE" or "TODO".
+#
+# Beware that this configuration is quite insecure. By default, it will make
+# CockroachDB accessible on port 26257 on your Kubernetes nodes' network
+# interfaces, meaning that if your nodes are reachable from the Internet, then
+# this CockroachDB cluster will be too. To disable this behavior, remove the
+# `hostNetwork` configuration field below.
+#
+# To use this file, customize all the parts labeled "TODO" before running:
+# kubectl create -f cockroachdb-statefulset-insecure.yaml
+#
+# You will then have to initialize the cluster as described in the parent
+# directory's README.md file.
+#
+# If you don't see any pods being created, it's possible that your cluster was
+# not able to meet the resource requests asked for, whether it was the amount
+# of CPU, memory, or disk or the disk type. To find information about why pods
+# haven't been created, you can run:
+# kubectl get events
+#
+# For more information on improving CockroachDB performance in Kubernetes, see
+# our docs:
+# https://www.cockroachlabs.com/docs/stable/kubernetes-performance.html
+apiVersion: v1
+kind: Service
+metadata:
+ # This service is meant to be used by clients of the database. It exposes a ClusterIP that will
+ # automatically load balance connections to the different database pods.
+ name: cockroachdb-public
+ labels:
+ app: cockroachdb
+spec:
+ ports:
+ # The main port, served by gRPC, serves Postgres-flavor SQL, internode
+ # traffic and the cli.
+ - port: 26257
+ targetPort: 26257
+ name: grpc
+ # The secondary port serves the UI as well as health and debug endpoints.
+ - port: 8080
+ targetPort: 8080
+ name: http
+ selector:
+ app: cockroachdb
+---
+apiVersion: v1
+kind: Service
+metadata:
+ # This service only exists to create DNS entries for each pod in the stateful
+ # set such that they can resolve each other's IP addresses. It does not
+ # create a load-balanced ClusterIP and should not be used directly by clients
+ # in most circumstances.
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+ annotations:
+ # Use this annotation in addition to the actual publishNotReadyAddresses
+ # field below because the annotation will stop being respected soon but the
+ # field is broken in some versions of Kubernetes:
+ # https://github.com/kubernetes/kubernetes/issues/58662
+ service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
+ # Enable automatic monitoring of all instances when Prometheus is running in the cluster.
+ prometheus.io/scrape: "true"
+ prometheus.io/path: "_status/vars"
+ prometheus.io/port: "8080"
+spec:
+ ports:
+ - port: 26257
+ targetPort: 26257
+ name: grpc
+ - port: 8080
+ targetPort: 8080
+ name: http
+ # We want all pods in the StatefulSet to have their addresses published for
+ # the sake of the other CockroachDB pods even before they're ready, since they
+ # have to be able to talk to each other in order to become ready.
+ publishNotReadyAddresses: true
+ clusterIP: None
+ selector:
+ app: cockroachdb
+---
+apiVersion: policy/v1
+kind: PodDisruptionBudget
+metadata:
+ name: cockroachdb-budget
+ labels:
+ app: cockroachdb
+spec:
+ selector:
+ matchLabels:
+ app: cockroachdb
+ maxUnavailable: 1
+---
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: cockroachdb
+spec:
+ serviceName: "cockroachdb"
+ replicas: 3
+ selector:
+ matchLabels:
+ app: cockroachdb
+ template:
+ metadata:
+ labels:
+ app: cockroachdb
+ spec:
+ # NOTE: Running with `hostNetwork: true` means that CockroachDB will use
+ # the host machines' IP address and hostname, and that nothing else on
+ # the machines will be able to use the same ports. This means that only 1
+ # CockroachDB pod will ever be schedulable on the same machine, because
+ # otherwise their ports would conflict.
+ #
+ # If your client pods generate a lot of network traffic to and from the
+ # CockroachDB cluster, you may see a benefit to doing the same thing in
+ # their configurations.
+ hostNetwork: true
+ dnsPolicy: ClusterFirstWithHostNet
+ # NOTE: If you are running clients that generate heavy load, you may find
+ # it useful to copy this anti-affinity policy into the client pods'
+ # configurations as well to avoid running them on the same machines as
+ # CockroachDB and interfering with each other's performance.
+ affinity:
+ podAntiAffinity:
+ preferredDuringSchedulingIgnoredDuringExecution:
+ - weight: 100
+ podAffinityTerm:
+ labelSelector:
+ matchExpressions:
+ - key: app
+ operator: In
+ values:
+ - cockroachdb
+ topologyKey: kubernetes.io/hostname
+ containers:
+ - name: cockroachdb
+ # NOTE: Always use the most recent version of CockroachDB for the best
+ # performance and reliability.
+ image: cockroachdb/cockroach:latest
+ imagePullPolicy: IfNotPresent
+ # TODO: Change these to appropriate values for the hardware that you're running. You can see
+ # the resources that can be allocated on each of your Kubernetes nodes by running:
+ # kubectl describe nodes
+ # Note that requests and limits should have identical values.
+ resources:
+ requests:
+ cpu: "2"
+ memory: "8Gi"
+ limits:
+ cpu: "2"
+ memory: "8Gi"
+ ports:
+ - containerPort: 26257
+ name: grpc
+ - containerPort: 8080
+ name: http
+# We recommend that you do not configure a liveness probe on a production environment, as this can impact the availability of production databases.
+# livenessProbe:
+# httpGet:
+# path: "/health"
+# port: http
+# initialDelaySeconds: 30
+# periodSeconds: 5
+ readinessProbe:
+ httpGet:
+ path: "/health?ready=1"
+ port: http
+ initialDelaySeconds: 10
+ periodSeconds: 5
+ failureThreshold: 2
+ volumeMounts:
+ - name: datadir
+ mountPath: /cockroach/cockroach-data
+ env:
+ - name: COCKROACH_CHANNEL
+ value: kubernetes-insecure
+ command:
+ - "/bin/bash"
+ - "-ecx"
+ # The use of qualified `hostname -f` is crucial:
+ # Other nodes aren't able to look up the unqualified hostname.
+ - "exec /cockroach/cockroach start --logtostderr --insecure --advertise-host $(hostname -f) --http-addr 0.0.0.0 --join cockroachdb-0.cockroachdb,cockroachdb-1.cockroachdb,cockroachdb-2.cockroachdb --cache 25% --max-sql-memory 25%"
+ # No pre-stop hook is required, a SIGTERM plus some time is all that's
+ # needed for graceful shutdown of a node.
+ terminationGracePeriodSeconds: 60
+ volumes:
+ - name: datadir
+ persistentVolumeClaim:
+ claimName: datadir
+ podManagementPolicy: Parallel
+ updateStrategy:
+ type: RollingUpdate
+ volumeClaimTemplates:
+ - metadata:
+ name: datadir
+ spec:
+ accessModes:
+ - "ReadWriteOnce"
+ # TODO: This specifically asks for a storage class with the name "ssd". A
+ # storage class of this name doesn't exist by default. See our docs for
+ # more information on how to create an optimized storage class for use here:
+ # https://www.cockroachlabs.com/docs/stable/kubernetes-performance.html#disk-type
+ storageClassName: ssd
+ resources:
+ requests:
+ # TODO: This asks for a fairly large disk by default because on
+ # certain popular clouds there is a direct correlation between disk
+ # size and the IOPS provisioned to the disk. Change this as necessary
+ # to suit your needs, but be aware that smaller disks will typically
+ # mean worse performance.
+ storage: 1024Gi
diff --git a/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-secure.yaml b/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-secure.yaml
new file mode 100644
index 00000000000..44f48abbf36
--- /dev/null
+++ b/src/current/files/cockroach/cloud/kubernetes/performance/cockroachdb-statefulset-secure.yaml
@@ -0,0 +1,312 @@
+# This configuration file sets up a secure StatefulSet running CockroachDB with
+# tweaks to make it more performant than our default configuration files. All
+# changes from the default secure configuration have been marked with a comment
+# starting with "NOTE" or "TODO".
+#
+# To use it, customize all the parts of the file labeled "TODO" before running:
+# kubectl create -f cockroachdb-statefulset-secure.yaml
+#
+# You will then have to approve certificate-signing requests and initialize the
+# cluster as described in the parent directory's README.md file.
+#
+# If you don't see any pods being created, it's possible that your cluster was
+# not able to meet the resource requests asked for, whether it was the amount
+# of CPU, memory, or disk or the disk type. To find information about why pods
+# haven't been created, you can run:
+# kubectl get events
+#
+# For more information on improving CockroachDB performance in Kubernetes, see
+# our docs:
+# https://www.cockroachlabs.com/docs/stable/kubernetes-performance.html
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+rules:
+- apiGroups:
+ - ""
+ resources:
+ - secrets
+ verbs:
+ - create
+ - get
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+rules:
+- apiGroups:
+ - certificates.k8s.io
+ resources:
+ - certificatesigningrequests
+ verbs:
+ - create
+ - get
+ - watch
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+roleRef:
+ apiGroup: rbac.authorization.k8s.io
+ kind: Role
+ name: cockroachdb
+subjects:
+- kind: ServiceAccount
+ name: cockroachdb
+ namespace: default
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+roleRef:
+ apiGroup: rbac.authorization.k8s.io
+ kind: ClusterRole
+ name: cockroachdb
+subjects:
+- kind: ServiceAccount
+ name: cockroachdb
+ namespace: default
+---
+apiVersion: v1
+kind: Service
+metadata:
+ # This service is meant to be used by clients of the database. It exposes a ClusterIP that will
+ # automatically load balance connections to the different database pods.
+ name: cockroachdb-public
+ labels:
+ app: cockroachdb
+spec:
+ ports:
+ # The main port, served by gRPC, serves Postgres-flavor SQL, internode
+ # traffic and the cli.
+ - port: 26257
+ targetPort: 26257
+ name: grpc
+ # The secondary port serves the UI as well as health and debug endpoints.
+ - port: 8080
+ targetPort: 8080
+ name: http
+ selector:
+ app: cockroachdb
+---
+apiVersion: v1
+kind: Service
+metadata:
+ # This service only exists to create DNS entries for each pod in the stateful
+ # set such that they can resolve each other's IP addresses. It does not
+ # create a load-balanced ClusterIP and should not be used directly by clients
+ # in most circumstances.
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+ annotations:
+ # Use this annotation in addition to the actual publishNotReadyAddresses
+ # field below because the annotation will stop being respected soon but the
+ # field is broken in some versions of Kubernetes:
+ # https://github.com/kubernetes/kubernetes/issues/58662
+ service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
+ # Enable automatic monitoring of all instances when Prometheus is running in the cluster.
+ prometheus.io/scrape: "true"
+ prometheus.io/path: "_status/vars"
+ prometheus.io/port: "8080"
+spec:
+ ports:
+ - port: 26257
+ targetPort: 26257
+ name: grpc
+ - port: 8080
+ targetPort: 8080
+ name: http
+ # We want all pods in the StatefulSet to have their addresses published for
+ # the sake of the other CockroachDB pods even before they're ready, since they
+ # have to be able to talk to each other in order to become ready.
+ publishNotReadyAddresses: true
+ clusterIP: None
+ selector:
+ app: cockroachdb
+---
+apiVersion: policy/v1
+kind: PodDisruptionBudget
+metadata:
+ name: cockroachdb-budget
+ labels:
+ app: cockroachdb
+spec:
+ selector:
+ matchLabels:
+ app: cockroachdb
+ maxUnavailable: 1
+---
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: cockroachdb
+spec:
+ serviceName: "cockroachdb"
+ replicas: 3
+ selector:
+ matchLabels:
+ app: cockroachdb
+ template:
+ metadata:
+ labels:
+ app: cockroachdb
+ spec:
+ serviceAccountName: cockroachdb
+ # NOTE: Running with `hostNetwork: true` means that CockroachDB will use
+ # the host machines' IP address and hostname, and that nothing else on
+ # the machines will be able to use the same ports. This means that only 1
+ # CockroachDB pod will ever be schedulable on the same machine, because
+ # otherwise their ports would conflict.
+ #
+ # If your client pods generate a lot of network traffic to and from the
+ # CockroachDB cluster, you may see a benefit to doing the same thing in
+ # their configurations.
+ hostNetwork: true
+ dnsPolicy: ClusterFirstWithHostNet
+ # Init containers are run only once in the lifetime of a pod, before
+ # it's started up for the first time. It has to exit successfully
+ # before the pod's main containers are allowed to start.
+ initContainers:
+ # The init-certs container sends a certificate signing request to the
+ # kubernetes cluster.
+ # You can see pending requests using: kubectl get csr
+ # CSRs can be approved using: kubectl certificate approve
+ #
+ # All addresses used to contact a node must be specified in the --addresses arg.
+ #
+ # In addition to the node certificate and key, the init-certs entrypoint will symlink
+ # the cluster CA to the certs directory.
+ - name: init-certs
+ image: cockroachdb/cockroach-k8s-request-cert:0.4
+ imagePullPolicy: IfNotPresent
+ command:
+ - "/bin/ash"
+ - "-ecx"
+ - "/request-cert -namespace=${POD_NAMESPACE} -certs-dir=/cockroach-certs -type=node -addresses=localhost,127.0.0.1,$(hostname -f),$(hostname -f|cut -f 1-2 -d '.'),cockroachdb-public,cockroachdb-public.$(hostname -f|cut -f 3- -d '.'),cockroachdb-public.$(hostname -f|cut -f 3-4 -d '.'),cockroachdb-public.$(hostname -f|cut -f 3 -d '.') -symlink-ca-from=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
+ env:
+ - name: POD_NAMESPACE
+ valueFrom:
+ fieldRef:
+ fieldPath: metadata.namespace
+ volumeMounts:
+ - name: certs
+ mountPath: /cockroach-certs
+ # NOTE: If you are running clients that generate heavy load, you may find
+ # it useful to copy this anti-affinity policy into the client pods'
+ # configurations as well to avoid running them on the same machines as
+ # CockroachDB and interfering with each other's performance.
+ affinity:
+ podAntiAffinity:
+ preferredDuringSchedulingIgnoredDuringExecution:
+ - weight: 100
+ podAffinityTerm:
+ labelSelector:
+ matchExpressions:
+ - key: app
+ operator: In
+ values:
+ - cockroachdb
+ topologyKey: kubernetes.io/hostname
+ containers:
+ - name: cockroachdb
+ # NOTE: Always use the most recent version of CockroachDB for the best
+ # performance and reliability.
+ image: cockroachdb/cockroach:latest
+ imagePullPolicy: IfNotPresent
+ # TODO: Change these to appropriate values for the hardware that you're running. You can see
+ # the resources that can be allocated on each of your Kubernetes nodes by running:
+ # kubectl describe nodes
+ # Note that requests and limits should have identical values.
+ resources:
+ requests:
+ cpu: "2"
+ memory: "8Gi"
+ limits:
+ cpu: "2"
+ memory: "8Gi"
+ ports:
+ - containerPort: 26257
+ name: grpc
+ - containerPort: 8080
+ name: http
+# We recommend that you do not configure a liveness probe on a production environment, as this can impact the availability of production databases.
+# livenessProbe:
+# httpGet:
+# path: "/health"
+# port: http
+# scheme: HTTPS
+# initialDelaySeconds: 30
+# periodSeconds: 5
+ readinessProbe:
+ httpGet:
+ path: "/health?ready=1"
+ port: http
+ scheme: HTTPS
+ initialDelaySeconds: 10
+ periodSeconds: 5
+ failureThreshold: 2
+ volumeMounts:
+ - name: datadir
+ mountPath: /cockroach/cockroach-data
+ - name: certs
+ mountPath: /cockroach/cockroach-certs
+ env:
+ - name: COCKROACH_CHANNEL
+ value: kubernetes-secure
+ command:
+ - "/bin/bash"
+ - "-ecx"
+ # The use of qualified `hostname -f` is crucial:
+ # Other nodes aren't able to look up the unqualified hostname.
+ - "exec /cockroach/cockroach start --logtostderr --certs-dir /cockroach/cockroach-certs --advertise-host $(hostname -f) --http-addr 0.0.0.0 --join cockroachdb-0.cockroachdb,cockroachdb-1.cockroachdb,cockroachdb-2.cockroachdb --cache 25% --max-sql-memory 25%"
+ # No pre-stop hook is required, a SIGTERM plus some time is all that's
+ # needed for graceful shutdown of a node.
+ terminationGracePeriodSeconds: 60
+ volumes:
+ - name: datadir
+ persistentVolumeClaim:
+ claimName: datadir
+ - name: certs
+ emptyDir: {}
+ podManagementPolicy: Parallel
+ updateStrategy:
+ type: RollingUpdate
+ volumeClaimTemplates:
+ - metadata:
+ name: datadir
+ spec:
+ accessModes:
+ - "ReadWriteOnce"
+ # TODO: This specifically asks for a storage class with the name "ssd". A
+ # storage class of this name doesn't exist by default. See our docs for
+ # more information on how to create an optimized storage class for use here:
+ # https://www.cockroachlabs.com/docs/stable/kubernetes-performance.html#disk-type
+ storageClassName: ssd
+ resources:
+ requests:
+ # TODO: This asks for a fairly large disk by default because on
+ # certain popular clouds there is a direct correlation between disk
+ # size and the IOPS provisioned to the disk. Change this as necessary
+ # to suit your needs, but be aware that smaller disks will typically
+ # mean worse performance.
+ storage: 1024Gi
diff --git a/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml b/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml
new file mode 100644
index 00000000000..275dbe1b81b
--- /dev/null
+++ b/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml
@@ -0,0 +1,205 @@
+# GENERATED FILE - DO NOT EDIT
+apiVersion: monitoring.coreos.com/v1
+kind: PrometheusRule
+metadata:
+ labels:
+ app: cockroachdb
+ prometheus: cockroachdb
+ role: alert-rules
+ name: prometheus-cockroachdb-rules
+spec:
+ groups:
+ - name: rules/dummy.rules
+ rules:
+ - alert: TestAlertManager
+ expr: vector(1)
+ - name: rules/aggregation.rules
+ rules:
+ - expr: sum without(store) (capacity{job="cockroachdb"})
+ record: node:capacity
+ - expr: sum without(instance) (node:capacity{job="cockroachdb"})
+ record: cluster:capacity
+ - expr: sum without(store) (capacity_available{job="cockroachdb"})
+ record: node:capacity_available
+ - expr: sum without(instance) (node:capacity_available{job="cockroachdb"})
+ record: cluster:capacity_available
+ - expr: capacity_available{job="cockroachdb"} / capacity{job="cockroachdb"}
+ record: capacity_available:ratio
+ - expr: node:capacity_available{job="cockroachdb"} / node:capacity{job="cockroachdb"}
+ record: node:capacity_available:ratio
+ - expr: cluster:capacity_available{job="cockroachdb"} / cluster:capacity{job="cockroachdb"}
+ record: cluster:capacity_available:ratio
+ - expr: rate(txn_durations_bucket{job="cockroachdb"}[1m])
+ record: txn_durations_bucket:rate1m
+ - expr: histogram_quantile(0.5, txn_durations_bucket:rate1m)
+ record: txn_durations:rate1m:quantile_50
+ - expr: histogram_quantile(0.75, txn_durations_bucket:rate1m)
+ record: txn_durations:rate1m:quantile_75
+ - expr: histogram_quantile(0.9, txn_durations_bucket:rate1m)
+ record: txn_durations:rate1m:quantile_90
+ - expr: histogram_quantile(0.95, txn_durations_bucket:rate1m)
+ record: txn_durations:rate1m:quantile_95
+ - expr: histogram_quantile(0.99, txn_durations_bucket:rate1m)
+ record: txn_durations:rate1m:quantile_99
+ - expr: rate(exec_latency_bucket{job="cockroachdb"}[1m])
+ record: exec_latency_bucket:rate1m
+ - expr: histogram_quantile(0.5, exec_latency_bucket:rate1m)
+ record: exec_latency:rate1m:quantile_50
+ - expr: histogram_quantile(0.75, exec_latency_bucket:rate1m)
+ record: exec_latency:rate1m:quantile_75
+ - expr: histogram_quantile(0.9, exec_latency_bucket:rate1m)
+ record: exec_latency:rate1m:quantile_90
+ - expr: histogram_quantile(0.95, exec_latency_bucket:rate1m)
+ record: exec_latency:rate1m:quantile_95
+ - expr: histogram_quantile(0.99, exec_latency_bucket:rate1m)
+ record: exec_latency:rate1m:quantile_99
+ - expr: rate(round_trip_latency_bucket{job="cockroachdb"}[1m])
+ record: round_trip_latency_bucket:rate1m
+ - expr: histogram_quantile(0.5, round_trip_latency_bucket:rate1m)
+ record: round_trip_latency:rate1m:quantile_50
+ - expr: histogram_quantile(0.75, round_trip_latency_bucket:rate1m)
+ record: round_trip_latency:rate1m:quantile_75
+ - expr: histogram_quantile(0.9, round_trip_latency_bucket:rate1m)
+ record: round_trip_latency:rate1m:quantile_90
+ - expr: histogram_quantile(0.95, round_trip_latency_bucket:rate1m)
+ record: round_trip_latency:rate1m:quantile_95
+ - expr: histogram_quantile(0.99, round_trip_latency_bucket:rate1m)
+ record: round_trip_latency:rate1m:quantile_99
+ - expr: rate(sql_exec_latency_bucket{job="cockroachdb"}[1m])
+ record: sql_exec_latency_bucket:rate1m
+ - expr: histogram_quantile(0.5, sql_exec_latency_bucket:rate1m)
+ record: sql_exec_latency:rate1m:quantile_50
+ - expr: histogram_quantile(0.75, sql_exec_latency_bucket:rate1m)
+ record: sql_exec_latency:rate1m:quantile_75
+ - expr: histogram_quantile(0.9, sql_exec_latency_bucket:rate1m)
+ record: sql_exec_latency:rate1m:quantile_90
+ - expr: histogram_quantile(0.95, sql_exec_latency_bucket:rate1m)
+ record: sql_exec_latency:rate1m:quantile_95
+ - expr: histogram_quantile(0.99, sql_exec_latency_bucket:rate1m)
+ record: sql_exec_latency:rate1m:quantile_99
+ - expr: rate(raft_process_logcommit_latency_bucket{job="cockroachdb"}[1m])
+ record: raft_process_logcommit_latency_bucket:rate1m
+ - expr: histogram_quantile(0.5, raft_process_logcommit_latency_bucket:rate1m)
+ record: raft_process_logcommit_latency:rate1m:quantile_50
+ - expr: histogram_quantile(0.75, raft_process_logcommit_latency_bucket:rate1m)
+ record: raft_process_logcommit_latency:rate1m:quantile_75
+ - expr: histogram_quantile(0.9, raft_process_logcommit_latency_bucket:rate1m)
+ record: raft_process_logcommit_latency:rate1m:quantile_90
+ - expr: histogram_quantile(0.95, raft_process_logcommit_latency_bucket:rate1m)
+ record: raft_process_logcommit_latency:rate1m:quantile_95
+ - expr: histogram_quantile(0.99, raft_process_logcommit_latency_bucket:rate1m)
+ record: raft_process_logcommit_latency:rate1m:quantile_99
+ - expr: rate(raft_process_commandcommit_latency_bucket{job="cockroachdb"}[1m])
+ record: raft_process_commandcommit_latency_bucket:rate1m
+ - expr: histogram_quantile(0.5, raft_process_commandcommit_latency_bucket:rate1m)
+ record: raft_process_commandcommit_latency:rate1m:quantile_50
+ - expr: histogram_quantile(0.75, raft_process_commandcommit_latency_bucket:rate1m)
+ record: raft_process_commandcommit_latency:rate1m:quantile_75
+ - expr: histogram_quantile(0.9, raft_process_commandcommit_latency_bucket:rate1m)
+ record: raft_process_commandcommit_latency:rate1m:quantile_90
+ - expr: histogram_quantile(0.95, raft_process_commandcommit_latency_bucket:rate1m)
+ record: raft_process_commandcommit_latency:rate1m:quantile_95
+ - expr: histogram_quantile(0.99, raft_process_commandcommit_latency_bucket:rate1m)
+ record: raft_process_commandcommit_latency:rate1m:quantile_99
+ - name: rules/alerts.rules
+ rules:
+ - alert: InstanceDown
+ annotations:
+ description: '{{ $labels.instance }} for cluster {{ $labels.cluster }} has
+ been down for more than 5 minutes.'
+ summary: Instance {{ $labels.instance }} down
+ expr: up{job="cockroachdb"} == 0
+ for: 5m
+ - alert: InstanceDead
+ annotations:
+ description: '{{ $labels.instance }} for cluster {{ $labels.cluster }} has
+ been down for more than 15 minutes.'
+ summary: Instance {{ $labels.instance }} dead
+ expr: up{job="cockroachdb"} == 0
+ for: 15m
+ - alert: InstanceRestart
+ annotations:
+ description: '{{ $labels.instance }} for cluster {{ $labels.cluster }} restarted
+ {{ $value }} time(s) in 10m'
+ summary: Instance {{ $labels.instance }} restarted
+ expr: resets(sys_uptime{job="cockroachdb"}[10m]) > 0 and resets(sys_uptime{job="cockroachdb"}[10m])
+ < 5
+ - alert: InstanceFlapping
+ annotations:
+ description: '{{ $labels.instance }} for cluster {{ $labels.cluster }} restarted
+ {{ $value }} time(s) in 10m'
+ summary: Instance {{ $labels.instance }} flapping
+ expr: resets(sys_uptime{job="cockroachdb"}[10m]) > 5
+ - alert: LivenessMismatch
+ annotations:
+ description: Prometheus and {{ $labels.instance }} disagree on liveness
+ summary: Liveness mismatch for {{ $labels.instance }}
+ expr: (liveness_livenodes{job="cockroachdb"}) != ignoring(instance) group_left()
+ (count by(cluster, job) (up{job="cockroachdb"} == 1))
+ for: 5m
+ labels:
+ severity: testing
+ - alert: VersionMismatch
+ annotations:
+ description: Cluster {{ $labels.cluster }} running {{ $value }} different
+ versions
+ summary: Binary version mismatch on {{ $labels.cluster }}
+ expr: count by(cluster) (count_values by(tag, cluster) ("version", build_timestamp{job="cockroachdb"}))
+ > 1
+ for: 30m
+ - alert: StoreDiskLow
+ annotations:
+ summary: Store {{ $labels.store }} on node {{ $labels.instance }} at {{ $value
+ }} available disk fraction
+ expr: capacity_available:ratio{job="cockroachdb"} < 0.15
+ - alert: ClusterDiskLow
+ annotations:
+ summary: Cluster {{ $labels.cluster }} at {{ $value }} available disk fraction
+ expr: cluster:capacity_available:ratio{job="cockroachdb"} < 0.2
+ - alert: ZeroSQLQps
+ annotations:
+ summary: Instance {{ $labels.instance }} has SQL connections but no queries
+ expr: sql_conns{job="cockroachdb"} > 0 and rate(sql_query_count{job="cockroachdb"}[5m])
+ == 0
+ for: 10m
+ - alert: UnavailableRanges
+ annotations:
+ summary: Instance {{ $labels.instance }} has {{ $value }} unavailable ranges
+ expr: (sum by(instance, cluster) (ranges_unavailable{job="cockroachdb"})) >
+ 0
+ for: 10m
+ labels:
+ severity: testing
+ - alert: NoLeaseRanges
+ annotations:
+ summary: Instance {{ $labels.instance }} has {{ $value }} ranges without leases
+ expr: (sum by(instance, cluster) (replicas_leaders_not_leaseholders{job="cockroachdb"}))
+ > 0
+ for: 10m
+ labels:
+ severity: testing
+ - alert: CACertificateExpiresSoon
+ annotations:
+ summary: CA certificate for {{ $labels.instance }} expires in less than a
+ year
+ expr: (security_certificate_expiration_ca{job="cockroachdb"} > 0) and (security_certificate_expiration_ca{job="cockroachdb"}
+ - time()) < 86400 * 366
+ labels:
+ frequency: daily
+ - alert: NodeCertificateExpiresSoon
+ annotations:
+ summary: Node certificate for {{ $labels.instance }} expires in less than
+ six months
+ expr: (security_certificate_expiration_node{job="cockroachdb"} > 0) and (security_certificate_expiration_node{job="cockroachdb"}
+ - time()) < 86400 * 183
+ labels:
+ frequency: daily
+ - alert: HighOpenFDCount
+ annotations:
+ summary: 'Too many open file descriptors on {{ $labels.instance }}: {{ $value
+ }} fraction used'
+ expr: sys_fd_open{job="cockroachdb"} / sys_fd_softlimit{job="cockroachdb"} >
+ 0.8
+ for: 10m
+ labels:
+ severity: testing
diff --git a/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager-config.yaml b/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager-config.yaml
new file mode 100644
index 00000000000..b558d5f6f6d
--- /dev/null
+++ b/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager-config.yaml
@@ -0,0 +1,14 @@
+global:
+ resolve_timeout: 5m
+route:
+ group_by: ['job']
+ group_wait: 30s
+ group_interval: 5m
+ repeat_interval: 12h
+ receiver: 'webhook'
+receivers:
+# Note that this is a dummy configuration just to allow AlertManager to start
+- name: 'webhook'
+ webhook_configs:
+ - url: 'http://alertmanagerwh:30500/'
+
diff --git a/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml b/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml
new file mode 100644
index 00000000000..5c451a8557f
--- /dev/null
+++ b/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml
@@ -0,0 +1,27 @@
+# Have prometheus-operator run an AlertManager cluster
+apiVersion: monitoring.coreos.com/v1
+kind: Alertmanager
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+spec:
+ replicas: 3
+---
+# Create a Service to allow Prometheus to talk to AlertManager
+apiVersion: v1
+kind: Service
+metadata:
+ name: alertmanager-cockroachdb
+ labels:
+ app: cockroachdb
+spec:
+ type: ClusterIP
+ ports:
+ - name: web
+ port: 9093
+ protocol: TCP
+ targetPort: web
+ selector:
+ alertmanager: cockroachdb
+
diff --git a/src/current/files/cockroach/cloud/kubernetes/prometheus/prometheus.yaml b/src/current/files/cockroach/cloud/kubernetes/prometheus/prometheus.yaml
new file mode 100644
index 00000000000..c2073a9cbce
--- /dev/null
+++ b/src/current/files/cockroach/cloud/kubernetes/prometheus/prometheus.yaml
@@ -0,0 +1,94 @@
+# Create a service account for prometheus to run under
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+ name: prometheus
+ labels:
+ app: cockroachdb
+---
+# Define the access permissions that prometheus will run with
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+ name: prometheus
+ labels:
+ app: cockroachdb
+rules:
+- apiGroups: [""]
+ resources:
+ - nodes
+ - services
+ - endpoints
+ - pods
+ verbs: ["get", "list", "watch"]
+- apiGroups: [""]
+ resources:
+ - configmaps
+ verbs: ["get"]
+- nonResourceURLs: ["/metrics"]
+ verbs: ["get"]
+---
+# Associate the service account with the role
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+ name: prometheus
+ labels:
+ app: cockroachdb
+roleRef:
+ apiGroup: rbac.authorization.k8s.io
+ kind: ClusterRole
+ name: prometheus
+subjects:
+- kind: ServiceAccount
+ name: prometheus
+ namespace: default
+---
+# Select any services with the prometheus:cockroachdb label
+apiVersion: monitoring.coreos.com/v1
+kind: ServiceMonitor
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+ prometheus: cockroachdb
+spec:
+ selector:
+ matchLabels:
+ prometheus: cockroachdb
+ endpoints:
+ - port: http
+ path: /_status/vars
+ tlsConfig:
+ ca:
+ secret:
+ key: ca.crt
+ # This is the secret name used by the CockroachDB Kubernetes Operator.
+ # When using a custom CA, replace this with your secret name
+ name: cockroachdb-node
+ serverName: "127.0.0.1"
+---
+# Have prometheus-operator run a replicated Prometheus cluster
+apiVersion: monitoring.coreos.com/v1
+kind: Prometheus
+metadata:
+ name: cockroachdb
+ labels:
+ app: cockroachdb
+spec:
+ serviceAccountName: prometheus
+ alerting:
+ alertmanagers:
+ - namespace: default
+ name: alertmanager-cockroachdb
+ port: web
+ serviceMonitorSelector:
+ matchLabels:
+ prometheus: cockroachdb
+ resources:
+ requests:
+ memory: 400Mi
+ ruleSelector:
+ matchLabels:
+ role: alert-rules
+ prometheus: cockroachdb
diff --git a/src/current/files/cockroach/docs/RFCS/20160421_distributed_sql.md b/src/current/files/cockroach/docs/RFCS/20160421_distributed_sql.md
new file mode 100644
index 00000000000..1e48ea2bb8e
--- /dev/null
+++ b/src/current/files/cockroach/docs/RFCS/20160421_distributed_sql.md
@@ -0,0 +1,1517 @@
+- Feature Name: Distributing SQL queries
+- Status: completed
+- Start Date: 2015/02/12
+- Authors: andreimatei, knz, RaduBerinde
+- RFC PR: [#6067](https://github.com/cockroachdb/cockroach/pull/6067)
+- Cockroach Issue:
+
+# Table of Contents
+
+
+ * [Table of Contents](#table-of-contents)
+ * [Summary](#summary)
+ * [Vocabulary](#vocabulary)
+ * [Motivation](#motivation)
+ * [Detailed design](#detailed-design)
+ * [Overview](#overview)
+ * [Logical model and logical plans](#logical-model-and-logical-plans)
+ * [Example 1](#example-1)
+ * [Example 2](#example-2)
+ * [Back propagation of ordering requirements](#back-propagation-of-ordering-requirements)
+ * [Example 3](#example-3)
+ * [Types of aggregators](#types-of-aggregators)
+ * [From logical to physical](#from-logical-to-physical)
+ * [Processors](#processors)
+ * [Joins](#joins)
+ * [Join-by-lookup](#join-by-lookup)
+ * [Stream joins](#stream-joins)
+ * [Inter-stream ordering](#inter-stream-ordering)
+ * [Execution infrastructure](#execution-infrastructure)
+ * [Creating a local plan: the ScheduleFlows RPC](#creating-a-local-plan-the-scheduleflows-rpc)
+ * [Local scheduling of flows](#local-scheduling-of-flows)
+ * [Mailboxes](#mailboxes)
+ * [On-the-fly flows setup](#on-the-fly-flows-setup)
+ * [Retiring flows](#retiring-flows)
+ * [Error handling](#error-handling)
+ * [A more complex example: Daily Promotion](#a-more-complex-example-daily-promotion)
+ * [Implementation strategy](#implementation-strategy)
+ * [Logical planning](#logical-planning)
+ * [Physical planning](#physical-planning)
+ * [Processor infrastructure and implementing processors](#processor-infrastructure-and-implementing-processors)
+ * [Joins](#joins-1)
+ * [Scheduling](#scheduling)
+ * [KV integration](#kv-integration)
+ * [Implementation notes](#implementation-notes)
+ * [Test infrastructure](#test-infrastructure)
+ * [Visualisation/tracing](#visualisationtracing)
+ * [Alternate approaches considered (and rejected)](#alternate-approaches-considered-and-rejected)
+ * [More logic in the KV layer](#more-logic-in-the-kv-layer)
+ * [Complexity](#complexity)
+ * [Applicability](#applicability)
+ * [SQL2SQL: Distributed SQL layer](#sql2sql-distributed-sql-layer)
+ * [Sample high-level query flows](#sample-high-level-query-flows)
+ * [Complexity](#complexity-1)
+ * [Applicability](#applicability-1)
+ * [Spark: Compiling SQL into a data-parallel language running on top of a distributed-execution runtime](#spark-compiling-sql-into-a-data-parallel-language-running-on-top-of-a-distributed-execution-runtime)
+ * [Sample program](#sample-program)
+ * [Complexity](#complexity-2)
+ * [Unresolved questions](#unresolved-questions)
+
+
+# Summary
+
+In this RFC we propose a general approach for distributing SQL processing and
+moving computation closer to the data. The goal is to trigger an initial
+discussion and not a complete detailed design.
+
+## Vocabulary
+
+- KV - the KV system in cockroach, defined by its key-value, range and batch API
+- k/v - a key-value pair, usually used to refer to an entry in KV
+- Node - machine in the cluster
+- Client / Client-side - the SQL client
+- Gateway node / Gateway-side - the cluster node to which the client SQL query is delivered first
+- Leader node / Leader-side - the cluster node which resolves a KV operation and
+ has local access to the respective KV data
+
+Most of the following text reads from the gateway-side perspective, where the query parsing and planning currently runs.
+
+# Motivation
+
+The desired improvements are listed below.
+
+1. Remote-side filtering
+
+ When querying for a set of rows that match a filtering expression, we
+ currently query all the keys in certain ranges and process the filters after
+ receiving the data on the gateway node over the network. Instead, we want the
+ filtering expression to be processed by the lease holder or remote node, saving on
+ network traffic and related processing.
+
+ The remote-side filtering does not need to support full SQL expressions - it
+ can support a subset that includes common expressions (e.g. everything that
+ can be translated into expressions operating on strings) with the requesting
+ node applying a "final" filter.
+
+2. Remote-side updates and deletes
+
+ For statements like `UPDATE .. WHERE` and `DELETE .. WHERE` we currently
+ perform a query, receive results at the gateway over the network, and then
+ perform the update or deletion there. This involves too many round-trips;
+ instead, we want the query and updates to happen on the node which has access
+ to the data.
+
+ Again, this does not need to cover all possible SQL expressions (we can keep
+ a working "slow" path for some cases). However, to cover the most important
+ queries we still need more than simple filtering expressions (`UPDATE`
+ commonly uses expressions and functions for calculating new values).
+
+3. Distributed SQL operations
+
+ Currently SQL operations are processed by the entry node and thus their
+performance does not scale with the size of the cluster. We want to be able to
+distribute the processing on multiple nodes (parallelization for performance).
+
+ 1. Distributed joins
+
+ In joins, we produce results by matching the values of certain columns
+ among multiple tables. One strategy for distributing this computation is
+ based on hashing: `K` nodes are chosen and each of the nodes with fast
+ access to the table data sends the results to the `K` nodes according to a
+ hash function computed on the join columns (guaranteeing that all results
+ with the same values on these columns go to the same node). Hash-joins are
+ employed e.g. by F1.
+
+
+ Distributed joins and remote-side filtering can be needed together:
+ ```sql
+ -- find all orders placed around the customer's birthday. Notice the
+ -- filtering needs to happen on the results. I've complicated the filtering
+ -- condition because a simple equality check could have been made part of
+ -- the join.
+ SELECT * FROM Customers c INNER JOIN Orders o ON c.ID = i.CustomerID
+ WHERE DayOfYear(c.birthday) - DayOfYear(o.date) < 7
+ ```
+
+ 2. Distributed aggregation
+
+ When using `GROUP BY` we aggregate results according to a set of columns or
+ expressions and compute a function on each group of results. A strategy
+ similar to hash-joins can be employed to distribute the aggregation.
+
+ 3. Distributed sorting
+
+ When ordering results, we want to be able to distribute the sorting effort.
+ Nodes would sort their own data sets and one or more nodes would merge the
+ results.
+
+# Detailed design
+
+## Overview
+
+The proposed approach was originally inspired by [Sawzall][1] - a project by
+Rob Pike et al. at Google that proposes a "shell" (high-level language
+interpreter) to ease the exploitation of MapReduce. Its main innovation is a
+concise syntax to define “local” processes that take a piece of local data and
+emit zero or more results (these get translated to Map logic); then another
+syntax which takes results from the local transformations and aggregates them
+in different ways (this gets translated to Reduce logic). In a nutshell:
+Sawzall = MapReduce + high-level syntax + new terminology (conveniently hiding
+distributed computations behind a simple set of conceptual constructs).
+
+We propose something somewhat similar, but targeting a different execution
+model than MapReduce.
+
+1. A predefined set of *aggregators*, performing functionality required by SQL.
+ Most aggregators are configurable, but not fully programmable.
+2. One special aggregator, the 'evaluator', is programmable using a very simple
+ language, but is restricted to operating on one row of data at
+ a time.
+3. A routing of the results of an aggregator to the next aggregator in the
+ query pipeline.
+4. A logical model that allows for SQL to be compiled in a data-location-agnostic
+ way, but that captures enough information so that we can distribute the
+ computation.
+
+Besides accumulating or aggregating data, the aggregators can feed their results
+to another node or set of nodes, possibly as input for other programs. Finally,
+aggregators with special functionality for batching up results and performing KV
+commands are used to read data or make updates to the database.
+
+The key idea is that we can map SQL to a well-defined logical model which we
+then transform into a distributed execution plan.
+
+[1]: http://research.google.com/archive/sawzall.html
+
+## Logical model and logical plans
+
+We compile SQL into a *logical plan* (similar on the surface to the current
+`planNode` tree) which represents the abstract data flow through computation
+stages. The logical plan is agnostic to the way data is partitioned and
+distributed in the cluster; however, it contains enough information about the
+structure of the planned computation to allow us to exploit data parallelism
+later - in a subsequent phase, the logical plan will be converted into a
+*physical plan*, which maps the abstract computation and data flow to concrete
+data processors and communication channels between them.
+
+The logical plan is made up **aggregators**. Each aggregator consumes an **input
+stream** of rows (or more streams for joins, but let's leave that aside for now)
+and produces an **output stream** of rows. Each row is a tuple of column values;
+both the input and the output streams have a set schema. The schema is a set of
+columns and types, with each row having a datum for each column. Again, we
+emphasize that the streams are a logical concept and might not map to a single
+data stream in the actual computation.
+
+We introduce the concept of **grouping** to characterize a specific aspect of
+the computation that happens inside an aggregator. The groups are defined based
+on a **group key**, which is a subset of the columns in the input stream schema.
+The computation that happens for each group is independent of the data in the
+other groups, and the aggregator emits a concatenation of the results for all
+the groups. The ordering between group results in the output stream is not
+fixed - some aggregators may guarantee a certain ordering, others may not.
+
+More precisely, we can define the computation in an aggregator using a function
+`agg` that takes a sequence of input rows that are in a single group (same group
+key) and produces a set of output rows. The output of an aggregator is
+the concatenation of the outputs of `agg` on all the groups, in some order.
+
+The grouping characteristic will be useful when we later decide how to
+distribute the computation that is represented by an aggregator: since results
+for each group are independent, different groups can be processed on different
+nodes. The more groups we have, the better. At one end of the spectrum there are
+single-group aggregators (group key is the empty set of columns - `Group key:
+[]`, meaning everything is in the same group) which cannot be distributed. At
+the other end there are no-grouping aggregators which can be parallelized
+arbitrarily. Note that no-grouping aggregators are different than aggregators
+where the group key is the full set of columns - the latter still requires rows
+that are equal to be processed on a single node (this would be useful for an
+aggregator implementing `DISTINCT` for example). An aggregator with no grouping
+is a special but important case in which we are not aggregating multiple pieces
+of data, but we may be filtering, transforming, or reordering individual pieces
+of data.
+
+Aggregators can make use of SQL expressions, evaluating them with various inputs
+as part of their work. In particular, all aggregators can optionally use an
+**output filter** expression - a boolean function that is used to discard
+elements that would have otherwise been part of the output stream.
+
+(Note: the alternative of restricting use of SQL expressions to only certain
+aggregators was considered; that approach makes it much harder to support outer
+joins, where the `ON` expression evaluation must be part of the internal join
+logic and not just a filter on the output.)
+
+A special type of aggregator is the **evaluator** aggregator which is a
+"programmable" aggregator which processes the input stream sequentially (one
+element at a time), potentially emitting output elements. This is an aggregator
+with no grouping (group key is the full set of columns); the processing of each
+row independent. An evaluator can be used, for example, to generate new values from
+arbitrary expressions (like the `a+b` in `SELECT a+b FROM ..`); or to filter
+rows according to a predicate.
+
+Special **table reader** aggregators with no inputs are used as data sources; a
+table reader can be configured to output only certain columns, as needed.
+A special **final** aggregator with no outputs is used for the results of the
+query/statement.
+
+Some aggregators (final, limit) have an **ordering requirement** on the input
+stream (a list of columns with corresponding ascending/descending requirements).
+Some aggregators (like table readers) can guarantee a certain ordering on their
+output stream, called an **ordering guarantee** (same as the `orderingInfo` in
+the current code). All aggregators have an associated **ordering
+characterization** function `ord(input_order) -> output_order` that maps
+`input_order` (an ordering guarantee on the input stream) into `output_order`
+(an ordering guarantee for the output stream) - meaning that if the rows in the
+input stream are ordered according to `input_order`, then the rows in the output
+stream will be ordered according to `output_order`.
+
+The ordering guarantee of the table readers along with the characterization
+functions can be used to propagate ordering information across the logical plan.
+When there is a mismatch (an aggregator has an ordering requirement that is not
+matched by a guarantee), we insert a **sorting aggregator** - this is a
+non-grouping aggregator with output schema identical to the input schema that
+reorders the elements in the input stream providing a certain output order
+guarantee regardless of the input ordering. We can perform optimizations wrt
+sorting at the logical plan level - we could potentially put the sorting
+aggregator earlier in the pipeline, or split it into multiple nodes (one of
+which performs preliminary sorting in an earlier stage).
+
+To introduce the main types of aggregators we use of a simple query.
+
+### Example 1
+
+```sql
+TABLE Orders (OId INT PRIMARY KEY, CId INT, Value DECIMAL, Date DATE)
+
+SELECT CID, SUM(VALUE) FROM Orders
+ WHERE DATE > 2015
+ GROUP BY CID
+ ORDER BY 1 - SUM(Value)
+```
+
+This is a potential description of the aggregators and streams:
+
+```
+TABLE-READER src
+ Table: Orders
+ Table schema: Oid:INT, Cid:INT, Value:DECIMAL, Date:DATE
+ Output filter: (Date > 2015)
+ Output schema: Cid:INT, Value:DECIMAL
+ Ordering guarantee: Oid
+
+AGGREGATOR summer
+ Input schema: Cid:INT, Value:DECIMAL
+ Output schema: Cid:INT, ValueSum:DECIMAL
+ Group Key: Cid
+ Ordering characterization: if input ordered by Cid, output ordered by Cid
+
+EVALUATOR sortval
+ Input schema: Cid:INT, ValueSum:DECIMAL
+ Output schema: SortVal:DECIMAL, Cid:INT, ValueSum:DECIMAL
+ Ordering characterization:
+ ValueSum -> ValueSum and -SortVal
+ Cid,ValueSum -> Cid,ValueSum and Cid,-SortVal
+ ValueSum,Cid -> ValueSum,Cid and -SortVal,Cid
+ SQL Expressions: E(x:INT) INT = (1 - x)
+ Code {
+ EMIT E(ValueSum), CId, ValueSum
+ }
+
+AGGREGATOR final:
+ Input schema: SortVal:DECIMAL, Cid:INT, ValueSum:DECIMAL
+ Input ordering requirement: SortVal
+ Group Key: []
+
+Composition: src -> summer -> sortval -> final
+```
+
+Note that the logical description does not include sorting aggregators. This
+preliminary plan will lead to a full logical plan when we propagate ordering
+information. We will have to insert a sorting aggregator before `final`:
+```
+src -> summer -> sortval -> sort(OrderSum) -> final
+```
+Each arrow is a logical stream. This is the complete logical plan.
+
+In this example we only had one option for the sorting aggregator. Let's look at
+another example.
+
+
+### Example 2
+
+```sql
+TABLE People (Age INT, NetWorth DECIMAL, ...)
+
+SELECT Age, Sum(NetWorth) FROM v GROUP BY AGE ORDER BY AGE
+```
+
+Preliminary logical plan description:
+```
+TABLE-READER src
+ Table: People
+ Table schema: Age:INT, NetWorth:DECIMAL
+ Output schema: Age:INT, NetWorth:DECIMAL
+ Ordering guarantee: XXX // will consider different cases later
+
+AGGREGATOR summer
+ Input schema: Age:INT, NetWorth:DECIMAL
+ Output schema: Age:INT, NetWorthSum:DECIMAL
+ Group Key: Age
+ Ordering characterization: if input ordered by Age, output ordered by Age
+
+AGGREGATOR final:
+ Input schema: Age:INT, NetWorthSum:DECIMAL
+ Input ordering requirement: Age
+ Group Key: []
+
+Composition: src -> summer -> final
+```
+
+The `summer` aggregator can perform aggregation in two ways - if the input is
+not ordered by Age it will use an unordered map with one entry per `Age` and the
+results will be output in arbitrary order; if the input is ordered by `Age` it can
+aggregate on one age at a time and it will emit the results in age order.
+
+Let's take two cases:
+
+1. src is ordered by `Age` (we use a covering index on `Age`)
+
+ In this case, when we propagate the ordering
+ information we will notice that `summer` preserves ordering by age and we
+ won't need to add sorting aggregators.
+
+2. src is not ordered by anything
+
+ In this case, summer will not have any output ordering guarantees and we will
+ need to add a sorting aggregator before `final`:
+ ```
+ src -> summer -> sort(Age) -> final
+ ```
+ We could also use the fact that `summer` would preserve the order by `Age`
+ and put the sorting aggregator before `summer`:
+ ```
+ src -> sort(Age) -> summer -> final
+ ```
+ We would choose between these two logical plans.
+
+There is also the possibility that `summer` uses an ordered map, in which case
+it will always output the results in age order; that would mean we are always in
+case 1 above, regardless of the ordering of `src`.
+
+### Back propagation of ordering requirements
+
+In the previous example we saw how we could use an ordering on a table reader
+stream along with an order preservation guarantee to avoid sorting. The
+preliminary logical plan will try to preserve ordering as much as possible to
+minimize any additional sorting.
+
+However, in some cases preserving ordering might come with some cost; some
+aggregators could be configured to either preserve ordering or not. To avoid
+preserving ordering unnecessarily, after the sorting aggregators are put in
+place we post-process the logical plan to relax the ordering on the streams
+wherever possible. Specifically, we inspect each logical stream (in reverse
+topological order) and check if removing its ordering still yields a correct
+plan; this results in a back-propagation of the ordering requirements.
+
+To recap, the logical planning has three stages:
+ 1. preliminary logical plan, with ordering preserved as much as possible and no
+ sort nodes,
+ 2. order-satisfying logical plan, with sort nodes added as necessary,
+ 3. final logical plan, with ordering requirements relaxed where possible.
+
+### Example 3
+
+```sql
+TABLE v (Name STRING, Age INT, Account INT)
+
+SELECT COUNT(DISTINCT(account)) FROM v
+ WHERE age > 10 and age < 30
+ GROUP BY age HAVING MIN(Name) > 'k'
+```
+
+```
+TABLE-READER src
+ Table: v
+ Table schema: Name:STRING, Age:INT, Account:INT
+ Filter: (Age > 10 AND Age < 30)
+ Output schema: Name:STRING, Age:INT, Account:INT
+ Ordering guarantee: Name
+
+AGGREGATOR countdistinctmin
+ Input schema: Name:String, Age:INT, Account:INT
+ Group Key: Age
+ Group results: distinct count as AcctCount:INT
+ MIN(Name) as MinName:STRING
+ Output filter: (MinName > 'k')
+ Output schema: AcctCount:INT
+ Ordering characterization: if input ordered by Age, output ordered by Age
+
+AGGREGATOR final:
+ Input schema: AcctCount:INT
+ Input ordering requirement: none
+ Group Key: []
+
+Composition: src -> countdistinctmin -> final
+```
+
+### Types of aggregators
+
+- `TABLE READER` is a special aggregator, with no input stream. It's configured
+ with spans of a table or index and the schema that it needs to read.
+ Like every other aggregator, it can be configured with a programmable output
+ filter.
+- `EVALUATOR` is a fully programmable no-grouping aggregator. It runs a "program"
+ on each individual row. The evaluator can drop the row, or modify it
+ arbitrarily.
+- `JOIN` performs a join on two streams, with equality constraints between
+ certain columns. The aggregator is grouped on the columns that are
+ constrained to be equal. See [Stream joins](#stream-joins).
+- `JOIN READER` performs point-lookups for rows with the keys indicated by the
+ input stream. It can do so by performing (potentially remote) KV reads, or by
+ setting up remote flows. See [Join-by-lookup](#join-by-lookup) and
+ [On-the-fly flows setup](#on-the-fly-flows-setup).
+- `MUTATE` performs insertions/deletions/updates to KV. See section TODO.
+- `SET OPERATION` takes several inputs and performs set arithmetic on them
+ (union, difference).
+- `AGGREGATOR` is the one that does "aggregation" in the SQL sense. It groups
+ rows and computes an aggregate for each group. The group is configured using
+ the group key. `AGGREGATOR` can be configured with one or more aggregation
+ functions:
+ - `SUM`
+ - `COUNT`
+ - `COUNT DISTINCT`
+ - `DISTINCT`
+
+ `AGGREGATOR`'s output schema consists of the group key, plus a configurable
+ subset of the generated aggregated values. The optional output filter has
+ access to the group key and all the aggregated values (i.e. it can use even
+ values that are not ultimately outputted).
+- `SORT` sorts the input according to a configurable set of columns. Note that
+ this is a no-grouping aggregator, hence it can be distributed arbitrarily to
+ the data producers. This means, of course, that it doesn't produce a global
+ ordering, instead it just guarantees an intra-stream ordering on each
+ physical output streams). The global ordering, when needed, is achieved by an
+ input synchronizer of a grouped processor (such as `LIMIT` or `FINAL`).
+- `LIMIT` is a single-group aggregator that stops after reading so many input
+ rows.
+- `INTENT-COLLECTOR` is a single-group aggregator, scheduled on the gateway,
+ that receives all the intents generated by a `MUTATE` and keeps track of them
+ in memory until the transaction is committed.
+- `FINAL` is a single-group aggregator, scheduled on the gateway, that collects
+ the results of the query. This aggregator will be hooked up to the pgwire
+ connection to the client.
+
+## From logical to physical
+
+To distribute the computation that was described in terms of aggregators and
+logical streams, we use the following facts:
+
+ - for any aggregator, groups can be partitioned into subsets and processed in
+ parallel, as long as all processing for a group happens on a single node.
+
+ - the ordering characterization of an aggregator applies to *any* input stream
+ with a certain ordering; it is useful even when we have multiple parallel
+ instances of computation for that logical node: if the physical input streams
+ in all the parallel instances are ordered according to the logical input
+ stream guarantee (in the logical plan), the physical output streams in all
+ instances will have the output guarantee of the logical output stream. If at
+ some later stage these streams are merged into a single stream (merge-sorted,
+ i.e. with the ordering properly maintained), that physical stream will have
+ the correct ordering - that of the corresponding logical stream.
+
+ - aggregators with empty group keys (`limit`, `final`) must have their final
+ processing on a single node (they can however have preliminary distributed
+ stages).
+
+So each logical aggregator can correspond to multiple distributed instances, and
+each logical stream can correspond to multiple physical streams **with the same
+ordering guarantees**.
+
+We can distribute using a few simple rules:
+
+ - table readers have multiple instances, split according to the ranges; each
+ instance is processed by the raft leader of the relevant ranges and is the
+ start of a physical stream.
+
+ - streams continue in parallel through programs. When an aggregator is reached,
+ the streams can be redistributed to an arbitrary number of instances using
+ hashing on the group key. Aggregators with empty group keys will have a
+ single physical instance, and the input streams are merged according to the
+ desired ordering. As mentioned above, each physical stream will be already
+ ordered (because they all correspond to an ordered logical stream).
+
+ - sorting aggregators apply to each physical stream corresponding to the
+ logical stream it is sorting. A sort aggregator by itself will *not* result
+ in coalescing results into a single node. This is implicit from the fact that
+ (like evaluators) it requires no grouping.
+
+It is important to note that correctly distributing the work along range
+boundaries is not necessary for correctness - if a range gets split or moved
+while we are planning the query, it will not cause incorrect results. Some key
+reads might be slower because they actually happen remotely, but as long as
+*most of the time, most of the keys* are read locally this should not be a
+problem.
+
+Assume that we run the Example 1 query on a **Gateway** node and the table has
+data that on two nodes **A** and **B** (i.e. these two nodes are masters for all
+the relevant range). The logical plan is:
+
+```
+TABLE-READER src
+ Table: Orders
+ Table schema: Oid:INT, Cid:INT, Value:DECIMAL, Date:DATE
+ Output filter: (Date > 2015)
+ Output schema: Cid:INT, Value:DECIMAL
+ Ordering guarantee: Oid
+
+AGGREGATOR summer
+ Input schema: Cid:INT, Value:DECIMAL
+ Output schema: Cid:INT, ValueSum:DECIMAL
+ Group Key: Cid
+ Ordering characterization: if input ordered by Cid, output ordered by Cid
+
+EVALUATOR sortval
+ Input schema: Cid:INT, ValueSum:DECIMAL
+ Output schema: SortVal:DECIMAL, Cid:INT, ValueSum:DECIMAL
+ Ordering characterization: if input ordered by [Cid,]ValueSum[,Cid], output ordered by [Cid,]-ValueSum[,Cid]
+ SQL Expressions: E(x:INT) INT = (1 - x)
+ Code {
+ EMIT E(ValueSum), CId, ValueSum
+ }
+```
+
+
+
+This logical plan above could be instantiated as the following physical plan:
+
+
+
+Each box in the physical plan is a *processor*:
+ - `src` is a table reader and performs KV Get operations and forms rows; it is
+ programmed to read the spans that belong to the respective node. It evaluates
+ the `Date > 2015` filter before outputting rows.
+ - `summer-stage1` is the first stage of the `summer` aggregator; its purpose is
+ to do the aggregation it can do locally and distribute the partial results to
+ the `summer-stage2` processes, such that all values for a certain group key
+ (`CId`) reach the same process (by hashing `CId` to one of two "buckets").
+ - `summer-stage2` performs the actual sum and outputs the index (`CId`) and
+ corresponding sum.
+ - `sortval` calculates and emits the additional `SortVal` value, along with the
+ `CId` and `ValueSum`
+ - `sort` sorts the stream according to `SortVal`
+ - `final` merges the two input streams of data to produce the final sorted
+ result.
+
+Note that the second stage of the `summer` aggregator doesn't need to run on the
+same nodes; for example, an alternate physical plan could use a single stage 2
+processor:
+
+
+
+The processors always form a directed acyclic graph.
+
+### Processors
+
+Processors are generally made up of three components:
+
+
+
+1. The *input synchronizer* merges the input streams into a single stream of
+ data. Types:
+ * single-input (pass-through)
+ * unsynchronized: passes rows from all input streams, arbitrarily
+ interleaved.
+ * ordered: the input physical streams have an ordering guarantee (namely the
+ guarantee of the corresponding logical stream); the synchronizer is careful
+ to interleave the streams so that the merged stream has the same guarantee.
+
+2. The *data processor* core implements the data transformation or aggregation
+ logic (and in some cases performs KV operations).
+
+3. The *output router* splits the data processor's output to multiple streams;
+ types:
+ * single-output (pass-through)
+ * mirror: every row is sent to all output streams
+ * hashing: each row goes to a single output stream, chosen according
+ to a hash function applied on certain elements of the data tuples.
+ * by range: the router is configured with range information (relating to a
+ certain table) and is able to send rows to the nodes that are lease holders for
+ the respective ranges (useful for `JoinReader` nodes (taking index values
+ to the node responsible for the PK) and `INSERT` (taking new rows to their
+ lease holder-to-be)).
+
+## Joins
+
+### Join-by-lookup
+
+The join-by-lookup method involves receiving data from one table and looking up
+corresponding rows from another table. It is typically used for joining an index
+with the table, but they can be used for any join in the right circumstances,
+e.g. joining a small number of rows from one table ON the primary key of another
+table. We introduce a variant of `TABLE-READER` which has an input stream. Each
+element of the input stream drives a point-lookup in another table or index,
+with a corresponding value in the output. Internally the aggregator performs
+lookups in batches, the way we already do it today. Example:
+
+```sql
+TABLE t (k INT PRIMARY KEY, u INT, v INT, INDEX(u))
+SELECT k, u, v FROM t WHERE u >= 1 AND u <= 5
+```
+Logical plan:
+```
+TABLE-READER indexsrc
+Table: t@u, span /1-/6
+Output schema: k:INT, u:INT
+Output ordering: u
+
+JOIN-READER pksrc
+Table: t
+Input schema: k:INT, u:INT
+Output schema: k:INT, u:INT, v:INT
+Ordering characterization: preserves any ordering on k/u
+
+AGGREGATOR final
+Input schema: k:INT, u:INT, v:INT
+
+indexsrc -> pksrc -> final
+```
+
+Example of when this can be used for a join:
+```
+TABLE t1 (k INT PRIMARY KEY, v INT, INDEX(v))
+TABLE t2 (k INT PRIMARY KEY, w INT)
+SELECT t1.k, t1.v, t2.w FROM t1 INNER JOIN t2 ON t1.k = t2.k WHERE t1.v >= 1 AND t1.v <= 5
+```
+
+Logical plan:
+```
+TABLE-READER t1src
+Table: t1@v, span /1-/6
+Output schema: k:INT, v:INT
+Output ordering: v
+
+JOIN-READER t2src
+Table: t2
+Input schema: k:INT, v:INT
+Output schema: k:INT, v:INT, w:INT
+Ordering characterization: preserves any ordering on k
+
+AGGREGATOR final
+Input schema: k:INT, u:INT, v:INT
+
+t1src -> t2src -> final
+```
+
+Note that `JOIN-READER` has the capability of "plumbing" through an input column
+to the output (`v` in this case). In the case of index joins, this is only
+useful to skip reading or decoding the values for `v`; but in the general case
+it is necessary to pass through columns from the first table.
+
+In terms of the physical implementation of `JOIN-READER`, there are two possibilities:
+
+ 1. it can perform the KV queries (in batches) from the node it is receiving the
+ physical input stream from; the output stream continues on the same node.
+
+ This is simple but involves round-trips between the node and the range
+ lease holders. We will probably use this strategy for the first implementation.
+
+ 2. it can use routers-by-range to route each input to an instance of
+ `JOIN-READER` on the node for the respective range of `t2`; the flow of data
+ continues on that node.
+
+ This avoids the round-trip but is problematic because we may be setting up
+ flows on too many nodes (for a large table, many/all nodes in the cluster
+ could hold ranges). To implement this effectively, we require the ability to
+ set up the flows "lazily" (as needed), only when we actually find a row that
+ needs to go through a certain flow. This strategy can be particularly useful
+ when the ordering of `t1` and `t2` are correlated (e.g. t1 could be ordered
+ by a date, `t2` could be ordered by an implicit primary key).
+
+
+ Even with this optimization, it would be wasteful if we involve too many
+ remote nodes but we only end up doing a handful of queries. We can
+ investigate a hybrid approach where we batch some results and depending on
+ how many we have and how many different ranges/nodes they span, we choose
+ between the two strategies.
+
+
+### Stream joins
+
+The join aggregator performs a join on two streams, with equality constraints
+between certain columns. The aggregator is grouped on the columns that are
+constrained to be equal.
+
+```
+TABLE People (First STRING, Last STRING, Age INT)
+TABLE Applications (College STRING PRIMARY KEY, First STRING, Last STRING)
+SELECT College, Last, First, Age FROM People INNER JOIN Applications ON First, Last
+
+TABLE-READER src1
+Table: People
+Output Schema: First:STRING, Last:STRING, Age:INT
+Output Ordering: none
+
+TABLE_READER src2
+Table: Applications
+Output Schema: College:STRING, First:STRING, Last:STRING
+Output Ordering: none
+
+JOIN AGGREGATOR join
+Input schemas:
+ 1: First:STRING, Last:STRING, Age:INT
+ 2: College:STRING, First:STRING, Last:STRING
+Output schema: First:STRING, Last:STRING, Age:INT, College:STRING
+Group key: (1.First, 1.Last) = (2.First, 2.Last) // we need to get the group key from either stream
+Order characterization: no order preserved // could also preserve the order of one of the streams
+
+AGGREGATOR final
+ Ordering requirement: none
+ Input schema: First:STRING, Last:STRING, Age:INT, College:STRING
+```
+
+
+
+At the heart of the physical implementation of the stream join aggregators sits
+the join processor which (in general) puts all the rows from one stream in a
+hash map and then processes the other stream. If both streams are ordered by the
+group key, it can perform a merge-join which requires less memory.
+
+
+Using the same join processor implementation, we can implement different
+distribution strategies depending how we set up the physical streams and
+routers:
+
+ - the routers can distribute each row to one of multiple join processors based
+ on a hash on the elements for the group key; this ensures that all elements
+ in a group reach the same instance, achieving a hash-join. An example
+ physical plan:
+
+ 
+
+ - the routers can *duplicate* all rows from the physical streams for one table
+ and distribute copies to all processor instances; the streams for the other
+ table get processed on their respective nodes. This strategy is useful when
+ we are joining a big table with a small table, and can be particularly useful
+ for subqueries, e.g. `SELECT ... WHERE ... AND x IN (SELECT ...)`.
+
+ For the query above, if we expect few results from `src2`, this plan would
+ be more efficient:
+
+ 
+
+ The difference in this case is that the streams for the first table stay on
+ the same node, and the routers after the `src2` table readers are configured
+ to mirror the results (instead of distributing by hash in the previous case).
+
+
+## Inter-stream ordering
+
+**This is a feature that relates to implementing certain optimizations, but does
+not alter the structure of logical or physical plans. It will not be part of the
+initial implementation but we keep it in mind for potential use at a later
+point.**
+
+Consider this example:
+```sql
+TABLE t (k INT PRIMARY KEY, v INT)
+SELECT k, v FROM t WHERE k + v > 10 ORDER BY k
+```
+
+This is a simple plan:
+
+```
+READER src
+ Table: t
+ Output filter: (k + v > 10)
+ Output schema: k:INT, v:INT
+ Ordering guarantee: k
+
+AGGREGATOR final:
+ Input schema: k:INT, v:INT
+ Input ordering requirement: k
+ Group Key: []
+
+Composition: src -> final
+```
+
+Now let's say that the table spans two ranges on two different nodes - one range
+for keys `k <= 10` and one range for keys `k > 10`. In the physical plan we
+would have two streams starting from two readers; the streams get merged into a
+single stream before `final`. But in this case, we know that *all* elements in
+one stream are ordered before *all* elements in the other stream - we say that
+we have an **inter-stream ordering**. We can be more efficient when merging
+(before `final`): we simply read all elements from the first stream and then all
+elements from the second stream. Moreover, we would also know that the reader
+and other processors for the second stream don't need to be scheduled until the
+first stream is consumed, which is useful information for scheduling the query.
+In particular, this is important when we have a query with `ORDER BY` and
+`LIMIT`: the limit would be represented by an aggregator with a single group,
+with physical streams merging at that point; knowledge of the inter-stream
+ordering would allow us to potentially satisfy the limit by only reading from
+one range.
+
+We add the concept of inter-physical-stream ordering to the logical plan - it is
+a property of a logical stream (even though it refers to multiple physical
+streams that could be associated with that logical stream). We annotate all
+aggregators with an **inter-stream ordering characterization function** (similar
+to the ordering characterization described above, which can be thought of as
+"intra-stream" ordering). The inter-stream ordering function maps an input
+ordering to an output ordering, with the meaning that if the physical streams
+that are inputs to distributed instances of that aggregator have the
+inter-stream ordering described by input, then the output streams have the given
+output ordering.
+
+Like the intra-stream ordering information, we can propagate the inter-stream
+ordering information starting from the table readers onward. The streams coming
+out of a table reader have an inter-stream order if the spans each reader "works
+on" have a total order (this is always the case if each table reader is
+associated to a separate range).
+
+The information can be used to apply the optimization above - if a logical
+stream has an appropriate associated inter-stream ordering, the merging of the
+physical streams can happen by reading the streams sequentially. The same
+information can be used for scheduling optimizations (such as scheduling table
+readers that eventually feed into a limit sequentially instead of
+concurrently).
+
+## Execution infrastructure
+
+Once a physical plan has been generated, the system needs to divvy it up
+between the nodes and send it around for execution. Each node is responsible
+for locally scheduling data processors and input synchronizers. Nodes also need
+to be able to communicate with each other for connecting output routers to
+input synchronizers. In particular, a streaming interface is needed for
+connecting these actors. To avoid paying extra synchronization costs, the
+execution environment providing all these needs to be flexible enough so that
+different nodes can start executing their pieces in isolation, without any
+orchestration from the gateway besides the initial request to schedule a part
+of the plan.
+
+### Creating a local plan: the `ScheduleFlows` RPC
+
+Distributed execution starts with the gateway making a request to every node
+that's supposed to execute part of the plan asking the node to schedule the
+sub-plan(s) it's responsible for (modulo "on-the-fly" flows, see below). A node
+might be responsible for multiple disparate pieces of the overall DAG. Let's
+call each of them a *flow*. A flow is described by the sequence of physical
+plan nodes in it, the connections between them (input synchronizers, output
+routers) plus identifiers for the input streams of the top node in the plan and
+the output streams of the (possibly multiple) bottom nodes. A node might be
+responsible for multiple heterogeneous flows. More commonly, when a node is the
+lease holder for multiple ranges from the same table involved in the query, it will
+be responsible for a set of homogeneous flows, one per range, all starting with
+a `TableReader` processor. In the beginning, we'll coalesce all these
+`TableReader`s into one, configured with all the spans to be read across all
+the ranges local to the node. This means that we'll lose the inter-stream
+ordering (since we've turned everything into a single stream). Later on we
+might move to having one `TableReader` per range, so that we can schedule
+multiple of them to run in parallel.
+
+A node therefore implements a `ScheduleFlows` RPC which takes a set of flows,
+sets up the input and output mailboxes (see below), creates the local
+processors and starts their execution. We might imagine at some point
+implementing admission control for flows at the node boundary, in which case
+the RPC response would have the option to push back on the volume of work
+that's being requested.
+
+### Local scheduling of flows
+
+The simplest way to schedule the different processors locally on a node is
+concurrently: each data processor, synchronizer and router can be run as a
+goroutine, with channels between them. The channels can be buffered to
+synchronize producers and consumers to a controllable degree.
+
+### Mailboxes
+
+Flows on different nodes communicate with each other over GRPC streams. To
+allow the producer and the consumer to start at different times,
+`ScheduleFlows` creates named mailboxes for all the input and output streams.
+These message boxes will hold some number of tuples in an internal queue until
+a GRPC stream is established for transporting them. From that moment on, GRPC
+flow control is used to synchronize the producer and consumer.
+A GRPC stream is established by the consumer using the `StreamMailbox` RPC,
+taking a mailbox id (the same one that's been already used in the flows passed
+to `ScheduleFlows`). This call might arrive to a node even before the
+corresponding `ScheduleFlows` arrives. In this case, the mailbox is created on
+the fly, in the hope that the `ScheduleFlows` will follow soon. If that doesn't
+happen within a timeout, the mailbox is retired.
+Mailboxes present a channel interface to the local processors.
+If we move to a multiple `TableReader`s/flows per node, we'd still want one
+single output mailbox for all the homogeneous flows (if a node has 1mil ranges,
+we don't want 1mil mailboxes/streams). At that point we might want to add
+tagging to the different streams entering the mailbox, so that the inter-stream
+ordering property can still be used by the consumer.
+
+A diagram of a simple query using mailboxes for its execution:
+
+
+### On-the-fly flows setup
+
+In a couple of cases, we don't want all the flows to be setup from the get-go.
+`PointLookup` and `Mutate` generally start on a few ranges and then send data
+to arbitrary nodes. The amount of data to be sent to each node will often be
+very small (e.g. `PointLookup` might perform a handful of lookups in total on
+table *A*, so we don't want to set up receivers for those lookups on all nodes
+containing ranges for table *A*. Instead, the physical plan will contain just
+one processor, making the `PointLookup` aggregator single-stage; this node can
+chose whether to perform KV operations directly to do the lookups (for ranges
+with few lookups), or setup remote flows on the fly using the `ScheduleFlows`
+RPC for ranges with tons of lookups. In this case, the idea is to push a bunch
+of the computation to the data, so the flow passed to `ScheduleFlows` will be a
+copy of the physical nodes downstream of the aggregator, including filtering
+and aggregation. We imagine the processor will take the decision to move to
+this heavyweight process once it sees that it's batching a lot of lookups for
+the same range.
+
+## Retiring flows
+
+Processors and mailboxes needs to be destroyed at some point. This happens
+under a number of circumstances:
+
+1. A processor retires when it receives a sentinel on all of its input streams
+ and has outputted the last tuple (+ a sentinel) on all of its output
+ streams.
+2. A processor retires once either one of its input or output streams is
+ closed. This can be used by a consumer to inform its producers that it's
+ gotten all the data it needs.
+3. An input mailbox retires once it has put the sentinel on the wire or once
+ its GRPC stream is closed remotely.
+4. An output mailbox retires once it has passed on the sentinel to the reader,
+ which it does once all of its input channels are closed (remember that an
+ output mailbox may receive input from many channels, one per member of a
+ homogeneous flow family). It also retires if its GRPC stream is closed
+ remotely.
+5. `TableReader` retires once it has delivered the last tuple in its range (+ a
+ sentinel).
+
+### Error handling
+
+At least initially, the plan is to have no error recovery (anything goes wrong
+during execution, the query fails and the transaction is rolled back).
+The only concern is releasing all resources taken by the plan nodes.
+This can be done by propagating an error signal when any GRPC stream is
+closed abruptly.
+Similarly, cancelling a running query can be done by asking the `FINAL` processor
+to close its input channel. This close will propagate backwards to all plan nodes.
+
+
+# A more complex example: Daily Promotion
+
+Let's draw a possible logical and physical plan for a more complex query. The
+point of the query is to help with a promotion that goes out daily, targeting
+customers that have spent over $1000 in the last year. We'll insert into the
+`DailyPromotion` table rows representing each such customer and the sum of her
+recent orders.
+
+```SQL
+TABLE DailyPromotion (
+ Email TEXT,
+ Name TEXT,
+ OrderCount INT
+)
+
+TABLE Customers (
+ CustomerID INT PRIMARY KEY,
+ Email TEXT,
+ Name TEXT
+)
+
+TABLE Orders (
+ CustomerID INT,
+ Date DATETIME,
+ Value INT,
+
+ PRIMARY KEY (CustomerID, Date),
+ INDEX date (Date)
+)
+
+INSERT INTO DailyPromotion
+(SELECT c.Email, c.Name, os.OrderCount FROM
+ Customers AS c
+ INNER JOIN
+ (SELECT CustomerID, COUNT(*) as OrderCount FROM Orders
+ WHERE Date >= '2015-01-01'
+ GROUP BY CustomerID HAVING SUM(Value) >= 1000) AS os
+ ON c.CustomerID = os.CustomerID)
+```
+
+Logical plan:
+
+```
+TABLE-READER orders-by-date
+ Table: Orders@OrderByDate /2015-01-01 -
+ Input schema: Date: Datetime, OrderID: INT
+ Output schema: Cid:INT, Value:DECIMAL
+ Output filter: None (the filter has been turned into a scan range)
+ Intra-stream ordering characterization: Date
+ Inter-stream ordering characterization: Date
+
+JOIN-READER orders
+ Table: Orders
+ Input schema: Oid:INT, Date:DATETIME
+ Output filter: None
+ Output schema: Cid:INT, Date:DATETIME, Value:INT
+ // TODO: The ordering characterizations aren't necessary in this example
+ // and we might get better performance if we remove it and let the aggregator
+ // emit results out of order. Update after the section on backpropagation of
+ // ordering requirements.
+ Intra-stream ordering characterization: same as input
+ Inter-stream ordering characterization: Oid
+
+AGGREGATOR count-and-sum
+ Input schema: CustomerID:INT, Value:INT
+ Aggregation: SUM(Value) as sumval:INT
+ COUNT(*) as OrderCount:INT
+ Group key: CustomerID
+ Output schema: CustomerID:INT, OrderCount:INT
+ Output filter: sumval >= 1000
+ Intra-stream ordering characterization: None
+ Inter-stream ordering characterization: None
+
+JOIN-READER customers
+ Table: Customers
+ Input schema: CustomerID:INT, OrderCount: INT
+ Output schema: e-mail: TEXT, Name: TEXT, OrderCount: INT
+ Output filter: None
+ // TODO: The ordering characterizations aren't necessary in this example
+ // and we might get better performance if we remove it and let the aggregator
+ // emit results out of order. Update after the section on backpropagation of
+ // ordering requirements.
+ Intra-stream ordering characterization: same as input
+ Inter-stream ordering characterization: same as input
+
+INSERT inserter
+ Table: DailyPromotion
+ Input schema: email: TEXT, name: TEXT, OrderCount: INT
+ Table schema: email: TEXT, name: TEXT, OrderCount: INT
+
+INTENT-COLLECTOR intent-collector
+ Group key: []
+ Input schema: k: TEXT, v: TEXT
+
+AGGREGATOR final:
+ Input schema: rows-inserted:INT
+ Aggregation: SUM(rows-inserted) as rows-inserted:INT
+ Group Key: []
+
+Composition:
+order-by-date -> orders -> count-and-sum -> customers -> inserter -> intent-collector
+ \-> final (sum)
+```
+
+A possible physical plan:
+
+
+# Implementation strategy
+
+There are five streams of work. We keep in mind two initial milestones to track
+the extent of progress we must achieve within each stream:
+- Milestone M1: offloading filters to remote side
+- Milestone M2: offloading updates to remote side
+
+### Logical planning
+
+Building a logical plan for a statement involves many aspects:
+ - index selection
+ - query optimization
+ - choosing between various strategies for sorting, aggregation
+ - choosing between join strategies
+
+This is a very big area where we will see a long tail of improvements over time.
+However, we can start with a basic implementation based on the existing code.
+For a while we can use a hybrid approach where we implement table reading and
+filtering using the new framework and make the results accessible via a
+`planNode` so we can use the existing code for everything else. This would be
+sufficient for M1. The next stage would be implementing the mutation aggregators
+and refactoring the existing code to allow using them (enabling M2).
+
+### Physical planning
+
+A lot of the decisions in the physical planning stage are "forced" - table
+readers are distributed according to ranges, and much of the physical planning
+follows from that.
+
+One place where physical planning involves difficult decisions is when
+distributing the second stage of an aggregation or join - we could set up any
+number of "buckets" (and subsequent flows) on any nodes. E.g. see the `summer`
+example. Fortunately there is a simple strategy we can start with - use as many
+buckets as input flows and distribute them among the same nodes. This strategy
+scales well with the query size: if a query draws data from a single node, we
+will do all the aggregation on that node; if a query draws data from many nodes,
+we will distribute the aggregation among those nodes.
+
+We will also support configuring things to minimize the distribution - getting
+everything back on the single gateway node as quickly as possible. This will be
+useful to compare with the current "everything on the gateway" approach; it is
+also a conservative step that might avoid some problems when distributing
+queries between too many nodes.
+
+A "stage 2" refinement would be detecting when a computation (and subsequent
+stages) might be cheap enough to not need distribution and automatically switch
+to performing the aggregation on the gateway node. Further improvements
+(statistics based) can be investigated later.
+
+We should add extended SQL syntax to allow the query author to control some of
+these parameters, even if only as a development/testing facility that we don't
+advertise.
+
+### Processor infrastructure and implementing processors
+
+This involves building the framework within which we can run processors and
+implementing the various processors we need. We can start with the table readers
+(enough for M1). If necessary, this work stream can advance faster than the
+logical planning stream - we can build processors even if the logical plan isn't
+using them yet (as long as there is a good testing framework in place); we can
+also potentially use the implementations internally, hidden behind a `planNode`
+interface and running non-distributed.
+
+##### Joins
+
+A tangential work stream is to make progress toward supporting joins (initially
+non-distributed). This involves building the processor that will be at the heart
+of the hash join implementation and integrating that code with the current
+`planNode` tree.
+
+### Scheduling
+
+The problem of efficiently queuing and scheduling processors will also involve a
+long tail of improvements. But we can start with a basic implementation using
+simple strategies:
+ - the queue ordering is by transactions; we don't order individual processors
+ - limit the number of transactions for which we run processors at any one time;
+ we can also limit the total number of processors running at any one time, as
+ long as we allow all the processors needed for at least one txn
+ - the txn queuing order is a function of the txn timestamp and its priority,
+ allowing the nodes to automatically agree on the relative ordering of
+ transactions, eliminating deadlock scenarios (example of a deadlock: txn A
+ has some processors running on node 1, and some processors on node 2 that are
+ queued behind running processors of txn B; and txn B also has some processors
+ that are queued behind txn A on node 1)
+
+We won't need anything fancier in this area to reach M1 and M2.
+
+### KV integration
+
+We do not propose introducing any new KV Get/Put APIs. The current APIs are to
+be used; we simply rely on the fact that when run from the lease holder node they will
+be faster as the work they do is local.
+
+However, we still require some integration with the KV layer:
+
+1. Range information lookup
+
+ At the physical planning stage we need to break up key spans into ranges and
+ determine who is the lease holder for each range. We may also use range info at the
+ logical planning phase to help estimate table sizes (for index selection,
+ join order, etc). The KV layer already has a range cache that maintains this
+ information, but we will need to make changes to be more aggressive in terms
+ of how much information we maintain, and how we invalidate/update it.
+
+2. Distributed reads
+
+ There is very little in the KV layer that needs to change to allow
+ distributed reads - they are currently prevented only because of a fix
+ involving detecting aborted transactions (PR #5323).
+
+3. Distributed writes
+
+ The txn coordinator currently keeps track of all the modified keys or key
+ ranges. The new sql distribution layer is designed to allow funneling the
+ modified key information back to the gateway node (which acts as the txn
+ coordinator). There will need to be some integration here, to allow us to
+ pass this information to the KV layer. There are also likely various cases
+ where checking for error cases must be relaxed.
+
+The details of all these need to be further investigated. Only 1 and 2 are
+required for M1; 3 is required for M2.
+
+Another potential improvement is fixing the impedance mismatch between the
+`TableReader`, which produces a stream, and the underlying KV range reads,
+which do batch reading. Eventually we'll need a streaming reading interface for
+range reads, although in the beginning we can use what we have.
+
+## Implementation notes
+
+Capturing various notes and suggestions.
+
+#### Test infrastructure
+
+Either extend the logictest framework to allow specification of additional
+system attributes, or create new framework. We must have tests that can control
+various settings (number of nodes, range distribution etc) and examine the
+resulting query plans.
+
+#### Visualisation/tracing
+
+Detailed information about logical and physical plans must be available, as well
+as detailed traces for all phases of queries, including execution timings,
+stats, etc.
+
+At the SQL we will have to present data, plans in textual representations. One
+idea to help with visualisation is to build a small web page where we can paste
+a textual representation to get a graphical display.
+
+# Alternate approaches considered (and rejected)
+
+We outline a few different approaches we considered but eventually decided
+against.
+
+## More logic in the KV layer
+
+In this approach we would build more intelligence in the KV layer. It would
+understand rows, and it would be able to process expressions (either SQL
+expressions, or some kind of simplified language, e.g. string based).
+
+### Complexity
+
+Most of the complexity of this approach is around building APIs and support for
+expressions. For full SQL expressions, we would need a KV-level language that
+is able to read and modify SQL values without being part of the SQL layer. This
+would mean a compiler able to translate SQL expressions to programs in a
+KV-level VM that perform the SQL-to-bytes and bytes-to-SQL translations
+explicitly (i.e. translate/migrate our data encoding routines from Go to that
+KV-level VM's instructions).
+
+### Applicability
+
+The applicability of this technique is limited: it would work well for
+filtering and possibly for remote-side updates, but it is hard to imagine
+building the logic necessary for distributed SQL operations (joins,
+aggregation) into the KV layer.
+
+It seems that if we want to meet all described goals, we need to make use of a
+smarter approach. With this in mind, expending any effort toward this approach
+seems wasteful at this point in time. We may want to implement some of these
+ideas in the future if it helps make things more efficient, but for now we
+should focus on initial steps towards a more encompassing solution.
+
+
+## SQL2SQL: Distributed SQL layer
+
+In this approach we would build a distributed SQL layer, where the SQL layer of
+a node can make requests to the SQL layer of any other node. The SQL layer
+would "peek" into the range information in the KV layer to decide how to split
+the workload so that data is processed by the respective raft range lease holders.
+Achieving a correct distribution to range lease holders would not be necessary for
+correctness; thus we wouldn't need to build extra coordination with the KV
+layer to synchronize with range splits/merges or lease holdership changes during an
+SQL operation.
+
+
+### Sample high-level query flows
+
+Sample flow of a “simple” query (select or update with filter):
+
+| **Node A** | **Node B** | **Node C** | **Node D** |
+|--------------------------------------------|--------------|------------|------------|
+| Receives statement | | | |
+| Finds that the table data spans three ranges on **B**, **C**, **D** | | |
+| Sends scan requests to **B**, **C**, **D** | | | |
+| | Starts scan (w/filtering, updates) | Starts scan (w/filtering, updates) | Starts scan (w/filtering, updates) |
+| | Sends results back to **A** | Sends results back to **A** | Sends results back to **A** |
+| Aggregates and returns results. | | | |
+
+Sample flow for a hash-join:
+
+| **Node A** | **Node B** | **Node C** | **Node D** |
+|--------------------------------------------|--------------|------------|------------|
+| Receives statement | | | |
+| Finds that the table data spans three ranges on **B**, **C**, **D** | | |
+| Sets up 3 join buckets on B, C, D | | | |
+| | Expects join data for bucket 0 | Expects join data for bucket 1 | Expects join data for bucket 2 |
+| Sends scan requests to **B**, **C**, **D** | | | |
+| | Starts scan (w/ filtering). Results are sent to the three buckets in batches | Starts scan (w/ filtering) Results are sent to the three buckets in batches | Starts scan (w/ filtering). Results are sent to the three buckets in batches |
+| | Tells **A** scan is finished | Tells **A** scan is finished | Tells **A** scan is finished |
+| Sends finalize requests to the buckets | | | |
+| | Sends bucket data to **A** | Sends bucket data to **A** | Sends bucket data to **A** |
+| Returns results | | | |
+
+### Complexity
+
+We would need to build new infrastructure and APIs for SQL-to-SQL. The APIs
+would need to support SQL expressions, either as SQL strings (which requires
+each node to re-parse expressions) or a more efficient serialization of ASTs.
+
+The APIs also need to include information about what key ranges the request
+should be restricted to (so that a node processes the keys that it is lease holder
+for - or at least was, at the time when we started the operation). Since tables
+can span many raft ranges, this information can include a large number of
+disjoint key ranges.
+
+The design should not be rigid on the assumption that for any key there is a
+single node with "fast" access to that key. In the future we may implement
+consensus algorithms like EPaxos which allow operations to happen directly on
+the replicas, giving us multiple choices for how to distribute an operation.
+
+Finally, the APIs must be designed to allow overlap between processing, network
+transfer, and storage operations - it should be possible to stream results
+before all of them are available (F1 goes as far as streaming results
+out-of-order as they become available from storage).
+
+### Applicability
+
+This general approach can be used for distributed SQL operations as well as
+remote-side filtering and updates. The main drawback of this approach is that it
+is very general and not prescriptive on how to build reusable pieces of
+functionality. It is not clear how we could break apart the work in modular
+pieces, and it has the potential of evolving into a monster of unmanageable
+complexity.
+
+
+## Spark: Compiling SQL into a data-parallel language running on top of a distributed-execution runtime
+
+The idea here is to introduce a new system - an execution environment for
+distributed computation. The computations use a programming model like M/R, or
+more pipeline stuff - Spark, or Google's [Dataflow][1] (parts of it are an
+Apache project that can run on top of other execution environments - e.g.
+Spark).
+
+In these models, you think about arrays of data, or maps on which you can
+operate in parallel. The storage for these is distributed. And all you do is
+operation on these arrays or maps - sort them, group them by key, transform
+them, filter them. You can also operate on pairs of these datasets to do joins.
+
+These models try to have *a)* smart compilers that do symbolic execution, e.g.
+fuse as many operations together as possible - `map(f, map(g, dataset)) == map(f
+● g, dataset)` and *b)* dynamic runtimes. The runtimes probably look at operations
+after their input have been at least partially computed and decide which nodes
+participate in this current operation based on who has the input and who needs
+the output. And maybe some of this work has already been done for us in one of
+these open source projects.
+
+The idea would be to compile SQL into this sort of language, considering that we
+start execution with one big sorted map as a dataset, and run it.If the
+execution environment is good, it takes advantage of the data topology. This is
+different from "distributed sql" because *a)* the execution environment is
+dynamic, so you don't need to come up with an execution plan up front that says
+what node is gonna issues what command to what other node and *b)* data can be
+pushed from one node to another, not just pulled.
+
+We can start small - no distributed runtime, just filtering for `SELECTS` and
+filtering with side effects for `UPDATE, DELETE, INSERT FROM SELECT`. But we
+build this outside of KV; we build it on top of KV (these programs call into KV,
+as opposed to KV calling a filtering callback for every k/v or row).
+
+[1]: https://cloud.google.com/dataflow/model/programming-model
+
+### Sample program
+
+Here's a quick sketch of a program that does remote-side filtering and deletion
+for a table with an index.
+
+Written in a language for (what I imagine to be) Spark-like parallel operations.
+The code is pretty tailored to this particular table and this particular query
+(which is a good thing). The idea of the exercise is to see if it'd be feasible
+at all to generate such a thing, assuming we had a way to execute it.
+
+The language has some data types, notably maps and tuples, besides the
+distributed maps that the computation is based on. It interfaces with KV through
+some `builtin::` functions.
+
+It starts with a map corresponding to the KV map, and then munges and aggregates
+the keys to form a map of "rows", and then generates the delete KV commands.
+
+The structure of the computation would stay the same and the code would be a lot
+shorter if it weren't tailored to this particular table, and instead it used
+generic built-in functions to encode and decode primary key keys and index keys.
+
+```sql
+TABLE t (
+ id1 int
+ id2 string
+ a string
+ b int DEFAULT NULL
+
+ PRIMARY KEY id1, id2
+ INDEX foo (id1, b)
+)
+
+DELETE FROM t WHERE id1 >= 100 AND id2 < 200 AND len(id2) == 5 and b == 77
+```
+
+```go
+func runQuery() {
+ // raw => Map. The key is a primary key string - table id, id1,
+ // id2, col id. This map is also sorted.
+ raw = builtin::readRange("t/primary_key/100", "t/primary_key/200")
+
+ // m1 => Map<(int, string), (int, string)>. This map is also sorted because the
+ // input is sorted and the function maintains sorting.
+ m1 = Map(raw, transformPK).
+
+ // Now build something resembling SQL rows. Since m1 is sorted, ReduceByKey is
+ // a simple sequential scan of m1.
+ // m2 => Map<(int, string), Map>. These are the rows.
+ m2 = ReduceByKey(m1, buildColMap)
+
+ // afterFilter => Map<(int, string), Map>. Like m2, but only the rows that passed the filter
+ afterFilter = Map(m2, filter)
+
+ // now we batch up all delete commands, for the primary key (one KV command
+ // per SQL column), and for the indexes (one KV command per SQL row)
+ Map(afterFilter, deletePK)
+ Map(afterFilter, deleteIndexFoo)
+
+ // return the number of rows affected
+ return len(afterFilter)
+}
+
+func transformPK(k, v) {
+ #pragma maintainsSort // important, keys remain sorted. So future
+ // reduce-by-key operations are efficient
+ id1, id2, colId = breakPrimaryKey(k)
+ return (key: {id1, id2}, value: (colId, v))
+}
+
+func breakPrimaryKey(k) {
+ // remove table id and the col_id
+ tableId, remaining = consumeInt(k)
+ id1, remaining = consumeInt(remaining)
+ id2, remaining = consumeInt(remaining)
+ colId = consumeInt(remaining)
+ returns (id1, id2, colId)
+}
+
+func BuildColMap(k, val) {
+ colId, originalVal = val // unpack
+ a, remaining = consumeInt(originalVal)
+ b, remaining = consumeString(remaining)
+ // output produces a result. Can appear 0, 1 or more times.
+ output (k, {'colId': colId, 'a': a, 'b': b})
+}
+
+func Filter(k, v) {
+ // id1 >= 100 AND id2 < 200 AND len(id2) == 5 and b == 77
+ id1, id2 = k
+ if len(id2) == 5 && v.getWithDefault('b', NULL) == 77 {
+ output (k, v)
+ }
+ // if filter doesn't pass, we don't output anything
+}
+
+
+func deletePK(k, v) {
+ id1, id2 = k
+ // delete KV row for column a
+ builtIn::delete(makePK(id1, id2, 'a'))
+ // delete KV row for column b, if it exists
+ if v.hasKey('b') {
+ builtIn::delete(makePK(id1, id2, 'b'))
+ }
+}
+
+func deleteIndexFoo(k, v) {
+ id1, id2 = k
+ b = v.getWithDefault('b', NULL)
+
+ builtIn::delete(makeIndex(id1, b))
+}
+```
+
+### Complexity
+
+This approach involves building the most machinery; it is probably overkill
+unless we want to use that machinery in other ways than SQL.
+
+# Unresolved questions
+
+The question of what unresolved questions there are is, as of yet, unresolved.
diff --git a/src/current/files/cockroach/docs/RFCS/20160706_expressive_zone_config.md b/src/current/files/cockroach/docs/RFCS/20160706_expressive_zone_config.md
new file mode 100644
index 00000000000..95abb4b5d19
--- /dev/null
+++ b/src/current/files/cockroach/docs/RFCS/20160706_expressive_zone_config.md
@@ -0,0 +1,241 @@
+- Feature Name: Expressive ZoneConfig
+- Status: obsolete
+- Start Date: 2016-07-06
+- Authors: @d4l3k
+- RFC PR: [#7660](https://github.com/cockroachdb/cockroach/pull/7660)
+- Cockroach Issue: [#4868](https://github.com/cockroachdb/cockroach/issues/4868)
+
+# Summary
+
+This document has been made partially obsolete by more recent changes to
+ZoneConfig constraints. For the latest information, please see [the
+docs](https://www.cockroachlabs.com/docs/stable/configure-replication-zones.html).
+
+# Motivation
+
+The current ZoneConfig format has a number of drawbacks that make it
+hard to specify the number of replicas and more complicated constraints.
+
+Current ZoneConfig format:
+```yaml
+replicas:
+- attrs: [comma-separated attribute list]
+- attrs: [comma-separated attribute list]
+- attrs: [comma-separated attribute list]
+range_min_bytes:
+range_max_bytes:
+gc:
+ ttlseconds:
+```
+https://www.cockroachlabs.com/docs/v2.0/configure-replication-zones#replication-zone-format
+
+## Number of replicas
+
+Currently, the number of replicas is controlled by adding extra `- attrs: []`
+elements to the replicas array.
+
+Comment from @bdarnell on #4866:
+
+> I'd be in favor of introducing a num_replicas field which would be used in
+> place of len(attrs). If attrs are present they would be used to constrain the
+> replica selection in the way that they are today, otherwise it is as if there
+> are num_replicas empty attr entries. No need to actually address the issues
+> around negative or diversity constraints at this point.
+
+## Replication Constraints
+
+Currently it's only possible to specify positive constraints, such as a replica
+must be placed on a store with an SSD or in `us-west-1`. Each node/store must
+have the list of tags provided in the command line arguments to the server.
+There's also no way to do wild card matches such as `us-.*`.
+
+### Types of constraints
+
+* Hardware
+ * What type of backing store `hdd`, `ssd`, `mem`.
+* Location
+ * Cloud provider `gce`, `aws`
+ * Data center `us-west-1`
+ * Sub data center `rack-12`
+ * These could also all be combined into one `gce-us-west-1-rack-12`.
+
+### Potentially desired constraints
+
+* Positive (must match)
+ * on `ssd`
+ * in `us-.*`.
+* Negative (must not match)
+ * not on `hdd`
+ * not in `us-.*`
+* Diversity (replicas distributed to minimize risk)
+ * 3 replicas with no 2 replicas on the same rack unless there are less than 3
+ racks
+* Compound
+ * in `us` or `canada`
+ * not in `us` or `canada`
+ * in `us` and on `ssd`
+ * in `us` and not on `ssd`
+
+# Detailed design
+
+Note: This is one potential design. See [Alternatives](#alternatives) and
+[Unresolved Questions](#unresolved-questions).
+
+## New ZoneConfig Format
+```yaml
+num_replicas:
+constraints: [comma-separated constraint list]
+range_min_bytes:
+range_max_bytes:
+gc:
+ ttlseconds:
+```
+
+## Constraint system
+
+This approach would be to extend the current simple tag matching with modifiers
+and the ability to do regex matches.
+
+Comment from @petermattis on #4868:
+
+> For example, instead of a single expression we could imagine there are a
+> series of expressions. `diversity(us-.*), require(hdd), prohibit(eu-.*)` would
+> provide diversity across nodes containing an attribute matching `us-.*`,
+> require nodes to have the attribute `hdd` and prohibit nodes from containing
+> the attribute `eu-.*`. To make this more terse we could provide short-hand
+> operators: `~us-.*, +hdd, !eu.*`. We'd want some rule about how to drop
+> constraints when they can't be achieved.
+
+This would be implemented as a simple array of string tags (similar to the
+current implementation) with three types of constraints.
+
+### Positive constraints `us-.*`.
+
+Positive constraints are sufficient for most use cases. The allocator would
+take care of automatically maximizing diversity of these constraints so the
+user doesn't have to worry about having duplicate replicas on the same rack
+unless it's unavoidable. If these constraints can't be matched, the allocator
+will fall back to the next closest match.
+
+Since the allocator will maximize diversity, this allows us to have `or`
+statements. Specifying `[ssd, mem]`, will have all replicas put on an SSD or
+in-memory store since there won't be any stores that are both. Likewise,
+specifying `[us-.*, ssd]` will have all replicas in the US and on SSDs.
+
+### Required constraints `+us-.*` / Prohibited constraints `-us-.*`
+
+Required and Prohibited constraints are useful for legal situations where you
+need a certain data restriction. This may be used for cases where data has to be
+local to a country, or shouldn't be in certain ones. The failure mode would be
+blocking new replicas from being added while there isn't enough capacity that
+matches the constraints.
+
+### Node Locality
+
+The suggested format for locality would be a set of keys and values that would
+then be diversified across. This could look like
+`--locality cloud=gce,country=us,region=west,datacenter=us-west-1,rack=12`.
+Constraints would be applied like `[cloud=gce, country=us]`.
+
+We would try to maximize diversity within each KV tag. This would make it
+easier to tell which tags match each other instead of just the index in the
+hierarchy list.
+
+We would use the order the tags were defined in as a hierarchy for which levels to
+prioritize hierarchy for. If the order of tags on one node doesn't match those
+on others, we would warn the user. In such a case, we will use the order that the
+majority of nodes have.
+
+If we have three datacenters and each datacenter has 3 racks
+`datacenter=us-{1,2,3},rack={{1,2,3},{1,2,3},{1,2,3}}` there's no way to
+distinguish the primary diversity factor automatically, but we can maximize
+diversity on both levels. In this situation, the best thing to do would be to
+have 3 replicas
+`datacenter=us-1,rack=1`, `datacenter=us-2,rack=2` and `datacenter=us-3,rack=3`.
+
+While maximizing diversity in a mostly empty cluster is easy to do, as servers
+fill up we might be in the situation of putting two replicas on the same rack or
+two replicas in the same datacenter. This means that an ordering is needed to
+know which tag diversity is more important.
+
+### Examples
+
+Prefer SSDs and do not put on in-memory stores.
+```yaml
+constraints: [ssd, -mem]
+```
+
+Require data to be stored in the US.
+```yaml
+constraints: [+country=us]
+```
+
+In-memory store, and not in aws.
+```yaml
+constraints: [mem, -cloud=aws]
+```
+
+## Remove per replica attributes
+
+Remove the ability to specify per replica attributes as they're a poor stand in
+for proper diversity constraints.
+
+With a first class diversity system we can get rid of the attributes arrays.
+This also allows for specifying a candidate sets that are bigger than the exact
+number of replicas.
+
+### Old format
+
+```yaml
+replicas:
+- attrs: [us-1]
+- attrs: [eur-2]
+- attrs: [asia-3]
+```
+
+### New Format
+
+```yaml
+num_replicas: 3
+constraints: [us-1, eur-2, asia-3]
+```
+
+## Constraint failure modes
+
+By default, the allocator will try to maximize the number of constraints it
+satisfies. If it can't find a perfect match it'll find the best matching store
+and use that one instead.
+
+The only exception is that for hard constraints and prohibited constraints,
+replication will fail and an alert will show up in the admin UI. In the future,
+we could try and migrate other ranges off of matching stores to free space and
+allow for replication to continue.
+
+## How do you notify users of constraint failures or issues?
+
+Any issues with constraints will show up in logs and in the admin UI. While
+typically a constraint issue shouldn't break the system, it could operate in a
+manner that a user doesn't expect.
+
+## Debugging
+
+A simple tool that will allow users to specify constraints and view the matching
+stores will be added to the web UI. This will allow users to vet any changes
+before actually using them. In addition, it will also highlight any existing
+invalid configurations such as out of order locality tags, and failing
+constraints.
+
+# Drawbacks
+
+One drawback is that the new system prevents explicit control over the diversity
+of a replica set. There's no way to say "one ssd and two hdds".
+
+# Alternatives
+
+# Unresolved questions
+
+## Weak negative constraints?
+
+Might be useful to have weak negative constraints such as `!ssd` that won't stop
+allocation such as how a prohibited constraint would. There's currently not a
+strong use case for it, but should be fairly easy to add later on.
diff --git a/src/current/files/cockroach/docs/RFCS/20200331_enums.md b/src/current/files/cockroach/docs/RFCS/20200331_enums.md
new file mode 100644
index 00000000000..e894bdd0620
--- /dev/null
+++ b/src/current/files/cockroach/docs/RFCS/20200331_enums.md
@@ -0,0 +1,398 @@
+- Feature Name: Enum Data Types in CockroachDB
+- Status: accepted
+- Start Date: 2020-03-31
+- Authors: Rohan Yadav, Lucy Zhang, Andrew Werner, Jordan Lewis
+- RFC PR: #47070
+- Cockroach Issue: #24873
+
+# Summary
+
+This RFC proposes adding enum types to CockroachDB.
+
+# Background
+
+Enum types are a class of user defined types where the values in
+the type are constrained to a fixed set
+of user specified values. The system then ensures type safety over operations
+on this type. This includes ensuring that only values that are members of the
+enum can be inserted into a column of the enum type, and that enums can only
+be compared to other values of the same enum type. For example, consider an
+application that needs to store events and the days of the week that they happen.
+This application could use an enum to represent the days of the week.
+
+```sql
+CREATE TYPE day ENUM AS ('monday', 'tuesday', 'wednesday'...);
+CREATE TABLE events (id INT, dayofweek day);
+INSERT INTO events VALUES (1, 'monday');
+```
+
+
+# Overview
+
+To implement enum types in CockroachDB, we have to touch many layers
+of the system. In particular, we need to introduce a way of storing
+metadata about enums durably in the database. We then need a way to
+cache this metadata so that lookups on this metadata is fast, as well
+as a way to invalidate this cache when enum metadata changes. When
+enum metadata changes, we need to ensure that these changes do not
+result in some nodes in the cluster entering a situation where
+they are unable to process enum values they find. Lastly, we need
+to define a physical layout for enums and integrate enums within
+the type system and SQL execution stack.
+
+# Detailed Explanation
+
+## Metadata Storage
+
+Enums themselves are a special case of user-defined types. In order
+to lay the groundwork for future work in this area, we propose storing
+metadata about an enum in a new descriptor called a `TypeDescriptor`.
+This descriptor will be added to the descriptor union alongside table and
+database descriptors. The descriptor will store metadata about the type,
+including the parent database and schema IDs, a unique ID for the type, and
+the name of the type. The descriptor will also include specific information
+for the kind of type being stored in the descriptor (as of now there
+would only be enums). For enums, this information would include the mapping
+of the enum's values to their physical representations. A proposal of the
+descriptor's contents is below:
+
+```proto
+message TypeDescriptor {
+ // Used by all kinds of user-defined types.
+ // Parent database and schema.
+ uint32 parent_id;
+ uint32 parent_schema_id;
+ // ID and Postgres compatible OID of the type.
+ uint32 id;
+ uint32 oid;
+ // Visible name of the type.
+ string name;
+
+ // Enum specific fields.
+ message enum_members {
+ byte[] physical_encoding;
+ string name;
+ };
+ enum_members[] members;
+}
+```
+
+These descriptors
+will be stored in the `system.descriptor` table and will use the leasing
+and versioning system being built. There is ongoing work on unifying
+the leasing interface so that components are easily shared across
+different descriptor types, and we will take advantage of these
+systems once they are available. The leasing system will enable caching
+and cache invalidation of type descriptors. Until the leasing system
+is ready for integration, we will first implement a prototype
+that either doesn't use a cache or uses a simple incoherent cache for
+`TypeDescriptor` access.
+
+## Name Resolution
+
+Enums are scoped within a database and a schema. In Postgres, enums
+cannot be accessed from other databases -- they can only be accessed from
+different schemas in the same database. However, there is no core reason
+that CockroachDB cannot support this. In fact, we might need to support
+references of types across databases to be in line with other cross
+database references that we currently support. The topic of cross database
+references has come up in discussion about
+[user defined schemas](https://github.com/cockroachdb/cockroach/pull/48276)
+as well. The direction that we take in allowing cross database references
+vs allowing only cross schema references will follow what has been decided
+in that context.
+
+Table and type names exist within the same namespace in Postgres. This means
+that it is possible to create a type and table of the same name within
+the same schema. Additionally, tables in Postgres are types themselves
+as a record type where each field is typed like the tables columns. Therefore,
+we will store type namespace entries along with table namespace entries
+in the `system.namespace` table. This allows namespace conflicts between
+types and tables to be properly detected, as well as allowing us to reuse
+a large amount of name resolution logic that exists for table name lookup.
+This strategy also will allow the user defined types implementation to
+adapt to new features like user defined schemas without extra work.
+
+## ID's and OID's
+
+All user defined types will need a stable ID that they are uniquely addressable
+by from within CockroachDB, as well as an OID that can be used for Postgres
+compliant operations. Importantly, the OIDs cannot conflict
+with existing type OIDs. Our strategy is to construct OIDs from the stable ID.
+In particular, the OID of a user defined type is equal to
+`ID + oidext.CockroachPredefinedOIDMax`. This strategy allows us to easily
+map back and forth between OIDs and IDs, and avoid using multiple counters for
+essentially the same information. The offset ensures that no user defined
+types have OIDs that conflict with any preexisting OIDs. This approach will
+naturally extend when we allow treating tables as types.
+
+## Changing Enum Definitions
+
+There are a few ways that enums can change over time.
+* The name can change.
+* The schema the enum is in can change.
+* A new enum member can be added to the set of values.
+* A member in the enum can be renamed.
+* The enum can be dropped.
+
+In order to rename an enum or a value in an enum can be done with a write
+to the enum descriptor and then waiting for all nodes to agree on the new value.
+There are plans to lift operations on descriptor names off of the individual
+descriptors, because such operations are common to all of them. This work
+would involve moving the draining names off of descriptors as well. It's
+possible that this work would be part of or take advantage of this effort.
+
+The case of adding a new enum element is more difficult. The key difficulty comes
+from ensuring that a node does not attempt to translate a physical layout that it
+does not know about yet into a user facing representation of the enum. If we naively
+just add the new enum value to the enum metadata, it is possible that another node
+reads a newly written enum from disk and is unsure how to decode it. Consider the
+following sequence of events:
+* Node 1 receives a new enum element `foo` to its enum descriptor and blocks on
+ `WaitForOneVersion`
+* Node 2 receives the new enum descriptor update and writes a value with `foo`
+* Node 3 tries to read the value of `foo` before receiving the update to
+ its enum descriptor.
+
+In order to avoid these situations, we propose an extension of the strategy
+used for performing online schema changes. As a reminder, when we add a new
+schema object to a table, it moves through a series of states before becoming
+usable. As the object moves through these states, the types of operations
+that are allowed upon the object change. Between each state, we require that
+all nodes in the cluster agree on the new version of the schema object.
+For more details, refer to the
+[online schema changes RFC](https://github.com/cockroachdb/cockroach/blob/master/docs/RFCS/20151014_online_schema_change.md).
+We propose a similar state
+progression to adding new elements to an enum type.
+1. When a new value is added
+to an enum, it is instead placed into a "read only" state.
+2. After all nodes agree on the "read only" state, the new enum value
+is promoted into the set of writeable values in the enum.
+
+This process ensures that all nodes know
+about all potential enum values before they have a chance to be written.
+This approach has the drawback of not being able to add an enum value and
+then insert that value in the same transaction. This drawback is similar
+to our existing limitation of not being able to add a column and insert
+into it in the same transaction.
+
+This enum schema change will be implemented with a new job, rather than
+trying to build off of the existing table schema changer. While conceptually
+similar to a table schema change, there is not much implementation to share.
+This new job will
+1. Collect all "read only" enum values and wait for one version in the cluster.
+2. Transition these values to "public", and then wait for one version in the cluster.
+
+A rollback of this job can just remove the "read-only" values.
+Additionally, enums don't really need a concept of mutations like tables. The
+members of an enum in the enum's `TypeDescriptor` can be tagged with whether
+the member is "read only" or public.
+
+In Postgres, if an enum is dropped without `CASCADE`, the operation will not succeed
+if there are any tables that use the enum. If an enum is dropped with
+`CASCADE`, all dependent columns are dropped as well. If the database
+that an enum is created within is dropped, then the enum
+is dropped as well. In order to maintain this information, the
+descriptors that represent an enum need to hold back-references to
+the tables and columns that use them. We expect the descriptor leasing
+system being developed to manage invalidation of cached enums when enums
+are destroyed in these cases.
+
+## Physical Layout
+
+At first, it may seem that a valid implementation of enum values is
+to map each to an integer, and then store these integers on disk.
+This implementation seems like it would supply all the ordering
+guarantees needed of enums. However, Postgres allows for adding
+new enums and specifying the order of the newly created enum
+with respect to an existing value of the enum. This looks like:
+```sql
+CREATE TYPE t ENUM AS ('v1', 'v2');
+ALTER TYPE t ADD VALUE 'v1.5' AFTER 'v1'
+```
+This means add the value `v1.5` to the enum `t` and order it
+after the value `v1`. Using just integers as the backing value
+for enums would not allow us to handle this sort of case.
+Postgres implements this feature on enums by storing a sorting
+order for enums as a float. When a new value is added like this,
+Postgres takes the sort orders of the enums that the new enum is
+being inserted in between, and creates a float that bisects the
+range between the two orders. Concretely, if `v1` had a sort order
+of `1.5` and `v2` had a sort order of `2.0`, then `v1.5` would be
+inserted with a sort order of `1.75`. However, once the floating
+point precision limit has been reached, Postgres rewrites all
+sort orders to integral values. Postgres can do this because it
+doesn't require a stable disk encoding for enums. In our case,
+we need to have a stable encoding to store data on disk if an enum
+is used in an index, and cannot afford to rewrite all tables using an
+enum if the enum sort order has changed.
+
+We propose a different strategy that is related to this idea of
+bisecting ranges, but doesn't suffer from problems due to floating
+point arithmetic precision. The general idea is to use byte arrays
+to hold the sort order of our enums, and reserve some bytes in the
+arrays to create the ordering that we need. In particular we reserve
+the minimum byte (`0`) and have a maximum allowed byte. In practice
+this will be `255`. An example of the encoding scheme is below.
+
+Assume we started with 3 elements (`a`, `b`, `c`), and let the maximum byte value be 3.
+The sort order byte arrays for each element would be:
+```
+a 1/
+b 2/
+c 3/
+```
+To add an element after `b` we can create a new key that sits in the middle of the range
+between `b` and `c`.
+```
+a 1/
+b 2/
+d 2/2/
+c 3/
+```
+Now lets add more values before `d`. The first one is easy:
+```
+a 1/
+b 2/
+e 2/1/
+d 2/2/
+c 3/
+```
+The tricky case is adding a value before `e`. Because we reserved the minimum byte, we can
+append it and then bisect the range again.
+```
+a 1/
+b 2/
+f 2/0/2
+e 2/1/
+d 2/2/
+c 3/
+```
+This strategy can be extended indefinitely as long as this pattern is followed to reserve
+the minimum byte. A prototype of the exact algorithm is included as part of the RFC PR.
+
+This sort order byte array will be the physical layout and identifier of the enum. We expect
+that for small enums only a byte or two will be used to hold all the values, and that our
+compression strategies at the storage layer will compress this data well.
+
+Since the common case of adding members to an enum is to add a member at the beginning
+or end of the set of values, we can adjust the algorithm slightly to better
+handle this case. When generating a new key byte where one of the endpoints is
+the min or max element, the algorithm can add or subtract a small constant from
+the existing key rather than bisecting the range. This allows for adding many
+more elements to the beginning or end of the range without increasing the
+number of bytes used to store the enum. The algorithm can be found implemented in
+[this PR](https://github.com/cockroachdb/cockroach/pull/47939).
+
+## Parsing
+
+Currently, the CockroachDB grammar is not equipped to handle type names
+that are qualified due to changes made in the past that separated parsing of
+object and type identifiers. Some of these changes will have to be
+reverted/adapted in order to allow for types to have qualifications again.
+The work to allow the parser to recognize qualified names has been done in
+[this PR](https://github.com/cockroachdb/cockroach/pull/47216).
+
+## Type System Changes
+
+The type system of CockroachDB currently makes an assumption that anywhere
+a type is present in an AST, that type is statically known. In code, this
+means that every AST object that holds a type (like a `CastExpr` or
+`ColumnDef`) holds a `*types.T`, which is constructed at parse time.
+As part of implementing user defined types, the type system must be taught
+that all types are no longer statically known. The general idea is to change
+the types in AST nodes to a new interface representing an unresolved type
+reference. These type references can then be resolved into `*types.T` through
+type resolution. Additionally, we must enforce that types are only attempted
+to be accessed after type checking, when all type references have been resolved.
+A prototype of this approach can be found in
+[this PR](https://github.com/cockroachdb/cockroach/pull/47386).
+
+After the process of type resolution, enums need a `types.T` for interaction
+with other components of the system. We will introduce a new family for enums,
+and the `types.T` for an enums will contain the stable ID for the
+`TypeDescriptor` that backs the type. The `types.T` will also contain extra
+fields for an enum like the mapping of names to values. Importantly, these
+extra fields will not be serialized as part of the proto. Instead, when a
+type is resolved, the returned `*types.T` will be hydrated to populate these
+fields.
+
+A potential option was to avoid using
+a `TypeDescriptor` and instead just extend the `types.T` proto to contain
+necessary fields for user defined types. However, this is not feasible because
+the `types.T` proto's are stored on disk in various descriptors. It is too
+expensive to update all descriptors that contain a type every time the type
+is altered.
+
+A new `Datum` `DEnum` will be introduced to represent values of the
+enums at runtime. A `DEnum` will store the physical representation of the
+enum as well as the hydrated `*types.T` of its type. The extra fields in the
+`*types.T` that hold information about enum values will be used for datum
+operations without the need to thread ID resolution capabilities to evaluation
+of operations on datums.
+
+When a user-defined type is created in Postgres, Postgres will automatically
+create an alias for an array of the new type. For example, if a user creates
+a type `days`, the system would also create the type `_days` as an alias for
+`days[]`. This type tracks changes made to the referenced type as it
+moves through schemas and is dropped.
+
+## Semantic Analysis Changes
+
+The optimizer will need to be taught about the check constraint implied by
+a column being of an enum type. Additionally, it will need to be taught how
+to convert enum values from their input string representation into their
+`Datum` physical representation.
+
+The `Catalog` that is used by the optimizer will need to be extended to support
+resolution of types. The way that the catalog represents user defined types is
+important for invalidation of cached plans. If a type is updated, all plans
+containing data sources using the type need to be invalidated.
+
+## DistSQL
+The gateway node that plans a SQL query has access to all resolved type
+information for the query. Remote nodes that different parts of the query
+are planned on need access this information in order to correctly execute
+the query. In particular, these nodes need to hydrate their `*types.T`
+containers with metadata and they need to parse and type check serialized
+expressions. The hydration of `*types.T` objects can be done at operator
+initialization. The trickier problem is type checking serialized expressions --
+we don't want to pay the cost of name resolution again. Our strategy is to
+serialize user defined type references with their OIDs similar to how column
+references are serialized. All explicit references to user defined types (i.e.
+in casts or user defined type value literals) will be serialized like `@`.
+The expression initialization process will resolve these OID references to the
+correct `TypeDescriptor`. To actually resolve these references, we access the set
+of leased descriptors through a `descs.Collection` that is initialized for each
+flow.
+
+# Alternatives
+
+## Namespacing and Metadata Storage
+During discussion of the RFC, some alternatives were debated. In particular,
+the ideas of using a separate namespace table for types and/or a separate
+descriptor table for metadata storage. The benefit of a separate namespace
+table is that it has the potential of making future work in allowing tables
+to be interpreted as types more straightforward. However, using a separate
+namespace table complicates existing name resolution and conflict detection
+strategies. A separate descriptor table allows for scans over all tables or
+types to not have to touch descriptors of different types, which is a
+performance improvement for catalog table operations. However, this problem
+is somewhat orthogonal to this work, and would be better solved by building
+some sort of indexing structure on the `system.descriptor` table.
+Using the existing namespace table allows most of the existing name resolution
+code to be used directly, and using the same descriptor table allows for
+leasing primitives to be built on only one system table.
+
+## Overall Alternative
+One alternative approach to this physical layout was to store just an
+enum ID on disk, and store ordering and representation information in
+a separate lookup table. When operations like on enums would involve
+joining or rendering the enums, a join would be produced against this
+reference table. This allows for easy changing of enum data, but
+results in a variety of complexity during planning.
+
+# Unresolved questions
+
+It is unclear what interactions will arise between this work and the
+planned/ongoing work with user defined schemas.
diff --git a/src/current/files/cockroach/docs/RFCS/20200811_non_blocking_txns.md b/src/current/files/cockroach/docs/RFCS/20200811_non_blocking_txns.md
new file mode 100644
index 00000000000..fdb5d3841de
--- /dev/null
+++ b/src/current/files/cockroach/docs/RFCS/20200811_non_blocking_txns.md
@@ -0,0 +1,1096 @@
+- Feature Name: Non-Blocking Transactions
+- Status: in-progress
+- Start Date: 2020-08-11
+- Authors: Nathan VanBenschoten
+- RFC PR: #52745
+- Cockroach Issue: None
+
+# Summary
+
+Non-blocking transactions are a variant of CockroachDB's standard read-write
+transaction protocol that permit low-latency, global reads of read-mostly and
+read-only (excluding maintenance events) data. The transaction protocol and the
+replication schema that it is paired with differ from standard read-write
+transactions in two important ways:
+- non-blocking transactions support a replication scheme over Ranges that they
+ operate on which allows all followers in these Ranges to serve **consistent**
+ (non-stale) follower reads.
+- non-blocking transactions are **minimally disruptive** to reads over the data
+ that they modify, even in the presence of read/write contention.
+
+The ability to serve reads from follower and/or learner replicas is beneficial
+both because it can reduce read latency in geo-distributed deployments and
+because it can serve as a form of load-balancing for concentrated read traffic
+in order to reduce tail latencies. The ability to serve **consistent**
+(non-stale) reads from any replica in a Range makes the functionality accessible
+to a larger class of read-only transactions and accessible for the first time to
+read-write transactions.
+
+The ability to perform writes on read-heavy data without causing conflicting
+reads to block is beneficial for providing predictable read latency. Such
+predictability is doubly important in global deployments, where the cost of
+read/write contention can delay reads for 100's of ms as they are forced to
+navigate wide-area network latencies in order to resolve conflicts.
+
+These properties combine to prioritize read latency over write latency for some
+configurable subset of data, recognizing that there exists a sizable class of
+data which is heavily skewed towards read traffic.
+
+Non-blocking transactions are provided through extensions to existing concepts
+in the CockroachDB architecture (i.e. uncertainty intervals, read refreshes,
+closed timestamps, learner replicas) and compose with CockroachDB's standard
+transaction protocol intuitively and effectively.
+
+This proposal serves as an alternative to the [Consistent Read Replicas
+proposal](https://github.com/cockroachdb/cockroach/pull/39758). Whereas the
+Consistent Read Replicas proposal enforces consistency through communication,
+this proposal enforces consistency through semi-synchronized clocks with bounded
+uncertainty.
+
+# Motivation / Making Global Easy
+
+Various efforts over the past two years have recognized the need for a
+simplified and more prescriptive approach towards global deployment of
+CockroachDB. These efforts have called out a lack of high-level abstractions,
+incomplete topology patterns, and missing architectural support for read-heavy
+global (non-localized) data as blockers towards this goal.
+
+Working within the framework discussed in the [CockroachDB Multi-Region
+Abstractions](https://docs.google.com/document/d/16ON-HPv5qFjK4sXuJQRqI1prfZYh4sBBAa2F0gkPkwQ/edit?usp=sharing)
+document, this proposal identifies non-blocking transactions as the answer to
+this missing architectural component and a foundation upon which we can build
+upon to define sensible topology patterns and provide much needed high-level
+abstractions.
+
+The CockroachDB Multi-Region Abstractions doc suggested that SQL tables are
+split into two categories: "geo-partitioned" and "reference". It then discussed
+a "hub-and-spokes" replication topology, wherein each Range in the system is
+composed of a limited set of nearby voter replicas, whose diameter is based on
+the desired failure tolerance (e.g. zone or region failure tolerance), combined
+with one or more learner replicas in every other region. This topology minimizes
+consensus latency while establishing a covering of data across all regions to
+minimize read latency, subject to transaction constraints. Working within this
+model, this RFC considers non-blocking transactions to be the key architectural
+advancement needed to support "reference" tables.
+
+With non-blocking transactions and a hub-and-spokes replication topology, the two
+categories of SQL tables have the following behavior:
+
+| | geo-partitioned | reference |
+|--------------------------------------|-------------------------|---------------------------|
+| data locality | local | global |
+| data access | read-often, write-often | read-mostly, write-rarely |
+| local read latency | fast | N/A |
+| local write latency | fast | N/A |
+| remote/global read latency | slow, fast if stale | fast |
+| remote/global write latency | slow | slow |
+| reads block on local writes | yes, fast | N/A |
+| writes block on local writes | yes, fast | N/A |
+| reads block on remote/global writes | yes, slow | no, fast |
+| writes block on remote/global writes | yes, slow | yes, slow |
+
+_where: fast < 5ms, slow > 100ms_
+
+We can see that data within geo-partitioned tables remains fast to read and
+write within its local region, but slow to access from remote regions. Remote
+reads in read-only transactions have the opportunity to downgrade to a lower
+consistency level (i.e. exact or bounded staleness reads) to improve latency,
+but read-write transactions that read from or write to remote data are slow.
+Meanwhile, data within reference tables is fast to read from any region, even
+within read-write transactions and even with read/write contention. However,
+data within reference tables is consistently slow to write to from any region.
+
+This structure makes it easy to identify which category a given table falls
+into. If its data has geographic access locality then it should be set to
+"geo-partitioned". If not, then it should be set to "reference".
+
+This effectively handles all forms of data except that which has no access
+locality and is also write-heavy. Such data is fundamentally incompatible with
+low-latency modifications under linearizability across large geographic
+distances. These access patterns require either synchronous coordination or
+weakened [consistency levels](https://en.wikipedia.org/wiki/Eventual_consistency) and limited
+[operational generality](https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type).
+Both of these compromises make global data harder to work with and appear
+antithetical to the goal of "making global easy", so we deem the communication
+latency necessary for linearizable writes to global data to be an acceptable cost.
+
+To summarize, this proposal considers non-blocking transactions to be a key
+architectural advancement necessary to support low-latency reads of global data,
+which itself is a must-have for many, if not most, global deployments.
+
+## What's wrong with follower reads?
+
+Closed timestamps and [follower
+reads](https://github.com/cockroachdb/cockroach/blob/master/docs/RFCS/20180603_follower_reads.md)
+provide a mechanism to serve *consistent stale reads* from follower replicas of
+a Range without needing to interact with the leaseholder of that Range. There
+are two primary reasons why a user of CockroachDB may want to use follower
+reads:
+1. to avoid wide-area network hops: If a follower for a Range is in the same
+ region as a gateway and the leaseholder for that Range is in a separate
+ region as the gateway, follower reads provides the ability to avoid an
+ expensive wide-area network jump on each read. This can dramatically reduce
+ the latency of these reads.
+2. to distribute concentrated read traffic: Range splitting provides the ability to
+ distribute heavy read and write traffic over multiple machines. However, Range
+ splitting cannot be used to spread out concentrated load on individual keys.
+ Follower reads provides a solution to the problem of concentrated read
+ traffic by allowing the followers of a Range, in addition to its leaseholder,
+ to serve reads for its data.
+
+However, this capability comes with a large asterisk. Follower reads are only
+suitable for serving _**historical**_ reads from followers. They have no ability
+to serve consistent reads at the current time from followers. Even with
+[attempts](https://github.com/cockroachdb/cockroach/pull/39643) to reduce the
+staleness of follower reads, their historical nature will always necessarily
+come with large UX hurdles that limit the situations in which they can be used.
+
+The most damaging of these hurdles is that, for all intents and purposes, follower
+reads cannot be used in any read-write transaction - their use is limited to read-only
+transactions. This dramatically reduces their usefulness, which has caused us to
+look for other solutions to this problem, such as [duplicated indexes](https://www.cockroachlabs.com/docs/stable/topology-duplicate-indexes.html)
+to avoid WAN hops on foreign key checks.
+
+Another hurdle is that the staleness they permit requires buy-in, so accessing
+them from SQL [requires application-level changes](https://github.com/cockroachdb/cockroach/blob/master/docs/RFCS/20181227_follower_reads_implementation.md).
+Users need to specify an `AS OF SYSTEM TIME` clause on either their statements
+or on their transaction declarations. This can be awkward to get right and
+imposes a large cost on the use of follower reads. Furthermore, because a
+statement is the smallest granularity that a user can buy-in to follower reads
+at, there is a strong desire to support [mixed timestamp statements](https://github.com/cockroachdb/cockroach/issues/35712),
+which CockroachDB does not currently support.
+
+Because of all of these reasons, follower reads in its current form remains a
+specialized feature that, while powerful, is also not usable in a lot of
+important cases. It's becoming increasingly clear that to continue improving
+upon our global offering, we need a solution that solves the same kinds of
+problems that follower reads solves, but for all forms of transaction and
+without extensive client buy-in. This RFC presents a proposal that fits within a
+spectrum of possible solutions to address these shortcomings.
+
+## What's wrong with the "duplicate index" pattern?
+
+CRDB's current "solution" for people wanting local consistent reads is the
+[duplicated index pattern](https://www.cockroachlabs.com/docs/stable/topology-duplicate-indexes.html).
+This RFC proposes replacing that pattern with capabilities built into the lower
+layers of the system, on the argument that it will result in a simpler, cheaper
+and more reliable system. This section discusses the current pattern (in
+particular, its problems).
+
+The duplicate indexes idea is that you create a bunch of indexes on your table,
+one per region, with each index containing all the columns in the table (using
+the `STORING ,...` syntax). Then you create a zone config per region,
+pinning the leaseholders of each respective index to a different region. Our query
+optimizer is locality-aware, and so, for each query trying to access the table
+in question, it selects an index collocated with the gateway in the same
+locality. We're thus taking advantage of our transactional update semantics to
+keep all the indexes in sync and get the desired read latencies without any help
+from the KV layer.
+
+While neat, there's some major problems with this scheme which make it a tough
+proposition for many applications. Some of them are fixable with more work, at
+an engineering cost that seems greater than the alternative in this RFC, others
+seem more fundamental assuming no cooperation from KV.
+
+### Problem 1: Read Latency under Contention
+
+Without contention, reads using the duplicate index pattern are served locally
+at local latencies. With an index + pinned leaseholder in each region, any read
+on a "duplicate index" table can be served in a few milliseconds without a WAN
+hop. Better yet, these reads are not stale, so they can be comfortably used in
+read-only and read-write transactions (often during foreign key checks).
+
+But these local read latencies can provide a false sense of performance
+predictability. While the duplicate index pattern optimizes for read latency, it
+badly pessimizes write latency. On the surface, this seems to be the expected
+tradeoff. The problem is that when reads and writes contend, reads can get
+blocked on writes and have to wait for writing transactions to complete. If the
+writing transaction is implicit, this results in at least 1 WAN RTT of blocking.
+If the writing transaction is explicit, this results in at least 2 WAN RTTs of
+blocking.
+
+Reads that users expect to be fast (1ms-3ms) can quickly get two orders of
+magnitude slower (120ms-240ms or more) due to any read/write contention. As
+we've seen, this can be a shocking violation of expectations for users.
+
+Over the past few months, we've seen users recognize this issue, both with the
+duplicate index pattern and with standard CockroachDB transactions. The
+corresponding ask has often been to support a `READ COMMITTED` isolation level.
+Our response is to perform stale reads using `AS OF SYSTEM TIME` to avoid
+contention, but this pushes the burden onto the readers to accept stale data in
+exchange for more predictable read latency. This seems backwards, as avoiding
+blocking should really be the responsibility of the infrequent writers. It also
+only works if the readers are read-only transactions, as read-write transactions
+cannot use `AS OF SYSTEM TIME`, which removes a large class of use cases like
+foreign key checks in read-write transactions.
+
+### Problem 2: Ergonomics
+
+So you’ve got 15 regions and 20 of these global tables. You’ve created 15x20=300
+indexes on them. And now you want to take a region away, or add a new one. You
+have to remember to delete or add 20 indexes. Tall order. And it gets better:
+say you want to add a column to one of the tables. If, naively, you just do your
+`ALTER TABLE` and add that column to the primary key, then, if you had any
+`SELECT *` queries (or queries autogenerated by an ORM) which used to be served
+locally from all regions, now all of a sudden all those indexes we’ve created
+are useless. Your application falls off a cliff because, until you fix all the
+indexes, all your queries travel to one central location. The scheme proves to
+be very fragile, breaking surprisingly and spectacularly.
+
+What would it take to improve this? Focusing just on making the column
+components of each index eventually consistent, I guess we'd need a system that
+automatically performs individual schema changes on all the indexes to bring
+them up to date, responding to table schema changes and regions coming and
+going. Tracking these schema changes, which essentially can't be allowed to
+fail, seems like a big burden; we have enough trouble with sporadic
+user-generated schema changes without a colossal amplification of their number
+by the system. We'd also presumably need to hide these schema changes from the
+user (in fact, we need to hide all these indexes too), and so a lot of parts of
+the system would need to understand about regular vs hidden indexes/schema
+changes. That alone seems like major unwanted pollution. And then there's the
+issue of wanting stronger consistency between the indexes, not just eventual
+consistency; I'm not sure how achievable that is.
+
+### Problem 3: Fault tolerance
+
+You’ve got 15 regions - so 15 indexes for each reference table - and you’ve used
+replication settings to “place each index in a region”. How exactly do you do that?
+You've got two options:
+1. Set a constraint on each index saying `+region=`. The problems with this
+is that when one region goes down (you’ve got 15 of them, they can’t all be
+good) you lose write-availability for the table (because any writes needs to
+update all the indexes). So that's no good.
+
+2. Instead of setting a constraint which keeps all of an index’s replicas in one
+region, one can let the system diversify the replicas but set a “leaseholder
+preference” such that reads of the index from that region are fast, but the
+replicas of the index are more diverse. You got to be careful to also constrain
+one replica to the desired leaseholder region, otherwise the leaseholder
+preference by itself doesn't do anything. Of course, as things currently stand,
+you have to do this 15 times. This configuration is resilient to region failure,
+but there's still a problem: any leaseholder going away means the lease goes out
+of region - thus making reads in the now lease-less region non-local (because we
+only set up one replica per region).
+
+So, all replicas in one region is no good, a single replica in one region is not
+good, and, if the replication factor is 3, having 2 replicas in one region is
+also not good because it’s equivalent to having all of them in one region. What
+you have to do, I guess, is set a replication factor of at least 5 *for each
+index* and then configure two replicas in the reader region (that’s 3x5=15
+replicas for every index for a 3-region configuration; 10x5=50 replicas in a
+10-region configuration). In contrast, with this RFC's proposal, you'd have 2
+replicas per region (so 2 per region vs 5 per region). Our documentation doesn't
+talk about replication factors and fault tolerance. Probably for the better,
+because these issues are hard to reason about if you don't spend your days
+counting quorums. This point shows the problem with doing such replication
+schemes at the SQL level: we hide from the system the redundancy between the
+indexes (and between the replicas of different indexes), and so then we need to
+pay a lot to duplicate storage and replication work just to maintain the
+independent availability of each index.
+
+# Working Assumptions
+
+The bulk of this proposal is founded on the idea that at some time in the
+future, we will be able to dramatically reduce the clock uncertainty that we
+feel comfortable running with, either across the board or at least within
+CockroachCloud.
+
+Wide area network (WAN) round-trip time (RTT) latencies can be as large as 200ms
+in the global deployment scenarios that we are interested in (ref:
+[AWS](https://docs.aviatrix.com/_images/inter_region_latency.png),
+[Azure](https://docs.aviatrix.com/_images/arm_inter_region_latency.png),
+[GCP](https://docs.aviatrix.com/_images/gcp_inter_region_latency.png)). The
+current default clock uncertainty offset is 500ms. This proposal explores the
+half of the design space in which clock uncertainty bounds drop below WAN RTT
+time. Within this regime, it becomes cheaper to establish causality between
+events using clocks and waiting than by communicating with distant nodes.
+
+As such, the uncertainty bounds necessary for this proposal to make sense are
+much lower than what we currently use, but are also well within the realm of
+possibility for software-only solutions. Clouds are continuing to invest into
+the software-level time synchronization services that they provide. For
+instance, Amazon released a [Time Sync
+Service](https://aws.amazon.com/about-aws/whats-new/2017/11/introducing-the-amazon-time-sync-service/)
+in 2017 that strives to provide reliable time references for EC2 instances.
+Similarly, software-level time synchronization has seen interest from the
+academic community in recent years [4], with research even leading to
+[productionized service offerings](https://www.ticktocknetworks.com/) in some
+cases. Lastly, users are already running CockroachDB with battle-tested time
+synchronization services like [Kronos](https://github.com/rubrikinc/kronos)
+which we may eventually explore using or extending.
+
+All of this means that dramatically reducing the clock uncertainty bounds that
+we run with seems realistic over the next few years. Combined with that fact
+that this proposal makes sense even for fairly large clock uncertainty values
+(up to 200ms), it is safe to say that atomic clocks and TrueTime are not
+prerequisites for this proposal!
+
+For the rest of this proposal, we will assume that the clock uncertainty offset
+is configured to 30ms. We will also assume that the global cluster topologies
+that we are interested in have diameters of 120ms RTTs. The proposal will still
+work with uncertainty offsets larger than this value, but with a larger offset,
+writers and contended readers (those that observe values in their uncertainty
+interval) will be forced to wait longer in some situations. So while a low clock
+uncertainty offset is not a prerequisite for this proposal, one does more
+effectively demonstrate the benefits that the proposal provides.
+
+# Guide-level explanation
+
+## Non-Blocking Transactions
+
+The RFC introduces the term "non-blocking transaction", which is a variant of
+the standard read-write transaction that performs locking in a manner such that
+contending reads by other transactions can avoid waiting on its locks.
+
+The RFC then introduces the term "non-blocking Range", which is a Range in which
+any transaction that writes to it is turned into a non-blocking transaction, if
+it is not one already. In doing so, these non-blocking Ranges are able to
+propagate [closed timestamps](20180603_follower_reads.md) in the future of
+present time. The effect of this is that all replicas in a non-blocking Range
+are expected to be able to serve transactionally-consistent reads at the present
+time. In this sense, all followers and all learners (non-voting followers) in a
+non-blocking Range implicitly behave as "consistent read replicas".
+
+In the reference-level explanation, we will explore how non-blocking
+transactions are implemented, how their implementation combines with
+non-blocking Ranges to provide consistent read replicas, how they interact with
+other standard read-only and read-write transactions, and how they interact with
+each other.
+
+For now, it is sufficient to say that "non-blocking transactions" are hidden
+from DML statements. Instead, users interact with "non-blocking Ranges" by
+choosing which key ranges should consist of non-blocking Ranges. These Ranges
+are configured using zone configurations, though we'll likely end up adding a
+nicer abstraction on top of this (e.g. `CREATE REFERENCE TABLE ...`).
+
+In the future, we could explore running non-blocking transactions on standard
+Ranges, which would still provide the "non-blocking property" (i.e. less
+disruptive to conflicting reads) without providing the "consistent reads from
+any follower" property. However, doing so is not explored in this RFC.
+
+## Example Use Cases
+
+### Reference Tables
+
+It is often the case that a schema contains one or more tables composed of
+immutable or rarely modified data. These tables fall into the category of "read
+always". In a geo-distributed cluster where these tables are read across
+geographical regions, it is highly desirable for reads of the tables to be
+servable locally in all regions, not just in the single region that the tables'
+leaseholders happens to land. A common example of this arises with foreign keys
+on static tables.
+
+Until now, our best solution to this problem has been [duplicated indexes](https://www.cockroachlabs.com/docs/v19.1/pology-duplicate-indexes.html).
+This solution requires users to create a collection of secondary SQL indexes that
+each contain every single column in the table. Users then use zone configurations
+to place a leaseholder for each of these indexes in every geographical locality.
+With buy-in from the SQL optimizer, foreign key checks on these tables are able
+to use the local index to avoid wide-area network hops.
+
+A better solution to this problem would be to use non-blocking Ranges for these
+tables. This would reduce operational burden, reduce write amplification, avoid
+blocking on contention, and increase availability.
+
+### Read-Heavy Tables
+
+In addition to tables that are immutable, it is common for schemas to have some
+tables that are mutable but only rarely mutated. An example of this is a user
+profile table in an application for a global user base. This table may be read
+frequently across the world and yet it is expected to be updated infrequently.
+
+## Transaction Latency Comparison
+
+To get a feel for the interactions between standard and non-blocking
+transactions, we estimate the latency that different transaction types would see
+with and without contention. Here, we continue to make the clock synchronization
+and geographic latency [assumptions](#Working-Assumptions) we made earlier (30ms
+uncertainty, 120ms WAN RTT).
+
+We compare under two different cluster topologies. The first is a "hub and
+spokes" cluster topology that places long-lived learners for every Range in each
+region but keeps a Range's write quorum within a region (~5ms replication). The
+second is a "global replication" topology that places a voter for every Range in
+each region, which increases replication latency to 120ms. We also add a fixed
+3ms latency for the evaluation of each transaction.
+
+We assume the remote reads are able to read from follower replicas, except where
+otherwise specified.
+
+### Hubs and Spokes
+
+| | without contention | contending with standard txn | read contending with non-blocking txn | write contending with non-blocking txn |
+|------------------------------------------------------|-----------------|--------------------|------------------|------------------|
+| local read-only txn | 3ms | 21ms (3+8+10) | 33ms (3+30) | N/A |
+| remote read-only txn | 3ms | 141ms (3+8+10+120) | 33ms (3+30) | N/A |
+| remote read-only txn (no follower reads) | 123ms (3+120) | 141ms (3+8+10+120) | N/A | N/A |
+| read-write txn with local reads | 8ms (3+5) | 26ms (8+8+10) | 38ms (8+30) | 138ms (8+130) |
+| read-write txn with remote reads | 8ms (3+5) | 146ms (8+8+10+120) | 38ms (8+30) | 138ms (8+130) |
+| read-write txn with remote reads (no follower reads) | 128ms (3+5+120) | 146ms (8+8+10+120) | N/A | N/A |
+| non-blocking read-write txn | 138ms (3+5+130) | 156ms (138+8+10) | 156ms (138+8+10) | 156ms (138+8+10) |
+
+NOTE: 8+10 in various places is read-write txn's sync contention footprint plus
+its async contention footprint.
+
+NOTE: assume `non_blocking_duration` is equal to RTT/2 + uncertainty + some padding = 130ms.
+
+### Global Replication
+
+| | without contention | contending with standard txn | read contending with non-blocking txn | write contending with non-blocking txn |
+|------------------------------------------------------|-------------------|-------------------------|---------------------|---------------------|
+| local read-only txn | 3ms | 366ms (3+123+240) | 33ms (3+30) | N/A |
+| remote read-only txn | 3ms | 486ms (3+123+240+120) | 33ms (3+30) | N/A |
+| remote read-only txn (no follower reads) | 123ms (3+120) | 486ms (3+123+240+120) | N/A | N/A |
+| read-write txn with local reads | 123ms (3+120) | 486ms (123+123+240) | 153ms (123+30) | 253ms (123+130) |
+| read-write txn with remote reads | 128ms (3+120) | 606ms (123+123+240+120) | 153ms (123+30) | 253ms (123+130) |
+| read-write txn with remote reads (no follower reads) | 243ms (3+120+120) | 606ms (123+123+240+120) | N/A | N/A |
+| non-blocking read-write txn | 253ms (3+120+130) | 501ms (138+123+240) | 501ms (138+123+240) | 501ms (138+123+240) |
+
+NOTE: 123+240 in various places is read-write txn's sync contention footprint plus
+its async contention footprint.
+
+# Reference-level explanation
+
+## Detailed design
+
+High-level overview:
+
+- Non-blocking transactions read and write at `non_blocking_duration` in the
+ future of present time.
+- After committing, they wait out the `non_blocking_duration` before
+ acknowledging the commit to the client.
+- Conflicting non-stale readers ignore future writes until they enter their
+ uncertainty interval, after which they wait until the write's timestamp is
+ above the reader's clock before reading the value (maximum wait of max clock
+ offset = 30ms).
+- non-blocking Range leaseholders close out timestamps in the future of present
+ time.
+- non-blocking Range followers receive these future-time closed timestamps and
+ can serve present time follower reads.
+- KV clients are aware that they can route read requests to any replica in a
+ non-blocking Range.
+
+### Aside: "Present Time" and Uncertainty
+
+Today, all transactions in CockroachDB pick a provisional commit timestamp from
+their gateway's local HLC when starting. This provisional commit timestamp is
+where the transaction performs its reads and where it initially performs its
+writes. For various reasons, a transaction may move its timestamp forward, which
+eventually dictates its final commit timestamp. However, a standard transaction
+maintains the invariant that its commit timestamp is always lagging "present"
+time.
+
+The meaning of present time is somewhat ambiguous in a system with
+semi-synchronized clocks. For the purpose of this document, we can define it as
+the time observed on the node in the system with the fastest clock. In a well
+behaved cluster that respects its configured maximum clock offset bounds,
+present time must be no more than `max_offset` in the future of the time
+observed on any other node in the system. This guarantee is the foundation upon
+which uncertainty intervals enforce single key linearizability within
+CockroachDB. If a write were to commit from the fastest node in the cluster at
+"present time", a causally dependent read from the slowest node would be
+guaranteed to see the write either at a lower timestamp than its provisional
+timestamp or at least in its uncertainty interval. Therefore, if reading
+transactions makes sure to observe all values in their uncertainty interval,
+stale reads are not possible and linearizability is preserved.
+
+### Writing in the Future
+
+As defined above, "non-blocking transactions" perform locking in such a way that
+contending reads do not need to wait on their locks. In practice, this means
+that non-blocking transactions **perform their writes at a timestamp in advance
+of present time**. In a sense, we can view this as scheduling a change to happen
+at some time in the future.
+
+The result of this is that the values written by a non-blocking transaction are
+committed and their locks removed (i.e. intents resolved) by the time that a
+read of conflicting keys needs to observe the effect of a non-blocking
+transaction. If the locks protecting a non-blocking transaction are removed by
+the time that its effects drop below the read timestamp of a conflicting read
+then the read can avoid the questions: "Did this conflicting transaction commit?
+Should I trust its effects?". Instead, it can simply go ahead and read the value
+(\*) because it knows that the transaction committed, no coordination required.
+
+This need for coordination to determine whether a read should observe the
+effects of a conflicting write is the fundamental reason why writes can be so
+disruptive to conflicting reads. We saw that in [Problem 1: Read Latency under
+Contention](#Problem-1:-Read-Latency-under-Contention). A read that conflicts
+with the lock and the provisional value of a write cannot determine locally
+whether that lock and value are pending, committed, or aborted. Instead, it
+needs to reach out to that transaction's record. Transaction priorities dictate
+whether the read waits or whether it can force a pending writer out of its way.
+Either way, coordination is required, and as a result, the read may need to
+perform a remote network hop (or several). Depending on the location of the
+conflicting write's transaction record, this network hop may be over the WAN and
+may cost 100's of ms.
+
+By scheduling the writes sufficiently far in the future, we ensure that reads
+never observe locks or provisional values, only committed values that have been
+scheduled to go into effect at a specific point in time. Coordination on the
+part of the read is no longer required. As a result, non-blocking writes are no
+longer disruptive to conflicting reads.
+
+\* Things are never quite this easy. See [Uncertainty Intervals: To Wait or Not to Wait](#Uncertainty-Intervals:-To-Wait-or-Not-to-Wait).
+
+#### Aside: Communication vs. Conflict Boundaries
+
+It is interesting to note that this approach of moving "communication outside of
+conflict boundaries" is present in other attempts to provide serializable,
+low-latency, geo-replication transactions. Specifically, it is found in
+deterministic database architectures like SLOG, which establish an ordering of
+transactions before evaluating their result [5]. In doing so, transactions avoid
+wide-area network latencies while holding locks or blocking other transactions,
+minimizing their contention footprint and maximizing throughput under
+contention.
+
+Interactive transactions are largely incompatible with deterministic execution,
+as they submit transactions to the database in pieces. However, we can still
+recover some of the benefits of deterministic execution by writing in the
+future. Specifically, writing in the future moves WAN communication outside of
+read/write conflict boundaries. However, it does not move the WAN communication
+outside of write/write conflict boundaries.
+
+#### Synthetic Timestamps
+
+Writing in the future is new for CockroachDB. In fact, talking about time in the
+future in any capacity has been traditionally frowned upon. Instead, we try to
+only ever pass around HLC timestamps that were pulled from a real HLC clock.
+This ensures that if the timestamp is ever used to
+[Update](https://github.com/cockroachdb/cockroach/blob/bbbfdbd1919c6de411742c8442cfc3903d33ee86/pkg/util/hlc/hlc.go#L326-L332)
+an HLC clock, the resulting clock is guaranteed to still be within the
+`max_offset` of all other nodes.
+
+So we need to be careful with deriving future timestamps from timestamps pulled
+from an HLC. To that end, this RFC proposes the introduction of "synthetic
+timestamps". Synthetic timestamps are identical to standard timestamps (64 bit
+physical, 32 bit logical), except that they make no claim about the value of the
+clock that they came from, or even that they came from a clock at all. As such,
+they are allowed to be arbitrarily disconnected from the clocks in the system.
+
+##### Representation
+
+To distinguish between "synthetic" and "real" timestamps, this RFC proposes that
+we reserve a high-order bit in either the physical or the logical component of
+the existing timestamp structure to denote this fact.
+
+##### Timestamp.Forward
+
+Merging a "synthetic" and "real" timestamp, typically done using a `Forward`
+operation obeys the following rule: the `synthetic` bit from the larger of the
+two timestamps is carried over to the result. In the case of a tie between a
+"synthetic" and "real" timestamp, the `synthetic` bit is not carried over to the
+result. This is because a timestamp with a "real" bit is a firmer guarantee and
+carries more information than a timestamp with a "synthetic" – such a timestamp
+not only describes a time, but also guarantees that the time is less than or
+equal to "present time".
+
+
+Examples:
+```
+Forward({synth, 5}, {synth, 6}) = {synth, 6}
+Forward({real, 5}, {synth, 6}) = {synth, 6}
+Forward({synth, 5}, {real, 6}) = {real, 6}
+Forward({real, 5}, {real, 6}) = {real, 6}
+
+Forward({synth, 6}, {synth, 6}) = {synth, 6}
+Forward({real, 6}, {synth, 6}) = {real, 6}
+Forward({synth, 6}, {real, 6}) = {real, 6}
+Forward({real, 6}, {real, 6}) = {real, 6}
+```
+
+##### hlc.Update
+
+Outside of identifying synthetic vs. real timestamps, we must make one more
+change. The hybrid-logical clock structure exposes a method called `Update` that
+forwards its value to that of a real timestamp received from another member of
+the cluster. The best way to understand this method and its usage is that
+`Update` guarantees that after it returns, the local HLC clock will have a value
+equal to or greater than the provided timestamp and that no other node's HLC
+clock can have a value less that `timestamp - max_offset`.
+
+For "real" timestamps, it is cheap to implement this operation by ratcheting a
+few integers within the HLC because we know that the timestamp must have come
+from another node, which itself must have been ahead of the slowest node by less
+that `max_offset`, so the ratcheting operation will not push our HLC further
+than the `max_offset` away from the slowest node.
+
+For "synthetic" timestamps, we have to be more careful. Since the timestamp says
+nothing about the "present time", we can't just ratchet our clock's state.
+Instead, `hlc.Update` with a synthetic timestamp needs to wait until the HLC
+advances past the synthetic timestamp either on its own or due to updates from
+real timestamps. By waiting, we once again ensure that by the time the method
+call returns, no other node's HLC clock can have a value less that
+`timestamp-max_offset`.
+
+#### Commit Wait: Not Just for the Cool Kid With the Fancy Hardware
+
+The original [Spanner paper](https://storage.googleapis.com/pub-tools-public-publication-data/pdf/65b514eda12d025585183a641b5a9e096a3c4be5.pdf)
+discussed how the system added a delay to all read-write transactions before
+returning to the client to ensure that all nodes in the system had clocks in
+excess of the commit timestamp of a transaction before that transaction was
+acknowledged. This was discussed in section 4.1.2 and section 4.2.1 of the
+paper:
+
+> Before allowing any coordinator replica to apply the commit record, the coordinator
+> leader waits until TT.after(s), so as to obey the commit-wait rule described in
+> Section 4.1.2. Because the coordinator leader chose s based on TT.now().latest, and
+> now waits until that timestamp is guaranteed to be in the past, the expected wait
+> is at least 2 ∗ . This wait is typically overlapped with Paxos communication. After
+> commit wait, the coordinator sends the commit timestamp to the client and all other
+> participant leaders.
+
+Up to this point, CockroachDB has avoided this concern because of its use of
+uncertainty intervals. Instead of transactions ensuring that all nodes in the
+system have clocks in excess of the commit timestamp of a transaction before
+acknowledging it to the client, **CockroachDB ensures that all nodes in the
+system have clocks in excess of the commit timestamp of a transaction minus the
+`max_offset` (a weaker guarantee) before acknowledging it to the client**. This
+is satisfied trivially because, to this point, transactions have always written
+at or below "present time", which is by definition no further than `max_offset`
+ahead of the slowest node in the cluster. CockroachDB then implements
+uncertainty intervals on subsequent transactions to make up for this weakened
+guarantee and ensure linearizability just like Spanner.
+
+If we start committing transactions in the future, this guarantee is no longer
+trivially satisfied. If we want to ensure linearizability for the writes of
+these non-blocking transactions, we need to do a little extra work to ensure
+that the clocks on all nodes are sufficiently close to the commit timestamp of
+the transactions before acknowledging their success to clients.
+
+Conveniently, we already have a formalism for this – `hlc.Update`. Commit Wait
+would naturally fall out of calling `hlc.Update(txn.CommitTimestamp)` before
+acknowledging any transaction commit to a client. This would reduce to a no-op
+for standard transactions, and would result in a wait of up to
+`non_blocking_duration` for non-blocking transactions. This wait artificially
+increases the latency of non-blocking transactions, but critically, it ensures
+that we continue to preserve linearizability.
+
+It is worth noting that, like Spanner, this CommitTimestamp is chosen before the
+final round of consensus replication. This means that the wait will overlap with
+consensus communication and will therefore be less disruptive to non-blocking
+transactions than it may initially seem. So instead of the
+`non_blocking_duration` adding latency to the end of a non-blocking transaction,
+it will simply hide the rest of the transaction's latency. This means that given
+our estimate for the `non_blocking_duration` of 130ms in a global cluster, we
+would expect non-blocking transaction's to have a latency of roughly 130ms.
+
+#### Uncertainty Intervals: To Wait or Not to Wait
+
+Non-blocking transactions also force us to rethink uncertainty intervals and
+question both how and why they work. Uncertainty intervals are timestamp ranges
+within which a transaction is unable to make claims of causality. Given a
+bounded clock offset of `max_offset` between nodes, a transaction cannot
+definitively claim that a write that it observes within a `max_offset` window on
+either side of its original timestamp if causally ordered before it. But of
+course, it would be impossible for the write to be causally ordered after it. So
+to ensure that all possible causal dependencies are captured and linearizability
+is enforced, a transaction will bump its timestamp to ensure that it observes
+all values in its uncertainty interval. This either incurs a transaction refresh
+or a full transaction restart.
+
+The writes performed by non-blocking transactions are no exception. When a read
+observes a write performed by a non-blocking transaction in its uncertainty
+interval, it will need to bump its timestamp so that it observes the write.
+
+The complication here is that if a reading transaction bumped its timestamp
+above a value written by a non-blocking transaction in its uncertainty interval,
+it could end up with a timestamp in the future of "present time". This could
+risk resulting in a stale read if this read immediately committed and then
+triggered a causally dependent read that hit a slower gateway where the written
+value was not considered in the second read's uncertainty interval. The following
+example demonstrates this hazard:
+
+```
+params:
+ clock_offset = 10
+ non_blocking_duration = 50
+
+clocks:
+ node A's clock = 100
+ node B's clock = 95
+
+- txn 1 begins on node A
+- txn 1 writes in future @ time 150
+- txn 1 commits
+- txn 1 begins waiting out non_blocking_duration before returning to client
+
+- time advances by 40
+clocks:
+ node A's clock = 140
+ node B's clock = 135
+
+- txn 2 begins on node A
+- txn 2 reads at 140 and observes write in uncertainty interval
+- txn 2 returns write to client
+- txn 2 commits and acks client
+
+- txn 3 is triggered by commit of txn 2 (causal dependency)
+- txn 3 begins on node B
+- txn 3 reads at 135 and observed no write in uncertainty interval
+- txn 3 returns nothing to client
+- txn 3 commits and acks client
+
+- time advances by 10
+clocks:
+ node A's clock = 150
+ node B's clock = 145
+
+- txn1 acks client
+
+HAZARD! txn3 just saw stale read. It was triggered by txn2, which had observed
+ the value written by txn1, but it did not observe the value. This is a
+ violation of the monotonic reads property, and therefore a violation of
+ single-key linearizability.
+```
+
+Careful readers may notice that the first read-only transaction (txn2) would
+have had to bump its timestamp to 150 upon observing a value in its uncertainty
+interval. So if it had respected the Commit Wait rule then it would have been
+delayed before committing, after which point the stale read would not be
+possible because node B's clock would have advanced far enough that the value
+would have been in the second read-only transaction's (txn3) uncertainty
+interval.
+
+But relying on Commit Wait here violates a desirable property of our transaction
+model – that committing a read-only transaction is a no-op. To recover this
+property, we'll want to wait out the uncertainty before the refresh/retry.
+Again, the easiest way to do this is to `hlc.Update(txn.ReadTimestamp)` before
+allowing a refreshed/retried transaction to read at its new timestamp. This
+again becomes a no-op when the new ReadTimestamp is "real" and not "synthetic".
+
+So while reading transactions which conflict with writes from non-blocking
+transactions never block on locks, because those locks have already been
+released by the time the read conflicts with the write, these reading
+transactions do occasionally need to wait to resolve uncertainty and ensure that
+linearizability is maintained. We present the full picture of how non-blocking
+transactions interact with standard transactions [later
+on](#non-blocking-and-standard-transaction-interactions).
+
+### Future-Time Closed Timestamps
+
+To this point, we've talked about writing in the future as a means to reduce
+contention, but doing so has a second major benefit – it enables non-stale
+follower reads if coupled with a closed timestamp mechanism that closes off
+timestamps in the future.
+
+This RFC proposes that "non-blocking Ranges" use a separate closed timestamp
+tracker that is configured to close time out at least `WAN_RTT / 2 + max_offset`
+in the future. This ensures that all followers in these ranges will hear about
+and active closed timestamps above "present time" so that they can serve all
+"present time" reads locally.
+
+The details of this change are TBD, but all complications here appear to be
+engineering related and not foundational. For instance, there currently exists
+only a single closed timestamp tracker per store. Future-Time Closed Timestamps
+would mandate the existence of at least two closed timestamp trackers.
+
+#### Aside: Writing in Future vs. Closing Time in Future
+
+It may be evident to readers that there is some inherent but fuzzy connection
+between the idea of writing in the future and closing time in the future. It is
+interesting to point out that in systems like Spanner and YB which assign
+monotonically increasing timestamps to all writes within a replication group and
+therefore "instantaneously" close out time, these two operations are one and the
+same.
+
+CockroachDB's transaction and replication model are not quite set up to support
+this. Transactions in CockroachDB are optimized for committing at their original
+read timestamp, so they don't like when their writes get bumped to higher
+timestamps. This allows for optimistic, lockless reads. It also avoids a layer
+of write amplification, wherein provisional values must be moved to a higher
+timestamp during intent resolution. This write amplification is concerning
+enough that systems like Google's Percolator introduce a specialized storage
+component (see the "write metadata" column) to perform the translation without
+needing to rewrite the data [6].
+
+Additionally, replication in CockroachDB is below the level of evaluation (see
+[proposer evaluated kv](20160420_proposer_evaluated_kv.md)), so the timestamp of
+a replicated value need to be determined before replication-level sequencing.
+This allows for evaluation-level parallelism and less need for determinism
+during evaluation. This evaluation-level parallelism helps avoid the problem of
+the "parallelism gap" seen between compute and/or storage intensive processing
+before replication and after replication [7].
+
+For these reasons, the closed timestamp of a Range lags the timestamps of writes
+in a Range and the closed timestamp is more of a Range-level concern and less of
+a Request-level concern in CockroachDB (i.e. a single request cannot dictate the
+closed timestamp for a Range). This is why the closed timestamp dictates the
+time at which requests perform writes (as we'll see in the next section) instead
+of the other way around.
+
+### Becoming a Non-Blocking Transaction
+
+As stated earlier, this proposal does not expose "non-blocking transactions" to
+users directly. Instead, it exposes "non-blocking Ranges" to users. It was
+also mentioned earlier that a "non-blocking Range" will have a closed
+timestamp tracker that is leading "present time" by some duration. As it turns
+out, configuring this closed timestamp tracker to lead "present time" by
+`non_blocking_duration` is enough to implement "non-blocking transactions"
+without any other server-side changes.
+
+Any standard transaction that writes to a "non-blocking Range" will naturally
+get pushed into the future. If it had performed writes in the past, these will
+eventually get moved up to the new commit timestamp. If it had performed reads
+in the past, these will eventually get refreshed up to the new (synthetic)
+commit timestamp before committing. The new Commit Wait logic in the
+`txnCommitter` will ensure that these transactions naturally wait before
+retuning to the client.
+
+So in a very real sense, "non-blocking transactions" will not exist in the code,
+although it is still useful to classify any transaction that commits with a
+commit timestamp in the future of "present time" as non-blocking (i.e. those
+with synthetic commit timestamps).
+
+### Non-blocking and Standard Transaction Interactions
+
+There are a number of interesting interactions that can happen if we allow
+non-blocking transactions to touch (read from and/or write to) standard ranges
+and interact with standard transactions. These interactions can increase the
+latency of conflicting writes made by standard transactions. However, they will
+not meaningfully increase the latency of conflicting reads made by standard
+transactions.
+
+first \ second | standard read | standard write | non-blocking txn's read | non-blocking txn's write
+-------------------------|---------------|----------------|-------------------------|-------------------------
+standard read | both bump ts cache, no interaction | bump ts cache, may bump write ts, but still below present time | both bump ts cache, no interaction | bump ts cache, no interaction
+standard write | read ignores write if below, read waits on write if above | 2nd write waits on 1st write, bumps write ts above 1st write | read waits on write | non-blocking write waits on standard write
+non-blocking txn's read | both bump ts cache, no interaction | **bump ts cache, bump write ts, standard write becomes non-blocking write, must wait on commit** | both bump ts cache, no interaction | bump ts cache, may bump write ts
+non-blocking txn's write | **no interaction if write above uncertainty, read waits up to max_offset if write within uncertainty** | **standard write waits on non-blocking write, bumps write ts, standard write becomes non-blocking write, must wait on commit** | read ignores write if below, read waits on write if above | 2nd write waits on 1st write, bumps write ts above 1st write
+
+Interesting interactions in bold.
+
+Most of these interactions go away if we don't allow non-blocking transactions
+to touch standard ranges, but such a restriction would be a serious limitation
+to their utility.
+
+It should be noted that the contention footprint of a non-blocking transaction
+is only the duration that it holds its locks and does not include its Commit
+Wait latency. This means that while conflicting non-blocking transactions will
+each need to perform a Commit Wait, they need only wait out their own
+`non_blocking_duration`, so this latency is not additive.
+
+It should also be noted that while this "infection" of non-blocking transactions
+to other conflicting read-write transactions is undesirable, it is no more
+undesirable in practice than read-write transactions that touch global data and
+hold locks for long periods of time. This is what we would expect to see with
+alternative proposals such as the [Consistent Read Replicas
+proposal](https://github.com/cockroachdb/cockroach/pull/39758). Under the same
+workloads that would cause an "infection" of synthetic timestamps due to a
+non-blocking transaction that also touches contended data on standard Ranges, we
+would expect any proposal that uses locks to exhibit buildups of dependent
+transaction chains waiting on the global transaction's locks to be released.
+This would be similarly disruptive, so in many ways, such workloads would be an
+anti-pattern under either proposal.
+
+### Implementation Touch Points
+
+* Implement synthetic timestamps
+ * add bit on hlc.Timestamp
+ * wait on hlc.Update with synthetic timestamps
+* Implement CommitWait
+ * wait in txnCommitter after transaction commit
+ * hlc.Update(txn.CommitTimestamp)
+ * reduces to no-op for standard transactions
+* Introduce non-blocking Range concept
+ * Closed timestamp `non_blocking_duration` in future
+ * To do this, need to split up per-Store closed timestamp tracker
+ * Tune, maybe based on a Range's replication latency
+ * Hook up to cluster setting
+ * Add to zone configs
+* Route present-time reads to followers in non-blocking Ranges
+ * Similar to existing follower read logic, but Range specific
+ * Add bit on RangeDescriptor? Or to Lease? Both are cached in client
+* Introduce [long-lived learner replicas](https://github.com/cockroachdb/cockroach/issues/51943)
+ * Biggest work item, but...
+ * Already being worked on and generally useful outside of this proposal
+ * Also, not a hard requirement for the rest of this proposal
+
+#### Tuning non_blocking_duration
+
+The `non_blocking_duration` is the distance in the future at which non-blocking
+transactions acquire locks and write provisional values. The non-blocking
+property of these transactions is provided when this is far enough in the future
+such that a non-blocking transaction is able to navigate its entire operation
+set, commit, and resolve intents before its commit timestamp its passed by
+"present time + max clock offset". Further, the non-blocking property of these
+transaction is provided on followers when all of these effects are replicated
+and applied on followers by the time that the non-blocking transaction's commit
+timestamp is passed by "present time + max clock offset".
+
+As such, some tuning of this value will need to be performed, likely taking into
+account transaction latency and replication latency.
+
+This tuning will also need to take into account the "effective" clock skew
+between nodes to account for a follower that is leading a leaseholder by some
+duration. In this case, the follower's view of "present time" will lead that of
+the leaseholder by this skew. To avoid needing to wait for a sufficiently recent
+closed timestamp on the follower, the leaseholder will need to close time a
+little further in the future. In practice, we'll likely add some small buffer to
+the `non_blocking_duration` to account for effective clock skew.
+
+#### Non-Blocking Transaction Pushing
+
+A difficulty we face with tuning the `non_blocking_duration` is that we don't
+know how long a non-blocking transaction is going to take between the time that
+it writes its first intent until the time that it commits and resolves all of
+its intents. This means that if we tune too low, the transaction's intents may
+not be resolved by the time they drop below "present time" + uncertainty (i.e.
+become visible to readers) and this could cause unintentional blocking of
+readers. A potential mitigation to this is to actively push these intents
+forward using a mechanism similar to the `rangefeedTxnPusher`. This mechanism
+would monitor active intents on non-blocking Ranges and push them and their
+transaction forward if they ever got too close to present time without being
+resolved.
+
+Such a mechanism would also help ensure that even if the coordinator for a
+non-blocking transaction died, it would be cleaned up before a read on one of
+the followers conflicted with it and was forced to perform a WAN RTT.
+
+An alternative approach that would have a similar effect is to buffer a
+non-blocking transactions writes in its coordinator until it is ready to commit.
+This would mean that we would only need to tune `non_blocking_duration` to the
+expected commit latency of a transaction, instead of its expected evaluation and
+commit latency combined.
+
+We'll probably want to do one or both of these approaches.
+
+## Drawbacks
+
+Complexity. But not overly so, especially compared to Consistent Read Replicas.
+
+Only effective if we can reduce the clock uncertainty interval dramatically.
+
+## Alternatives
+
+### Consistent Read Replicas
+
+See the [Consistent Read Replicas proposal](https://github.com/cockroachdb/cockroach/pull/39758).
+
+Consistent Read Replicas provide non-stale reads from followers, which is one of
+the two major wins of this proposal. However, they do not provide the
+non-blocking property of this proposal. Consistent Read Replicas are still
+disruptive to reads when reads and writes contend.
+
+#### Read and Write Availability
+
+Another area where the two proposals diverge is in fault tolerance and
+availability in the presence of node failure.
+
+The use of closed timestamps in this proposal means that if a closed timestamp
+is delayed, present-time follower reads may not be servable for a period of
+time. This means that leaseholder failures can delay follower reads. However,
+the failure of a follower will never disrupt the ability of another follower to
+server follower reads. Similarly, it will never disrupt the ability of the
+leaseholder to serve writes (assuming a quorum of replicas is alive). Put
+simply, this proposal is only susceptible to leaseholder unavailability, which
+is part of why it needs to make no distinction between "consistent read
+replicas" and normal "followers/learners". In that sense, the proposals
+relationship to unavailability is identical to that of follower reads.
+
+The Consistent Read Replica proposal reacts differently to various forms of
+unavailability. The failure of a leaseholder will delay writes, as usual. Unlike
+this proposal, the failure of a consistent read replica will also delay writes,
+as the consistent read replica's lease needs to be revoked. However, it seems
+possible (but challenging) for consistent read replicas to continue serving
+reads in the presence of a follower **or leaseholder** failure if we ensured
+that new leaseholders properly adopted the same set of existing read replicas.
+So this write-everywhere, read-anywhere approach appears to reduce write
+availability but increase read availability. Yet, it's worth noting that the
+increased susceptibility to write unavailability can also cause an increased
+susceptibility to read unavailability for any reads that contend with blocked
+writes.
+
+#### Load Balancing Applications OR Applicability to Single Region Deployment
+
+Most of the discussion in both of these proposals was centered around follower
+reads to reduce latency. However, follower reads as a means of load balancing is
+another valid use case. This is true even if the followers are in the same data
+center or region. In these cases, communication once again becomes cheaper than
+waiting out clock uncertainty, which changes the comparison.
+
+For writes, consistent read replicas are now faster because they don't have a
+Commit Wait stage. For uncontended reads, the two proposals are still the same –
+there is no blocking. For contended reads, reads will still need to wait out the
+uncertainty interval under this proposal. Under consistent read replicas, they
+will still need to wait out the contention footprint of the contending
+transaction, but this is expected to be lower because replication is faster.
+
+### Optimistic Global Transactions
+
+Link: https://docs.google.com/document/d/17SC35GR-3G61JCUl-lTnZ4o-UMMEQNMS_p4Ps66EkSI.
+
+Optimistic Global Transactions was a second proposal that grew organically out
+of the Consistent Read Replicas proposal. The ideas were complex but powerful.
+As it turns out, there are a large number of parallels that can be drawn to this
+proposal if we imagine that all Ranges are placed in this "non-blocking mode".
+The most important of these parallels is that reads in transactions begin local
+and only need to go global to perform verification at the end. However, the big
+improvement we've made here is that we've solved the initially stale reads
+problem by adding the Commit Wait at the end of other writing transactions!
+
+In fact, with just two small extensions to this non-blocking transactions
+proposal, we arrive at exactly the same place as the Optimistic Global
+Transactions proposal:
+- defer writes until commit time, only "upgrade" to a non-blocking transaction
+ at commit time so that all reads can be local during transaction evaluation.
+- adapt commit protocol to acquire read locks after read validation, perform
+ GlobalParallelCommit to parallelize read validation with writes.
+
+The "hubs and spokes" architecture proposed in the Optimistic Global
+Transactions is just as applicable to this proposal as it was to that one.
+
+### Non-Monotonic Reads
+
+The goal of this proposal is to avoid cases where reads need to block on writes.
+However, as written, reads still do have to wait out an uncertainty interval (up
+to the maximum clock offset) when they conflict with writes. This is necessary
+in order to preserve single-key linearizability across all reads and writes to
+non-blocking ranges. This is explored in [Uncertainty Intervals: To Wait or Not to Wait](#Uncertainty-Intervals:-To-Wait-or-Not-to-Wait).
+
+If we look at this closely, we see that the waiting that reads perform is not
+strictly necessary to coordinate with the corresponding writes. Put differently,
+by making the writes wait a little longer before acknowledging the client, we
+could still ensure ["read your writes"](http://jepsen.io/consistency/models/read-your-writes).
+
+Instead, the waiting that reads perform is necessary to coordinate with other
+reads. If this waiting was not performed, we could see cases where a read observes
+a state after a change and then a causally dependent read observes the state before
+the change. This would be a violation of ["monotonic reads"](http://jepsen.io/consistency/models/monotonic-reads).
+To this point, we've considered such an anomaly to be disqualifying.
+
+Maybe that's not the case. If we were ok with violations of monotonic reads on
+non-blocking transactions then we could avoid cases where reads need to block on
+writes entirely by ignoring writes with synthetic timestamps in a read's
+uncertainty interval.
+
+This doesn't seem like a realistic default, but may be a very useful opt-in
+option.
+
+#### With Effective Uncertainty
+
+A slight variation of this idea is to split the notion of "max clock skew"
+(O(10-100ms)) from "effective clock skew" (O(1ms)) and allow non-monotonic reads
+only when clock skew exceeds this "effective clock skew". The benefit of this is
+that we would still be able to place some bound on the cases where non-monotonic
+read anomalies are possible (i.e. only if the effective clock skew is exceeded),
+but would be able to drop the maximum uncertainty interval delay down an about
+an order of magnitude.
+
+Again, it's difficult to think that this would be a realistic default because it
+complicates our consistency story, but may make a very useful option.
+
+## References
+
+[1] Spanner: https://storage.googleapis.com/pub-tools-public-publication-data/pdf/65b514eda12d025585183a641b5a9e096a3c4be5.pdf
+ * Specifically, see _Section 4.2.3. Schema-Change Transactions_, which briefly mentions a scheme that sounds similar to this, although for the purpose of running large schema changes.
+
+[2] Logical Physical Clocks and Consistent Snapshots in Globally Distributed Databases: https://cse.buffalo.edu/tech-reports/2014-04.pdf
+
+[3] Beyond TrueTime: Using AugmentedTime for Improving Spanner: https://cse.buffalo.edu/~demirbas/publications/augmentedTime.pdf
+
+[4] Exploiting a Natural Network Effect for Scalable, Fine-grained Clock Synchronization: https://www.usenix.org/system/files/conference/nsdi18/nsdi18-geng.pdf
+
+[5] SLOG: Serializable, Low-latency, Geo-replicated Transactions: https://www.cs.umd.edu/~abadi/papers/1154-Abadi.pdf
+
+[6] Large-scale Incremental Processing Using Distributed Transactions and Notifications: https://storage.googleapis.com/pub-tools-public-publication-data/pdf/36726.pdf
+
+[7] KuaFu: Closing the Parallelism Gap in Database Replication: https://www.cs.cmu.edu/afs/cs/Web/People/dongz/papers/KuaFu.pdf
diff --git a/src/current/files/cockroach/docs/RFCS/20230122_read_committed_isolation.md b/src/current/files/cockroach/docs/RFCS/20230122_read_committed_isolation.md
new file mode 100644
index 00000000000..080e343d48d
--- /dev/null
+++ b/src/current/files/cockroach/docs/RFCS/20230122_read_committed_isolation.md
@@ -0,0 +1,2471 @@
+- Feature Name: Read Committed Isolation
+- Status: completed
+- Start Date: 2023-01-22
+- Authors: Nathan VanBenschoten, Michael Erickson, Drew Kimball
+- RFC PR: #100608
+- Cockroach Issue: None
+
+# Summary
+
+**Concurrency control** comprises the mechanisms that a database system employs
+to guarantee the "correct" execution of concurrent operations. **Isolation
+levels** provide concurrency control with the requirements for correctness —
+defining how and when the changes made by one transaction become visible to
+other transactions.
+
+Strong isolation levels provide a high degree of isolation between concurrent
+transactions. They limit or eliminate the forms of concurrency effects that
+transactions may observe.
+
+Weak isolation levels are more permissive. They trade off isolation guarantees
+for improved performance. Transactions run under weaker isolation levels block
+less and encounter fewer aborts (retry errors). In some systems, they also
+perform less work.
+
+This RFC proposes the implementation of the **Read Committed** isolation level
+in CockroachDB.
+
+```
+BEGIN TRANSACTION ISOLATION LEVEL READ COMMITTED
+```
+
+Read Committed is a common, relatively weak isolation level found in most legacy
+database systems, including PostgreSQL. In fact, it is the default isolation
+level in PostgreSQL and most other legacy database systems.
+
+Critically for this proposal, it is the strongest isolation level present in
+PostgreSQL that does not experience [serialization failure
+errors](https://www.postgresql.org/docs/current/mvcc-serialization-failure-handling.html)
+and require applications to handle these errors using application-side retry
+logic.
+
+By providing users with the option to run transactions under Read Committed, we
+provide them with the option to avoid transaction retry logic in their
+applications and make migrations to CockroachDB easier.
+
+# Motivation
+
+As alluded to above, the main impetus behind the introduction of Read Committed
+is that the isolation level is expected to **ease application migrations**.
+
+Existing applications that were built for Read Committed (the default isolation
+level in many legacy databases, including PostgreSQL) often struggle to move to
+CockroachDB's Serializable isolation level. Doing so requires the introduction
+of transaction retry logic in the application to handle isolation-related
+"serialization" errors. These errors are a consequence of strong isolation
+guarantees. While adding retry logic is merely inconvenient for new
+applications, it is often infeasible for existing applications. Implementing
+Read Committed in CockroachDB will allow these applications to migrate to
+CockroachDB without also migrating to a stronger isolation level at the same
+time.
+
+For much the same reason, applications also struggle to move to CockroachDB
+because they must now tolerate consistency-related "clock uncertainty" errors.
+While not a direct consequence of strong isolation, these errors cannot commonly
+be handled transparently by CockroachDB because of strong isolation guarantees.
+Read Committed's isolation guarantees are weak enough that CockroachDB can
+commonly handle consistency-related retry errors internally without involvement
+from the application, eliminating the other reason for application-side retry
+logic.
+
+Performance-sensitive applications may also prefer Read Committed over
+Serializable. The isolation guarantees made by Read Committed are weak enough
+that implementations can prevent readers from blocking writers (or causing them
+to retry) and writers from blocking readers (or causing them to retry). This
+limits the forms of transaction contention that can cause performance
+degradation to just write-write contention.
+
+Transaction contention is notoriously difficult to understand, predict, and
+mitigate. This is because contention is a global property of an entire workload,
+not a local property of single transactions. It emerges from the conflicts
+between two transactions, their relative timing, and the degree to which the
+transactions are impacted by contention with other transactions. As a result, it
+is often associated with meta-stable failures, where a system behaves well until
+contention reaches a tipping point, beyond which throughput collapses. Limiting
+contention concerns to write-write contention is therefore a major win for
+**performance predictability**. In exchange, Read Committed is permissive of
+concurrency anomalies, but this may be the right trade-off for some
+applications.
+
+In addition, while the primary beneficiaries of the new isolation level are
+users that run their apps entirely under the weaker isolation level, users
+running their apps under Serializable isolation may still benefit from this
+work. Read Committed will provide these users with a tool to run **bulk
+read-write transactions** alongside their applications without risk of these
+bulk transactions being starved due to serialization errors. This has been a
+common struggle for users of CockroachDB.
+
+Arguably, these bulk transactions would be just as well served (or in some cases
+better served) by Snapshot isolation, an isolation level between Read Committed
+and Serializable. However, CockroachDB does not support this isolation level
+either. This reveals the fourth benefit of this work — that it **paves the road
+to and then past Snapshot isolation** (Repeatable Read, in PostgreSQL terms).
+Almost all engineering work performed in service of Read Committed will also
+benefit a potential future Snapshot isolation implementation. At that point, the
+work to support Snapshot isolation will be predominantly an exercise in testing,
+as many of the pieces inside the database will already be in place.
+
+# Background
+
+The following two subsections present the high-level landscape of isolation
+levels and then focus on PostgreSQL's location in this landscape. This
+background helps to contextualize the design choices made later on in this
+proposal. However, readers that are comfortable with the topic can feel free to
+jump to the [technical design](#technical-design).
+
+## Isolation Level Theory Primer
+
+[ANSI SQL](https://www.contrib.andrew.cmu.edu/~shadow/sql/sql1992.txt) (1992)
+defined four isolation levels: READ UNCOMMITTED, READ COMMITTED, REPEATABLE
+READ, and SERIALIZABLE. The levels were defined in terms of three _phenomena_:
+Dirty Reads, Non-Repeatable Reads, and Phantom Reads. Stronger isolation levels
+allow fewer phenomena to occur. As a result, they permit less anomalous behavior
+(permit fewer anomalies). Weaker isolation levels allow more phenomena to occur.
+
+[A Critique of ANSI SQL Isolation
+Levels](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/tr-95-51.pdf)
+(1995) demonstrated that the ANSI SQL standard definitions of isolation levels
+were insufficient. Some phenomena were ambiguous, while others were missing
+entirely. The work provided a new characterization of isolation levels, defining
+the levels using a set of eight different phenomena. The expanded
+characterization also made room for a new isolation level: SNAPSHOT.
+
+While more complete, these definitions were still based on preventing
+conflicting operations that could lead to anomalies from executing concurrently.
+Adya's dissertation ["Weak Consistency: A Generalized Theory and Optimistic
+Implementations for Distributed
+Transactions"](https://pmg.csail.mit.edu/papers/adya-phd.pdf) (1999) argued that
+this _preventative_ approach is overly restrictive. The definitions were
+"disguised versions of locking" and therefore disallow optimistic and
+multi-versioning schemes. Adya's work generalizes existing isolation levels in
+terms of conflicts, serialization graphs, and the forms of phenomena allowed in
+the serialization graphs of different isolation levels.
+
+Because Adya's classification of isolation levels and phenomenon is general
+enough to allow both locking and optimistic transaction implementations (of
+which CockroachDB is a hybrid), we use its formalization where appropriate in
+this proposal. Readers are encouraged to familiarize themselves with the thesis.
+Once they notice the page count, they are encouraged to familiarize themselves
+with the [summary paper](https://pmg.csail.mit.edu/papers/icde00.pdf) instead.
+
+Conveniently, [Elle](https://github.com/jepsen-io/elle), a transactional
+consistency checker that we intend to use as part of the [testing for this
+work](#testing), also talks in the language of Adya-style transaction histories
+and anomalies.
+
+For a different take on transaction isolation, readers can familiarize
+themselves with the work of Crooks et al. in [Seeing is Believing: A
+Client-Centric Specification of Database
+Isolation](http://www.cs.cornell.edu/lorenzo/papers/Crooks17Seeing.pdf). The
+work presents a formalization of isolation guarantees from the perspective of
+the users of a database system, instead of from the perspective of the system
+itself. Crook's handling of different variants of Snapshot isolation is
+particularly novel.
+
+## Isolation Levels in PostgreSQL
+
+Transaction isolation in PostgreSQL is an interesting and relevant topic that is
+well documented [here](https://www.postgresql.org/docs/current/transaction-iso.html).
+
+PostgreSQL provides all four standard transaction isolation levels: READ
+UNCOMMITTED, READ COMMITTED, REPEATABLE READ, and SERIALIZABLE. However, READ
+UNCOMMITTED is not implemented and maps to READ COMMITTED, so internally the
+system only supports three distinct isolation levels. Furthermore, for a variety
+of reasons and in a variety of ways, the three isolation levels that are present
+deviate from their ANSI SQL definitions. Let's explore those briefly, from
+strongest to weakest.
+
+`SERIALIZABLE` is the strongest isolation level, and PostgreSQL has supported
+true Serializable isolation since its 9.1 release (2011). Being the only
+unambiguously defined ANSI SQL isolation level, PostgreSQL's implementation of
+Serializable is true Serializable (PL-3 in Adya).
+
+However, PostgreSQL deviates from ANSI SQL's assumed use of [two-phase
+locking](https://en.wikipedia.org/wiki/Two-phase_locking) (2PL) to implement
+Serializable isolation. Instead, it uses an optimistic concurrency control
+protocol called [Serializable Snapshot
+Isolation](https://courses.cs.washington.edu/courses/cse444/08au/544M/READING-LIST/fekete-sigmod2008.pdf)
+(SSI). SSI extends the multi-versioning present in Snapshot isolation with
+additional runtime conflict detection to provide Serializable isolation. The
+implementation of SSI in PostgreSQL is recounted by Ports and Grittner in
+[Serializable Snapshot Isolation in
+PostgreSQL](https://drkp.net/papers/ssi-vldb12.pdf).
+
+Notably, CockroachDB also implements Serializable isolation using SSI.
+
+`REPEATABLE READ` is the next strongest standard isolation level. Formally, the
+isolation level (PL-2.99 in Adya) permits Phantom Reads but does not permit
+Write Skew. In its place and under the same name, PostgreSQL implements Snapshot
+isolation (PL-SI in Adya). Snapshot isolation does not permit Phantom Reads but
+does permit Write Skew. This deviation is not strictly wrong from a standards
+conformance perspective due to ANSI SQL's ambiguous definitions (which is what
+prompted Berenson, Bernstein et al.'s
+[critique](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/tr-95-51.pdf)),
+but it is confusing.
+
+Architecturally, the deviation is a consequence of PostgreSQL's multi-version
+concurrency control scheme and general avoidance of read locks. PostgreSQL's
+docs explain the [benefit of this
+approach](https://www.postgresql.org/docs/current/mvcc-intro.html):
+
+> The main advantage of using the MVCC model of concurrency control rather than
+> locking is that in MVCC locks acquired for querying (reading) data do not
+> conflict with locks acquired for writing data, and so reading never blocks
+> writing and writing never blocks reading.
+
+When viewed as traditional Snapshot isolation, PostgreSQL's REPEATABLE READ
+isolation level is the most straightforward of the bunch. Transactions establish
+a consistent read snapshot of previously committed data when they begin. All
+reads across statements in the transaction are served out of this snapshot
+(ignoring read-your-writes). Mutation statements evaluate their row search using
+the read snapshot. They then lock and modify qualifying rows, aborting if any
+locked row has been changed since the transaction began ("first committer
+wins"). These locks then block other writers, but not readers, until the locking
+transaction commits.
+
+`READ COMMITTED` is the second weakest standard isolation level, stronger only
+than `READ UNCOMMITTED` (which PG does not implement). It is also the focus of
+this RFC, so we will give its implementation in PostgreSQL extra attention.
+
+Formally, the isolation level (PL-2 in Adya) permits Non-Repeatable Reads,
+Phantom Reads, Lost Updates, and Write Skew, but does not permit Dirty Reads or
+Dirty Writes. PostgreSQL's implementation of Read Committed adheres to the
+requirements of the isolation level. It also includes additional monotonicity
+and recency properties beyond what is required by Read Committed. As a result,
+it may be more accurately characterized as an implementation of [Monotonic
+Atomic View](https://www.vldb.org/pvldb/vol7/p181-bailis.pdf).
+
+Each statement in a Read Committed transaction establishes a consistent read
+snapshot of previously committed data when that statement began. For SELECT
+statements, all reads are served out of this per-statement consistent snapshot
+(ignoring read-your-writes). As a result, the statement never sees either
+uncommitted data or changes committed by concurrent transactions during
+statement execution. However, successive SELECT statements in the same
+transaction receive different read snapshots and can see different data. Later
+statements always receive newer read snapshots.
+
+Mutation statements (e.g. `UPDATE` and `DELETE`) evaluate their row search using
+the per-statement read snapshot. They then lock qualifying rows, receiving the
+latest version of the row. Once locked, rows that were unchanged since the
+statement's read snapshot are mutated. Rows that were changed since the
+statement's read snapshot are passed back through the mutation's search
+predicate to determine whether they still qualify for mutation. This process of
+re-evaluating the query predicate after locking is referred to as
+`EvalPlanQual`, and is described
+[here](https://github.com/postgres/postgres/blob/f03bd5717eaf31569ca797a2f7d65608f88ac2a2/src/backend/executor/README#L350).
+
+Re-evaluating query predicates when skew is detected between a mutation's read
+snapshot and its locked snapshot avoids the need for serialization errors on
+write-write conflicts. However, it also permits concurrency anomalies like Lost
+Updates. One way to understand this is that anomalies can occur because the
+mutation's row search is performed at a different MVCC snapshot than its
+mutations, so interleaving mutations from other transactions can be lost.
+
+An example of this is presented in [PG's
+docs](https://www.postgresql.org/docs/current/transaction-iso.html#XACT-READ-COMMITTED:~:text=Because%20of%20the,with%20transactions%20like%3A),
+near the text "it is possible for an updating command to see an inconsistent
+snapshot".
+
+Explicit row-level locking statements (e.g. `SELECT FOR UPDATE`) behave
+similarly to mutations in Read Committed transactions. The initial row search of
+the statement is performed using a non-locking read. Qualifying rows are then
+locked and their latest version is retrieved. Rows that were changed since the
+statement's read snapshot are re-qualified using the same EvalPlanQual approach
+and rows that still match the query qualifications are returned.
+
+An added complexity of `SELECT FOR UPDATE` compared to other mutations is that
+multiple tables can be locked at once when the query contains a join. In these
+cases, base rows that contribute to the returned join rows in each table are
+locked. If re-qualification is needed, individual join rows are passed back
+through the join condition to determine whether they satisfy the condition with
+the latest versions of the contributing rows. If so, the join row is still
+returned. Otherwise, it is discarded.
+
+For readers that want to learn more about isolation levels in PostgreSQL,
+[Jepsen's 2020 analysis of PostgreSQL
+12.3](https://jepsen.io/analyses/postgresql-12.3) is an enjoyable read.
+
+### Data Consistency Checks at the Application Level
+
+PostgreSQL [documents](https://www.postgresql.org/docs/current/applevel-consistency.html)
+approaches that application developers can use to enforce application-level data
+integrity rules at the different isolation levels.
+
+As expected, the rules are trivial for Serializable isolation. By emulating
+serial transaction execution, Serializable isolation has the unique property
+that any interleaving of correct transactions will preserve application-level
+data constraints, even if the database system is not aware of the constraints.
+This property is a major simplification for application developers, and it is
+why some are willing to pay a performance penalty.
+
+Repeatable Read and Read Committed are both susceptible to concurrency
+anomalies, so the rules and strategies to ensure correctness are more complex.
+PostgreSQL recommends the use of explicit row-level (`SELECT FOR UPDATE`,
+`SELECT FOR SHARE`) and table-level (`LOCK TABLE`) locks to protect against
+concurrent operations that could violate data integrity.
+
+The use of explicit locking to enforce correctness [has been
+demonstrated](http://www.bailis.org/papers/acidrain-sigmod2017.pdf) to be subtle
+and error-prone, and yet, it is a task bestowed upon the application developer
+by these weaker isolation levels. You get what you pay for.
+
+Of note is that explicit locking is never necessary to preserve data integrity
+under Serializable isolation. However, it is necessary in some cases for weaker
+isolation levels. This elevates the importance of a correct implementation of
+explicit locking; data integrity depends on it. This will be discussed later in
+the RFC.
+
+### Data Consistency Checks at the System Level
+
+While not externally documented, PostgreSQL enforces system-level data integrity
+rules (foreign key constraints, etc.) under weak isolation levels in the same
+manner. Explicit row-level locks are acquired when performing constraint checks
+and these locks are held for the remainder of the transaction.
+
+Foreign key existence checks are an example of such a system-level consistency
+check. Under the hood, PostgreSQL runs a `SELECT FOR SHARE` query that [looks
+like this](https://github.com/postgres/postgres/blob/75f49221c22286104f032827359783aa5f4e6646/src/backend/utils/adt/ri_triggers.c#L363)
+(specifically, the query uses `SELECT FOR KEY SHARE`).
+
+### Serialization Failure Handling
+
+PostgreSQL also [documents](https://www.postgresql.org/docs/current/mvcc-serialization-failure-handling.html)
+the approaches that application developers can use to handle serialization
+failures. As discussed earlier, both Repeatable Read and Serializable isolation
+can produce serializable errors (`code 40001 - serialization_failure`).
+Repeatable Read will raise them on write-write conflicts. Serializable will also
+raise them on certain conflict cycles involving only read-write conflicts.
+
+The documentation recommends that application developers using either of these
+isolation levels add retry loops in their application around transactions, and
+that these loops should unconditionally retry on `serialization_failure` errors.
+
+Read Committed transactions are not susceptible to these errors, so such retry
+loops are not needed. This again is the [primary motivation](#motivation) for
+the introduction of Read Committed into CockroachDB.
+
+The documentation does mention that all three isolation levels are susceptible
+to locking deadlock errors (`code 40P01 - deadlock_detected`). However, it makes
+a weaker recommendation about how application developers should handle this
+class of error. Retry loops "may be advisable", but in practice, these deadlocks
+are typically handled by structuring transaction logic to use a consistent
+locking order.
+
+# Technical Design
+
+The design of the Read Committed isolation level is sprawling. However, the bulk
+of the design is split across three major areas of focus:
+- [Transaction Model](#transaction-model)
+- [Row-Level Locks](#row-level-locks)
+- [Query Planning and Execution](#query-planning-and-execution)
+
+## Transaction Model
+
+This RFC proposes a Read Committed implementation in CockroachDB based on a
+"Per-Statement Snapshot Isolation" transaction model. To understand this model,
+it is helpful to start at CockroachDB's current Serializability model,
+resuscitate the corresponding form of Snapshot isolation[^1], and then
+generalize this model with the ability to operate over multiple read snapshots
+to arrive at Read Committed.
+
+[^1]: CockroachDB originally supported Snapshot isolation. However, the
+ isolation level was removed in v2.1 (October, 2018) by
+ [#26475](https://github.com/cockroachdb/cockroach/issues/26475) due to its
+ multiple bugs. Many of these bugs were a result of CockroachDB's minimal
+ support for row-level locking at the time.
+
+To do so, we start by defining two properties:
+
+**Write skew tolerance**: Does the isolation level permit write skew? In
+CockroachDB, this property can be expressed as whether the isolation level
+allows transactions to write and commit at an MVCC timestamp above the MVCC
+timestamp of its read snapshot(s).
+
+**Read snapshot scope**: Does the isolation level allow transactions to operate
+across multiple read snapshots? If not, a single read snapshot is used for the
+entire transaction. If so, what is the scope of each read snapshot?
+
+With these two properties, we can then construct a unifying framework for the
+three isolation levels:
+
+| Isolation Level | Write Skew Tolerance | Read Snapshot Scope |
+|---------------------|----------------------|---------------------|
+| Serializable (SSI) | No | Per-Transaction |
+| Snapshot (SI) | Yes | Per-Transaction |
+| Read Committed (RC) | Yes | Per-Statement |
+
+Interested readers can find a proof of correctness of this transaction model [in
+the appendix](#appendix-proof-of-correctness).
+
+### Write Skew Tolerance
+
+The primary difference between a Serializable implementation and a hypothetical
+Snapshot implementation is Snapshot's tolerance of write skew.
+
+Like in other MVCC systems that support Snapshot isolation, Snapshot isolation
+in CockroachDB would permit a transaction's read timestamp and its write
+timestamp to diverge and still allow that transaction to commit. Reads performed
+at the transaction's read timestamp would never be validated through a refresh
+at the transaction's eventual commit timestamp.
+
+Consequently, it would be possible for a key-value that was read at a
+transaction's read timestamp to be updated by a second, concurrent transaction
+at an MVCC timestamp greater than the first transaction's read timestamp but
+less than the first transaction's commit timestamp. The initial read would "no
+longer be valid" at the first transaction's commit timestamp. This setup forms
+the basis for the write skew anomaly.
+
+Because skew is permitted between (unlocked) reads T a transaction's read
+timestamp and its final commit timestamp, reads do not need to be tracked by a
+transaction's coordinator, and read refreshes are not needed. The transaction
+coordinator can forgo the use of a `txnSpanRefresher` entirely, which is the
+component responsible for read span tracking and refreshing.
+
+This tolerance to write skew applies to Read Committed transactions as well.
+
+### Per-Statement Read Snapshots
+
+Write skew tolerance is the major difference between a Serializable
+implementation and a hypothetical Snapshot implementation. The major difference
+between a hypothetical Snapshot implementation and a Read Committed
+implementation is per-statement read snapshots.
+
+Like in PostgreSQL, each statement in a Read Committed transaction will
+establish a new read snapshot and will use that snapshot (i.e. MVCC timestamp)
+for all of its reads. This will be accomplished by forwarding a `kv.Txn`'s
+`ReadTimestamp` to `hlc.Now()` on each statement boundary (i.e. each call to
+`kv.Txn.Step`).
+
+By establishing a new read snapshot for each statement, `SELECT` statements and
+unlocked portions of mutation statements will observe only data committed before
+the statement began. They will not observe uncommitted data or changes committed
+concurrently with statement execution.
+
+From this description, it is evident that Read Committed does not provide
+repeatable reads. `SELECT` statements in the same transaction can return
+different versions of the same data.
+
+Statements will see the effects of writes performed by previous statements
+within the same transaction. However, they won't see the effects of writes
+performed by the same statement in the same transaction. This avoids the
+"Halloween Problem"[^2]. There are no differences between the Serializable and
+Read Committed isolation levels as they relate to intra-transaction visibility,
+so no changes are needed here.
+
+[^2]: Note that there are currently known bugs in the area of intra-transaction
+ visibility (e.g. [#70731](https://github.com/cockroachdb/cockroach/issues/70731))
+ which this work does not intend to fix.
+
+#### Monotonicity and Atomicity Guarantees
+
+The use of a globally consistent per-statement read snapshot that advances
+between statements is a stronger consistency model than is strictly required by
+Read Committed. Notably, it prevents a transaction from observing some of a
+committed transaction's effects and then later failing to observe other effects
+from the same committed transaction.
+
+With these guarantees, the isolation level could be considered an implementation
+of [Monotonic Atomic View](https://www.vldb.org/pvldb/vol7/p181-bailis.pdf).
+This closely matches the behavior of PostgreSQL, which has also been
+[characterized similarly](https://github.com/ept/hermitage).
+
+#### Recency Guarantees
+
+The use of a per-statement read snapshot that includes all data committed before
+(in real-time) the statement began is also a stronger consistency model than is
+required by Read Committed. However, this again would match the behavior of
+PostgreSQL and other legacy database systems.
+
+### Read Uncertainty Intervals
+
+Providing strong recency guarantees in a distributed system requires more effort
+than it does in a single-machine system.
+
+CockroachDB already provides such real-time constraints across transactions.
+Formally, the system provides a single-object consistency model of
+[linearizability](https://jepsen.io/consistency/models/linearizable). Translated
+to a multi-object, multi-operation consistency model, CockroachDB has been
+characterized informally as providing [a guarantee of "no stale
+reads"](https://www.cockroachlabs.com/blog/consistency-model/). A transaction
+observes all data committed before the transaction began.
+
+To provide this guarantee in a distributed system without perfectly synchronized
+clocks but with the ability to place _some_ bound on clock skew between nodes,
+the system employs "read uncertainty intervals". Readers unfamiliar with the
+concept are encouraged to read [this blog
+post](https://www.cockroachlabs.com/blog/living-without-atomic-clocks/) and then
+[this tech
+note](https://github.com/cockroachdb/cockroach/blob/8e24570fa366ed038c6ae65f50db5d8e22826db0/pkg/kv/kvserver/uncertainty/doc.go).
+
+Read Committed transactions have the option to provide the same "no stale reads"
+guarantee at the level of each individual statement. Doing so would require
+transactions to reset their `GlobalUncertaintyLimit` and `ObservedTimestamps` on
+each statement boundary, setting their `GlobalUncertaintyLimit` to `hlc.Now() +
+hlc.MaxOffset()` and clearing all `ObservedTimestamps`.
+
+We propose that Read Committed transactions do not do this and instead use the
+same uncertainty interval across all statements. The cost of resetting a
+transaction's uncertainty interval on each statement boundary is likely greater
+than the benefit. Doing so increases the chance that individual statements retry
+due to `ReadWithinUncertaintyInterval` errors. In the worst case, each statement
+will need to traverse (through retries) an entire uncertainty interval before
+converging to a "certain" read snapshot. While these retries will be scoped to a
+single statement and [should not escape to the client](#transaction-retries),
+they do still have a latency cost.
+
+We make this decision because we do not expect that applications rely on strong
+consistency guarantees between the commit of one transaction and the start of an
+individual statement within another in-progress transaction. To rely on such
+guarantees would require complex and surprising application-side
+synchronization.
+
+This design decision can be revisited if it proves to be problematic for
+applications. We've been surprised before.
+
+For a discussion of how uncertainty errors are handled, see
+[below](#consistency-related-retries).
+
+### Write-Write Conflict Handling (or Lost Update Intolerance)
+
+While write skew is permitted under Snapshot isolation, lost updates are not.
+Like under Serializable isolation, transactions throw errors on write-write
+conflicts. This is commonly referred to as a "first writer wins" or "first
+committer wins" conflict resolution policy.
+
+Protection against lost updates is weaker under Read Committed isolation, where
+a limited form of the anomaly is permitted. Updates cannot be lost between a
+single statement's read and write timestamp, but can be lost across one
+statement's read timestamp and a subsequent statement's write timestamp. This is
+a consequence of the use of [per-statement read
+snapshots](#per-statement-read-snapshots) in Read Committed.
+
+To understand this, we first decompose the definition of a write-write conflict
+as follows:
+
+**Write-Write Locking Conflict**: any case where a transaction attempts to lock
+or write to a key that is locked with an exclusive lock by a different
+transaction, where intents are considered to be a combination of an exclusive
+lock and a provisional value.
+
+**Write-Write Version Conflict**: any case where a transaction attempts to lock
+or write to a key that has a _committed version_ with an MVCC timestamp greater
+than the locking/writing transaction's current read snapshot.
+
+We define _write-write version conflict_ in terms of a transaction's "current
+read snapshot" (i.e. `txn.ReadTimestamp`) to afford flexibility in the
+definition to transactions that change their read snapshot across their
+execution. For example, Read Committed transactions advance their read snapshot
+on each statement boundary, so a committed version that would cause a
+write-write version conflict for one statement may not cause a write-write
+version conflict for a later statement in the same transaction.
+
+Snapshot and Read Committed transactions handle _write-write locking conflicts_
+identically to Serializable transactions. The prospective locker
+[waits](#blocking-write-write-conflicts) for the existing exclusive lock to be
+released before acquiring it. In cases where multiple transactions wait for
+exclusive access to the same key, they form an orderly queue through the
+`lockWaitQueue` mechanism.
+
+Once a transaction (regardless of isolation level) has navigated any potential
+_write-write locking conflict_, it may experience a _write-write version
+conflict_. This occurs when the holder of the lock performs a write on the
+locked row before committing. In such cases, a `WriteTooOld` error is thrown.
+
+For Snapshot and Serializable transactions, this requires the entire transaction
+to retry. We will see below that this is not the case for Read Committed
+transactions. A write-write version conflict will raise a `WriteTooOld` error
+that will cause the Read Committed transaction to retry the individual statement
+that experienced the write-write version conflict at a new read snapshot, but
+will not require the entire Read Committed transaction to retry. This
+adaptability is possible because Read Committed transactions use per-statement
+read snapshots.
+
+This behavior deviates from PostgreSQL, which does not require statements in
+Read Committed to restart on write-write conflicts. We [explore an alternative
+approach](#appendix-postgres-compatible-intra-mutation-consistency) that behaves
+more similarly to PostgreSQL in the appendix, where we also explain why that
+approach was rejected in favor of the stronger model presented here.
+
+### Blocking Behavior
+
+A transaction's _blocking behavior_ defines the cases in which contention with
+other transactions does or does not block.
+
+Isolation levels do not directly mandate a specific blocking behavior — the same
+isolation level can be implemented differently and lead to different blocking
+behavior while providing the same isolation guarantees. However, weaker
+isolation levels often make it easier to avoid blocking with fewer consequences
+(e.g. potential aborts).
+
+In the following subsections, we outline the blocking behavior of Read Committed
+transactions for the four different classes of conflicts composed of read and
+write operations performed by two different transactions on the same row(s).
+
+In all cases, exclusive locking reads (`SELECT FOR UPDATE`) are treated as a
+type of write. Shared locking reads complicate the discussion for little
+immediate benefit, so [their discussion is deferred](#shared-locks).
+
+The name of each conflict refers to the order in which operations take place.
+For example, a "write-read conflict" refers to a write to a row, followed by a
+read of the same row by a different transaction. The later operation is said to
+encounter the conflict, so we focus on the blocking behavior of that operation.
+The earlier operation never blocks because there was no conflict at the time of
+its execution.
+
+#### Non-Blocking Read-Read Conflicts
+
+Two reads to the same row from concurrent transactions never block. Furthermore,
+in an MVCC system like CockroachDB, the two reads do not interact at all. The
+later read is not influenced by the earlier read.
+
+#### Blocking Write-Write Conflicts
+
+Two writes to the same row cause blocking. The later writer will wait for the
+lock acquired by the earlier writer to be released. This will occur when the
+earlier writer's transaction is committed or rolled back.
+
+#### Non-Blocking Read-Write Conflicts
+
+A read of a row followed by a write to that row by a different transaction does
+not block. The writer is allowed to proceed with its write, unencumbered. This
+is an [oft-cited benefit of MVCC
+systems](https://docs.oracle.com/en/database/oracle/oracle-database/19/cncpt/data-concurrency-and-consistency.html#GUID-1D60EFCC-03F4-4A04-B099-1B4DE5D02C47).
+
+However, while there is no blocking, the two operations do still interact. The
+writer is barred from committing at an MVCC timestamp at or below the timestamp
+of the reader. Failure to do so would violate the consistency of the reader's
+snapshot. In CockroachDB, this coordination is facilitated through the
+[TimestampCache](https://github.com/cockroachdb/cockroach/blob/8e24570fa366ed038c6ae65f50db5d8e22826db0/pkg/kv/kvserver/tscache/cache.go#L31).
+
+#### Non-Blocking Write-Read Conflicts
+
+Finally, a write of a row followed by a read of that row by a different
+transaction does not block. The reader is allowed to proceed with its read after
+determining whether it should observe or ignore the conflicting write. Either
+way, it does not block and wait for the writer to commit. This is the other
+[oft-cited benefit of MVCC
+systems](https://docs.oracle.com/en/database/oracle/oracle-database/19/cncpt/data-concurrency-and-consistency.html#GUID-1D60EFCC-03F4-4A04-B099-1B4DE5D02C47).
+
+Non-blocking write-read conflicts are the **only blocking behavior interaction
+that will change with the Read Committed work**. As such, they deserve focused
+attention.
+
+Currently, a subset of write-read conflicts are blocking in CockroachDB. This
+has been a repeated source of confusion and frustration for users of the system.
+This is especially true for migrations from PostgreSQL, which provides
+non-blocking write-read conflicts in all isolation levels. Unexpected blocking
+can cause performance problems, unpredictability, and application deadlocks.
+
+Preliminary discussions with users of CockroachDB revealed that a desire for
+this property is one of the underlying reasons why they ask for Read Committed,
+alongside the desire for fewer transaction aborts.
+
+And yet, the property is not strictly an artifact of isolation. As PostgreSQL
+demonstrates, it can be provided at all isolation levels. CockroachDB's
+engineering team has [previously
+explored](https://docs.google.com/document/d/1ji6C0aDI6n61sVKPjf5-YUucbtgBlwfpsrNdcidW5a0/edit?usp=sharing)
+what it would take to support non-blocking write-read conflicts under
+Serializable isolation.
+
+That earlier proposal outlined a long-term vision for changes that would
+primarily benefit transaction contention, but would also benefit multi-region
+transactions, one-phase commits, foreign key lookups, and more. It was ambitious
+but difficult to decompose. It also struggled with increasing the risk of
+transaction starvation without sufficient recourse for users, at least until
+either SHARED locks or weaker isolation levels are introduced.
+
+This RFC proposes two deviations from that original proposal. First, we will
+start by supporting the (non-)blocking behavior **only at the Read Committed
+isolation level**, where there are fewer trade-offs. Read Committed transactions
+are not subject to the increased risk of starvation in order to provide this
+property, so there are fewer risks to the work. Second, we will implement the
+(non-)blocking behavior using **a simpler approach** that has fewer auxiliary
+benefits and shifts work from writers to readers but still achieves the
+sought-after non-blocking property.
+
+After the mechanisms to support non-blocking write-read conflicts are built as
+part of this work, we can then revisit their use for Serializable transactions.
+At that point, engineering efforts can focus solely on the problem of read
+predictability vs. fairness. If the property is eventually extended to
+Serializable transactions, negatively impacted users will be able to combat
+starvation by using SHARED locks during select reads or by running select
+transactions under Read Committed isolation.
+
+##### Status Check RPC on Intent Conflicts
+
+To implement non-blocking write-read conflicts, the conflict resolution rules
+for non-locking readers that encounter conflicting intents will be adjusted.
+Note here that "conflicting" means that the read timestamp of the reader is
+equal to or greater than the write timestamp of the intent and that the reader
+is attempting to read the intent's key. The reader must determine whether the
+intent should be included in its read snapshot.
+
+In such cases, the presence of the intent alone cannot be used to determine
+whether the intent's writer is still running or whether it has already been
+committed. Distinguishing between these two cases as a reader is necessary to
+ensure both atomicity and (CAP) consistency.
+
+Instead of waiting on these conflicting intents until the intent's transaction
+completes, the non-locking readers will perform a "status check" RPC in the form
+of a `PushTxn(PUSH_TIMESTAMP)` to the transaction record of encountered intent
+to determine its visibility. If the intent's transaction has already been
+committed or aborted, the reader must initiate and wait for intent resolution
+and then observe the post-resolution state[^3]. However, if the intent's
+transaction is in progress, the reader can [push the minimum commit
+timestamp](https://github.com/cockroachdb/cockroach/pull/95911) of the intent
+holder to prevent it from committing at or before the reader's read timestamp.
+The reader can then proceed to ignore the intent and continue with its scan.
+
+[^3]: there are other projects like
+ https://github.com/cockroachdb/cockroach/issues/91848 that aim to reduce
+ blocking between intent resolution and conflicting KV operations. These are
+ out of the scope of this RFC.
+
+To make the performance of this approach reasonable, two changes will be needed.
+
+First, requests must cache status check results for transactions that have been
+pushed, were seen to be `PENDING` after the request began, and are uncommittable
+below the request's read timestamp. This cache can be used to prevent a request
+from needing to perform a status check for the same transaction multiple times.
+
+The cache must be request-scoped and not scoped more broadly (e.g.
+range-scoped). This is because: Requests have different read timestamps. The
+cache can only be used to avoid a PushTxn request if a prior PushTxn advanced
+the intent holder's minimum commit timestamp above a request's read timestamp.
+Requests originate at different times. A PushTxn that previously observed a
+PENDING intent holder _before_ a request arrived is no proof that the intent
+holder has not committed before the request's transaction began. Ignoring such
+intents because they _used to be_ PENDING (without additional uncertainty
+interval logic) could lead to violations of real-time ordering.
+
+For ease of implementation, a new request-scoped cache of non-visible
+transactions will be added to `concurrency.Guard`, making it per-request and
+per-range scoped. This is convenient because it avoids wire-protocol changes and
+because `concurrency.Guard` objects are scoped below transaction read_timestamp
+adjustments, so cache eviction logic on transaction refresh will be implicit. As
+a result of this scoping, a given read request may query the status of a given
+pending transaction at most O(ranges) times, which is deemed to be a reasonable
+cost.
+
+Second, readers must be taught to ignore conflicting intents that are known to
+be PENDING without traversing consensus to resolve (rewrite) them. This can be
+achieved through either an augmentation of the `intentInterleavingIter` or by
+providing the MVCC scanner with another view into the `concurrency.Guard`. The
+approach is outlined in
+[#94730](https://github.com/cockroachdb/cockroach/issues/94730) and has been
+prototyped in
+[nvanbenschoten/nonBlockingReads](https://github.com/nvanbenschoten/cockroach/commits/nvanbenschoten/nonBlockingReads).
+
+##### Ignored Exclusive Lock Conflicts
+
+The conflict resolution rules for non-locking readers that encounter exclusive
+locks will also be adjusted. This adjustment is simpler. Exclusive locks that do
+not protect provisional values (e.g. those acquired by `SELECT FOR UPDATE`) will
+no longer block non-locking readers. Further, because they have no associated
+provisional value that may require visibility, readers need not concern
+themselves with determining the status of the lock holder's transaction.
+Instead, readers can simply ignore these locks.
+
+This matches the behavior of PostgreSQL, even under Serializable isolation.
+However, as with the other half of Non-Blocking Write-Read Conflicts, this
+behavior change will only initially apply to Read Committed readers until we can
+be sure it does not regress performance for Serializable transactions.
+
+### Transaction Retries
+
+CockroachDB's Serializable implementation combines elements of a pessimistic and
+an optimistic transaction model. Write-write conflicts are eagerly detected
+during statement execution and lead to immediate transaction aborts. However,
+"dangerous structures" (see [Serializable Snapshot
+Isolation](https://courses.cs.washington.edu/courses/cse444/08au/544M/READING-LIST/fekete-sigmod2008.pdf))
+that could otherwise lead to non-serializable histories are detected at commit
+time.
+
+This optimistic validation is accomplished through the combination of a
+commit-time condition that a transaction's `ReadTimestamp` equals its
+`WriteTimestamp`
+(https://github.com/cockroachdb/cockroach/blob/8e24570fa366ed038c6ae65f50db5d8e22826db0/pkg/kv/kvserver/batcheval/cmd_end_transaction.go#L504)
+and a [read
+refresh](https://github.com/cockroachdb/cockroach/blob/8e24570fa366ed038c6ae65f50db5d8e22826db0/pkg/kv/kvclient/kvcoord/txn_interceptor_span_refresher.go#L45)
+mechanism that advances the transaction's `ReadTimestamp`. The first of these
+checks fails if the transaction has established an inbound read-write
+anti-dependency (e.g. its `WriteTimestamp` is pushed by timestamp cache). The
+second of these checks fails if the transaction has established an outbound
+read-write anti-dependency (e.g. its read refresh finds a newer version). If
+both fail, the transaction aborts.
+
+These commit-time isolation checks are not needed for Read Committed
+transactions. This is because the "dangerous structures" that can create
+non-serializable histories are permitted under the Read Committed isolation
+level. In practice, this means that a Read Committed transaction can commit even
+if its `ReadTimestamp` is skewed from its `WriteTimestamp`.[^4]
+
+[^4]: This would be true of a Snapshot isolation implementation as well.
+
+#### Isolation-Related Retries
+
+We [alluded earlier](#write-write-conflict-handling-or-lost-update-intolerance)
+that write-write version conflicts do not cause Read Committed transactions to
+abort. This is despite the fact that write-write version conflicts raise
+`WriteTooOld` errors, as they do in other isolation levels.
+
+When Serializable transactions encounter such situations, they restart from the
+beginning to establish a new read snapshot that includes the committed version
+that caused the write-write version conflict. Read Committed transactions have
+the flexibility to do better, thanks to the use of [per-statement read
+snapshots](#per-statement-read-snapshots). Because each statement in a Read
+Committed transaction can observe a different read snapshot, `WriteTooOld`
+errors can be handled at the statement level and not at the transaction level.
+Critically, this allows the statement to retry at a new read snapshot without
+involving the client.
+
+To facilitate these statement-level retries of write-write conflict errors, each
+SQL statement will be run inside of a retry loop. `WriteTooOld` errors will be
+caught before they escape to the client and the statement will be retried. All
+prior effects of the statement will be rolled back on retry through the use of
+[transaction
+savepoints](https://www.cockroachlabs.com/docs/stable/savepoint.html). A
+savepoint will be created at the beginning of each statement, released on
+success, and rolled back on uncertainty error.
+
+However, there is a caveat here. If the statement has already begun streaming a
+partial result set back to the client, it cannot retry transparently. By
+default, the result set will be buffered up to 16KiB before overflowing and
+being streamed to the client. However, this result buffer size can be configured
+using the `sql.defaults.results_buffer.size` cluster setting or the
+`results_buffer_size` session variable. This condition is analogous to the
+[automatic retry
+behavior](https://www.cockroachlabs.com/docs/stable/transactions.html#automatic-retries)
+of implicit transactions.
+
+The only remaining source of isolation-related transaction aborts is locking
+deadlocks. Unlike write-write conflicts, which can be scoped to a single
+statement and retried without client involvement, locking deadlocks will result
+in a transaction abort and an error returned to the client, as [is also the case
+in PostgreSQL](#serialization-failure-handling). This is because locks acquired
+earlier in the transaction may need to be dropped to break the deadlock.
+
+#### Consistency-Related Retries
+
+While Read Committed is a weak enough isolation level to avoid most
+isolation-related causes of retries, its implementation in a distributed system
+leads to a uniquely distributed concern: consistency-related retries.
+
+As [previously discussed](#read-uncertainty-intervals), Read Committed
+transactions will use uncertainty intervals to provide real-time ordering
+guarantees between transactions. The use of uncertainty intervals implies the
+possibility of `ReadWithinUncertaintyInterval` errors. While reading, a Read
+Committed transaction may observe an MVCC version above its read snapshot but
+within its uncertainty interval. Ignoring such values could violate
+linearizability, so the value must be returned. However, merging the uncertain
+value into the existing read snapshot would create an inconsistent view.
+
+When Serializable transactions encounter such situations, they restart from the
+beginning to establish a new read snapshot that includes the uncertain value.
+Like with `WriteTooOld` errors, Read Committed transactions have the flexibility
+to do better, thanks to the use of [per-statement read
+snapshots](#per-statement-read-snapshots). Because each statement in a Read
+Committed transaction can observe a different read snapshot,
+`ReadWithinUncertaintyInterval` error can be handled at the statement level and
+not at the transaction level. Like with write-write conflict errors, this allows
+the statement to retry at a new read snapshot without involving the client.
+
+The same per-statement retry loop used to retry write-write conflicts will also
+be used to facilitate statement-level retries of uncertainty errors.
+
+As with transaction-level retries of Serializable transactions, statement-level
+retries of Read Committed transactions will not reset the transaction's
+uncertainty interval. Even as the read snapshot advances across retries, the
+upper bound of the uncertainty interval will remain fixed. This eliminates the
+possibility of starvation and bounds the retry duration to the configured
+maximum clock offset (i.e. the size of the uncertainty interval). In other
+words, even in the worst case, this statement-level retry loop will converge.
+
+The same caveat about result set streaming that was mentioned in the previous
+section applies to uncertainty errors as well.
+
+#### Retry Avoidance Through Read Refreshes
+
+While a per-statement retry loop limits the cases where isolation and
+consistency-related retry errors escape from the SQL executor to the client, it
+is better if these errors never emerge from the key-value layer in the first
+place. The avoidance of such retry errors is possible in limited but important
+cases through a familiar mechanism: read refreshes.
+
+This proposal previously mentioned that read refreshes are not needed for
+isolation levels that tolerate write skew. While read refreshes are less
+important for weak isolation levels, these levels can still benefit from the
+mechanism in the limited cases where transactions need to adjust their read
+snapshot. Specifically, even under weak isolation levels, read refreshes can be
+used to handle write-write version conflicts (`WriteTooOld` errors) and read
+uncertainty conflicts (`ReadWithinUncertaintyInterval` errors). Read refreshes
+do so by proving that a transaction's current read snapshot is equivalent to a
+later read snapshot for the set of key spans previously read by the transaction,
+allowing the transaction to dynamically adjust its read snapshot without a full
+restart.
+
+Read refreshes come in two flavors: client-side and server-side. Both forms of
+refreshes will be supported for Read Committed through the use of the
+`txnSpanRefresher` and the `CanForwardReadTimestamp` protocol.
+
+The only difference between Read Committed transactions and
+Snapshot/Serializable transactions in this area is that Read Committed
+transactions will clear their refresh spans when establishing new per-statement
+read snapshots. Doing so will increase the success rate of read refreshes Read
+Committed transactions. Additionally, it will increase the number of cases where
+server-side refreshes are possible — each statement will begin with the
+opportunity to perform server-side refreshes. Consequently, many simple
+statement will never perform per-statement retries, even if they experience
+contention.
+
+#### Structural Commit Conditions
+
+The remaining commit conditions that Read Committed transactions must check are
+structural.
+
+First, Read Committed transactions must ensure that their intent writes have all
+succeeded. If any pipelined intent writes have failed, a
+`RETRY_ASYNC_WRITE_FAILURE` error will be returned.
+
+Second, Read Committed transactions must commit before their schema-imposed
+deadline. If long-running Read Committed transactions fail to update schema
+leases before attempting a commit, a `RETRY_COMMIT_DEADLINE_EXCEEDED` error will
+be returned.
+
+None of these structural errors are user-controllable. They should not be
+produced outside of extraordinary cluster conditions like node failures. Users
+are not expected to handle them gracefully.
+
+#### Parallel Commits Protocol
+
+The [parallel
+commits](https://github.com/cockroachdb/cockroach/blob/8e24570fa366ed038c6ae65f50db5d8e22826db0/pkg/kv/kvclient/kvcoord/txn_interceptor_committer.go#L51)
+atomic commit protocol also deserves discussion.
+
+The protocol extends traditional two-phase commit with a distributed commit
+condition called the "implicit commit" state. This state is a function of the
+staging timestamp of a transaction record and the version timestamp of each of
+the intents listed in the transaction record's in-flight write set.
+
+The condition does not change for Read Committed transactions, and the atomic
+commit protocol remains equally applicable to the weaker isolation level.
+
+## Row-Level Locks
+
+Serializable transactions in CockroachDB use read refreshes as a form of
+optimistic validation to ensure the integrity of reads on commit. As discussed
+above, this will not be the case for Read Committed transactions.
+
+This is a deliberate design decision. By permitting skew between the one or more
+read snapshots observed during a transaction and the MVCC snapshot into which
+the transaction installs its writes, Read Committed transactions can avoid
+retries and blocking. In some sense, this flexibility is what enables the Read
+Committed isolation level to accomplish its goals. In exchange, these reads
+provide weak isolation guarantees.
+
+However, there are cases where stronger guarantees are needed for the reads in a
+Read Committed transaction to enforce data integrity rules. These situations
+arise at both the application level (outside the DB) and the system level
+(inside the DB). In these cases, row-level locks serve as an important tool to
+"upgrade the isolation" of select read operations in an otherwise weakly
+isolated transaction.
+
+A useful mental model for those familiar with MVCC is that standard reads
+performed during a Read Committed transaction are not guaranteed to remain valid
+up to the point when the transaction commits. However, by acquiring locks on
+specific rows when reading, writes by other transactions that would invalidate
+those reads are blocked from completing until the reader commits[^5]. As a
+result, stronger guarantees can be made about the relationship between certain
+reads in a transaction and that transaction's writes. These guarantees then
+serve as the foundation for building higher-level data integrity constraints.
+
+[^5]: Ignoring phantoms inserted between rows or assuming the use of gap locks.
+
+While CockroachDB already has limited support for row-level locks (e.g. `SELECT
+FOR UPDATE`), the support lacks the expressivity and reliability that users of
+Read Committed will demand. New forms of row-level locks must be introduced.
+These locks must then be improved to provide strong correctness guarantees.
+
+### Shared Locks
+
+CockroachDB added support for `SELECT FOR UPDATE` in v20.1. `SELECT FOR UPDATE`
+acquires an exclusive lock on each row returned from a `SELECT` statement. At
+the time, the tool was primarily meant to provide fine-grained control over lock
+ordering. Users could avoid transaction retries in certain cases by acquiring
+locks for rows that they intended to update earlier in their transaction. This
+motivation was explored in [this introductory blog
+post](https://www.cockroachlabs.com/blog/when-and-why-to-use-select-for-update-in-cockroachdb/).
+
+`FOR UPDATE` refers to the strength of the lock acquired on rows returned from
+the `SELECT` statement. If a transaction intends to later `UPDATE` a row, it
+benefits from acquiring an
+[Exclusive](https://github.com/cockroachdb/cockroach/blob/e67e3961a8f4314dd7d92a0bcbe0986f3ce8cee9/pkg/kv/kvserver/concurrency/lock/locking.proto#L91)
+lock on the row when it initially reads the row's value.
+
+Under Read Committed, transactions may want to lock a row to prevent concurrent
+writes even if they don't intend to `UPDATE` the row themselves. In these cases,
+blocking concurrent readers is unnecessary and undesirable, so
+[Shared](https://github.com/cockroachdb/cockroach/blob/e67e3961a8f4314dd7d92a0bcbe0986f3ce8cee9/pkg/kv/kvserver/concurrency/lock/locking.proto#L53)
+locks are a better alternative.
+
+SQL provides a tool for these situations in the form of `SELECT FOR SHARE`.
+`SELECT FOR SHARE` behaves identically to `SELECT FOR UPDATE`, except that it
+acquires a
+[weaker](https://github.com/cockroachdb/cockroach/blob/e67e3961a8f4314dd7d92a0bcbe0986f3ce8cee9/pkg/kv/kvserver/concurrency/lock/locking.proto#L142)
+Shared lock on each of the returned rows.
+
+The design of Shared locks was explored in: [#101799](https://github.com/cockroachdb/cockroach/pull/101799).
+
+Read Committed will depend on the implementation of the Shared locking strength
+for two reasons. First, applications that use `SELECT FOR SHARE` will behave
+incorrectly if we continue to treat the locking strength as a no-op. Second,
+CockroachDB's SQL layer will begin to use `SELECT FOR SHARE` internally to
+enforce referential integrity constraints in Read Committed transactions, as we
+will see later on in this proposal.
+
+### Reliability and Enforcement
+
+Row-level locks acquired by `SELECT FOR UPDATE` are currently best-effort.
+
+The locks are maintained only on the leaseholder (they are "unreplicated") and
+are discarded in a handful of circumstances, including lease transfers, range
+splits, range merges, node failures, and memory limits. This has caused user
+confusion, but has not been a correctness concern to date because Serializable
+transactions never rely on these locks to ensure isolation guarantees.
+
+With the introduction of Read Committed, best-effort row-level locks are no
+longer sufficient for `SELECT FOR UPDATE`. We will need these locks to provide
+stronger guarantees.
+
+#### Properties of Reliability
+
+We can split the meaning of "reliable" row-level locks into two properties:
+
+**Isolation**: If a lock is held on a row by a transaction, that row's value
+must not be changed by any other transaction before the lock holder commits.
+Here, "before" is defined in the MVCC timestamp domain.
+
+**Mutual Exclusion**: If a lock is held on a row by a transaction, no other
+transaction may acquire a
+[conflicting](https://github.com/cockroachdb/cockroach/blob/e67e3961a8f4314dd7d92a0bcbe0986f3ce8cee9/pkg/kv/kvserver/concurrency/lock/locking.proto#L142)
+lock on that same row before the original lock holder commits. Here, "before" is
+defined in the wall clock domain, as perceived by a client of the database.
+
+Without additional constraints or assumptions, neither property implies the
+other. This is inconsequential today because CockroachDB's row-level locks
+currently provide neither property.
+
+Isolation is enforced in Serializable transactions using read refreshes,
+allowing the locks to be best-effort. If a lock is lost and a conflicting
+version is installed on a previously locked row before the lock holder is
+committed, the lock holder's refresh will fail and the transaction will abort.
+
+Mutual Exclusion is not provided by any other mechanism. Instead, it is a use
+case for row-level locks that CockroachDB has resisted. At a theoretical level,
+fault-tolerant mutual exclusion in a distributed system [is
+impossible](https://martin.kleppmann.com/2016/02/08/how-to-do-distributed-locking.html).
+The best a distributed system like CockroachDB can do is strive to minimize the
+cases where mutual exclusion is violated and introduce commit-time validation
+that mutual exclusion was not violated (e.g. preventing a transaction from
+committing if its lock was removed because the system incorrectly deemed its
+coordinator to have failed).
+
+To support Read Committed, only the Isolation property is necessary. However,
+some solutions move us closer to achieving the Mutual Exclusion property as
+well.
+
+#### Reliability Improvement Alternatives
+
+There are a collection of approaches that we could take to strengthen row-level
+locking to a degree that the row-level locks could be used for Read Committed.
+To compare these alternatives, we first define a set of tests to evaluate each
+approach.
+
+- **(T1) Provides Isolation**: Does the approach provide the **Isolation**
+ property?
+
+- **(T2) Provides Mutual Exclusion**: Does the approach provide the **Mutual
+ Exclusion** property?
+
+- **(T3) Lock Persistence**: Does the approach require a disk write per
+ row-level lock?
+
+- **(T4) Lock Replication**: Does the approach require replication per row-level
+ lock?
+
+- **(T5) Commit-Time Validation**: Does the approach require commit-time
+ validation of lock validity?
+
+- **(T6) Commit Starvation**: Is the approach susceptible to commit starvation
+ if a committer is pushed during a commit-time validation phase?
+
+- **(T7) Completeness**: Will the approach support all transactions?
+
+- **(T8) Abort Conditions**: What conditions will cause a transaction to abort
+ and return an error?
+
+Using these tests, we can then compare the base alternatives and their
+specialized variants.
+
+At a high level, there are three alternative approaches:
+- **Refresh Under Locks**
+- **Validate Locks**
+- **Replicate Locks**
+
+##### Alternative: Refresh Under Locks
+
+The **Refresh Under Locks** approach extends the existing Read Refresh
+mechanism. While Read Committed transactions no longer refresh all of their
+reads, the keys on which they acquired locks will be tracked in their refresh
+span set and will be refreshed on commit.
+
+This approach provides isolation (**T1**) for the set of keys locked by a
+transaction in the same way that Serializable transactions provide isolation for
+all keys read, using a commit-time refresh (**T5**). However, it makes no
+attempt to ensure that locks are still held at commit time, so it does not
+provide mutual exclusion (**T2**).
+
+Because locks can be lost without violating isolation, the most basic version of
+this approach does not need lock persistence (**T3**) or lock replication
+(**T4**). However, a lost lock can lead to a refresh failure and a transaction
+abort (**T8**) if the loss of the lock allows for a conflicting write to
+succeed.
+
+Thanks to refresh span condensing, the approach is close to complete (**T7**),
+assuming no contention. Even transactions that acquire millions of locks like
+`SELECT * FROM billion_rows FOR UPDATE` should be able to commit, though span
+condensing does create the opportunity for false positive refresh validation
+failures. However, the approach is not complete with contention. A transaction
+that acquires millions of locks will quickly exceed the maximum in-memory lock
+limit and the locks will be lost. This can allow conflicting writes on
+should-be-locked keys, causing the refresh to fail.
+
+Finally, assuming non-blocking write-read conflicts, the approach is subject to
+starvation (**T6**). When a transaction begins to commit, it will refresh to its
+current provisional commit timestamp. If its minimum commit timestamp is pushed
+by a conflicting reader during this time, the transaction will need to refresh
+again. This can starve, as is described in
+[#95227](https://github.com/cockroachdb/cockroach/issues/95227).
+
+While the approach builds upon the existing Read Refresh mechanism, that
+mechanism will need generalization to work with Read Committed. This is because
+Read Committed transactions operate across many read snapshots. The refresh span
+tracking must be augmented with a timestamp-per-span, which complicates the data
+structure and the span condensing algorithm. The Read Refresh API already
+supports per-span `RefreshFrom` timestamps, so this will be strictly a
+client-side bookkeeping change.
+
+##### Alternative: Validate Locks
+
+The **Validate Locks** approach is similar in spirit to the **Refresh Under
+Locks** approach. Both use a commit-time validation step to verify isolation.
+However, the two approaches differ in what they verify. While **Refresh Under
+Locks** looks for violations of isolation in the MVCC history of keys that were
+locked, **Validate Locks** simply verifies that the locks that were acquired are
+still held at commit time.
+
+The approach then provides isolation (**T1**) by construction — if locks protect
+against conflicting writes that could violate isolation, then a commit-time
+proof (**T5**) that all acquired locks still exist acts as a proof that
+isolation has not (yet) been violated. Additionally, this serves as a proof that
+mutual exclusion (**T2**) has not been violated, because no conflicting lock
+could have been acquired since the committing transaction acquired its locks.
+
+The assumption with this approach is that neither lock persistence (**T3**) nor
+lock replication (**T4**) would be used. As a result, lease transfers, node
+crashes, memory limits (**T7**), or other events could lead to lost locks, which
+would cause validation to fail and the lock acquirer to abort (**T8**). Unlike
+**Refresh Under Locks**, this abort would occur even if there was no real
+transaction contention.
+
+This approach also has a similar downside to **Refresh Under Locks**, which is
+that the commit-time validation only provides a proof of isolation up to some
+MVCC timestamp. The lock could be lost immediately after validation and the best
+the validating transaction could do is bump the timestamp cache below the lock
+during validation to its current provisional commit timestamp (in fact, it must
+do this). However, if the committing transaction is then pushed, it would need
+to re-validate its locks. This means that the approach is also subject to
+starvation (**T6**) with non-blocking write-read conflicts.
+
+##### Alternative: Replicate Locks
+
+The **Replicate Locks** approach is similar to the **Validate Locks** approach
+in that it also uses locks themselves to provide isolation (**T1**) and mutual
+exclusion (**T2**) over the keys that it protects. However, it does not use a
+commit-time validation step (**T5**) to determine whether locks are still held.
+Instead, it replicates locks eagerly during transaction execution to eliminate
+cases where they could be lost.
+
+There are a handful of variants of this approach, all of which include some form
+of lock persistence (**T3**) and lock replication (**T4**). The simplest
+approach would be to synchronously replicate locks during the execution of
+`SELECT FOR {UPDATE/SHARE}` statements. A more sophisticated variant would be to
+pipeline this lock acquisition like we do with [intent
+pipelining](https://www.cockroachlabs.com/blog/transaction-pipelining/), at the
+risk of pipelining errors. A third variant would be to eagerly acquire
+unreplicated locks during the execution of `SELECT FOR {UPDATE/SHARE}`
+statements and then promote these locks to replicated locks at commit-time, at
+the risk of the unreplicated locks having been lost by this time. The variants
+differ in performance but are otherwise similar.
+
+Replicating locks and storing them in persistent storage (using the replicated
+lock table keyspace which was designed for such storage) prevents the locks from
+being lost during lease transfers, node crashes, or memory limits (**T8**). As a
+result, this approach is complete (**T7**). It can use client-side span
+coalescing to scale to an arbitrary number of locks in roughly the same way that
+a transaction can write to an arbitrary number of keys.
+
+Replicating locks also avoids the risk of commit starvation (**T6**) because it
+forgoes commit-time validation entirely. Once a transaction has persisted all
+locks, it is free to commit at any MVCC timestamp in the future. It can continue
+to be pushed by contending readers without consequence. The only requirement is
+that when locks are released after a transaction has committed and the locks are
+not replaced by a committed version, the timestamp cache must be bumped to the
+commit timestamp of the transaction to prevent conflicting writers from
+re-writing portions of the "locked MVCC history".
+
+##### Variants and Optimizations
+
+As implied above, variants of the approaches exist and the approaches are not,
+for lack of a better word, mutually exclusive. Aspects of some approaches could
+be used to augment other approaches to arrive at strong alternatives. For
+example, the commit-time lock replication variant discussed above can be viewed
+as a hybrid of Lock Validation and Lock Replication. Lock Validation schemes
+could also employ Lock Replication as a mechanism to spill locks on memory
+limits or ship locks during lease transfers.
+
+One additional hybrid approach that deserves mention is **Lock Validation with
+Validity Windows**. This approach exploits the fact that a stable leaseholder
+can provide _some_ guarantees on lock validity, even if it can't guarantee lock
+validity indefinitely. For instance, a leaseholder could make a guarantee that a
+lock will only be lost if the leaseholder crashes. In doing so, it could
+guarantee isolation on locked keys up to its lease expiration. As a result,
+unreplicated lock acquisition would provide a validity window for each lock, and
+a transaction would only need to validate its locks if it ran for long enough to
+exceed this validity window. This hybrid approach provides many of the benefits
+of each alternative — namely, it avoids synchronous lock replication but it also
+avoids commit starvation. However, for a leaseholder to make such a guarantee,
+it would likely employ lock replication in certain cases (e.g. memory limits),
+so it makes sense to consider this hybrid approach as a future optimization.
+
+#### Reliability for Preview Release
+
+For the initial version of Read Committed, we propose a limited form of
+synchronous **Lock Replication** as a mechanism to ensure lock reliability. This
+is primarily due to the completeness **(T7)** and starvation **(T6)** properties
+provided by the alternative. Performance is a secondary consideration, and the
+performance of replicated locks can be optimized in future releases using the
+suggestions outlined above.
+
+However, one performance optimization will be incorporated into the initial
+design. While `SELECT FOR UPDATE` and `SELECT FOR SHARE` will synchronously
+replicate locks, "implicit select for update" performed during the search phase
+of certain mutation statements will continue to acquire unreplicated and
+unvalidated locks. These locking reads are immediately followed by a write, so
+they need not provide isolation on their own. Meanwhile, replicating these locks
+would incur a severe latency cost. This is true even if the replication is
+pipelined, because the subsequent write would immediately hit a pipeline stall
+while waiting for the replication of the lock to complete.
+
+The decision of whether to replicate locks or not will be expressed from SQL
+through the KV API using the existing `lock.Durability` flag. Only replicated
+locks will provide isolation guarantees.
+
+## Query Planning and Execution
+
+To support the more stringent locking requirements of Read Committed, several
+changes must be made to query planning and execution around lock acquisition.
+
+### Read-Only Queries (`SELECT`)
+
+Query planning and execution for read-only queries (i.e. `SELECT` statements)
+will not change for Read Committed transactions, despite the use of
+[per-statement read snapshots](#per-statement-read-snapshots). These queries
+will continue to operate on a consistent snapshot of the system, so all planning
+optimizations derived from SQL constraints remain valid.
+
+### Explicit Locking Queries (`SELECT FOR UPDATE`)
+
+Under our Serializable isolation, locking is not needed for correctness. Because
+of this, our current implementation of `SELECT FOR UPDATE` takes some liberties
+for better performance.
+
+- Locks are currently [only placed on the indexes scanned by the
+ query](https://github.com/cockroachdb/cockroach/issues/57031). If the `SELECT
+ FOR UPDATE` query never reads from the primary index of the table, it will not
+ place a lock there. This could prevent a `SELECT FOR UPDATE` query from
+ correctly blocking an `UPDATE` if the `SELECT FOR UPDATE` and the `UPDATE`
+ touch disjoint indexes.
+- Locks are acquired [during the initial index scans of the
+ query](https://github.com/cockroachdb/cockroach/issues/75457), even if some
+ rows are later eliminated (e.g. by a filter or a join). This could cause us to
+ acquire unnecessary locks, potentially causing artificial contention.
+- To avoid this artificial contention, locks are sometimes [not
+acquired](https://github.com/cockroachdb/cockroach/blob/48ef0d89e6179c0d348a5236ad308d81fa392f7c/pkg/sql/opt/exec/execbuilder/mutation.go#L987-L1009)
+ at all. This could prevent `SELECT FOR UPDATE` from working in some cases.
+- As described in [Reliability and Enforcement](#reliability-and-enforcement),
+ locks are best-effort, and may not persist until commit for various reasons.
+
+Under Read Committed these shortcuts could cause incorrect query execution. To
+fix them, when necessary we will add an extra locking join to the top of the
+query plan instead of locking during the initial row fetch. This will typically
+be an index join or a lookup join to the primary index of the table to lock (or
+multiple joins in the case of multiple tables to lock).
+
+This locking join will acquire fully-replicated locks to ensure the locks
+persist until commit, as described in [Reliability for Preview
+Release](#reliability-for-preview-release). The locking join will return a
+`WriteTooOld` error if there have been any new versions committed to locked rows
+after the statement read snapshot.
+
+#### Optimizer Locking Alternatives
+
+There are several alternative methods the optimizer could use to produce this
+extra locking join for `SELECT FOR UPDATE`:
+1. Add a new `Lock` operator.
+2. Add a locking property to the exsting `Select` operator.
+3. Add a new `LockedSelect` operator.
+4. Use a physical property enforcer.
+
+##### Alternative: Lock Operator
+
+TODO(michae2): describe lock operator alternative
+
+##### Alternative: Locking Property in Select
+
+TODO(michae2): describe locking property in select alternative
+
+##### Alternative: LockedSelect Operator
+
+TODO(michae2): describe lockedselect operator alternative
+
+##### Alternative: Physical Property Enforcer
+
+TODO(michae2): describe physical property enforcer alternative
+
+#### Optimizer Change for Preview Release
+
+For the initial version of Read Committed
+
+#### Locking Individual Column families
+
+Narrowing lock scope to an individual column family of a row can help ensure
+that the performance benefits of multiple column families are realized in
+workloads with contention. For very simple `SELECT FOR UPDATE` queries, our
+current implementation is able to lock an individual column family of a row,
+rather than every column family, depending on how the initial row fetch of the
+`SELECT FOR UPDATE` is constrained.
+
+With the changes for Read Committed, we expect that `SELECT FOR UPDATE` will be
+able to lock only the necessary individual column families in more cases. This
+is because the extra locking join will be able to use column-family-tight spans
+in cases where the initial row fetch cannot.
+
+#### Write-Write Version Conflicts
+
+As discussed in [Write-Write Conflict
+Handling](#write-write-conflict-handling-or-lost-update-intolerance), `SELECT
+FOR UPDATE` statements can experience write-write version conflicts if new
+versions of rows are discovered after acquiring locks. PostgreSQL uses a special
+`EvalPlanQual` mode to handle these write-write version conflicts, which
+re-evaluates some of the query logic on the new version of each locked row. We
+will not implement an EPQ mode. Instead, on discovering new committed versions,
+the locking join will fail with a `WriteTooOld` error which will cause the
+statement to retry.
+
+### Reading Mutation Statements (`INSERT ON CONFLICT`, `UPDATE`, `DELETE`, etc.)
+
+"Reading mutation statements" are DML statements that both read from and write
+to the database, such as most `UPDATE` statements. Under Serializable isolation
+the read sets of these statements are validated at transaction commit time, to
+avoid write skew. In our current implementation, reading mutation statements
+sometimes acquire implicit row locks to try and avoid retries, but as mentioned
+previously these locks are not needed for correctness.
+
+Surprisingly, query planning and execution for reading mutation statements do
+not need to change for Read Committed, despite their potential to incur
+write-write conflicts. This is because write-write version conflicts will be
+detected when these statements write new versions of each row (lay down
+intents). Any committed version newer than the mutation statement's read
+snapshot will generate a `WriteTooOld` error, causing at least one of the
+conflicting statements to retry, so mutation statement execution can remain
+unaware of write-write conflict handling.
+
+This means that, as for `SELECT FOR UPDATE`, we will not implement EPQ mode for
+mutation statements. Instead we will rely on statement retries to handle
+write-write conflicts.
+
+And as described in [Reliability for Preview
+Release](#reliability-for-preview-release), mutation statements can continue to
+use unreplicated locks during their initial row fetch, because the initial
+unreplicated locks do not need to persist until commit time for
+correctness.
+
+FK checks performed at the end of mutation statements, however, will have to use
+replicated locks to ensure we maintain FK constraints. (See [system-level SQL
+constraints](#system-level-sql-constraints) below.)
+
+### Blind Mutation Statements (`INSERT`, `UPSERT`, `DELETE`, etc.)
+
+"Blind mutation statements" are DML statements that write to the database
+without reading, such as most `UPSERT` statements. These statements cannot incur
+lost updates, so it should almost never be necessary to retry these statements
+at the conn_executor level. Fortunately, these statements can benefit from use
+of server-side read refreshes, [as outlined
+previously](#retry-avoidance-through-read-refreshes).
+
+### Common Table Expressions and User-Defined Functions
+
+Planning and execution for CTEs and UDFs does not need to change for Read
+Committed isolation.
+
+CTEs, and both `STABLE` and `IMMUTABLE` UDFs will perform their reads at the
+same read snapshot as the main statement. Any locking or mutation statements
+they contain will perform in the manner described above. If a CTE or a UDF
+encounters a `WriteTooOld` error due to a write-write conflict, the entire main
+statement will retry.
+
+`VOLATILE` UDFs will perform their reads at the same read snapshot as the main
+statement, but with [a later sequence
+number](https://github.com/cockroachdb/cockroach/blob/08ac8fde23e42cf26677a3dfd1c3a0fb60e40f65/pkg/sql/routine.go#L44-L59),
+allowing them to read writes performed by the main statement, but not writes
+committed from later transactions. If a `VOLATILE` UDF encounters a
+`WriteTooOld` error it will also cause the main statement to retry.
+
+### System-level SQL Constraints
+
+System-level constraints must be enforced correctly regardless of isolation
+level. Under Read Committed isolation, this will require holding replicated
+locks for all constraint checks and cascades (whether executed inline or as
+post-queries).
+
+[Foreign key checks](https://github.com/cockroachdb/cockroach/issues/80683) will
+change to use `SELECT FOR SHARE` locks.
+
+Foreign key cascades will not change. The intents they write will function as
+replicated locks for the duration of the transaction.
+
+`UNIQUE` checks will not change. These checks are always handled through a
+combination of `InitPut` or `CPut` commands and careful key encoding when
+mutating a row. The intents written by the row mutation will function as
+replicated locks.
+
+`UNIQUE WITHOUT INDEX` checks cannot easily be implemented correctly under Read
+Committed using only row locking, because they depend on the non-existence of a
+span of rows. Initially we will disallow the enforcement of `UNIQUE WITHOUT
+INDEX` checks in transactions run under Read Committed, so these transactions
+will be unable to insert into tables with this form of constraint. Consequently,
+`REGIONAL BY ROW` tables will be inaccessible to Read Committed transactions in
+the initial preview. Eventually we will allow `UNIQUE WITHOUT INDEX` checks if
+the check can be built using single-row spans (i.e. if there is an index which
+only has enum columns before the `UNIQUE WITHOUT INDEX` columns). This will
+require taking `SELECT FOR SHARE` locks on non-existent rows.
+
+`CHECK` constraint checks will have to acquire `SELECT FOR SHARE` locks on any
+unmodified column families if the constraint references multiple column
+families. This may negate the benefit of column families in some cases.
+
+### Transaction Orchestration
+
+TODO(nvanbenschoten): Work with Rafi to flesh this section out. Mention
+connExecutor and changes for:
+- Updating txn read snapshot on each statement. Done through existing calls to Txn.Step
+- Retrying statements on retry errors. Needs to error handling logic.
+
+## Configuration
+
+Configuration of the Read Committed isolation level will be handled through the
+standard session variable infrastructure.
+
+Individual transactions can be configured to run at the Read Committed isolation
+level using any of the following configuration methods:
+```sql
+-- set isolation in BEGIN statement
+BEGIN TRANSACTION ISOLATION LEVEL READ COMMITTED;
+
+-- set isolation of current transaction (statement style)
+BEGIN; SET TRANSACTION ISOLATION LEVEL READ COMMITTED;
+
+-- set isolation of current transaction (variable style)
+BEGIN; SET transaction_isolation = 'read committed';
+```
+
+The default isolation level for all future transactions can be set to Read
+Committed using any of the following configuration methods:
+```sql
+-- set default isolation level (statement style)
+SET SESSION CHARACTERISTICS AS TRANSACTION ISOLATION LEVEL READ COMMITTED;
+
+-- set default isolation level (variable style)
+SET default_transaction_isolation = 'read committed';
+
+-- set default isolation level (connection parameter style)
+cockroach sql -–url='postgres://root@hostname:26257/db?options=-c default_transaction_isolation=read%20committed'
+```
+
+Like in PostgreSQL, READ UNCOMMITTED will also be accepted by all of these
+configurations and will map to READ COMMITTED.
+
+## Configuration Introspection
+
+Observability into the current transaction's isolation level and the session's
+default isolation level will both also be handled through the standard session
+variable infrastructure:
+```sql
+SHOW transaction_isolation;
+
+SHOW default_transaction_isolation;
+```
+
+### Configuration Migration
+
+Currently, CockroachDB accepts many of these configurations. However, it ignores
+them and provides applications with Serializable isolation instead. This poses a
+risk that applications which use CockroachDB today are unwittingly asking for
+Read Committed, either directly or indirectly (e.g. through an ORM). These same
+applications may not be prepared to suddenly jump to Read Committed when
+upgrading CockroachDB to v23.2.
+
+To hedge against this risk and provide users that fall into this situation with
+quick remediation, a hidden cluster setting will be introduced to disable the
+use of Read Committed and retain the existing mappings in all configurations
+from "read committed" isolation to "serializable" isolation. The cluster setting
+can be removed in a future release.
+
+## Observability
+
+Details about transaction isolation will be added to the following three system
+tables:
+```
+crdb_internal.node_transactions
+crdb_internal.cluster_transactions
+crdb_internal.cluster_locks
+```
+
+Surfacing transaction isolation and its impact on transaction contention in the
+DB console is important, but it is beyond the scope of this RFC.
+
+## Miscellaneous Interactions
+
+### With Serializable Isolation
+
+Serializable and Read Committed transactions can coexist in relative harmony. As
+described earlier, the isolation levels differ primarily in their tolerance to
+write skew and in their selection of read snapshot(s). These differences are
+internal concerns that do not affect other concurrent transactions, so mixing
+transaction isolation levels will not cause problems. Regardless of isolation
+level, transactions will continue to read from consistent snapshots of the
+system and the commit of any transaction will remain atomic to all other
+transactions.
+
+A serializable history will still be possible to construct from all transactions
+running at Serializable isolation. However, it will not always be possible to
+place Read Committed transactions into that history.
+
+The one interaction between the two isolation levels that deserves discussion is
+write-read conflicts where the writer is a Serializable transaction and the
+reader is a Read Committed transaction. We discussed earlier that we intend to
+make write-read conflicts non-blocking for all isolation levels. However, we
+will only initially make that change for Read Committed transactions because
+there are fewer consequences to doing so. During this intermediate period, we
+could either block the Read Committed transaction on the Serializable
+transaction to avoid pushing the Serializable transaction and forcing it to
+refresh, or we could make the conflict non-blocking to avoid unexpectedly
+blocking the Read Committed transaction. We chose the latter option to ensure
+that reads in Read Committed transactions never block, potentially at the cost
+of Serializable transaction retries.
+
+### With Implicit Transactions
+
+Implicit transactions are single-statement transactions without `BEGIN` or
+`COMMIT` statements. The transactions benefit from [automatic
+retries](https://www.cockroachlabs.com/docs/stable/transactions.html#automatic-retries)
+thanks to the CockroachDB SQL gateway hearing about the entire transaction at
+once.
+
+Read Committed's "per-statement Snapshot isolation" transaction model implies
+that these implicit transactions will behave identically to a potential Snapshot
+isolation level. Namely, they will operate at a single read snapshot and will
+permit skew between their commit timestamp and this read snapshot's timestamp.
+
+### With One-Phase Commit
+
+One-Phase Commit is a fast-path for mutations that perform writes to a single
+range and issue these writes in a batch with their EndTxn request. These
+transactions can avoid two-phase commit, bypassing the use of intents or a
+transaction record and writing committed values directly.
+
+Read Committed transactions will have access to this fast-path. The only
+relevant difference between Read Committed transactions and Serializable
+transactions is that Read Committed transactions have a more lenient commit
+condition (skew between their read and write timestamps are permitted) and the
+one-phase commit fast-path must be made aware of this.
+
+### With AS OF SYSTEM TIME
+
+The [`AS OF SYSTEM
+TIME`](https://www.cockroachlabs.com/docs/stable/as-of-system-time.html) clause
+can be added to individual statements or to entire read-only transactions,
+instructing CockroachDB to read at a historical MVCC snapshot. This is an
+important feature that underpins follower reads.
+
+In some regard, such a feature is incompatible with the Read Committed isolation
+level, which captures a new MVCC snapshot for each statement in a transaction.
+For this reason, PostgreSQL disallows the use of `SET TRANSACTION SNAPSHOT`
+(analogous to `AS OF SYSTEM TIME`) in Read Committed transactions. When issued,
+an error is returned:
+```
+ERROR: a snapshot-importing transaction must have isolation level SERIALIZABLE or REPEATABLE READ
+```
+
+Proscribing `AS OF SYSTEM TIME` in Read Committed transactions would cause
+confusion and inconvenience for users of CockroachDB, especially when attempting
+to use follower reads. Instead of banning it, the syntax will be accepted in
+`BEGIN` and `SET` statements and the transaction will be promoted to a
+read-only, Serializable transaction[^6] with the specified fixed read timestamp
+across all statements, which matches the user's intent. Such transactions are
+not subject to retry errors.
+
+[^6]: read-only, Serializable isolation transactions are equivalent to
+ read-only, Snapshot isolation transactions.
+
+As an extension, the syntax can also be accepted on individual `SELECT`
+statements in a Read Committed transaction. This will instruct the transaction
+to run the individual statement at the specified fixed read timestamp. The
+utility of this may be limited, but the behavior matches expectations of Read
+Committed transactions.
+
+### With Non-Standard SELECT FOR UPDATE Wait Policies
+
+TODO(michae2): write this after the Explicit Row-Level Locking (SELECT FOR
+UPDATE) above.
+
+#### NOWAIT
+
+#### SKIP LOCKED
+
+### With Schema Changes
+
+The schema change protocol runs partially within user transactions, so special
+care must be taken before permitting Read Committed transactions to perform
+schema changes. The protocol depends on a serialization of versions for each
+individual descriptor and on cross-descriptor consistency. Consequently, lost
+updates on a single descriptor and write skew across descriptors must both be
+prohibited.
+
+The use of explicit row-level locking (`SELECT FOR SHARE`) during descriptor
+lookups may be sufficient to eliminate these hazards. However, for the preview
+release of Read Committed, schema changes in Read Committed transactions will be
+disabled.
+
+### With Column Families
+
+TODO(nvanbenschoten): prove that writes to disjoint column families cannot cause
+index corruption.
+
+### With CDC
+
+Read Committed has no meaningful interaction with CDC. Committed values from
+Read Committed transactions will be published on rangefeed subscriptions during
+Raft log application like in any other transaction.
+
+### With Multi-Region
+
+Read Committed has only limited interactions with multi-region. At a high level,
+this is because Read Committed weakens isolation guarantees but not consistency
+guarantees. Consistency is where much of the cost of multi-region comes from.
+
+With that said, Read Committed transactions run in a multi-region deployment
+will still see some benefit from their weaker isolation guarantees. For example,
+Read Committed transactions avoid refreshing reads on commit, which can be
+costly in a read-write transaction that reads from a remote leaseholder.
+
+On the other hand, the high network latencies in a multi-region cluster may also
+expand the timing window for isolation-related anomalies to occur in Read
+Committed transactions. Anomalies that may be rare in single-region clusters
+(because transactions commit with low latency) may become more common in
+multi-region clusters.
+
+TODO(michae2): mention `UNIQUE WITHOUT INDEX` constraints
+
+### With Multi-Tenancy
+
+Read Committed has no meaningful interaction with multi-tenancy. Tenants can run
+transactions at any isolation level without concern.
+
+However, the introduction of non-blocking write-read conflicts will simplify the
+resolution of [#71946](https://github.com/cockroachdb/cockroach/issues/71946).
+
+# Testing
+
+To gain confidence in an implementation of Read Committed, testing will be
+needed at multiple levels. As always, individual changes and components will be
+validated using targeted unit tests. However, ensuring transaction isolation
+guarantees is a cross-cutting concern, so integration testing across the
+database stack will be paramount.
+
+To verify the integrity of changes made for Read Committed to the key-value
+layer, [kvnemesis](https://github.com/cockroachdb/cockroach/blob/fadd137a98540a317a379e938a7545fa70590cb4/pkg/kv/kvnemesis/doc.go)
+will be enhanced in four ways:
+1. The framework will be updated to run transactions at weak isolation levels.
+ The framework's validator will be taught to permit write skew for
+ transactions run below Serializable isolation.
+2. The framework will be updated to step read committed transactions through
+ multiple read snapshots, mimicking the style of use expected from SQL when
+ encountering statement boundaries.
+3. The framework's generator will be updated to issue locking reads with a
+ SHARED lock strength.
+4. The framework's generator will be updated to issue locking reads with a
+ REPLICATED lock durability. The framework's validator will then be taught to
+ expect stronger (serializable-like) isolation guarantees from keys locked by
+ a transaction with replicated locks.
+
+Integration testing of Read Committed that exercises SQL will be also
+introduced.
+
+CockroachDB's existing suite of logictests will be extended to run transactions
+under Read Committed isolation. Very few logictests exercise concurrency, so
+they should behave no different than if run under Serializable isolation. The
+few logictests that do manipulate multiple session concurrency will need to be
+updated with isolation-specific expectations.
+
+[Elle](https://github.com/jepsen-io/elle) is a transactional consistency checker
+for black-box databases, which supports weak isolation levels has been
+integrated into [Jepsen](https://github.com/jepsen-io/jepsen). CockroachDB
+already integrates Jepsen into its nightly test suite. This testing will be
+expanded to exercise Read Committed and validate correctness.
+
+[Hermitage](https://github.com/ept/hermitage) is a test suite that consists of
+transaction histories that simulates various concurrency issues. The test suite
+will be manually run against CockroachDB's Read Committed isolation level to
+ensure that it is subject to expected concurrency anomalies and not subject to
+unexpected anomalies. It will also be updated to reflect the addition of new
+isolation levels to CockroachDB.
+
+PostgreSQL's [isolation test
+suite](https://github.com/postgres/postgres/blob/36f40ce2dc66f1a36d6a12f7a0352e1c5bf1063e/src/test/isolation/README)
+contains a set of tests for concurrent transaction behaviors running at
+different isolation levels in PostgreSQL. This test suite will be hooked up to
+CockroachDB. While the behavior of transactions differs between CockroachDB and
+PostgreSQL in multiple ways, it will be useful to see how much of the test suite
+passes against CockroachDB and to understand precisely why the tests that fail
+do so.
+
+Nightly testing of TPC-C will be adapted to run at the Read Committed isolation
+level. When doing so, transaction retry loops will be removed from the workload
+to validate that retry errors are rare or non-existent under weak isolation.
+TPC-C provides a useful sandbox to test weaker isolation levels because it
+contains three moderately complex read-write transactions, two read-only
+transactions, a diverse schema with referential integrity constraints, and
+twelve post-workload consistency checks.
+
+Finally, existing workloads that run against Read Committed in other DBMS
+systems will be solicited from customers. Where possible, these will be run
+against CockroachDB's implementation of Read Committed to validate correctness,
+sufficient completeness of the Read Committed implementation, and expected
+properties like no retry errors and non-blocking reads.
+
+# Performance
+
+TODO(nvanbenschoten): from @bdarnell:
+> I'd like to see a section on performance. There are a number of ways that this
+> proposal affects performance, both good (less wasted work doing retries, no
+> pre-commit span refreshes), and bad (lock replication, savepoint overhead,
+> explicit locks for FK checks). Aside from the correctness concerns, how can we
+> characterize the net expected performance of the two isolation levels?
+>
+> There are also more subtle performance-related risks: Many applications have
+> isolation-related bugs that go undetected because the app simply doesn't get
+> enough traffic to hit the necessary race conditions. If your sub-millisecond
+> operations suddenly start to take longer because foreign key checks now
+> involve multiple RPCs, these latent bugs may be exposed.
+
+# Variation from PostgreSQL
+
+The form of the Read Committed isolation level presented here is strictly
+stronger than what is found in PostgreSQL. The difference between these two
+implementations is in how they handle write-write conflicts during mutation
+statements.
+
+The "Per-Statement Snapshot Isolation" model presented here retries individual
+statements on write-write conflicts, ensuring that within a single statement, no
+lost updates are permitted. This stronger model avoids certain anomalies that
+could allow a mutation statement to perceive non-atomic commits of other
+transactions. In exchange, this stronger model is subject to internal
+per-statement retries.
+
+The "Postgres-Compatible Intra-Mutation Consistency" model breaks mutation
+statements into a search phase, a locking phase, and a predicate re-evaluation
+phase. This decomposition avoids any per-statement retries. In exchange, it can
+permit intra-statement lost updates and other anomalous behavior when
+write-write conflicts are experienced.
+
+A more complete comparison between the two models is presented in the
+[appendix](#appendix-postgres-compatible-intra-mutation-consistency). This
+comparison is accompanied by a discussion of how Postgres-Compatible
+Intra-Mutation Consistency might be implemented in CockroachDB.
+
+# Drawbacks
+
+The primary drawback of introducing the Read Committed isolation level is that
+it provides users with a tool to weaken the correctness guarantees of
+CockroachDB. Unwitting users may employ Read Committed for performance reasons
+without understanding the trade-offs, leading to unexpected correctness bugs or
+data corruption.
+
+This is a real risk. Yet, the risk of not providing users with this
+configuration and failing to support a large class of applications is greater.
+Instead, we will combat this concern with ample documentation to help users make
+well-informed decisions about the performance/correctness trade-off.
+
+# Unresolved questions
+
+## Should we implement Snapshot isolation at the same time?
+
+As alluded to in this proposal, the changes needed to implement Snapshot
+isolation are contained within the changes needed to implement Read Committed.
+As a result, it will be a small lift for us to expose Snapshot isolation after
+making these changes. Conceptually, Snapshot isolation is identical to Read
+Committed except that it chooses _not_ to advance a transaction's read snapshot
+at each statement boundary.
+
+We propose not to expose this isolation level immediately to keep engineering
+efforts focused and to reduce the scope of testing work needed to gain
+confidence in these changes. Still, we note that doing so will be a small lift
+if/when we decide that such an isolation level is needed.
+
+## If we implement Snapshot isolation, should we call it Repeatable Read?
+
+Strictly speaking, the two isolation levels are not the same. Repeatable Read
+(PL-2.99 in Adya) permits Phantom Reads but does not permit Write Skew. Snapshot
+isolation (PL-SI in Adya) does not permit Phantom Reads but does permit Write
+Skew.
+
+However, there is ample precedent for conflating the two with minimal concern.
+Chiefly, PostgreSQL itself implements a form of Snapshot isolation and calls it
+Repeatable Read to remain ANSI SQL compliant. Therefore, if we decide to
+implement Snapshot isolation, we propose that we also call it Repeatable Read.
+
+## Should Read Committed become the new default isolation level in CockroachDB?
+
+We do not plan to immediately change the default isolation level in CockroachDB.
+If such a decision is made at some later point, it will be separate from the
+initial design and implementation effort.
+
+# Appendix: Examples
+
+In the following examples, consider the schema:
+```sql
+create table kv (k int primary key, v int);
+```
+
+### SELECT behavior (without FOR UPDATE)
+
+```sql
+truncate table kv;
+insert into kv values (1, 5);
+```
+
+| Client 1 | Client 2 |
+| -------- | -------- |
+| `begin transaction isolation level read committed;` | |
+| | `begin transaction isolation level read committed;` |
+| `select * from kv;`
`k \| v`
`--+---`
`1 \| 5` | |
+| | `insert into kv values (2, 6);` |
+| `select * from kv;`
`k \| v`
`--+---`
`1 \| 5` | |
+| `insert into kv values (3, 7);` | |
+| `select * from kv;`
`k \| v`
`--+---`
`1 \| 5`
`3 \| 7` | |
+| | `commit;` |
+| `select * from kv;`
`k \| v`
`--+---`
`1 \| 5`
`2 \| 6`
`3 \| 7` | |
+| `commit;` | |
+
+### SELECT FOR UPDATE behavior
+
+```sql
+truncate table kv;
+insert into kv values (0, 5), (1, 5), (2, 5), (3, 5), (4, 1);
+```
+
+| Client 1 | Client 2 |
+| -------- | -------- |
+| `begin transaction isolation level read committed;` | |
+| | `begin transaction isolation level read committed;` |
+| | `insert into kv values (5, 5);` |
+| | `update kv set v = 10 where k = 4;` |
+| | `delete from kv where k = 3;` |
+| | `update kv set v = 10 where k = 2;` |
+| | `update kv set v = 1 where k = 1;` |
+| | `update kv set k = 10 where k = 0;` |
+| `select * from kv where v >= 5 for update;`
`... waits ...` | |
+| | `commit;` |
+| `... waiting completes`
`k \| v`
`--+---`
`2 \| 10`
`4 \| 10`
`5 \| 5`
`10 \| 5` | |
+| `commit;` | |
+
+### UPDATE and DELETE behavior
+
+```sql
+truncate table kv;
+insert into kv values (0, 5), (1, 5), (2, 5), (3, 5), (4, 1);
+```
+
+| Client 1 | Client 2 |
+| -------- | -------- |
+| `begin transaction isolation level read committed;` | |
+| | `begin transaction isolation level read committed;` |
+| | `insert into kv values (5, 5);` |
+| | `update kv set v = 10 where k = 4;` |
+| | `delete from kv where k = 3;` |
+| | `update kv set v = 10 where k = 2;` |
+| | `update kv set v = 1 where k = 1;` |
+| | `update kv set k = 10 where k = 0;` |
+| `update kv set v = 100 where v >= 5;`
`... waits ...` | |
+| | `commit;` |
+| `... waiting completes` | |
+| `select * from kv`
`k \| v`
`--+---`
`1 \| 1`
`2 \| 100`
`4 \| 100`
`5 \| 100`
`10 \| 100` | |
+| `commit;` | |
+
+### INSERT behavior
+
+Insert a new key that has just been changed by another transaction:
+
+```sql
+truncate table kv;
+insert into kv values (1, 1);
+```
+
+| Client 1 | Client 2 |
+| -------- | -------- |
+| `begin transaction isolation level read committed;` | |
+| | `begin transaction isolation level read committed;` |
+| | `update kv set k = 2 where k = 1;` |
+| `insert into kv values (2, 1);`
`... waits ...` | |
+| | `commit;` |
+| `... waiting completes`
`ERROR: duplicate key value violates unique constraint "kv_pkey"` | |
+| `rollback;` | |
+
+Insert a new key that has just been changed by another transaction, with `ON CONFLICT`:
+
+```sql
+truncate table kv;
+insert into kv values (1, 1);
+```
+
+| Client 1 | Client 2 |
+| -------- | -------- |
+| `begin transaction isolation level read committed;` | |
+| | `begin transaction isolation level read committed;` |
+| | `update kv set k = 2 where k = 1;` |
+| `insert into kv values (2, 1) on conflict (k) do update set v = 100;`
`... waits ...` | |
+| | `commit;` |
+| `... waiting completes` | |
+| `select * from kv`
`k \| v`
`--+---`
`2 \| 100` | |
+| `commit;` | |
+
+Insert an old key that has been removed by another transaction:
+
+```sql
+truncate table kv;
+insert into kv values (1, 1);
+```
+
+| Client 1 | Client 2 |
+| -------- | -------- |
+| `begin transaction isolation level read committed;` | |
+| | `begin transaction isolation level read committed;` |
+| | `update kv set k = 2 where k = 1;` |
+| `insert into kv values (1, 1);`
`... waits ...` | |
+| | `commit;` |
+| `... waiting completes` | |
+| `select * from kv`
`k \| v`
`--+---`
`1 \| 1`
`2 \| 1` | |
+| `commit;` | |
+
+Insert an old key that has been removed by another transaction, with `ON CONFLICT`:
+
+```sql
+truncate table kv;
+insert into kv values (1, 1);
+```
+
+| Client 1 | Client 2 |
+| -------- | -------- |
+| `begin transaction isolation level read committed;` | |
+| | `begin transaction isolation level read committed;` |
+| | `update kv set k = 2 where k = 1;` |
+| `insert into kv values (1, 1) on conflict (k) do update set v = 100;`
`... waits ...` | |
+| | `commit;` |
+| `... waiting completes` | |
+| `select * from kv`
`k \| v`
`--+---`
`1 \| 1`
`2 \| 1` | |
+| `commit;` | |
+
+# Appendix: Proof of Correctness
+
+We demonstrate that this model for Read Committed is stronger than[^7]
+Berenson's characterization of Read Committed and Adya's characterization of
+Read Committed (PL-2). We also demonstrate that it is equivalent to[^8] ANSI
+SQL's characterization of Read Committed (Degree 2).
+
+[^7]: Using [Berenson et al.'s
+ definition](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/tr-95-51.pdf)
+ of stronger than, denoted `L1 » L2`.
+[^8]: Using [Berenson et al.'s
+ definition](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/tr-95-51.pdf)
+ of equivalent to, denoted `L1 == L2`.
+
+Berenson defines Read Committed as proscribing Dirty Write (P0) and Dirty Read
+(P1) and permitting Non-repeatable Read (P2), Phantom Read (P3), Lost Update
+(P4), and Read/Write Skew (A5). Our transaction model does not permit P0 because
+transactions are not permitted to overwrite uncommitted writes from other
+transactions. Our transaction model also does not permit P1 because all reads
+are served out of the per-statement reads snapshots established at the beginning
+of each statement. However, P3 and P4 are both allowed because subsequent
+statements can use different read snapshots, and so different statements in the
+same transaction can see different data. A5 is also permitted, because read
+locks are not acquired during reads, allowing both forms of skew to occur.
+
+| | P0
Dirty Write | P1
Dirty Read | P2
Fuzzy Read | P3
Phantom | P4C
Cursor Lost Update | P4
Lost Update | A5A
Read Skew | A5B
Write Skew |
+| ------------ | ----------------- | ---------------- | ---------------- | ------------- | ------------------------- | ---------------------- | ---------------- | ----------------- |
+| Berenson RC | Not Possible | Not Possible | Possible | Possible | Possible | Possible | Possible | Possible |
+| CRDB RC | Not Possible | Not Possible | Possible* | Possible* | Possible* | Possible* | Possible* | Possible |
+
+While permitted by this formalization, our transaction model only allows some
+forms of P2, P3, P4, P4C, and A5A. Read anomalies are not permitted within a
+statement because each statement operates using a consistent snapshot.
+Similarly. lost updates between the reads and writes in a statement are not
+permitted because a first writer wins conflict resolution policy is applied to
+such conflicts. However, all of these anomalies are permitted across statements.
+
+Adya defines Read Committed (PL-2) as proscribing G0 (Write Cycles), G1a
+(Aborted Reads), G1b (Intermediate Reads), and G1c (Circular Information Flow)
+and permitting G-single (Single Anti-dependency Cycles), G2-item (Item
+Anti-dependency Cycles), and G2 (Anti-dependency Cycles). Our transaction model
+does not permit G0 because transactions are not permitted to overwrite
+uncommitted writes from other transactions. Our transaction model also does not
+permit G1a, G1b, or G1c because all reads are served out of the per-statement
+reads snapshots established at the beginning of each statement. These snapshots
+contain writes from committed transactions, and only the final version of those
+writes on any given row. However, G-single, G2-item, and G2 are all permitted
+because our model allows a Read Committed transaction's read timestamp(s) to
+skew from its commit timestamp ("Write Skew Tolerance").
+
+| | G0
Write Cycles | G1a
Aborted Reads | G1b
Intermediate Reads | G1c
Circular Information Flow | G-single
Single Anti-dependency Cycles | G2-item
Item Anti-dependency Cycles | G2
Anti-dependency Cycles |
+| --------- | ------------------ | ---------------- | ---------------- | ------------- | ------------- | ---------- | ----------- |
+| Adya PL-2 | Not Possible | Not Possible | Not Possible | Not Possible | Possible | Possible | Possible |
+| CRDB RC | Not Possible | Not Possible | Not Possible | Not Possible | Possible* | Possible | Possible |
+
+As with the previous analysis, our transaction model is stronger than required
+by Adya because it does not permit G-single within a single statement.
+
+Finally, ANSI SQL defines Read Committed as proscribing Dirty Read (P1) and
+permitting Non-repeatable Read (P2) and Phantom Read (P3). As demonstrated
+above, out transaction model does not permit P1 but does permit P2 and P3.
+Therefore, it is _equivalent to_ ANSI SQL's definition of Read Committed.
+
+| | P1
Dirty Read | P2
Non-repeatable Read | P3
Phantom Read |
+| ------- | ---------------- | ------------------------- | ------------------ |
+| ANSI RC | Not Possible | Possible | Possible |
+| CRDB RC | Not Possible | Possible | Possible |
+
+# Appendix: Postgres-Compatible Intra-Mutation Consistency
+
+This RFC proposed a "Per-Statement Snapshot Isolation" transaction model for
+Read Committed. An alternative approach considered was a "Postgres-Compatible
+Intra-Mutation Consistency" transaction model.
+
+## Comparison
+
+The difference between these two models is how they handle write-write conflicts
+during mutation statements.
+
+The "Per-Statement Snapshot Isolation" model presented here retries individual
+statements on write-write conflicts, ensuring that within a single statement, no
+lost updates are permitted. This stronger model avoids certain anomalies that
+could allow a mutation statement to perceive non-atomic commits of other
+transactions. In exchange, this stronger model is subject to internal
+per-statement retries.
+
+The "Postgres-Compatible Intra-Mutation Consistency" model breaks mutation
+statements into a search phase, a locking phase, and a predicate re-evaluation
+phase. This decomposition avoids any per-statement retries. In exchange, it can
+permit intra-statement lost updates and other anomalous behavior when
+write-write conflicts are experienced.
+
+## Write-Write Conflict Handling
+
+Per-statement read snapshots are one major difference between a Read Committed
+implementation and a Serializable (or hypothetical Snapshot) implementation. The
+other major difference is in the handling of write-write conflicts. Where a
+Serializable transaction would throw a serialization error on a write-write
+conflict, Read Committed transactions wait for the conflict to resolve (e.g. the
+conflicting transaction to commit or abort and release locks) and then continue
+running.
+
+To understand this, we first decompose the definition of a write-write conflict
+as follows:
+
+**Write-Write Locking Conflict**: any case where a transaction attempts to lock
+or write to a key that is locked with an exclusive lock by a different
+transaction, where intents are considered to be a combination of an exclusive
+lock and a provisional value.
+
+**Write-Write Version Conflict**: any case where a transaction attempts to lock
+or write to a key that has a _committed version_ with an MVCC timestamp greater
+than the locking/writing transaction's current read snapshot.
+
+We define _write-write version conflict_ in terms of a transaction's "current
+read snapshot" (i.e. `txn.ReadTimestamp`) to afford flexibility in the
+definition to transactions that change their read snapshot across their
+execution. For example, Read Committed transactions advance their read snapshot
+on each statement boundary, so a committed version that would cause a
+write-write version conflict for one statement may not cause a write-write
+version conflict for a later statement in the same transaction.
+
+Read Committed transactions handle _write-write locking conflicts_ identically
+to Serializable transactions. The prospective locker
+[waits](#blocking-write-write-conflicts) for the existing exclusive lock to be
+released before acquiring it. In cases where multiple transactions wait for
+exclusive access to the same key, they form an orderly queue through the
+`lockWaitQueue` mechanism.
+
+Once a Read Committed transaction has navigated any potential _write-write
+locking conflict_, it may experience a _write-write version conflict_. In such
+cases, locking read operations (e.g. `Get(key, Exclusive)`) return the latest
+committed version of the key, regardless of the reader's read snapshot. Writing
+operations (e.g. `Put(key, value)`) place an intent on the key with a version
+timestamp above the latest committed version.
+
+In both cases, the operations advance the locker/writer's `WriteTimestamp` above
+the committed version's timestamp. Recall that while a transaction is running,
+the `WriteTimestamp` serves as its provisional commit timestamp, forming a lower
+bound on the MVCC timestamp that the transaction can commit at.
+
+#### Impact of Write-Write Conflict Handling on KV API
+
+This necessitates a change in the KV API's handling of write-write version
+conflicts between transaction isolation levels. This difference is summarized
+below:
+
+| Operation type | SI Read Version | SI WW Version Conflict | RC Read Version | RC WW Version Conflict |
+| ---------------- | ----------------- | -------------------------- | ----------------- | -------------------------- |
+| Non-locking Read | [txn.ReadTimestamp](https://github.com/cockroachdb/cockroach/blob/8e24570fa366ed038c6ae65f50db5d8e22826db0/pkg/storage/pebble_mvcc_scanner.go#L791) | N/A | txn.ReadTimestamp | N/A |
+| Locking Read | [txn.ReadTimestamp](https://github.com/cockroachdb/cockroach/blob/8e24570fa366ed038c6ae65f50db5d8e22826db0/pkg/storage/pebble_mvcc_scanner.go#L791) | [WriteTooOldError](https://github.com/cockroachdb/cockroach/blob/8e24570fa366ed038c6ae65f50db5d8e22826db0/pkg/storage/pebble_mvcc_scanner.go#L832) | latest version | txn.WriteTimestamp.Forward |
+| Write-Only | N/A | [txn.WriteTimestamp.Forward](https://github.com/cockroachdb/cockroach/blob/8e24570fa366ed038c6ae65f50db5d8e22826db0/pkg/kv/kvserver/replica_evaluate.go#L371), then [WriteTooOldError](https://github.com/cockroachdb/cockroach/blob/8e24570fa366ed038c6ae65f50db5d8e22826db0/pkg/kv/kvclient/kvcoord/txn_interceptor_span_refresher.go#L259) | N/A | txn.WriteTimestamp.Forward |
+| Read-Write | [txn.ReadTimestamp](https://github.com/cockroachdb/cockroach/blob/8e24570fa366ed038c6ae65f50db5d8e22826db0/pkg/storage/mvcc.go#L2050) | [WriteTooOldError](https://github.com/cockroachdb/cockroach/blob/8e24570fa366ed038c6ae65f50db5d8e22826db0/pkg/kv/kvserver/replica_evaluate.go#L365) | latest version | txn.WriteTimestamp.Forward |
+
+Users of the KV API must be aware of this change in write-write version conflict
+handling or they will risk lost updates. Specifically, a non-locking read of a
+key followed by a write to that key in the same transaction could ignore a
+committed version between the transaction's read and write timestamp.
+
+Users of the KV API must also be aware that this change in write-write version
+conflict handling can expose locking read and read-write operations to an
+inconsistent snapshot of the system.
+
+We will later see how [SQL handles these
+situations](#mutations-update-delete-etc), exploiting the weakened consistency
+guarantees to avoid retries while still providing reasonably sound semantics.
+However, in general, we recommend that most users of the raw KV API continue to
+use Serializable transactions.
+
+## Post-Locking Predicate Re-evaluation
+
+A "Postgres-Compatible Intra-Mutation Consistency" transaction model requires
+additional SQL planning and execution logic beyond the "Per-Statement Snapshot
+Isolation" model in order to handle the predicate re-evaluation step. The
+following proposal was considered as an approach to closely (but not exactly)
+match the postgres predicate re-evaluation behavior. The complexity necessary to
+fully implement this behavior was a factor in the decision to instead implement
+the "Per-Statement Snapshot Isolation" model.
+
+The proposed design hoped to achieve a few desirable properties:
+1. There should not be significant overhead for the happy path (no conflicts).
+2. CRDB should not exhibit anomalies that are prevented in Postgres.
+3. The solution should be general, and shouldn’t require special execution-time
+ handling for individual operators (e.g. the Postgres
+ [EvalPlanQual](https://github.com/postgres/postgres/blob/0bc726d95a305ca923b1f284159f40f0d5cf5725/src/backend/executor/README#L350-L351)
+ step for scans).
+
+### Motivation
+
+Under the "Postgres-Compatible Intra-Mutation Consistency" model, predicate
+re-evaluation is necessary becuase a row may be updated between when it is read
+and filtered, and when it is locked. Note that this is not a concern when all
+predicates can be inlined into a locking scan, because locking and filtering
+happen at the same time in this case. Re-evaluation becomes necessary when the
+query scans, filters, and then locks rows in separate steps.
+
+### Requirements
+
+Postgres places the following restrictions on the syntax of a query that
+performs locking:
+
+1. Locking is [not permitted](https://www.postgresql.org/docs/current/sql-select.html) for
+ tables within GROUP BY, HAVING, WINDOW, DISTINCT, UNION, INTERSECT, or EXCEPT
+ clauses.
+2. In addition, postgres [doesn't allow](https://github.com/postgres/postgres/blob/eae0e20deffb0a73f7cb0e94746f94a1347e71b1/src/backend/optimizer/plan/initsplan.c#L1415-L1442)
+ locking to include tables from the null-extended side of an outer join.
+
+Note that this applies both to explicit locking specified using `SELECT FOR
+UPDATE` syntax, as well as the implicit locking added for mutations (applied
+only to the target table). Because of these restrictions, only mutations, WHERE
+clauses, and joins have to be considered for a re-evaluation approach.
+
+### The Approach
+
+Under the re-evaluation approach, a query with locking follows these steps:
+1. Execute the main query, as normal.
+2. Lock the qualifying rows of any locked tables.
+3. Retrieve the most up-to-date values for the locked rows.
+4. Logically re-evaluate the query for the rows that changed.
+
+The following sub-sections will specify how these steps might be implemented.
+
+Note that the examples shown will have some details omitted for clarity. Some of
+the SQL syntax is simplified for the same reason.
+
+#### Lock Operator
+
+Given a query that scans and then filters rows, locking can be handled by adding
+a lookup join onto the primary key of each locking table, which locks each row
+and returns its most recent version. This takes advantage of the aforementioned
+KV api changes. Re-evaluation is then performed using these updated values.
+
+Consider the following example query:
+```
+SELECT * FROM customers c
+WHERE NOT EXISTS (SELECT * FROM orders o WHERE o.id = c.order_id)
+FOR UPDATE OF c;
+```
+The rewrite to use a locking lookup join would look approximately like this:
+```
+SELECT locked.* FROM
+(
+ SELECT * FROM customers c
+ WHERE NOT EXISTS (SELECT * FROM orders o WHERE o.id = c.order_id)
+) INNER LOOKUP JOIN customers locked ON c.id = locked.id
+FOR UPDATE OF locked;
+```
+The remainder of the query (including predicate re-evaluation) would proceed
+with the updated values returned by the lookup join. This covers steps (1) and
+(2) of the approach.
+
+For convenience, assume that the locking lookup join also projects a boolean
+column that indicates whether a given row was updated.
+
+#### WHERE Clause
+
+WHERE clause handling is straightforward; given the updated and locked rows
+returned by the locking lookup join, we re-evaluate the WHERE clause predicates
+using the updated values.
+
+Consider again the example query:
+```
+SELECT * FROM customers c
+WHERE NOT EXISTS (SELECT * FROM orders o WHERE c.id = o.cust_id)
+FOR UPDATE OF c;
+```
+The predicate from the WHERE clause would be duplicated, and made to reference
+the updated values returned by the locking lookup join from the previous step:
+```
+SELECT locked.* FROM (...) locked
+WHERE NOT EXISTS (SELECT * FROM orders o WHERE o.id = locked.order_id)
+```
+This would remove any new values of `order_id` that don't pass the predicate.
+
+#### Joins
+
+A locking query can perform INNER or LEFT joins between the locked table and
+arbitrary subqueries. Note that the FROM and USING clauses of UPDATE and DELETE
+statements perform implicit INNER joins. For now, ignore the possibility of
+locking multiple tables.
+
+The difficulty is that the non-locking subqueries can be arbitrarily complex,
+and their results can depend on rows from the locking relation for correlated
+subqueries. Since the locking relation's rows can observe concurrent updates
+during re-evaluation, the result of a correlated subquery can change during
+re-evaluation. It may be possible to simplify and decorrelate specific examples,
+but (at least currently) we cannot guarantee this in the general case.
+
+Postgres actually re-executes the correlated subqueries with scan nodes rigged
+to return the same (single) row that contributed to the original output row (see
+[here](https://github.com/postgres/postgres/blob/eae0e20deffb0a73f7cb0e94746f94a1347e71b1/src/backend/executor/execScan.c#L27-L38)).
+This differs subtly from fully re-executing the correlated subqueries with
+updated values. The behavior does not fit well with CRDB’s vectorized and
+distributed execution model, and an implementation would likely impose a
+significant maintenance cost. The proposal departs from Postgres here by fully
+re-executing correlated subqueries during re-evaluation. Note that non-locked
+tables would still read at the old timestamp, and therefore would not observe
+concurrent updates.
+
+This is implemented by duplicating the subquery, and joining it back with the
+locked query. Take the following example:
+```
+SELECT * FROM customers c
+INNER JOIN orders o
+ON c.id = o.cust_id
+FOR UPDATE OF c;
+```
+The duplicated join would look like this:
+```
+SELECT * FROM (...) locked INNER JOIN (SELECT * FROM orders) o ON locked.id = o.cust_id;
+```
+
+##### Left Joins
+
+Handling a LEFT join in the original query only requires changing the duplicated
+join to a LEFT join as well. For the example query:
+```
+SELECT * FROM customers c
+LEFT JOIN orders o
+ON c.id = o.cust_id
+FOR UPDATE OF c;
+```
+The resulting plan would look like this:
+```
+SELECT * FROM (...) locked LEFT JOIN (SELECT * FROM orders) o ON locked.id = o.cust_id;
+```
+
+##### Multiple Locked Relations
+
+It is possible to lock multiple tables in a `SELECT FOR UPDATE` query. If this
+is the case, the locked relations will be connected by INNER joins, since it is
+not possible to lock rows in the right input of a LEFT join. This is implemented
+by duplicating the INNER join conditions into a combined WHERE clause that
+references the updated values. As always, we can skip evaluating the filters for
+non-updated rows using a CASE statement. This WHERE clause will then ensure that
+the updated join row passes all the join conditions.
+
+Example:
+```
+SELECT * FROM customers c
+INNER JOIN orders o
+ON c.id = o.cust_id
+INNER JOIN items i
+ON i.id = o.item_id
+FOR UPDATE;
+```
+Note that the `FOR UPDATE` clause here applies to all tables. For this query,
+re-evaluation for the join conditions would look like this:
+```
+SELECT * FROM (...) locked
+WHERE c_id = cust_id AND i_id = item_id;
+```
+
+Note that this implementation will reproduce Postgres’ behavior where an updated
+row will only match with rows that matched before and after the update, only for
+the case when multiple relations are locked and joined together. For non-locking
+sub-selects/relations, the behavior will still be total re-evaluation instead.
+
+##### Non-Unique Join Keys
+
+So far, this discussion has ignored the case when a join is performed on a
+non-unique key. In this case, the re-evaluation algorithm will duplicate
+results, since there may be more than one match for a given locked row during
+re-evaluation. This problem can be resolved using a DISTINCT operator,
+de-duplicating on a key for each locked table.
+
+Example:
+```
+CREATE TABLE xy (x INT PRIMARY KEY, y INT);
+CREATE TABLE ab (a INT PRIMARY KEY, b INT);
+
+SELECT * FROM xy INNER JOIN ab ON y = b FOR UPDATE OF xy;
+```
+In the above example, the DISTINCT would de-duplicate on column `x`, since it
+forms a key for the locked relation `xy`:
+```
+SELECT DISTINCT ON (x) * FROM (...) locked;
+```
+
+#### Mutations
+
+Mutations add implicit locking on updated rows in the target table. This is
+logically equivalent to first running a `SELECT FOR UPDATE` in the transaction
+before executing the mutation. Therefore, handling mutations can be reduced to
+handling the implicit `SELECT FOR UPDATE` that performs the locking.
+
+#### Optimization
+
+It is desirable to avoid re-evaluation for rows that weren't updated. For the
+filters used in the re-evaluation step, this is simple: wrap each filter in a
+CASE statement that checks whether the row was updated. Example:
+```
+CASE WHEN updated THEN ELSE True END;
+```
+CASE statements guarantee that a branch is not evaluated unless it is taken, so
+this method avoids overhead for filter re-evaluation in the happy case. This
+optimization applies to the filters for both WHERE clauses and join ON
+conditions.
+
+The other case that must be optimized is re-evaluation of correlated subqueries.
+CASE statements cannot be used here, because it is possible for the subquery to
+return multiple rows. However, strict UDFs have the required behavior - a
+set-returning strict UDF short-circuits and returns no rows when given a NULL
+argument.
+
+We can build a UDF that re-evaluates the correlated subquery, and join its
+result back to the main query. The UDF parameters will correspond to
+outer-column references, and the SQL body to the subquery. We can skip
+re-evaluation by adding an extra parameter that is non-NULL when the locked row
+changed, and NULL when it did not change. Example:
+```
+CREATE FUNCTION re_eval_subquery(val1, val2, ..., null_if_not_updated) RETURNS RECORD AS $$
+
+$$ STRICT LANGUAGE SQL;
+```
+Note that while the example shows a `CREATE` statement, only the UDF's execution
+machinery would be built here; it would not have a descriptor or become visible
+to users.
+
+A few corrections are necessary to make the UDF solution work:
+1. The old and new values for the subquery must be combined. This can be done
+ via another CASE statement.
+2. The join must preserve locked (left) rows where the UDF returned no rows,
+ since this is the case when there were no updates. This is handled by always
+ using a LEFT join. If the original join was an INNER join, we add an extra
+ filter to remove rows that were updated and did not pass the join filters.
+
+Example for correlated subquery re-evaluation:
+```
+SELECT * FROM (...) locked
+LEFT JOIN LATERAL (SELECT * FROM re_eval_subquery(locked.val, null_if_not_updated)) sub
+ON locked.id = sub.cust_id
+[WHERE (NOT updated) OR passed_join_condition]; -- Only for INNER joins.
+```
+
+It's unclear how to avoid the overhead of de-duplication for non-unique joins.
+
+#### Summary
+
+Ignoring optimizations, the proposed implementation of the "Postgres-Compatible
+Intra-Mutation Consistency" model follows these steps for a query with locking:
+1. Execute the original query, taking care to keep the primary key of each
+ target table.
+2. Perform a lookup join into the primary index for each target table, locking
+ each row and returning the most recent values.
+3. For each non-locking subquery joined to the target table(s), duplicate the
+ subquery, and join it back to the main query.
+4. Duplicate the query's WHERE clause, remapping it to refer to updated values,
+ and add it to the main query.
+5. Duplicate each join condition that joins target tables to one another, and
+ add it to the main query's WHERE clause.
+6. De-duplicate the result on the key of each target table.
+
+### Limitations
+
+* Complexity - many different steps are required in the algorithm in order to
+ handle the various edge cases around handling joins. The model is also more
+ complex to reason about and test than "Per-Statement Snapshot Isolation".
+* Overhead - in the fast path, the initial scan/filtering must preserve primary
+ keys for the locking tables in order to perform the locking lookup join. This
+ will add some execution-time overhead and possibly restrict the possible query
+ plans. In addition, the slow path is handled via correlated subqueries, which
+ can have very poor performance. This may not be unique to the predicate
+ re-evaluation design, however.
+* Compatibility - Postgres and CRDB handle row identifiers differently - in PG,
+ deleting and inserting a row with the same key and a different is not the same
+ as updating it. In CRDB, the two scenarios are indistinguishable. This means
+ it would be possible for re-evaluation to read a newly inserted row in CRDB
+ when it wouldn't be possible in PG. In addition, PG would "follow" an update
+ that changes the row's key during re-evaluation, while CRDB would not. In
+ addition, the proposal does not exactly replicate Postgres behavior for
+ correlated joins with non-locking subqueries.
diff --git a/src/current/files/cockroach/docs/tech-notes/admission_control.md b/src/current/files/cockroach/docs/tech-notes/admission_control.md
new file mode 100644
index 00000000000..9692bd94a60
--- /dev/null
+++ b/src/current/files/cockroach/docs/tech-notes/admission_control.md
@@ -0,0 +1,343 @@
+# Admission Control
+
+Author: Sumeer Bhola
+
+
+## Goals
+
+Admission control for a system decides when work submitted to that
+system begins executing, and is useful when there is some resource
+(e.g. CPU) that is saturated. The high-level goals of admission
+control in CockroachDB are to control resource overload such that it
+(a) does not degrade throughput or cause node failures, (b) achieve
+differentiation between work with different levels of importance
+submitted by users, and (c) allow for load-balancing among data
+replicas (when possible).
+
+For CockroachDB Serverless, where the shared cluster only runs the KV
+layer and below, admission control also encompasses achieving fairness
+across tenants (fairness is defined as equally allocating resources,
+e.g. CPU, across tenants that are competing for resources).
+Multi-tenant isolation for the shared cluster is included in the scope
+of admission control since many of the queuing and re-ordering
+mechanisms for work prioritization overlap with those for inter-tenant
+isolation.
+
+Even though we do not discuss per-tenant SQL nodes in the remainder of
+this document, the overload control mechanisms discussed below could
+potentially be applied in that context, though the code for it is
+incomplete. The scope of admission control excludes per-tenant cost
+controls, since per-tenant cost is not a system resource.
+
+The current implementation focuses on node-level admission control,
+for CPU and storage IO (specifically writes) as the bottleneck
+resources. The focus on node-level admission control is based on the
+observation that large scale systems may be provisioned adequately at
+the aggregate level, but since CockroachDB has stateful nodes,
+individual node hotspots can develop that can last for some time
+(until rebalancing). Such hotspots should not cause failures or
+degrade service for important work (or unfairly for tenants that are
+not responsible for the hotspot).
+
+Specifically, for CPU the goal is to shift queueing from inside the
+goroutine scheduler, where there is no differentiation, into various
+admission queues, where we can differentiate. This must be done while
+allowing the system to have high peak CPU utilization. For storage IO,
+the goal is to prevent the log-structured merge (LSM) tree based
+storage layer (Pebble) from getting into a situation with high read
+amplification due to many files/sub-levels in level 0, which slows
+down reads. This needs to be done while maintaining the ability to
+absorb bursts of writes. Both CPU overload and high read amplification
+in the LSM are areas where we have seen problems in real CockroachDB
+clusters.
+
+The notable omission here is memory as a bottleneck resource for
+admission control. The difficulty is that memory is non-preemptible
+(mostly, ignoring disk spilling), and is used in various layers of the
+system, and so slowing down certain activities (like KV processing)
+may make things worse by causing the SQL layer to hold onto memory it
+has already allocated. We want forward progress so that memory can be
+released, and we do not know what should make progress to release the
+most memory. We also currently do not have predictions for how much
+memory a SQL query will consume, so we cannot make reasonable
+reservations to make up for the non-preemptibility.
+
+## High-level Approach
+
+
+### Ordering Tuple
+
+Admission queues use the tuple **(tenant, priority, transaction start
+time)**, to order items that are waiting to be admitted. There is
+coarse-grained fair sharing across tenants (for the multi-tenant
+shared cluster). Priority is used within a tenant, and allows for
+starvation, in that if higher priority work is always consuming all
+resources, the lower priority work would wait forever. The transaction
+start time is used within a priority, and gives preference to earlier
+transactions. We currently do not have a way for end-users to assign
+priority to their SQL transactions. We also currently do not support
+multi-tenancy in dedicated clusters, even though many such customers
+would like to distinguish between different internal tenants. Both
+these limitations could be addressed by adding the requisite
+plumbing/integration code.
+
+
+### Possible solution for CPU Resource with scheduler change
+
+Let us consider the case of CPU as a bottleneck resource. If we had
+the ability to change the goroutine scheduler we could associate the
+above ordering tuple with each goroutine and could allocate the CPU
+(P) slots to the runnable goroutine that should be next according to
+that tuple. Such a scheme does not need to make any guesses about
+whether some admitted work is currently doing useful work or blocked
+on IO, since there is visibility into that state inside the
+scheduler. And if we are concerned about starting too much work, and
+not finishing already started work, one could add a **started**
+boolean as the first element of the tuple and first give preference to
+goroutines that had already started doing some work (there is a draft
+Cockroach Labs [internal
+doc](https://docs.google.com/document/d/18S4uE8O1nRxULhSg9Z1Zt4jUPBiLJgMh7X1I6shsbug/edit#heading=h.ssc9exx0epqo)
+that provides more details). However, we currently do not have the
+ability to make such scheduler changes. So we resort to more indirect
+control as outlined in the next section.
+
+### Admission for CPU Resource
+
+#### Kinds of Work and Queues
+
+There are various kinds of work that consume CPU, and we add admission
+interception points in various places where we expect the
+post-admission CPU consumption to be significant and (somewhat)
+bounded. Note that there is still extreme heterogeneity in work size,
+that we are not aware of at admission time, and we do not know the
+CPU/IO ratio for a work unit. Specifically, the interception points
+are:
+
+- **KV**: KV work admission, specifically the
+ `roachpb.InternalServer.Batch` API implemented by
+ [`Node`](https://github.com/cockroachdb/cockroach/blob/d10b3a5badf25c9e19ca84037f2426b03196b2ac/pkg/server/node.go#L938). This
+ includes work submitted by SQL, and internal operations like
+ garbage collection and node heartbeats.
+
+- **SQL-KV**: Admission of SQL processing for a response provided by
+ KV. For example, consider a distributed SQL scan of a large table
+ that is being executed at N nodes, where the SQL layer is issuing
+ local requests to the KV layer. The response from KV is subject to
+ admission control at each node, before processing by the SQL layer.
+
+- **SQL-SQL**: Distributed SQL runs as a two-level tree where the
+ lower level sends back responses to the root for further
+ processing. Admission control applies to the response processing
+ at the root.
+
+Each of these kinds of work has its own admission queue. Work is
+queued until admitted or the work deadline is exceeded. Currently
+there is no early rejection when encountering long queues. Under
+aggressive user-specified deadlines throughput may collapse because
+everything exceeds the deadline after doing part of the work. This is
+no different than what will likely happen without admission control,
+due to undifferentiated queueing inside the goroutine scheduler, but
+we note that this is a behavior that is not currently improved by
+admission control.
+
+The above list of kinds are ordered from lower level to higher level,
+and also serves as a hard-wired ordering from most important to least
+important. The high importance of KV work reduces the likelihood that
+non-SQL KV work will be starved. SQL-KV (response) work is prioritized
+over SQL-SQL since the former includes leaf DistSQL processing and we
+would like to release memory used up in RPC responses at lower layers
+of RPC tree. We expect that if SQL-SQL (response) work is delayed, it
+will eventually reduce new work being issued, which is a desirable
+form of natural backpressure. Note that this hard prioritization
+across kinds of work is orthogonal to the priority specified in the
+ordering tuple, and would ideally not be needed (one reason for
+introducing it is due to our inability to change the goroutine
+scheduler).
+
+Consider the example of a lower priority long-running OLAP query
+competing with higher priority small OLTP queries in a single node
+setting. Say the OLAP query starts first and uses up all the CPU
+resource such that the OLTP queries queue up in the KV work
+queue. When the OLAP query's KV work completes, it will queue up for
+SQL-KV work, which will not start because the OLTP queries are now
+using up all available CPU for KV work. When this OLTP KV work
+completes, their SQL-KV work will queue up. The queue for SQL-KV will
+first admit those for the higher priority OLTP queries. This will
+prevent or slow down admission of further work by the OLAP query.
+
+One weakness of this prioritization across kinds of work is that it
+can result in priority inversion: lower importance KV work, not
+derived from SQL, like GC of MVCC versions, will happen before
+user-facing SQL-KV work. This is because the backpressure via SQL-SQL,
+mentioned earlier, does not apply to work generated from within the KV
+layer. This could be addressed by introducing a **KV-background** work
+kind and placing it last in the above ordering.
+
+#### Slots and Tokens
+
+The above kinds of work behave differently in whether we know a work
+unit is completed or not. For KV work we know when the admitted work
+completes, but this is not possible to know for SQL-KV and SQL-SQL
+work because of the way the execution code is structured. Knowing
+about completion is advantageous since it allows for better control
+over resource consumption, sine we do not know how big each work unit
+actually is.
+
+We model these two different situations with different ways of
+granting admission: for KV work we grant a slot that is occupied while
+the work executes and becomes available when the work completes, and
+for SQL-KV and SQL-SQL we grant a token that is consumed. The slot
+terminology is akin to a scheduler, where a scheduling slot must be
+free for a thread to run. But unlike a scheduler, we do not have
+visibility into the fact that work execution may be blocked on IO. So
+a slot can also be viewed as a limit on concurrency of ongoing
+work. The token terminology is inspired by token buckets. Unlike a
+token bucket, which shapes the rate, the current implementation limits
+burstiness and does not do rate shaping -- this is because it is hard
+to predict what rate is appropriate given the difference in sizes of
+the work.
+
+#### Slot Adjustment for KV
+
+The current implementation makes no dynamic adjustments to token burst
+sizes since the lack of a completion indicator and heterogeneity in
+size makes it hard to figure out how to adjust these tokens. In
+contrast, the slots that limit KV work concurrency are adjusted. And
+because KV work must be admitted (and have no waiting requests) before
+admission of SQL-KV and SQL-SQL work, the slot adjustment also
+throttles the latter kinds of work.
+
+We monitor the following state of the goroutine scheduler: the number
+of processors, and the number of goroutines that are runnable (i.e.,
+they are ready to run, but not running). The latter represents
+queueing inside the goroutine scheduler, since these goroutines are
+waiting to be scheduled. KV work concurrency slots are adjusted by
+gradually decreasing or increasing the total slots (additive
+decrements or increments), when the runnable count, normalized by the
+number of processors, is too high or too low. The adjustment also
+takes into account current usage of slots. The exact heuristic can be
+found in `admission.kvSlotAdjuster`. It monitors the runnable count at
+1ms intervals. The motivation for this high frequency is that sudden
+shifts in CPU/IO ratio or lock contention can cause the current slot
+count to be inadequate, while leaving the CPU underutilized, which is
+undesirable.
+
+
+#### Instantaneous CPU feedback and limiting bursts
+
+Tokens are granted (up to the burst size) for SQL-KV when the KV work
+queue is empty and the CPU is not overloaded. For SQL-SQL the
+additional requirement is that the SQL-KV work queue must also be
+empty. It turns out that using the runnable goroutine count at 1ms
+intervals, as a signal for CPU load, is insufficient time granularity
+to properly control token grants. So we use two instantaneous
+indicators:
+
+- CPU is considered overloaded if all the KV slots are utilized.
+
+- Tokens are not directly granted to waiting requests up to the burst
+ size. Instead we setup a "grant chaining" system where the goroutine
+ that is granted a token has to run and grant the next token. This
+ gives instantaneous feedback into the overload state. In an
+ experiment, using such grant chains reduced burstiness of grants by
+ 5x and shifted ~2s of latency (at p99) from the goroutine scheduler
+ into admission control (which is desirable since the latter is where
+ we can differentiate between work).
+
+### Admission for IO Resource
+
+KV work that involves writing to a store is additionally subject to a
+per-store admission queue. Admission in this queue uses tokens
+(discussed below), and happens before the KV work is subject to the KV
+CPU admission queue (which uses slots), so that we do not have a
+situation where a KV slot is taken up by work that is now waiting for
+IO admission.
+
+KV work completion is not a good indicator of write work being
+complete in the LSM tree. This is because flushes of memtables, and
+compactions of sstables, which are the costly side-effect of writes,
+happen later. We use "IO tokens" to constrain how many KV work items
+are admitted. There is no limit on tokens when the LSM is healthy. Bad
+health is indicated using thresholds on two level 0 metrics: the
+sub-level count and file count. We do not consider other metrics for
+health (compaction backlog, high compaction scores on other levels
+etc.) since we are primarily concerned with increasing
+read-amplification, which is what will impact user-facing traffic. It
+is acceptable for the LSM to deteriorate in terms of compaction scores
+etc. as long as the read-amplification does not explode, because
+absorbing write bursts is important, and write bursts are often
+followed by long enough intervals of low activity that restore the LSM
+compaction scores to good health.
+
+When the LSM is considered overloaded, tokens are calculated by
+estimating the average bytes added per KV work, and using the outgoing
+compaction bytes to estimate how many KV work items it is acceptable
+to admit. This detailed logic can be found in
+`admission.ioLoadListener` which generates a new token estimate every
+15s. The 15s duration is based on experimental observations of
+compaction durations in level 0 when the number of sub-levels
+increases beyond the overload threshold. We want a duration that is
+larger than most compactions, but not too large (for
+responsiveness). These tokens are given out in 1s intervals (for
+smoothing). The code has comments with experimental details that
+guided some of the settings.
+
+### Priority adjustments and Bypassing admission control
+
+KV work that is not directly issued by SQL is never queued, though it
+does consume a slot, which means the available slots can become
+negative. This is a simple hack to prevent distributed deadlock that
+can happen if we queue KV work issued due to other KV work. This will
+need to be changed in the future to also queue low priority KV
+operations (e.g. GC can be considered lower priority than user facing
+work).
+
+Transactions that are holding locks, or have ongoing requests to
+acquire locks, have their subsequent work requests bumped to higher
+priority. This is a crude way to limit priority inversion where a
+transaction holding locks could be waiting in an admission queue while
+admitted requests are waiting in the lock table queues for this
+transaction to make progress and release locks. Such prioritization
+can also fare better than a system with no admission control, since
+work from transactions holding locks will get prioritized, versus no
+prioritization in the goroutine scheduler. A TPCC run with 3000
+warehouses showed 2x reduction in lock waiters and 10+% improvement in
+transaction throughput with this priority adjustment compared to no
+priority adjustment. See
+https://github.com/cockroachdb/cockroach/pull/69337#issue-978534871
+for comparison graphs.
+
+### Tuning Knobs
+
+Enabling admission control is done via cluster settings. It is
+currently disabled by default. There are three boolean settings,
+`admission.kv.enabled`, `admission.sql_kv_response.enabled`,
+`admission.sql_sql_response.enabled` and we have only experimented
+with all of them turned on.
+
+There are also some advanced tuning cluster settings, that adjust the
+CPU overload threshold and the level 0 store overload thresholds.
+
+### Results
+
+There are certain TPCC-bench and KV roachtests running regularly with
+admission control enabled (they have "admission" in the test name
+suffix). The TPCC performance is roughly equivalent to admission
+control disabled. Certain KV roachtests that used to overload IO, but
+were not running long enough to show the bad effects, are worse with
+admission control enabled when comparing the mean throughput. However
+the runs with admission control enabled are arguably better since they
+do not have a steadily worsening throughput over the course of the
+experimental run.
+
+For some graphs showing before and after effects of enabling admission control see:
+
+- CPU overload:
+ https://github.com/cockroachdb/cockroach/pull/65614#issue-651424608
+ and
+ https://github.com/cockroachdb/cockroach/pull/66891#issue-930351128
+
+- IO overload:
+ https://github.com/cockroachdb/cockroach/pull/65850#issue-656777155
+ and
+ https://github.com/cockroachdb/cockroach/pull/69311#issue-978297918
diff --git a/src/current/files/cockroach/docs/tech-notes/encoding.md b/src/current/files/cockroach/docs/tech-notes/encoding.md
new file mode 100644
index 00000000000..94d55360d25
--- /dev/null
+++ b/src/current/files/cockroach/docs/tech-notes/encoding.md
@@ -0,0 +1,561 @@
+Structured data encoding in CockroachDB SQL
+===========================================
+
+Like many databases, CockroachDB (CRDB) encodes SQL data into key-value
+(KV) pairs. The format evolves over time, with an eye toward backward
+compatibility. This document describes format version 3 in detail except
+for how CRDB encodes primitive values ([pkg/util/encoding/encoding.go]).
+
+The Cockroach Labs blog post [SQL in CockroachDB: Mapping Table Data to
+Key-Value Storage] covers format version 1, which predates column
+families, interleaving, and composite encoding. Format version 2
+introduced column families, covered in [Implementing Column Families in
+CockroachDB]. See also the [column families RFC] and the [interleaving
+RFC].
+
+This document was originally written by David Eisenstat
+<>.
+
+Tables (primary indexes)
+------------------------
+
+SQL tables consist of a rectangular array of data and some metadata. The
+metadata include a unique table ID; a nonempty list of primary key
+columns, each with an ascending/descending designation; and some
+information about each column. Each column has a numeric ID that is
+unique within the table, a SQL type, and a column family ID. A column
+family is a maximal subset of columns with the same column family ID.
+For more details, see [pkg/sql/catalog/descpb/structured.proto].
+
+Each row of a table gives rise to one or more KV pairs, one per column
+family as needed (see subsection NULL below). CRDB stores primary key
+data in KV keys and other data in KV values so that it can use the KV
+layer to prevent duplicate primary keys. For encoding, see
+[pkg/sql/row/writer.go]. For decoding, see
+[pkg/sql/row/fetcher.go].
+
+### Key encoding
+
+KV keys consist of several fields:
+
+1. The table ID
+2. The ID of the primary index (see section Indexes below)
+3. The primary key of the row, one field per primary key column in list
+ order
+4. The column family ID.
+5. When the previous field is nonzero (non-sentinel), its length in
+ bytes.
+
+CRDB encodes these fields individually and concatenates the resulting
+bytes. The decoder can determine the field boundaries because the field
+encoding is [prefix-free].
+
+Encoded fields start with a byte that indicates the type of the field.
+For primary key fields, this type has a one-to-many relationship with
+the SQL datum type. The SQL types `STRING` and `BYTES`, for example,
+share an encoding. The relationship will become many-to-many when CRDB
+introduces a [new `DECIMAL` encoding], since the old decoder will be
+retained for backward compatibility.
+
+The format of the remaining bytes depends on the field type. The details
+(in [pkg/util/encoding/encoding.go]) are irrelevant here except that,
+for primary key fields, these bytes have the following order property.
+Consider a particular primary key column and let enc be the mathematical
+function that maps SQL data in that column to bytes.
+
+- If the column has an ascending designation, then for data *x* and
+ *y*, enc(*x*) ≤ enc(*y*) if and only if *x* ≤ *y*.
+- If the column has a descending designation, then for data *x* and
+ *y*, enc(*x*) ≤ enc(*y*) if and only if *x* ≥ *y*.
+
+In conjunction with prefix freedom, the order property ensures that the
+SQL layer and the KV layer sort primary keys the same way.
+
+For more details on primary key encoding, see `EncodeTableKey`
+([pkg/sql/rowenc/column\_type\_encoding.go]). See also `EncDatum`
+([pkg/sql/rowenc/encoded\_datum.go]).
+
+### Value encoding
+
+KV values consist of
+
+1. A four-byte checksum covering the whole KV pair
+2. A one-byte value type (see the enumeration `ValueType` in
+ [pkg/roachpb/data.proto])
+3. Data from where the row specified in the KV key intersects the
+ specified column family, including composite encodings of primary
+ key columns that are members of the specified column family.
+
+The value type defaults to `TUPLE`, which indicates the following
+encoding. (For other values, see subsection Single-column column
+families below.) For each column in the column family sorted by column
+ID, encode the column ID difference and the datum encoding type
+(unrelated to the value type!) jointly, followed by the datum itself.
+The column ID difference is the column ID minus the previous column ID
+if this column is not the first, else the column ID. The joint encoding
+is commonly one byte, which displays conveniently in hexadecimal as the
+column ID difference followed by the datum encoding type.
+
+The Go function that performs the joint encoding is `encodeValueTag`
+([pkg/util/encoding/encoding.go]), which emits an unsigned integer with
+a variable-length encoding. The low four bits of the integer contain the
+datum encoding type. The rest contain the column ID difference. As an
+alternative for datum encoding types greater than 14, `encodeValueTag`
+sets the low four bits to `SentinelType` (15) and emits the actual datum
+encoding type next.
+
+**Note:** Values for sequences are a special case: the sequence value is
+encoded as if the sequence were a one-row, one-column table, with the
+key structured in the usual way: `/Table////`.
+However, the value is a bare int64; it doesn't use the encoding
+specified here. This is because it is incremented using the KV
+`Increment` operation so that the increment can be done in one
+roundtrip, not a read followed by a write as would be required by a
+normal SQL `UPDATE`.
+
+An alternative design would be to teach the KV Inc operation to
+understand SQL value encoding so that the sequence could be encoded
+consistently with tables, but that would break the KV/SQL abstraction
+barrier.
+
+The code that performs generation of keys and values for primary indexes
+can be found in `prepareInsertOrUpdateBatch`([pkg/sql/row/writer.go]).
+
+### Sentinel KV pairs
+
+The column family with ID 0 is special because it contains the primary
+key columns. The KV pairs arising from this column family are called
+sentinel KV pairs. CRDB emits sentinel KV pairs regardless of whether
+the KV value has other data, to guarantee that primary keys appear in at
+least one KV pair. (Even if there are other column families, their KV
+pairs may be suppressed; see subsection NULL below.)
+
+Note that in system tables that use multiple column families, such as
+system.zones or system.namespace, there may not be any sentinel KV pair at all.
+This is because of the fact that the database writes to these system tables
+using raw KV puts and does not include the logic to write a sentinel KV. KV
+decoding code that needs to understand system tables must be aware of this
+possibility.
+
+### Single-column column families
+
+Before column families (i.e., in format version 1), non-sentinel KV keys
+had a column ID where the column family ID is now. Non-sentinel KV
+values contained exactly one datum, whose encoding was indicated by the
+one-byte value type (see `MarshalColumnValue` in
+[pkg/sql/rowenc/column\_type\_encoding.go]). Unlike the `TUPLE` encoding, this encoding
+did not need to be prefix-free, which was a boon for strings.
+
+On upgrading to format version 2 or higher, CRDB puts each existing
+column in a column family whose ID is the same as the column ID. This
+allows backward-compatible encoding and decoding. The encoder uses the
+old format for single-column column families when the ID of that column
+equals the `DefaultColumnID` of the column family
+([pkg/sql/catalog/descpb/structured.proto]).
+
+### NULL
+
+SQL `NULL` has no explicit encoding in tables (primary indexes).
+Instead, CRDB encodes each row as if the columns where that row is null
+did not exist. If all of the columns in a column family are null, then
+the corresponding KV pair is suppressed. The motivation for this design
+is that adding a column does not require existing data to be re-encoded.
+
+### Example dump
+
+The commands below create a table and insert some data. An annotated KV
+dump follows.
+
+ CREATE TABLE accounts (
+ id INT PRIMARY KEY,
+ owner STRING,
+ balance DECIMAL,
+ FAMILY f0 (id, balance),
+ FAMILY f1 (owner)
+ );
+
+ INSERT INTO accounts VALUES
+ (1, 'Alice', 10000.50),
+ (2, 'Bob', 25000.00),
+ (3, 'Carol', NULL),
+ (4, NULL, 9400.10),
+ (5, NULL, NULL);
+
+Here is the relevant output from
+`cockroach debug rocksdb scan --value_hex`, with annotations.
+
+ /Table/51/1/1/0/1489427290.811792567,0 : 0xB244BD870A3505348D0F4272
+ ^- ^ ^ ^ ^-------^-^^^-----------
+ | | | | | | |||
+ Table ID (accounts) Checksum| |||
+ | | | | |||
+ Index ID Value type (TUPLE)
+ | | |||
+ Primary key (id = 1) Column ID difference
+ | ||
+ Column family ID (f0) Datum encoding type (Decimal)
+ |
+ Datum encoding (10000.50)
+
+ /Table/51/1/1/1/1/1489427290.811792567,0 : 0x30C8FBD403416C696365
+ ^- ^ ^ ^ ^ ^-------^-^---------
+ | | | | | | | |
+ Table ID (accounts) Checksum| |
+ | | | | | |
+ Index ID Value type (BYTES)
+ | | | |
+ Primary key (id = 1) Datum encoding ('Alice')
+ | |
+ Column family ID (f1)
+ |
+ Column family ID encoding length
+
+ /Table/51/1/2/0/1489427290.811792567,0 : 0x2C8E35730A3505348D2625A0
+ ^ ^-----------
+ 2 25000.00
+
+ /Table/51/1/2/1/1/1489427290.811792567,0 : 0xE911770C03426F62
+ ^ ^-----
+ 2 'Bob'
+
+ /Table/51/1/3/0/1489427290.811792567,0 : 0xCF8B38950A
+ ^
+ 3
+
+ /Table/51/1/3/1/1/1489427290.811792567,0 : 0x538EE3D6034361726F6C
+ ^ ^---------
+ 3 'Carol'
+
+ /Table/51/1/4/0/1489427290.811792567,0 : 0x247286F30A3505348C0E57EA
+ ^ ^-----------
+ 4 9400.10
+
+ /Table/51/1/5/0/1489427290.811792567,0 : 0xCB0644270A
+ ^
+ 5
+
+### Composite encoding
+
+There exist decimal numbers and collated strings that are equal but not
+identical, e.g., 1.0 and 1.000. This is problematic because in primary
+keys, 1.0 and 1.000 must have the same encoding, which precludes
+lossless decoding. Worse, the encoding of collated strings in primary
+keys is defined by the [Unicode Collation Algorithm], which may not even
+have [an efficient partial inverse].
+
+When collated strings and [(soon) decimals][new `DECIMAL` encoding]
+appear in primary keys, they have composite encoding. For collated
+strings, this means encoding data as both a key and value, with the
+latter appearing in the sentinel KV value (naturally, since the column
+belongs to the column family with ID 0).
+
+Example schema and data:
+
+ CREATE TABLE owners (
+ owner STRING COLLATE en PRIMARY KEY
+ );
+
+ INSERT INTO owners VALUES
+ ('Bob' COLLATE en),
+ ('Ted' COLLATE en);
+
+Example dump:
+
+ /Table/51/1/"\x16\x05\x17q\x16\x05\x00\x00\x00 \x00 \x00 \x00\x00\b\x02\x02"/0/1489502864.477790157,0 : 0xDC5FDAE10A1603426F62
+ ^--------------------------------------------------------------- ^-------
+ Collation key for 'Bob' 'Bob'
+
+ /Table/51/1/"\x18\x16\x16L\x161\x00\x00\x00 \x00 \x00 \x00\x00\b\x02\x02"/0/1489502864.477790157,0 : 0x8B30B9290A1603546564
+ ^------------------------------------------------------------ ^-------
+ Collation key for 'Ted' 'Ted'
+
+Indexes (secondary indexes)
+---------------------------
+
+To unify the handling of SQL tables and indexes, CRDB stores the
+authoritative table data in what is termed the primary index. SQL
+indexes are secondary indexes. All indexes have an ID that is unique
+within their table.
+
+The user-specified metadata for secondary indexes include a nonempty
+list of indexed columns, each with an ascending/descending designation,
+and a disjoint list of stored columns. The first list determines how the
+index is sorted, and columns from both lists can be read directly from
+the index.
+
+Users also specify whether a secondary index should be unique. Unique
+secondary indexes constrain the table data not to have two rows where,
+for each indexed column, the data therein are non-null and equal.
+
+As of #42073 (after version 19.2), secondary indexes have been extended to
+include support for column families. These families are the same as the ones
+defined upon the table. Families will apply to the stored columns in the index.
+Like in primary indexes, column family 0 on a secondary index will always be
+present for a row so that each row in the index has at least one k/v entry.
+
+### Key encoding
+
+The main encoding function for secondary indexes is
+`EncodeSecondaryIndex` in [pkg/sql/rowenc/index\_encoding.go]. Each row gives
+rise to one KV pair per secondary index, whose KV key has fields
+mirroring the primary index encoding:
+
+1. The table ID
+2. The index ID
+3. Data from where the row intersects the indexed columns
+4. If the index is non-unique or the row has a NULL in an indexed
+ column, data from where the row intersects the non-indexed primary
+ key (implicit) columns
+5. If the index is non-unique or the row has a NULL in an indexed
+ column, and the index uses the old format for stored columns, data
+ from where the row intersects the stored columns
+6. The column family ID.
+7. When the previous field is nonzero (non-sentinel), its length in bytes.
+
+Unique indexes relegate the data in extra columns to KV values so that
+the KV layer detects constraint violations. The special case for an
+indexed NULL arises from the fact that NULL does not equal itself, hence
+rows with an indexed NULL cannot be involved in a violation. They need a
+unique KV key nonetheless, as do rows in non-unique indexes, which is
+achieved by including the non-indexed primary key data. For the sake of
+simplicity, data in stored columns are also included.
+
+### Value encoding
+KV values for secondary indexes are encoded using the following rules:
+
+If the value corresponds to column family 0:
+
+The KV value will have value type bytes, and will consist of
+1. If the index is unique, data from where the row intersects the
+ non-indexed primary key (implicit) columns, encoded as in the KV key
+2. If the index is unique, and the index uses the old format for stored
+ columns, data from where the row intersects the stored columns,
+ encoded as in the KV key
+3. If needed, `TUPLE`-encoded bytes for non-null composite and stored
+ column data in family 0 (new format).
+
+Since column family 0 is always included, it contains extra information
+that the index stores in the value, such as composite column values and
+stored primary key columns. Note that this is different than the encoding of
+composite indexed columns values in primary key columns, where the composite
+value component of an indexed column is placed in the KV pair corresponding
+to the column family of the indexed column. All of these fields are optional,
+so the `BYTES` value may be empty. Note that, in a unique index, rows with
+a NULL in an indexed column have their implicit column data stored in both the
+KV key and the KV value. (Ditto for stored column data in the old format.)
+
+For indexes with more than one column family, the remaining column families'
+KV values will have value type `TUPLE` and will consist of all stored
+columns in that family in the `TUPLE` encoded format.
+
+### Backwards Compatibility With Indexes Encoded Without Families
+
+Index descriptors hold on to a version bit that denotes what encoding
+format the descriptor was written in. The default value of the bit denotes
+the original secondary index encoding, and indexes created when all
+nodes in a cluster are version 20.1 or greater will have the version representing
+secondary indexes with column families.
+
+### Example dump
+
+Example schema and data:
+
+ CREATE TABLE accounts (
+ id INT PRIMARY KEY,
+ owner STRING,
+ balance DECIMAL,
+ UNIQUE INDEX i2 (owner) STORING (balance),
+ INDEX i3 (owner) STORING (balance)
+ );
+
+ INSERT INTO accounts VALUES
+ (1, 'Alice', 10000.50),
+ (2, 'Bob', 25000.00),
+ (3, 'Carol', NULL),
+ (4, NULL, 9400.10),
+ (5, NULL, NULL);
+
+Index ID 1 is the primary index.
+
+ /Table/51/1/1/0/1489504989.617188491,0 : 0x4AAC12300A2605416C6963651505348D0F4272
+ /Table/51/1/2/0/1489504989.617188491,0 : 0x148941AD0A2603426F621505348D2625A0
+ /Table/51/1/3/0/1489504989.617188491,0 : 0xB1D0B5390A26054361726F6C
+ /Table/51/1/4/0/1489504989.617188491,0 : 0x247286F30A3505348C0E57EA
+ /Table/51/1/5/0/1489504989.617188491,0 : 0xCB0644270A
+
+#### Old STORING format
+
+Index ID 2 is the unique secondary index `i2`.
+
+ /Table/51/2/NULL/4/9400.1/0/1489504989.617188491,0 : 0x01CF9BB0038C2BBD011400
+ ^--- ^ ^----- ^-^-^---------
+ Indexed column | Stored column BYTES 4 9400.1
+ Implicit column
+
+ /Table/51/2/NULL/5/NULL/0/1489504989.617188491,0 : 0xE86B1271038D00
+ ^--- ^ ^--- ^-^-^-
+ Indexed column | Stored column BYTES 5 NULL
+ Implicit column
+
+ /Table/51/2/"Alice"/0/1489504989.617188491,0 : 0x285AC6F303892C0301016400
+ ^------ ^-^-^-----------
+ Indexed column BYTES 1 10000.5
+
+ /Table/51/2/"Bob"/0/1489504989.617188491,0 : 0x23514F1F038A2C056400
+ ^---- ^-^-^-------
+ Indexed column BYTES 2 2.5E+4
+
+ /Table/51/2/"Carol"/0/1489504989.617188491,0 : 0xE98BFEE6038B00
+ ^------ ^-^-^-
+ Indexed column BYTES 3 NULL
+
+Index ID 3 is the non-unique secondary index `i3`.
+
+ /Table/51/3/NULL/4/9400.1/0/1489504989.617188491,0 : 0xEEFAED0403
+ ^--- ^ ^----- ^-
+ Indexed column | Stored column BYTES
+ Implicit column
+
+ /Table/51/3/NULL/5/NULL/0/1489504989.617188491,0 : 0xBE090D2003
+ ^--- ^ ^--- ^-
+ Indexed column | Stored column BYTES
+ Implicit column
+
+ /Table/51/3/"Alice"/1/10000.5/0/1489504989.617188491,0 : 0x7B4964C303
+ ^------ ^ ^------ ^-
+ Indexed column | Stored column BYTES
+ Implicit column
+
+ /Table/51/3/"Bob"/2/2.5E+4/0/1489504989.617188491,0 : 0xDF24708303
+ ^---- ^ ^----- ^-
+ Indexed column | Stored column BYTES
+ Implicit column
+
+ /Table/51/3/"Carol"/3/NULL/0/1489504989.617188491,0 : 0x96CA34AD03
+ ^------ ^ ^--- ^-
+ Indexed column | Stored column BYTES
+ Implicit column
+
+#### New STORING format
+
+Index ID 2 is the unique secondary index `i2`.
+
+ /Table/51/2/NULL/4/0/1492010940.897101344,0 : 0x7F2009CC038C3505348C0E57EA
+ ^--- ^ ^-^-^-------------
+ Indexed column Implicit column BYTES 4 9400.10
+
+ /Table/51/2/NULL/5/0/1492010940.897101344,0 : 0x48047B1A038D
+ ^--- ^ ^-^-
+ Indexed column Implicit column BYTES 5
+
+ /Table/51/2/"Alice"/0/1492010940.897101344,0 : 0x24090BCE03893505348D0F4272
+ ^------ ^-^-^-------------
+ Indexed column BYTES 1 10000.50
+
+ /Table/51/2/"Bob"/0/1492010940.897101344,0 : 0x54353EB9038A3505348D2625A0
+ ^---- ^-^-^-------------
+ Indexed column BYTES 2 25000.00
+
+ /Table/51/2/"Carol"/0/1492010940.897101344,0 : 0xE731A320038B
+ ^------ ^-^-
+ Indexed column BYTES 3
+
+Index ID 3 is the non-unique secondary index `i3`.
+
+ /Table/51/3/NULL/4/0/1492010940.897101344,0 : 0x17C357B0033505348C0E57EA
+ ^--- ^ ^-^-------------
+ Indexed column Implicit column BYTES 9400.10
+
+ /Table/51/3/NULL/5/0/1492010940.897101344,0 : 0x844708BC03
+ ^--- ^ ^-
+ Indexed column Implicit column BYTES
+
+ /Table/51/3/"Alice"/1/0/1492010940.897101344,0 : 0x3AD2E728033505348D0F4272
+ ^------ ^ ^-^-------------
+ Indexed column Implicit column BYTES 10000.50
+
+ /Table/51/3/"Bob"/2/0/1492010940.897101344,0 : 0x7F1225A4033505348D2625A0
+ ^---- ^ ^-^-------------
+ Indexed column Implicit column BYTES 25000.00
+
+ /Table/51/3/"Carol"/3/0/1492010940.897101344,0 : 0x45C61B8403
+ ^------ ^ ^-
+ Indexed column Implicit column BYTES
+
+### Example dump with families
+```
+CREATE TABLE t (
+ a INT, b INT, c INT, d INT, e INT, f INT,
+ PRIMARY KEY (a, b),
+ UNIQUE INDEX i (d, e) STORING (c, f),
+ FAMILY (a, b, c), FAMILY (d, e), FAMILY (f)
+);
+
+INSERT INTO t VALUES (1, 2, 3, 4, 5, 6);
+
+/Table/52/2/4/5/0/1572546219.386986000,0 : 0xBDD6D93003898A3306
+ ^-- ^ ^_^_______
+ Indexed cols Column family 0 BYTES Stored PK cols + column c
+// Notice that /Table/52/2/4/5/1/1/ is not present, because these values are already indexed
+/Table/52/2/4/5/2/1/1572546219.386986000,0 : 0x46CC99AE0A630C
+ ^__ ^_^___
+ Column Family 2 TUPLE column f
+```
+
+### Composite encoding
+
+Secondary indexes use key encoding for all indexed columns, implicit
+columns, and stored columns in the old format. Every datum whose key
+encoding does not suffice for decoding (collated strings, floating-point
+and decimal negative zero, decimals with trailing zeros) is encoded
+again, in the same `TUPLE` that contains stored column data in the new
+format.
+
+Example schema and data:
+
+ CREATE TABLE owners (
+ id INT PRIMARY KEY,
+ owner STRING COLLATE en,
+ INDEX i2 (owner)
+ );
+
+ INSERT INTO owners VALUES
+ (1, 'Ted' COLLATE en),
+ (2, 'Bob' COLLATE en),
+ (3, NULL);
+
+Index ID 1 is the primary index.
+
+ /Table/51/1/1/0/1492008659.730236666,0 : 0x6CA87E2B0A2603546564
+ /Table/51/1/2/0/1492008659.730236666,0 : 0xE900EBB50A2603426F62
+ /Table/51/1/3/0/1492008659.730236666,0 : 0xCF8B38950A
+
+Index ID 2 is the secondary index `i2`.
+
+ /Table/51/2/NULL/3/0/1492008659.730236666,0 : 0xBDAA5DBE03
+ ^--- ^-
+ Indexed column BYTES
+
+ /Table/51/2/"\x16\x05\x17q\x16\x05\x00\x00\x00 \x00 \x00 \x00\x00\b\x02\x02"/2/0/1492008659.730236666,0 : 0x4A8239F6032603426F62
+ ^--------------------------------------------------------------- ^-^---------
+ Indexed column: Collation key for 'Bob' BYTES 'Bob'
+
+ /Table/51/2/"\x18\x16\x16L\x161\x00\x00\x00 \x00 \x00 \x00\x00\b\x02\x02"/1/0/1492008659.730236666,0 : 0x747DA39A032603546564
+ ^------------------------------------------------------------ ^-^---------
+ Indexed column: Collation key for 'Ted' BYTES 'Ted'
+
+ [pkg/util/encoding/encoding.go]: https://github.com/cockroachdb/cockroach/blob/master/pkg/util/encoding/encoding.go
+ [SQL in CockroachDB: Mapping Table Data to Key-Value Storage]: https://www.cockroachlabs.com/blog/sql-in-cockroachdb-mapping-table-data-to-key-value-storage/
+ [Implementing Column Families in CockroachDB]: https://www.cockroachlabs.com/blog/sql-cockroachdb-column-families/
+ [column families RFC]: https://github.com/cockroachdb/cockroach/blob/master/docs/RFCS/20151214_sql_column_families.md
+ [interleaving RFC]: https://github.com/cockroachdb/cockroach/blob/master/docs/RFCS/20160624_sql_interleaved_tables.md
+ [pkg/sql/catalog/descpb/structured.proto]: https://github.com/cockroachdb/cockroach/blob/master/pkg/sql/catalog/descpb/structured.proto
+ [pkg/sql/row/writer.go]: https://github.com/cockroachdb/cockroach/blob/master/pkg/sql/row/writer.go
+ [pkg/sql/row/fetcher.go]: https://github.com/cockroachdb/cockroach/blob/master/pkg/sql/row/fetcher.go
+ [prefix-free]: https://en.wikipedia.org/wiki/Prefix_code
+ [new `DECIMAL` encoding]: https://github.com/cockroachdb/cockroach/issues/13384#issuecomment-277120394
+ [pkg/sql/rowenc/column\_type\_encoding.go]: https://github.com/cockroachdb/cockroach/blob/master/pkg/sql/rowenc/column_type_encoding.go
+ [pkg/sql/rowenc/encoded\_datum.go]: https://github.com/cockroachdb/cockroach/blob/master/pkg/sql/rowenc/encoded_datum.go
+ [pkg/roachpb/data.proto]: https://github.com/cockroachdb/cockroach/blob/master/pkg/roachpb/data.proto
+ [Unicode Collation Algorithm]: http://unicode.org/reports/tr10/
+ [an efficient partial inverse]: http://stackoverflow.com/q/23609457/2144669
diff --git a/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/replication.json b/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/replication.json
new file mode 100644
index 00000000000..862b0efa73f
--- /dev/null
+++ b/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/replication.json
@@ -0,0 +1,1308 @@
+{
+ "annotations": {
+ "list": [
+ {
+ "builtIn": 1,
+ "datasource": {
+ "type": "datasource",
+ "uid": "grafana"
+ },
+ "enable": true,
+ "hide": true,
+ "iconColor": "rgba(0, 211, 255, 1)",
+ "name": "Annotations & Alerts",
+ "target": {
+ "limit": 100,
+ "matchAny": false,
+ "tags": [],
+ "type": "dashboard"
+ },
+ "type": "dashboard"
+ }
+ ]
+ },
+ "editable": true,
+ "fiscalYearStartMonth": 0,
+ "graphTooltip": 0,
+ "id": 10,
+ "links": [],
+ "liveNow": false,
+ "panels": [
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 0
+ },
+ "hiddenSeries": false,
+ "id": 2,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 2,
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "9.4.7",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(sum(ranges{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"})) by (instance)",
+ "hide": false,
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Ranges",
+ "refId": "A"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sum(sum(replicas_leaders{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}) by (instance))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Leaders",
+ "refId": "B"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sum(sum(replicas_leaseholders{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}) by (instance))",
+ "interval": "",
+ "legendFormat": "Lease Holders",
+ "refId": "G"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(sum(replicas_leaders_not_leaseholders{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}) by (instance))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Leaders w/o Lease",
+ "refId": "C"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sum(sum(ranges_unavailable{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}) by (instance))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Unavailable",
+ "refId": "D"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(sum(ranges_underreplicated{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}) by (instance))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Under-replicated",
+ "refId": "E"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sum(sum(ranges_overreplicated{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}) by (instance))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Over-replicated",
+ "refId": "F"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Ranges",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:133",
+ "format": "short",
+ "label": "ranges",
+ "logBase": 1,
+ "show": true
+ },
+ {
+ "$$hashKey": "object:134",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The number of replicas on each store.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 8
+ },
+ "hiddenSeries": false,
+ "id": 4,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 2,
+ "nullPointMode": "null as zero",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "9.4.7",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sum(replicas{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}) by (instance)",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "{{instance}}",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Replicas per Store",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:431",
+ "format": "short",
+ "label": "replicas",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:432",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The number of leaseholder replicas on each store. A leaseholder replica is the one that receives and coordinates all read and write requests for its range.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 16
+ },
+ "hiddenSeries": false,
+ "id": 6,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 2,
+ "nullPointMode": "null as zero",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "9.4.7",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sum(replicas_leaseholders{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}) by (instance)",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "{{instance}}",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Leaseholders per Store",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:581",
+ "format": "short",
+ "label": "leaseholders",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:582",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "Exponentially weighted moving average of the number of KV batch requests processed by leaseholder replicas on each store per second. Tracks roughly the last 30 minutes of requests. Used for load-based rebalancing decisions.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 24
+ },
+ "hiddenSeries": false,
+ "id": 14,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "9.3.1",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "rebalancing_queriespersecond{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}",
+ "interval": "",
+ "legendFormat": "{{instance}}",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Average Queries per Store",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:731",
+ "format": "short",
+ "label": "queries",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:732",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "Number of logical bytes stored in [key-value pairs](https://www.cockroachlabs.com/docs/v21.1/architecture/distribution-layer.html#table-data) on each node.\n\nThis includes historical and deleted data.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 32
+ },
+ "hiddenSeries": false,
+ "id": 16,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "9.3.1",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "totalbytes{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}",
+ "interval": "",
+ "legendFormat": "{{instance}}",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Logical Bytes per Store",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:807",
+ "format": "bytes",
+ "label": "logical store size",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:808",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 40
+ },
+ "hiddenSeries": false,
+ "id": 8,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 2,
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "9.3.1",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sum(sum(replicas{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}) by (instance)) ",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Replicas",
+ "refId": "A"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sum(sum(replicas_quiescent{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}) by (instance))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Quiescent",
+ "refId": "B"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Replica Quiescence",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:883",
+ "format": "short",
+ "label": "replicas",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:884",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 48
+ },
+ "hiddenSeries": false,
+ "id": 10,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null as zero",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "9.3.1",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sum(sum(rate(range_splits{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}[$rate_interval])) by (instance))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Splits",
+ "refId": "A"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sum(sum(rate(range_merges{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}[$rate_interval])) by (instance))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Merges",
+ "refId": "C"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sum(sum(rate(range_adds{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}[$rate_interval])) by (instance))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Adds",
+ "refId": "B"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sum(sum(rate(range_removes{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}[$rate_interval])) by (instance))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Removes",
+ "refId": "D"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(sum(rate(leases_transfers_success{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}[$rate_interval])) by (instance))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Lease Transfers",
+ "refId": "E"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(sum(rate(rebalancing_lease_transfers{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}[$rate_interval])) by (instance))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Load-based Lease Transfers",
+ "refId": "F"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(sum(rate(rebalancing_range_rebalances{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}[$rate_interval])) by (instance))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Load-based Range Rebalances",
+ "refId": "G"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Range Operations",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:959",
+ "format": "short",
+ "label": "ranges",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:960",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 56
+ },
+ "hiddenSeries": false,
+ "id": 12,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 2,
+ "nullPointMode": "null as zero",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "9.3.1",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sum(rate(range_snapshots_generated{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}[$rate_interval]))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Generated",
+ "refId": "A"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(rate(range_snapshots_applied_voter{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}[$rate_interval]))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Applied (Voters)",
+ "refId": "B"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(rate(range_snapshots_applied_initial{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}[$rate_interval]))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Applied (Initial Upreplication)",
+ "refId": "C"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(rate(range_snapshots_applied_initial{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}[$rate_interval]))",
+ "hide": false,
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Applied (Non-Voters)",
+ "refId": "D"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sum(rate(replicas_reserved{cluster=~\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}[$__rate_interval]))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Reserved Replicas",
+ "refId": "E"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Snapshots",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:1109",
+ "format": "short",
+ "label": "snapshots",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:1110",
+ "format": "short",
+ "label": "",
+ "logBase": 1,
+ "min": "0",
+ "show": false
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 64
+ },
+ "hiddenSeries": false,
+ "id": 18,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null as zero",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "9.3.1",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "editorMode": "code",
+ "expr": "rate(raft_rcvd_app{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}[$rate_interval])",
+ "hide": false,
+ "legendFormat": "__auto",
+ "range": true,
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Msg App",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:959",
+ "format": "short",
+ "label": "ranges",
+ "logBase": 2,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:960",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 72
+ },
+ "hiddenSeries": false,
+ "id": 19,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null as zero",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "9.3.1",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "editorMode": "code",
+ "expr": "rate(raft_rcvd_queued_bytes{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}[$rate_interval])",
+ "hide": false,
+ "legendFormat": "__auto",
+ "range": true,
+ "refId": "B"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "editorMode": "code",
+ "expr": "",
+ "hide": false,
+ "legendFormat": "__auto",
+ "range": true,
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Msg Queued Bytes",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:959",
+ "format": "short",
+ "label": "ranges",
+ "logBase": 2,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:960",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ }
+ ],
+ "refresh": "",
+ "revision": 1,
+ "schemaVersion": 38,
+ "style": "dark",
+ "tags": [],
+ "templating": {
+ "list": [
+ {
+ "current": {
+ "selected": false,
+ "text": "Prometheus",
+ "value": "Prometheus"
+ },
+ "hide": 0,
+ "includeAll": false,
+ "label": "datasource",
+ "multi": false,
+ "name": "DS_PROMETHEUS",
+ "options": [],
+ "query": "prometheus",
+ "queryValue": "",
+ "refresh": 1,
+ "regex": "",
+ "skipUrlSync": false,
+ "type": "datasource"
+ },
+ {
+ "current": {
+ "selected": false,
+ "text": "drew-demo",
+ "value": "drew-demo"
+ },
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "definition": "sys_uptime{job=\"cockroachdb\"}",
+ "hide": 0,
+ "includeAll": false,
+ "label": "Cluster",
+ "multi": false,
+ "name": "cluster",
+ "options": [],
+ "query": {
+ "query": "sys_uptime{job=\"cockroachdb\"}",
+ "refId": "Prometheus-cluster-Variable-Query"
+ },
+ "refresh": 1,
+ "regex": "/cluster=\"([^\"]+)\"/",
+ "skipUrlSync": false,
+ "sort": 1,
+ "tagValuesQuery": "",
+ "tagsQuery": "",
+ "type": "query",
+ "useTags": false
+ },
+ {
+ "allValue": "",
+ "current": {
+ "selected": false,
+ "text": "All",
+ "value": "$__all"
+ },
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "definition": "label_values(sys_uptime{job=\"cockroachdb\",cluster=\"$cluster\"},instance)",
+ "hide": 0,
+ "includeAll": true,
+ "label": "Node",
+ "multi": false,
+ "name": "node",
+ "options": [],
+ "query": {
+ "query": "label_values(sys_uptime{job=\"cockroachdb\",cluster=\"$cluster\"},instance)",
+ "refId": "Prometheus-node-Variable-Query"
+ },
+ "refresh": 1,
+ "regex": "",
+ "skipUrlSync": false,
+ "sort": 1,
+ "tagValuesQuery": "",
+ "tagsQuery": "",
+ "type": "query",
+ "useTags": false
+ },
+ {
+ "auto": false,
+ "auto_count": 30,
+ "auto_min": "10s",
+ "current": {
+ "selected": false,
+ "text": "30s",
+ "value": "30s"
+ },
+ "hide": 0,
+ "label": "Rate Interval",
+ "name": "rate_interval",
+ "options": [
+ {
+ "selected": true,
+ "text": "30s",
+ "value": "30s"
+ },
+ {
+ "selected": false,
+ "text": "1m",
+ "value": "1m"
+ },
+ {
+ "selected": false,
+ "text": "5m",
+ "value": "5m"
+ },
+ {
+ "selected": false,
+ "text": "15m",
+ "value": "15m"
+ },
+ {
+ "selected": false,
+ "text": "30m",
+ "value": "30m"
+ },
+ {
+ "selected": false,
+ "text": "1h",
+ "value": "1h"
+ },
+ {
+ "selected": false,
+ "text": "2h",
+ "value": "2h"
+ },
+ {
+ "selected": false,
+ "text": "1d",
+ "value": "1d"
+ }
+ ],
+ "query": "30s,1m,5m,15m,30m,1h,2h,1d",
+ "queryValue": "",
+ "refresh": 2,
+ "skipUrlSync": false,
+ "type": "interval"
+ }
+ ]
+ },
+ "time": {
+ "from": "now-1h",
+ "to": "now"
+ },
+ "timepicker": {},
+ "timezone": "America/New_York",
+ "title": "CRDB Console: Replication ",
+ "uid": "crdb-console-replications",
+ "version": 4,
+ "weekStart": ""
+}
diff --git a/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/runtime.json b/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/runtime.json
new file mode 100644
index 00000000000..1bb09fba55f
--- /dev/null
+++ b/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/runtime.json
@@ -0,0 +1,987 @@
+{
+ "annotations": {
+ "list": [
+ {
+ "builtIn": 1,
+ "datasource": {
+ "type": "datasource",
+ "uid": "grafana"
+ },
+ "enable": true,
+ "hide": true,
+ "iconColor": "rgba(0, 211, 255, 1)",
+ "name": "Annotations & Alerts",
+ "target": {
+ "limit": 100,
+ "matchAny": false,
+ "tags": [],
+ "type": "dashboard"
+ },
+ "type": "dashboard"
+ }
+ ]
+ },
+ "editable": true,
+ "fiscalYearStartMonth": 0,
+ "graphTooltip": 0,
+ "id": 8,
+ "links": [],
+ "liveNow": false,
+ "panels": [
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The number of live nodes in the cluster.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 0
+ },
+ "hiddenSeries": false,
+ "id": 2,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null as zero",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "9.4.7",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "min(liveness_livenodes{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"})",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Live Nodes",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Live Node Count",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:637",
+ "format": "short",
+ "label": "nodes",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:638",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "Memory in use across all nodes:\nRSS \nTotal memory in use by CockroachDB\n\nGo Allocated \nMemory allocated by the Go layer\n\nGo Total \nTotal memory managed by the Go layer\n\nC Allocated \nMemory allocated by the C layer\n\nC Total \nTotal memory managed by the C layer",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 8
+ },
+ "hiddenSeries": false,
+ "id": 4,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 2,
+ "nullPointMode": "null as zero",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "9.4.7",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(sys_rss{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"})",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Total memory (RSS)",
+ "refId": "A"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sum(sys_cgo_allocbytes{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"})",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Go Allocated",
+ "refId": "B"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sum(sys_go_totalbytes{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"})",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Go Total",
+ "refId": "D"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sum(sys_go_allocbytes{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"})",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "CGo Allocated",
+ "refId": "C"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sum(sys_cgo_totalbytes{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"})",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "CGo Total",
+ "refId": "E"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Memory Usage",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:863",
+ "format": "bytes",
+ "label": "memory usage",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:864",
+ "format": "bytes",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The number of Goroutines across all nodes. This count should rise and fall based on load.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 16
+ },
+ "hiddenSeries": false,
+ "id": 10,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null as zero",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "9.4.7",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(sys_goroutines{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"})",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Goroutine Count",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Goroutine Count",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:1235",
+ "format": "short",
+ "label": "goroutines",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:1236",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The number of Goroutines waiting for CPU. This count should rise and fall based on load.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 24
+ },
+ "hiddenSeries": false,
+ "id": 16,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "9.0.2",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sys_runnable_goroutines_per_cpu{job=\"cockroachdb\",cluster=~\"$cluster\",instance=~\"$node\"}",
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "{{instance}}",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Runnable Goroutines per CPU",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:391",
+ "format": "short",
+ "label": "goroutines",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:392",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The number of times that Go’s garbage collector was invoked per second across all nodes.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 32
+ },
+ "hiddenSeries": false,
+ "id": 8,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sum(rate(sys_gc_count{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}[$rate_interval]))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "GC Runs",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "GC Runs",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:1311",
+ "format": "short",
+ "label": "runs",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:1312",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The amount of processor time used by Go’s garbage collector per second across all nodes. During garbage collection, application code execution is paused.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 40
+ },
+ "hiddenSeries": false,
+ "id": 12,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 2,
+ "nullPointMode": "null as zero",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(rate(sys_gc_pause_ns{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}[$rate_interval]))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "GC Pause Time",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "GC Pause Time",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:1387",
+ "format": "ns",
+ "label": "pause time",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:1388",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 48
+ },
+ "hiddenSeries": false,
+ "id": 6,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 2,
+ "nullPointMode": "null as zero",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(rate(sys_cpu_user_ns{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}[$__rate_interval]))",
+ "instant": false,
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "User CPU Time",
+ "refId": "A"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(rate(sys_cpu_sys_ns{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}[$__rate_interval]))",
+ "instant": false,
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Sys CPU Time",
+ "refId": "B"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "CPU Time",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:1833",
+ "format": "ns",
+ "label": "cpu time",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:1834",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "Mean clock offset of each node against the rest of the cluster.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 56
+ },
+ "hiddenSeries": false,
+ "id": 14,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null as zero",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "clock_offset_meannanos{job=\"cockroachdb\",instance=~\"$node\",cluster=~\"$cluster\"}",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "{{instance}}",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Clock Offset",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:787",
+ "format": "ns",
+ "label": "offset",
+ "logBase": 1,
+ "show": true
+ },
+ {
+ "$$hashKey": "object:788",
+ "format": "short",
+ "label": "",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ }
+ ],
+ "refresh": "30s",
+ "revision": 1,
+ "schemaVersion": 38,
+ "style": "dark",
+ "tags": [],
+ "templating": {
+ "list": [
+ {
+ "current": {
+ "selected": false,
+ "text": "Prometheus",
+ "value": "Prometheus"
+ },
+ "hide": 0,
+ "includeAll": false,
+ "label": "datasource",
+ "multi": false,
+ "name": "DS_PROMETHEUS",
+ "options": [],
+ "query": "prometheus",
+ "refresh": 1,
+ "regex": "",
+ "skipUrlSync": false,
+ "type": "datasource"
+ },
+ {
+ "current": {
+ "selected": false,
+ "text": "drew-demo",
+ "value": "drew-demo"
+ },
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "definition": "sys_uptime{job=\"cockroachdb\"}",
+ "hide": 0,
+ "includeAll": false,
+ "label": "Cluster",
+ "multi": false,
+ "name": "cluster",
+ "options": [],
+ "query": {
+ "query": "sys_uptime{job=\"cockroachdb\"}",
+ "refId": "Prometheus-cluster-Variable-Query"
+ },
+ "refresh": 1,
+ "regex": "/cluster=\"([^\"]+)\"/",
+ "skipUrlSync": false,
+ "sort": 1,
+ "tagValuesQuery": "",
+ "tagsQuery": "",
+ "type": "query",
+ "useTags": false
+ },
+ {
+ "allValue": "",
+ "current": {
+ "selected": false,
+ "text": "All",
+ "value": "$__all"
+ },
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "definition": "label_values(sys_uptime{job=\"cockroachdb\",cluster=\"$cluster\"},instance)",
+ "hide": 0,
+ "includeAll": true,
+ "label": "Node",
+ "multi": false,
+ "name": "node",
+ "options": [],
+ "query": {
+ "query": "label_values(sys_uptime{job=\"cockroachdb\",cluster=\"$cluster\"},instance)",
+ "refId": "Prometheus-node-Variable-Query"
+ },
+ "refresh": 1,
+ "regex": "",
+ "skipUrlSync": false,
+ "sort": 1,
+ "tagValuesQuery": "",
+ "tagsQuery": "",
+ "type": "query",
+ "useTags": false
+ },
+ {
+ "auto": false,
+ "auto_count": 30,
+ "auto_min": "10s",
+ "current": {
+ "selected": false,
+ "text": "30s",
+ "value": "30s"
+ },
+ "hide": 0,
+ "label": "Rate Interval",
+ "name": "rate_interval",
+ "options": [
+ {
+ "selected": true,
+ "text": "30s",
+ "value": "30s"
+ },
+ {
+ "selected": false,
+ "text": "1m",
+ "value": "1m"
+ },
+ {
+ "selected": false,
+ "text": "5m",
+ "value": "5m"
+ },
+ {
+ "selected": false,
+ "text": "10m",
+ "value": "10m"
+ },
+ {
+ "selected": false,
+ "text": "30m",
+ "value": "30m"
+ },
+ {
+ "selected": false,
+ "text": "1h",
+ "value": "1h"
+ },
+ {
+ "selected": false,
+ "text": "6h",
+ "value": "6h"
+ },
+ {
+ "selected": false,
+ "text": "12h",
+ "value": "12h"
+ },
+ {
+ "selected": false,
+ "text": "1d",
+ "value": "1d"
+ }
+ ],
+ "query": "30s,1m,5m,10m,30m,1h,6h,12h,1d",
+ "queryValue": "",
+ "refresh": 2,
+ "skipUrlSync": false,
+ "type": "interval"
+ }
+ ]
+ },
+ "time": {
+ "from": "now-1h",
+ "to": "now"
+ },
+ "timepicker": {},
+ "timezone": "America/New_York",
+ "title": "CRDB Console: Runtime ",
+ "uid": "crdb-console-runtime",
+ "version": 4,
+ "weekStart": ""
+}
diff --git a/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/sql.json b/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/sql.json
new file mode 100644
index 00000000000..b90cbbe26dc
--- /dev/null
+++ b/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/sql.json
@@ -0,0 +1,1922 @@
+{
+ "annotations": {
+ "list": [
+ {
+ "builtIn": 1,
+ "datasource": {
+ "type": "datasource",
+ "uid": "grafana"
+ },
+ "enable": true,
+ "hide": true,
+ "iconColor": "rgba(0, 211, 255, 1)",
+ "name": "Annotations & Alerts",
+ "target": {
+ "limit": 100,
+ "matchAny": false,
+ "tags": [],
+ "type": "dashboard"
+ },
+ "type": "dashboard"
+ }
+ ]
+ },
+ "editable": true,
+ "fiscalYearStartMonth": 0,
+ "graphTooltip": 0,
+ "id": 7,
+ "links": [],
+ "liveNow": false,
+ "panels": [
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The total number of open SQL Sessions.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 0
+ },
+ "hiddenSeries": false,
+ "id": 2,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 2,
+ "nullPointMode": "null as zero",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "9.4.7",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sql_conns{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "{{instance}}",
+ "refId": "B"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Open SQL Sessions",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:108",
+ "format": "short",
+ "label": "connections",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:109",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The total number of SQL transactions currently open.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 8
+ },
+ "hiddenSeries": false,
+ "id": 30,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "9.4.7",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sql_txns_open{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}",
+ "interval": "",
+ "legendFormat": "{{instance}}",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Open SQL Transactions",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:279",
+ "format": "short",
+ "label": "transactions",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:280",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The total number of SQL statements currently running.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 16
+ },
+ "hiddenSeries": false,
+ "id": 10,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 2,
+ "nullPointMode": "null as zero",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "9.4.7",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(sql_distsql_queries_active{job=\"cockroachdb\",cluster=~\"$cluster\",instance=~\"$node\"})",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Active Statements",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Active SQL Statements",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:184",
+ "format": "short",
+ "label": "queries",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:185",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The total amount of SQL client network traffic in bytes per second.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 24
+ },
+ "hiddenSeries": false,
+ "id": 4,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "9.0.2",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(rate(sql_bytesin{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}[$__rate_interval]))",
+ "instant": false,
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Bytes In",
+ "refId": "A"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(rate(sql_bytesout{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}[$__rate_interval]))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Bytes Out",
+ "refId": "B"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "SQL Byte Traffic",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:260",
+ "format": "bytes",
+ "label": "byte traffic",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:261",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "A moving average of the # of SELECT, INSERT, UPDATE, and DELETE statements successfully executed per second.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 32
+ },
+ "hiddenSeries": false,
+ "id": 6,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "9.0.2",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(rate(sql_select_count{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}[$rate_interval]))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Selects",
+ "refId": "A"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(rate(sql_update_count{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}[$rate_interval]))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Updates",
+ "refId": "B"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(rate(sql_insert_count{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}[$rate_interval]))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Inserts",
+ "refId": "C"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(rate(sql_delete_count{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}[$rate_interval]))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Deletes",
+ "refId": "D"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "SQL Statements",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:336",
+ "format": "short",
+ "label": "queries",
+ "logBase": 1,
+ "show": true
+ },
+ {
+ "$$hashKey": "object:337",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The number of statements which returned a planning or runtime error.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 40
+ },
+ "hiddenSeries": false,
+ "id": 8,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(rate(sql_failure_count{job=\"cockroachdb\",cluster=~\"$cluster\",instance=~\"$node\"}[$__rate_interval]))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Errors",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "SQL Statement Errors",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:412",
+ "format": "short",
+ "label": "errors",
+ "logBase": 1,
+ "max": "1",
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:413",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The total number of SQL statements that experienced contention.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 48
+ },
+ "hiddenSeries": false,
+ "id": 32,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(rate(sql_distsql_contended_queries_count{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}[$__rate_interval]))",
+ "interval": "",
+ "legendFormat": "Contention",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "SQL Statement Contention",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:489",
+ "format": "short",
+ "label": "queries",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:490",
+ "format": "short",
+ "label": "",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The number of flows on each node contributing to currently running distributed SQL statements.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 56
+ },
+ "hiddenSeries": false,
+ "id": 12,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "rate(sql_distsql_flows_active{job=\"cockroachdb\",cluster=~\"$cluster\",instance=~\"$node\"}[$__rate_interval])",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "{{instance}}",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Active Flows for Distributed SQL Statements",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:107",
+ "format": "short",
+ "label": "flows",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:108",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "Over the last minute, this node executed 99% of queries within this time. This time does not include network latency between the node and client.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 64
+ },
+ "hiddenSeries": false,
+ "id": 14,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "histogram_quantile(0.99, rate(sql_service_latency_bucket{job=\"cockroachdb\",instance=~\"$node\",cluster=\"$cluster\"}[$__rate_interval]))",
+ "interval": "",
+ "legendFormat": "{{instance}}",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Service Latency: SQL, 99th percentile",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:710",
+ "format": "ns",
+ "label": "latency",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:711",
+ "format": "short",
+ "label": "",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "Over the last minute, this node executed 90% of queries within this time. This time does not include network latency between the node and client.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 72
+ },
+ "hiddenSeries": false,
+ "id": 16,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "histogram_quantile(0.90, rate(sql_service_latency_bucket{job=\"cockroachdb\",instance=~\"$node\",cluster=\"$cluster\"}[$__rate_interval]))",
+ "interval": "",
+ "legendFormat": "{{instance}}",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Service Latency: SQL, 90th percentile",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:934",
+ "format": "ns",
+ "label": "latency",
+ "logBase": 1,
+ "show": true
+ },
+ {
+ "$$hashKey": "object:935",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The 99th percentile of latency between query requests and responses over a 1 minute period. Values are displayed individually for each node.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 80
+ },
+ "hiddenSeries": false,
+ "id": 18,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 2,
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "histogram_quantile(0.99,rate(exec_latency_bucket{job=\"cockroachdb\",instance=~\"$node\"}[$rate_interval]))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "{{instance}}",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "KV Execution Latency: 99th percentile",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:1084",
+ "format": "µs",
+ "label": "latency",
+ "logBase": 1,
+ "show": true
+ },
+ {
+ "$$hashKey": "object:1085",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The 90th percentile of latency between query requests and responses over a 1 minute period. Values are displayed individually for each node.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 88
+ },
+ "hiddenSeries": false,
+ "id": 20,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "histogram_quantile(0.90, rate(exec_latency_bucket{job=\"cockroachdb\",instance=~\"$node\",cluster=\"$cluster\"}[$rate_interval]))",
+ "interval": "",
+ "legendFormat": "{{instance}}",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "KV Execution Latency: 90th percentile",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:1160",
+ "format": "ns",
+ "label": "latency",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:1161",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The total number of transactions initiated, committed, rolled back, or aborted per second.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 96
+ },
+ "hiddenSeries": false,
+ "id": 22,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 2,
+ "nullPointMode": "null as zero",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sum(rate(sql_txn_begin_count{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}[$rate_interval]))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Begin",
+ "refId": "A"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(rate(sql_txn_commit_count{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}[$rate_interval]))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Commits",
+ "refId": "B"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(rate( sql_txn_rollback_count{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}[$rate_interval]))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Rollbacks",
+ "refId": "D"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(rate(sql_txn_abort_count{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}[$rate_interval]))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Aborts",
+ "refId": "C"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Transactions",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:1458",
+ "format": "short",
+ "label": "latency",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:1459",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The 99th percentile of total transaction time over a 1 minute period. Values are displayed individually for each node.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 104
+ },
+ "hiddenSeries": false,
+ "id": 26,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "histogram_quantile(0.99,rate(sql_txn_latency_bucket{job=\"cockroachdb\",cluster=\"$cluster\"}[5m]))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "{{instance}}",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Transaction Latency: 99th percentile",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:1756",
+ "format": "µs",
+ "label": "latency",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:1757",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The 90th percentile of total transaction time over a 1 minute period. Values are displayed individually for each node.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 112
+ },
+ "hiddenSeries": false,
+ "id": 28,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 2,
+ "nullPointMode": "null as zero",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "histogram_quantile(0.90,rate(sql_txn_latency_bucket{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}[$__rate_interval]))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "{{instance}}",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Transaction Latency: 90th percentile",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:1832",
+ "format": "ns",
+ "label": "latency",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:1833",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The current amount of allocated SQL memory. This amount is what is compared against the node's --max-sql-memory flag.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 120
+ },
+ "hiddenSeries": false,
+ "id": 34,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sql_mem_root_current{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}",
+ "interval": "",
+ "legendFormat": "{{instance}}",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "SQL Memory",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:157",
+ "format": "bytes",
+ "label": "allocation bytes",
+ "logBase": 1,
+ "show": true
+ },
+ {
+ "$$hashKey": "object:158",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The total number of DDL statements per second",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 128
+ },
+ "hiddenSeries": false,
+ "id": 24,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 2,
+ "nullPointMode": "null as zero",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(rate(sql_ddl_count{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}[$rate_interval])) ",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "DDL Statements",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Schema Changes",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:1908",
+ "format": "short",
+ "label": "statements",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:1909",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The total number of statements denied per second due to a [cluster setting](https://www.cockroachlabs.com/docs/v21.1/cluster-settings.html) in the format feature.statement_type.enabled = FALSE.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 136
+ },
+ "hiddenSeries": false,
+ "id": 36,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "rate(sql_feature_flag_denial{cluster=\"$cluster\",job=\"cockroachdb\",instance=~\"$node\"}[$__rate_interval])",
+ "interval": "",
+ "legendFormat": "{{instance}}",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Statement Denials: Cluster Settings",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:214",
+ "format": "short",
+ "label": "statements",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:215",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ }
+ ],
+ "refresh": "30s",
+ "revision": 1,
+ "schemaVersion": 38,
+ "style": "dark",
+ "tags": [],
+ "templating": {
+ "list": [
+ {
+ "current": {
+ "selected": false,
+ "text": "Prometheus",
+ "value": "Prometheus"
+ },
+ "hide": 0,
+ "includeAll": false,
+ "label": "datasource",
+ "multi": false,
+ "name": "DS_PROMETHEUS",
+ "options": [],
+ "query": "prometheus",
+ "refresh": 1,
+ "regex": "",
+ "skipUrlSync": false,
+ "type": "datasource"
+ },
+ {
+ "current": {
+ "selected": false,
+ "text": "drew-demo",
+ "value": "drew-demo"
+ },
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "definition": "sys_uptime{job=\"cockroachdb\"}",
+ "hide": 0,
+ "includeAll": false,
+ "label": "Cluster",
+ "multi": false,
+ "name": "cluster",
+ "options": [],
+ "query": {
+ "query": "sys_uptime{job=\"cockroachdb\"}",
+ "refId": "Prometheus-cluster-Variable-Query"
+ },
+ "refresh": 1,
+ "regex": "/cluster=\"([^\"]+)\"/",
+ "skipUrlSync": false,
+ "sort": 1,
+ "tagValuesQuery": "",
+ "tagsQuery": "",
+ "type": "query",
+ "useTags": false
+ },
+ {
+ "allValue": "",
+ "current": {
+ "selected": false,
+ "text": "All",
+ "value": "$__all"
+ },
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "definition": "label_values(sys_uptime{job=\"cockroachdb\",cluster=\"$cluster\"},instance)",
+ "hide": 0,
+ "includeAll": true,
+ "label": "Node",
+ "multi": false,
+ "name": "node",
+ "options": [],
+ "query": {
+ "query": "label_values(sys_uptime{job=\"cockroachdb\",cluster=\"$cluster\"},instance)",
+ "refId": "Prometheus-node-Variable-Query"
+ },
+ "refresh": 1,
+ "regex": "",
+ "skipUrlSync": false,
+ "sort": 3,
+ "tagValuesQuery": "",
+ "tagsQuery": "",
+ "type": "query",
+ "useTags": false
+ },
+ {
+ "auto": false,
+ "auto_count": 30,
+ "auto_min": "10s",
+ "current": {
+ "selected": false,
+ "text": "1m",
+ "value": "1m"
+ },
+ "hide": 0,
+ "label": "Rate Interval",
+ "name": "rate_interval",
+ "options": [
+ {
+ "selected": false,
+ "text": "30s",
+ "value": "30s"
+ },
+ {
+ "selected": true,
+ "text": "1m",
+ "value": "1m"
+ },
+ {
+ "selected": false,
+ "text": "5m",
+ "value": "5m"
+ },
+ {
+ "selected": false,
+ "text": "10m",
+ "value": "10m"
+ },
+ {
+ "selected": false,
+ "text": "30m",
+ "value": "30m"
+ },
+ {
+ "selected": false,
+ "text": "1h",
+ "value": "1h"
+ },
+ {
+ "selected": false,
+ "text": "6h",
+ "value": "6h"
+ },
+ {
+ "selected": false,
+ "text": "12h",
+ "value": "12h"
+ },
+ {
+ "selected": false,
+ "text": "1d",
+ "value": "1d"
+ }
+ ],
+ "query": "30s,1m,5m,10m,30m,1h,6h,12h,1d",
+ "queryValue": "",
+ "refresh": 2,
+ "skipUrlSync": false,
+ "type": "interval"
+ }
+ ]
+ },
+ "time": {
+ "from": "now-1h",
+ "to": "now"
+ },
+ "timepicker": {},
+ "timezone": "America/New_York",
+ "title": "CRDB Console: SQL ",
+ "uid": "crdb-console-sql",
+ "version": 3,
+ "weekStart": ""
+}
diff --git a/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/storage.json b/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/storage.json
new file mode 100644
index 00000000000..d08cd5e5918
--- /dev/null
+++ b/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/storage.json
@@ -0,0 +1,1347 @@
+{
+ "annotations": {
+ "list": [
+ {
+ "builtIn": 1,
+ "datasource": {
+ "type": "datasource",
+ "uid": "grafana"
+ },
+ "enable": true,
+ "hide": true,
+ "iconColor": "rgba(0, 211, 255, 1)",
+ "name": "Annotations & Alerts",
+ "target": {
+ "limit": 100,
+ "matchAny": false,
+ "tags": [],
+ "type": "dashboard"
+ },
+ "type": "dashboard"
+ }
+ ]
+ },
+ "editable": true,
+ "fiscalYearStartMonth": 0,
+ "graphTooltip": 0,
+ "id": 6,
+ "links": [],
+ "liveNow": false,
+ "panels": [
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "Usage of disk space across all nodes\n\n**Capacity**: Maximum store size across all nodes. This value may be explicitly set per node using [--store](https://www.cockroachlabs.com/docs/v21.1/cockroach-start.html#store). If a store size has not been set, this metric displays the actual disk capacity.\n\n**Available**: Free disk space available to CockroachDB data across all nodes.\n\n**Used**: Disk space in use by CockroachDB data across all nodes. This excludes the Cockroach binary, operating system, and other system files.\n\n[How are these metrics calculated?](https://www.cockroachlabs.com/docs/v21.1/ui-storage-dashboard.html#capacity-metrics)",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 0
+ },
+ "hiddenSeries": false,
+ "id": 2,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 2,
+ "nullPointMode": "null as zero",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "9.4.7",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(sum(capacity{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}) by (instance))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Max",
+ "refId": "B"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": " sum(sum(capacity_available{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}) by (instance))",
+ "interval": "",
+ "legendFormat": "Available",
+ "refId": "C"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sum(sum(capacity{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}) by (instance)) - sum(sum(capacity_available{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}) by (instance))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Used",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Capacity",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:99",
+ "format": "bytes",
+ "label": "capacity",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:100",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "Amount of data that can be read by applications and CockroachDB.\n\n**Live**: Number of logical bytes stored in live [key-value pairs](https://www.cockroachlabs.com/docs/v21.1/architecture/distribution-layer.html#table-data) across all nodes. Live data excludes historical and deleted data.\n\n**System**: Number of physical bytes stored in [system key-value pairs](https://www.cockroachlabs.com/docs/v21.1/architecture/distribution-layer.html#table-data).",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 8
+ },
+ "hiddenSeries": false,
+ "id": 4,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 2,
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "9.4.7",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sum(sum(livebytes{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}) by (instance))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Live",
+ "refId": "A"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sum(sum(sysbytes{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}) by (instance))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "System",
+ "refId": "B"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Live Bytes",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:323",
+ "format": "bytes",
+ "label": "live bytes",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:324",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The 99th %ile latency for commits to the Raft Log. This measures essentially an fdatasync to the storage engine's write-ahead log.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 16
+ },
+ "hiddenSeries": false,
+ "id": 6,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null as zero",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "9.4.7",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "histogram_quantile(0.99,rate(raft_process_logcommit_latency_bucket{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}[$__rate_interval]))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "{{instance}}",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Log Commit Latency: 99th Percentile",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:474",
+ "format": "ns",
+ "label": "latency",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:475",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The 50th %ile latency for commits to the Raft Log. This measures essentially an fdatasync to the storage engine's write-ahead log.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 6,
+ "w": 24,
+ "x": 0,
+ "y": 24
+ },
+ "hiddenSeries": false,
+ "id": 8,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null as zero",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "histogram_quantile(0.50,rate(raft_process_logcommit_latency_bucket{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}[$__rate_interval]))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "{{instance}}",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Log Commit Latency: 50th Percentile",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:550",
+ "format": "ns",
+ "label": "latency",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:551",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The 99th %ile latency for commits of Raft commands. This measures applying a batch to the storage engine (including writes to the write-ahead log), but no fsync.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 30
+ },
+ "hiddenSeries": false,
+ "id": 10,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null as zero",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "histogram_quantile(0.99,rate(raft_process_commandcommit_latency_bucket{job=\"cockroachdb\", instance=~\"$node\", cluster=~\"$cluster\"}[$__rate_interval]))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "{{instance}}",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Command Commit Latency: 99th Percentile ",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:774",
+ "format": "ns",
+ "label": "latency",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:775",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The 50th %ile latency for commits of Raft commands. This measures applying a batch to the storage engine (including writes to the write-ahead log), but no fsync.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 38
+ },
+ "hiddenSeries": false,
+ "id": 12,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 2,
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "histogram_quantile(0.50,rate(raft_process_commandcommit_latency_bucket{job=\"cockroachdb\", instance=~\"$node\", cluster=~\"$cluster\"}[$__rate_interval]))",
+ "interval": "",
+ "legendFormat": "{{instance}}",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Command Commit Latency: 50th percentile ",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:850",
+ "format": "ns",
+ "label": "latency",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:851",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The average number of real read operations executed per logical read operation.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 46
+ },
+ "hiddenSeries": false,
+ "id": 20,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "avg(avg(rocksdb_read_amplification{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}) by (instance))",
+ "interval": "",
+ "legendFormat": "Read Amplification",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Read Amplification",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:926",
+ "format": "short",
+ "label": "factor",
+ "logBase": 1,
+ "show": true
+ },
+ {
+ "$$hashKey": "object:927",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The number of SSTables in use.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 54
+ },
+ "hiddenSeries": false,
+ "id": 16,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(rocksdb_num_sstables{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"})",
+ "interval": "",
+ "legendFormat": "SSTables",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "SSTables",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:1002",
+ "format": "short",
+ "label": "sstables",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:1003",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The number of open file descriptors, compared with the file descriptor limit.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 62
+ },
+ "hiddenSeries": false,
+ "id": 14,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sum(sum(sys_fd_open{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}) by (instance))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Open",
+ "refId": "A"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sum(sum(sys_fd_softlimit{job=\"cockroachdb\",cluster=\"$cluster\",instance=~\"$node\"}) by (instance))",
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "Limit",
+ "refId": "B"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "File Descriptors",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:1226",
+ "format": "short",
+ "label": "descriptors",
+ "logBase": 1,
+ "show": true
+ },
+ {
+ "$$hashKey": "object:1227",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The number of compactions and memtable flushes per second.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 70
+ },
+ "hiddenSeries": false,
+ "id": 18,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sum(rate(rocksdb_compactions{job=\"cockroachdb\",cluster=\"$cluster\"}[$__rate_interval]))",
+ "interval": "",
+ "legendFormat": "Compactions",
+ "refId": "A"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "expr": "sum(rate(rocksdb_flushes{job=\"cockroachdb\",cluster=\"$cluster\"}[$__rate_interval]))",
+ "interval": "",
+ "legendFormat": "Flushes",
+ "refId": "B"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Compactions/Flushes",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:1376",
+ "format": "short",
+ "label": "count",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:1377",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The number of successfully written time series samples, and number of errors attempting to write time series, per second.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 78
+ },
+ "hiddenSeries": false,
+ "id": 24,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(rate(timeseries_write_samples{job=\"cockroachdb\",cluster=~\"$cluster\",instance=~\"$node\"}[$__rate_interval]))",
+ "interval": "",
+ "legendFormat": "Samples Written",
+ "refId": "A"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(rate(timeseries_write_errors{job=\"cockroachdb\",cluster=~\"$cluster\",instance=~\"$node\"}[$__rate_interval]))",
+ "interval": "",
+ "legendFormat": "Errors",
+ "refId": "B"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Time Series Writes",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:1452",
+ "format": "short",
+ "label": "count",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:1453",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "description": "The number of bytes written by the time series system per second. \nNote that this does not reflect the rate at which disk space is consumed by time series; the data is highly compressed on disk. This rate is instead intended to indicate the amount of network traffic and disk activity generated by time series writes.\nSee the \"databases\" tab to find the current disk usage for time series data.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 86
+ },
+ "hiddenSeries": false,
+ "id": 22,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "",
+ "pointradius": 2,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(rate(timeseries_write_bytes{job=\"cockroachdb\",cluster=~\"$cluster\",instance=~\"$node\"}[$__rate_interval]))",
+ "interval": "",
+ "legendFormat": "Bytes Written",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeRegions": [],
+ "title": "Time Series Bytes Written",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "mode": "time",
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:1528",
+ "format": "bytes",
+ "label": "bytes",
+ "logBase": 1,
+ "min": "0",
+ "show": true
+ },
+ {
+ "$$hashKey": "object:1529",
+ "format": "short",
+ "logBase": 1,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false
+ }
+ }
+ ],
+ "refresh": "30s",
+ "revision": 1,
+ "schemaVersion": 38,
+ "style": "dark",
+ "tags": [],
+ "templating": {
+ "list": [
+ {
+ "current": {
+ "selected": false,
+ "text": "Prometheus",
+ "value": "Prometheus"
+ },
+ "hide": 0,
+ "includeAll": false,
+ "label": "datasource",
+ "multi": false,
+ "name": "DS_PROMETHEUS",
+ "options": [],
+ "query": "prometheus",
+ "refresh": 1,
+ "regex": "",
+ "skipUrlSync": false,
+ "type": "datasource"
+ },
+ {
+ "current": {
+ "selected": false,
+ "text": "drew-demo",
+ "value": "drew-demo"
+ },
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "definition": "sys_uptime{job=\"cockroachdb\"}",
+ "hide": 0,
+ "includeAll": false,
+ "label": "Cluster",
+ "multi": false,
+ "name": "cluster",
+ "options": [],
+ "query": {
+ "query": "sys_uptime{job=\"cockroachdb\"}",
+ "refId": "Prometheus-cluster-Variable-Query"
+ },
+ "refresh": 1,
+ "regex": "/cluster=\"([^\"]+)\"/",
+ "skipUrlSync": false,
+ "sort": 1,
+ "tagValuesQuery": "",
+ "tagsQuery": "",
+ "type": "query",
+ "useTags": false
+ },
+ {
+ "allValue": "",
+ "current": {
+ "selected": false,
+ "text": "All",
+ "value": "$__all"
+ },
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "definition": "label_values(sys_uptime{job=\"cockroachdb\",cluster=\"$cluster\"},instance)",
+ "hide": 0,
+ "includeAll": true,
+ "label": "Node",
+ "multi": false,
+ "name": "node",
+ "options": [],
+ "query": {
+ "query": "label_values(sys_uptime{job=\"cockroachdb\",cluster=\"$cluster\"},instance)",
+ "refId": "Prometheus-node-Variable-Query"
+ },
+ "refresh": 1,
+ "regex": "",
+ "skipUrlSync": false,
+ "sort": 1,
+ "tagValuesQuery": "",
+ "tagsQuery": "",
+ "type": "query",
+ "useTags": false
+ },
+ {
+ "auto": false,
+ "auto_count": 30,
+ "auto_min": "10s",
+ "current": {
+ "selected": false,
+ "text": "30s",
+ "value": "30s"
+ },
+ "hide": 0,
+ "label": "Rate Interval",
+ "name": "rate_interval",
+ "options": [
+ {
+ "selected": true,
+ "text": "30s",
+ "value": "30s"
+ },
+ {
+ "selected": false,
+ "text": "1m",
+ "value": "1m"
+ },
+ {
+ "selected": false,
+ "text": "5m",
+ "value": "5m"
+ },
+ {
+ "selected": false,
+ "text": "10m",
+ "value": "10m"
+ },
+ {
+ "selected": false,
+ "text": "30m",
+ "value": "30m"
+ },
+ {
+ "selected": false,
+ "text": "1h",
+ "value": "1h"
+ },
+ {
+ "selected": false,
+ "text": "6h",
+ "value": "6h"
+ },
+ {
+ "selected": false,
+ "text": "12h",
+ "value": "12h"
+ },
+ {
+ "selected": false,
+ "text": "1d",
+ "value": "1d"
+ }
+ ],
+ "query": "30s,1m,5m,10m,30m,1h,6h,12h,1d",
+ "queryValue": "",
+ "refresh": 2,
+ "skipUrlSync": false,
+ "type": "interval"
+ }
+ ]
+ },
+ "time": {
+ "from": "now-1h",
+ "to": "now"
+ },
+ "timepicker": {},
+ "timezone": "America/New_York",
+ "title": "CRDB Console: Storage ",
+ "uid": "crdb-console-storage",
+ "version": 3,
+ "weekStart": ""
+}
diff --git a/src/current/files/cockroach/monitoring/prometheus.yml b/src/current/files/cockroach/monitoring/prometheus.yml
new file mode 100644
index 00000000000..8e84600d4f2
--- /dev/null
+++ b/src/current/files/cockroach/monitoring/prometheus.yml
@@ -0,0 +1,35 @@
+# Prometheus configuration for cockroach clusters.
+# Requires prometheus 2.X
+#
+# Run with:
+# $ prometheus -config.file=prometheus.yml
+global:
+ scrape_interval: 10s
+ evaluation_interval: 10s
+
+rule_files:
+- "rules/alerts.rules.yml"
+- "rules/aggregation.rules.yml"
+
+# Alert manager running on the same host:
+alerting:
+ alertmanagers:
+ - path_prefix: "/alertmanager/"
+ static_configs:
+ - targets:
+ - localhost:9093
+
+scrape_configs:
+ - job_name: 'cockroachdb'
+ metrics_path: '/_status/vars'
+ # Insecure mode:
+ scheme: 'http'
+ # Secure mode:
+ # scheme: 'https'
+ tls_config:
+ insecure_skip_verify: true
+
+ static_configs:
+ - targets: ['localhost:8080']
+ labels:
+ cluster: 'my-cockroachdb-cluster'
diff --git a/src/current/files/cockroach/monitoring/rules/aggregation.rules.yml b/src/current/files/cockroach/monitoring/rules/aggregation.rules.yml
new file mode 100644
index 00000000000..56e0bc8e604
--- /dev/null
+++ b/src/current/files/cockroach/monitoring/rules/aggregation.rules.yml
@@ -0,0 +1,109 @@
+# This file contains aggregation rules, specifically:
+# "node:X" node-level aggregation of a per-store metric X
+# "cluster:X" cluster-level aggregation of a per-store or per-node metric X
+#
+# Most aggregation rules should use the "without (label1, label2, ...)" keyword
+# to keep all labels but the ones specified.
+
+groups:
+- name: rules/aggregation.rules
+ rules:
+ - record: node:capacity
+ expr: sum without(store) (capacity{job="cockroachdb"})
+ - record: cluster:capacity
+ expr: sum without(instance) (node:capacity{job="cockroachdb"})
+ - record: node:capacity_available
+ expr: sum without(store) (capacity_available{job="cockroachdb"})
+ - record: cluster:capacity_available
+ expr: sum without(instance) (node:capacity_available{job="cockroachdb"})
+ - record: capacity_available:ratio
+ expr: capacity_available{job="cockroachdb"} / capacity{job="cockroachdb"}
+ - record: node:capacity_available:ratio
+ expr: node:capacity_available{job="cockroachdb"} / node:capacity{job="cockroachdb"}
+ - record: cluster:capacity_available:ratio
+ expr: cluster:capacity_available{job="cockroachdb"} / cluster:capacity{job="cockroachdb"}
+ # Histogram rules: these are fairly expensive to compute live, so we precompute a few percetiles.
+ - record: txn_durations_bucket:rate1m
+ expr: rate(txn_durations_bucket{job="cockroachdb"}[1m])
+ - record: txn_durations:rate1m:quantile_50
+ expr: histogram_quantile(0.5, txn_durations_bucket:rate1m)
+ - record: txn_durations:rate1m:quantile_75
+ expr: histogram_quantile(0.75, txn_durations_bucket:rate1m)
+ - record: txn_durations:rate1m:quantile_90
+ expr: histogram_quantile(0.9, txn_durations_bucket:rate1m)
+ - record: txn_durations:rate1m:quantile_95
+ expr: histogram_quantile(0.95, txn_durations_bucket:rate1m)
+ - record: txn_durations:rate1m:quantile_99
+ expr: histogram_quantile(0.99, txn_durations_bucket:rate1m)
+ - record: exec_latency_bucket:rate1m
+ expr: rate(exec_latency_bucket{job="cockroachdb"}[1m])
+ - record: exec_latency:rate1m:quantile_50
+ expr: histogram_quantile(0.5, exec_latency_bucket:rate1m)
+ - record: exec_latency:rate1m:quantile_75
+ expr: histogram_quantile(0.75, exec_latency_bucket:rate1m)
+ - record: exec_latency:rate1m:quantile_90
+ expr: histogram_quantile(0.9, exec_latency_bucket:rate1m)
+ - record: exec_latency:rate1m:quantile_95
+ expr: histogram_quantile(0.95, exec_latency_bucket:rate1m)
+ - record: exec_latency:rate1m:quantile_99
+ expr: histogram_quantile(0.99, exec_latency_bucket:rate1m)
+ - record: round_trip_latency_bucket:rate1m
+ expr: rate(round_trip_latency_bucket{job="cockroachdb"}[1m])
+ - record: round_trip_latency:rate1m:quantile_50
+ expr: histogram_quantile(0.5, round_trip_latency_bucket:rate1m)
+ - record: round_trip_latency:rate1m:quantile_75
+ expr: histogram_quantile(0.75, round_trip_latency_bucket:rate1m)
+ - record: round_trip_latency:rate1m:quantile_90
+ expr: histogram_quantile(0.9, round_trip_latency_bucket:rate1m)
+ - record: round_trip_latency:rate1m:quantile_95
+ expr: histogram_quantile(0.95, round_trip_latency_bucket:rate1m)
+ - record: round_trip_latency:rate1m:quantile_99
+ expr: histogram_quantile(0.99, round_trip_latency_bucket:rate1m)
+ - record: sql_exec_latency_bucket:rate1m
+ expr: rate(sql_exec_latency_bucket{job="cockroachdb"}[1m])
+ - record: sql_exec_latency:rate1m:quantile_50
+ expr: histogram_quantile(0.5, sql_exec_latency_bucket:rate1m)
+ - record: sql_exec_latency:rate1m:quantile_75
+ expr: histogram_quantile(0.75, sql_exec_latency_bucket:rate1m)
+ - record: sql_exec_latency:rate1m:quantile_90
+ expr: histogram_quantile(0.9, sql_exec_latency_bucket:rate1m)
+ - record: sql_exec_latency:rate1m:quantile_95
+ expr: histogram_quantile(0.95, sql_exec_latency_bucket:rate1m)
+ - record: sql_exec_latency:rate1m:quantile_99
+ expr: histogram_quantile(0.99, sql_exec_latency_bucket:rate1m)
+ - record: raft_process_logcommit_latency_bucket:rate1m
+ expr: rate(raft_process_logcommit_latency_bucket{job="cockroachdb"}[1m])
+ - record: raft_process_logcommit_latency:rate1m:quantile_50
+ expr: histogram_quantile(0.5, raft_process_logcommit_latency_bucket:rate1m)
+ - record: raft_process_logcommit_latency:rate1m:quantile_75
+ expr: histogram_quantile(0.75, raft_process_logcommit_latency_bucket:rate1m)
+ - record: raft_process_logcommit_latency:rate1m:quantile_90
+ expr: histogram_quantile(0.9, raft_process_logcommit_latency_bucket:rate1m)
+ - record: raft_process_logcommit_latency:rate1m:quantile_95
+ expr: histogram_quantile(0.95, raft_process_logcommit_latency_bucket:rate1m)
+ - record: raft_process_logcommit_latency:rate1m:quantile_99
+ expr: histogram_quantile(0.99, raft_process_logcommit_latency_bucket:rate1m)
+ - record: raft_process_commandcommit_latency_bucket:rate1m
+ expr: rate(raft_process_commandcommit_latency_bucket{job="cockroachdb"}[1m])
+ - record: raft_process_commandcommit_latency:rate1m:quantile_50
+ expr: histogram_quantile(0.5, raft_process_commandcommit_latency_bucket:rate1m)
+ - record: raft_process_commandcommit_latency:rate1m:quantile_75
+ expr: histogram_quantile(0.75, raft_process_commandcommit_latency_bucket:rate1m)
+ - record: raft_process_commandcommit_latency:rate1m:quantile_90
+ expr: histogram_quantile(0.9, raft_process_commandcommit_latency_bucket:rate1m)
+ - record: raft_process_commandcommit_latency:rate1m:quantile_95
+ expr: histogram_quantile(0.95, raft_process_commandcommit_latency_bucket:rate1m)
+ - record: raft_process_commandcommit_latency:rate1m:quantile_99
+ expr: histogram_quantile(0.99, raft_process_commandcommit_latency_bucket:rate1m)
+ - record: storage_wal_fsync_latency_bucket:rate1m
+ expr: rate(storage_wal_fsync_latency_bucket{job="cockroachdb"}[1m])
+ - record: storage_wal_fsync_latency:rate1m:quantile_50
+ expr: histogram_quantile(0.5, storage_wal_fsync_latency_bucket:rate1m)
+ - record: storage_wal_fsync_latency:rate1m:quantile_75
+ expr: histogram_quantile(0.75, storage_wal_fsync_latency_bucket:rate1m)
+ - record: storage_wal_fsync_latency:rate1m:quantile_90
+ expr: histogram_quantile(0.9, storage_wal_fsync_latency_bucket:rate1m)
+ - record: storage_wal_fsync_latency:rate1m:quantile_95
+ expr: histogram_quantile(0.95, storage_wal_fsync_latency_bucket:rate1m)
+ - record: storage_wal_fsync_latency:rate1m:quantile_99
+ expr: histogram_quantile(0.99, storage_wal_fsync_latency_bucket:rate1m)
diff --git a/src/current/files/cockroach/monitoring/rules/alerts.rules.yml b/src/current/files/cockroach/monitoring/rules/alerts.rules.yml
new file mode 100644
index 00000000000..5d198a762bc
--- /dev/null
+++ b/src/current/files/cockroach/monitoring/rules/alerts.rules.yml
@@ -0,0 +1,157 @@
+groups:
+- name: rules/alerts.rules
+ rules:
+ # Alert for any instance that is unreachable for >15 minutes.
+ - alert: InstanceDead
+ expr: up{job="cockroachdb"} == 0
+ for: 15m
+ annotations:
+ description: '{{ $labels.instance }} for cluster {{ $labels.cluster }} has been
+ down for more than 15 minutes.'
+ summary: Instance {{ $labels.instance }} dead
+ # Alert for any instance that is not ready for a while.
+ - alert: InstanceNotReady
+ # This alert applies only to Kubernetes deployments and requires that you run kube-state-metrics: https://github.com/kubernetes/kube-state-metrics
+ expr: kube_statefulset_status_replicas_ready{statefulset="cockroachdb"} != kube_statefulset_status_replicas{statefulset="cockroachdb"}
+ for: 45m
+ annotations:
+ description: 'there has been an unready replica for cluster {{ $labels.cluster }}
+ for more than 15 minutes.'
+ summary: Instance not ready
+ # Alert on instance restarts.
+ - alert: InstanceRestart
+ expr: resets(sys_uptime{job="cockroachdb"}[24h]) > 1
+ annotations:
+ description: '{{ $labels.instance }} for cluster {{ $labels.cluster }} restarted
+ {{ $value }} time(s) in 24h'
+ summary: Instance {{ $labels.instance }} restarted
+ # Alert on flapping instances (frequent restarts).
+ - alert: InstancesFlapping
+ # Aggregated.
+ # This alert assumes that rolling restarts or rolling upgrades leave at least 3 minutes between each node being updated or restarted.
+ expr: sum by (cluster)(resets(sys_uptime{job="cockroachdb"}[5m])) > 2
+ annotations:
+ description: 'instances in cluster {{ $labels.cluster }} restarted
+ {{ $value }} time(s) in 5m'
+ summary: Instances in {{ $labels.cluster }} flapping
+ # Alert on flapping instances (frequent restarts).
+ - alert: InstanceFlapping
+ # Un-aggregated.
+ expr: resets(sys_uptime{job="cockroachdb"}[10m]) > 1
+ annotations:
+ description: '{{ $labels.instance }} for cluster {{ $labels.cluster }} restarted
+ {{ $value }} time(s) in 10m'
+ summary: Instance {{ $labels.instance }} flapping
+ # Alert on version mismatch.
+ # This alert is intentionally loose (4 hours) to allow for rolling upgrades.
+ # This may need to be adjusted for large clusters.
+ - alert: VersionMismatch
+ expr: count by(cluster) (count_values by(tag, cluster) ("version", build_timestamp{job="cockroachdb"}))
+ > 1
+ for: 4h
+ annotations:
+ description: Cluster {{ $labels.cluster }} running {{ $value }} different versions
+ summary: Binary version mismatch on {{ $labels.cluster }}
+ # Available capacity alerts.
+ - alert: StoreDiskLow
+ expr: capacity_available:ratio{job="cockroachdb"} < 0.15
+ annotations:
+ summary: Store {{ $labels.store }} on node {{ $labels.instance }} at {{ $value
+ }} available disk fraction
+ - alert: ClusterDiskLow
+ expr: cluster:capacity_available:ratio{job="cockroachdb"} < 0.2
+ annotations:
+ summary: Cluster {{ $labels.cluster }} at {{ $value }} available disk fraction
+ # Unavailable ranges.
+ - alert: UnavailableRanges
+ expr: (sum by(instance, cluster) (ranges_unavailable{job="cockroachdb"})) > 0
+ for: 10m
+ annotations:
+ summary: Instance {{ $labels.instance }} has {{ $value }} unavailable ranges
+ # Cockroach-measured clock offset nearing limit (by default, servers kill themselves at 400ms from the mean, so alert at 300ms)
+ - alert: ClockOffsetNearMax
+ expr: clock_offset_meannanos{job="cockroachdb"} > 300 * 1000 * 1000
+ for: 5m
+ annotations:
+ summary: Clock on {{ $labels.instance }} as measured by cockroach is offset by {{ $value }} nanoseconds from the cluster mean # Certificate expiration. Alerts are per node.
+ - alert: CACertificateExpiresSoon
+ expr: (security_certificate_expiration_ca{job="cockroachdb"} > 0) and (security_certificate_expiration_ca{job="cockroachdb"}
+ - time()) < 86400 * 366
+ labels:
+ frequency: daily
+ annotations:
+ summary: CA certificate for {{ $labels.instance }} expires in less than a year
+ - alert: ClientCACertificateExpiresSoon
+ expr: (security_certificate_expiration_client_ca{job="cockroachdb"} > 0) and (security_certificate_expiration_client_ca{job="cockroachdb"}
+ - time()) < 86400 * 366
+ labels:
+ frequency: daily
+ annotations:
+ summary: Client CA certificate for {{ $labels.instance }} expires in less than a year
+ - alert: UICACertificateExpiresSoon
+ expr: (security_certificate_expiration_ui_ca{job="cockroachdb"} > 0) and (security_certificate_expiration_ui_ca{job="cockroachdb"}
+ - time()) < 86400 * 366
+ labels:
+ frequency: daily
+ annotations:
+ summary: UI CA certificate for {{ $labels.instance }} expires in less than a year
+ - alert: NodeCertificateExpiresSoon
+ expr: (security_certificate_expiration_node{job="cockroachdb"} > 0) and (security_certificate_expiration_node{job="cockroachdb"}
+ - time()) < 86400 * 183
+ labels:
+ frequency: daily
+ annotations:
+ summary: Node certificate for {{ $labels.instance }} expires in less than six months
+ - alert: NodeClientCertificateExpiresSoon
+ expr: (security_certificate_expiration_node_client{job="cockroachdb"} > 0) and (security_certificate_expiration_node_client{job="cockroachdb"}
+ - time()) < 86400 * 183
+ labels:
+ frequency: daily
+ annotations:
+ summary: Client certificate for {{ $labels.instance }} expires in less than six months
+ - alert: UICertificateExpiresSoon
+ expr: (security_certificate_expiration_ui{job="cockroachdb"} > 0) and (security_certificate_expiration_ui{job="cockroachdb"}
+ - time()) < 86400 * 20
+ labels:
+ frequency: daily
+ annotations:
+ summary: UI certificate for {{ $labels.instance }} expires in less than 20 days
+ # Slow Latch/Lease/Raft requests.
+ - alert: SlowLatchRequest
+ expr: requests_slow_latch{job="cockroachdb"} > 0
+ for: 5m
+ labels:
+ severity: testing
+ annotations:
+ summary: '{{ $value }} slow latch requests on {{ $labels.instance }}'
+ - alert: SlowLeaseRequest
+ expr: requests_slow_lease{job="cockroachdb"} > 0
+ for: 5m
+ labels:
+ severity: testing
+ annotations:
+ summary: '{{ $value }} slow lease requests on {{ $labels.instance }}'
+ - alert: SlowRaftRequest
+ expr: requests_slow_raft{job="cockroachdb"} > 0
+ for: 5m
+ labels:
+ severity: testing
+ annotations:
+ summary: '{{ $value }} slow raft requests on {{ $labels.instance }}'
+ # Getting close to open file descriptor limit.
+ - alert: HighOpenFDCount
+ expr: sys_fd_open{job="cockroachdb"} / sys_fd_softlimit{job="cockroachdb"} > 0.8
+ for: 10m
+ annotations:
+ summary: 'Too many open file descriptors on {{ $labels.instance }}: {{ $value
+ }} fraction used'
+ # Prometheus disk getting full.
+ - alert: PrometheusDiskLow
+ expr: node_filesystem_free{cluster="prometheus",job="node_exporter_prometheus",mountpoint="/data"}
+ / node_filesystem_size{cluster="prometheus",job="node_exporter_prometheus",mountpoint="/data"}
+ < 0.2
+ for: 10m
+ labels:
+ severity: testing
+ annotations:
+ summary: 'Prometheus storage is almost full: {{ $value }} fraction free'
\ No newline at end of file
diff --git a/src/current/v23.1/monitor-cockroachdb-kubernetes.md b/src/current/v23.1/monitor-cockroachdb-kubernetes.md
index c36cdcabae0..6249e1f82b1 100644
--- a/src/current/v23.1/monitor-cockroachdb-kubernetes.md
+++ b/src/current/v23.1/monitor-cockroachdb-kubernetes.md
@@ -128,7 +128,7 @@ If you're on Hosted GKE, before starting, make sure the email address associated
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/prometheus.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/prometheus.yaml
~~~
{{site.data.alerts.callout_info}}
@@ -177,14 +177,14 @@ If you're on Hosted GKE, before starting, make sure the email address associated
## Configure Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
-1. Download our alertmanager-config.yaml configuration file:
+1. Download our alertmanager-config.yaml configuration file:
{% include_cached copy-clipboard.html %}
~~~ shell
$ curl -O \
- https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager-config.yaml
+ https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager-config.yaml
~~~
1. Edit the `alertmanager-config.yaml` file to [specify the desired receivers for notifications](https://prometheus.io/docs/alerting/configuration/#receiver). Initially, the file contains a placeholder web hook.
@@ -214,12 +214,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
The name of the secret, `alertmanager-cockroachdb`, must match the name used in the `alertmanager.yaml` file. If they differ, the Alertmanager instance will start without configuration, and nothing will happen.
{{site.data.alerts.end}}
-1. Use our [`alertmanager.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
+1. Use our [`alertmanager.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml
~~~
~~~
@@ -244,12 +244,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
-1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml):
+1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alert-rules.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml
~~~
~~~
diff --git a/src/current/v23.1/monitor-cockroachdb-with-prometheus.md b/src/current/v23.1/monitor-cockroachdb-with-prometheus.md
index 493b6cce1c0..0f59d33c8d5 100644
--- a/src/current/v23.1/monitor-cockroachdb-with-prometheus.md
+++ b/src/current/v23.1/monitor-cockroachdb-with-prometheus.md
@@ -15,7 +15,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
- Make sure you have already started a CockroachDB cluster, either [locally]({% link {{ page.version.version }}/start-a-local-cluster.md %}) or in a [production environment]({% link {{ page.version.version }}/manual-deployment.md %}).
-- Note that all files used in this tutorial can be found in the [`monitoring`](https://github.com/cockroachdb/cockroach/tree/master/monitoring) directory of the CockroachDB repository.
+- Note that all files used in this tutorial can be found in the `monitoring` directory of the CockroachDB repository.
## Step 1. Install Prometheus
@@ -39,11 +39,11 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
## Step 2. Configure Prometheus
-1. Download the starter [Prometheus configuration file](https://github.com/cockroachdb/cockroach/blob/master/monitoring/prometheus.yml) for CockroachDB:
+1. Download the starter [Prometheus configuration file](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/prometheus.yml) for CockroachDB:
{% include_cached copy-clipboard.html %}
~~~ shell
- $ wget https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/prometheus.yml \
+ $ wget https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/prometheus.yml \
-O prometheus.yml
~~~
@@ -61,7 +61,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
Production cluster | Change the `targets` field to include `':'` for each node in the cluster. Also, be sure your network configuration allows TCP communication on the specified ports.
Secure cluster | Uncomment `scheme: 'https'` and comment out `scheme: 'http'`.
-1. Create a `rules` directory and download the [aggregation rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/aggregation.rules.yml) and [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/alerts.rules.yml) for CockroachDB into it:
+1. Create a `rules` directory and download the [aggregation rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/aggregation.rules.yml) and [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml) for CockroachDB into it:
{% include_cached copy-clipboard.html %}
~~~ shell
@@ -75,12 +75,12 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
{% include_cached copy-clipboard.html %}
~~~ shell
- $ wget -P rules https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/rules/aggregation.rules.yml
+ $ wget -P rules https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/rules/aggregation.rules.yml
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- $ wget -P rules https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/rules/alerts.rules.yml
+ $ wget -P rules https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml
~~~
## Step 3. Start Prometheus
@@ -109,7 +109,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
## Step 4. Send notifications with Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. In step 2, you already downloaded CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/alerts.rules.yml). Now, download, configure, and start [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/).
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. In step 2, you already downloaded CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml). Now, download, configure, and start [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/).
1. Download the [latest Alertmanager tarball](https://prometheus.io/download/#alertmanager) for your OS.
@@ -174,29 +174,29 @@ Although Prometheus lets you graph metrics, [Grafana](https://grafana.com/) is a
Url | `http://:9090`
Access | Direct
-1. Download the starter [Grafana dashboards](https://github.com/cockroachdb/cockroach/tree/master/monitoring/grafana-dashboards/by-cluster) for CockroachDB:
+1. Download the starter Grafana dashboards for CockroachDB:
{% include_cached copy-clipboard.html %}
~~~ shell
- $ wget https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/runtime.json
+ $ wget https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/runtime.json
# runtime dashboard: node status, including uptime, memory, and cpu.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- $ wget https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/storage.json
+ $ wget https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/storage.json
# storage dashboard: storage availability.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- $ wget https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/sql.json
+ $ wget https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/sql.json
# sql dashboard: sql queries/transactions.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- $ wget https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/replication.json
+ $ wget https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/replication.json
# replicas dashboard: replica information and operations.
~~~
diff --git a/src/current/v23.1/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md b/src/current/v23.1/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
index 272d50f856a..2a37d12a609 100644
--- a/src/current/v23.1/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
+++ b/src/current/v23.1/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
@@ -71,7 +71,7 @@ Instead of using this approach, you can now [enable global access](https://cloud
Our multi-region deployment approach relies on pod IP addresses being routable across three distinct Kubernetes clusters and regions. Both the hosted Google Kubernetes Engine (GKE) and Amazon Elastic Kubernetes Service (EKS) satisfy this requirement.
-If you want to run on another cloud or on-premises, use this [basic network test](https://github.com/cockroachdb/cockroach/tree/master/cloud/kubernetes/multiregion#pod-to-pod-connectivity) to see if it will work.
+If you want to run on another cloud or on-premises, use this basic network test to see if it will work.
1. Complete the **Before You Begin** steps described in the [Google Kubernetes Engine Quickstart](https://cloud.google.com/kubernetes-engine/docs/quickstart) documentation.
@@ -322,21 +322,21 @@ This important rule enables node communication between Kubernetes clusters in di
The Kubernetes cluster in each region needs to have a [Network Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html) pointed at its CoreDNS service, which you will configure in the next step.
-1. Upload our load balancer manifest [`dns-lb-eks.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml) to the Kubernetes clusters in all 3 regions:
+1. Upload our load balancer manifest [`dns-lb-eks.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml) to the Kubernetes clusters in all 3 regions:
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
You should see the load balancer appear in the Load Balancers section of the EC2 console in each region. This load balancer will route traffic to CoreDNS in the region.
@@ -369,11 +369,11 @@ Each Kubernetes cluster has a [CoreDNS](https://coredns.io/) service that respon
To enable traffic forwarding to CockroachDB pods in all 3 regions, you need to [modify the ConfigMap](https://kubernetes.io/docs/tasks/administer-cluster/dns-custom-nameservers/#coredns-configmap-options) for the CoreDNS Corefile in each region.
-1. Download and open our ConfigMap template [`configmap.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/configmap.yaml):
+1. Download and open our ConfigMap template [`configmap.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/configmap.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/configmap.yaml
+ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/configmap.yaml
~~~
1. After [obtaining the IP addresses of the Network Load Balancers](#set-up-load-balancing) in all 3 regions, you can use this information to define a **separate ConfigMap for each region**. Each unique ConfigMap lists the forwarding addresses for the pods in the 2 other regions.
@@ -467,7 +467,7 @@ If you plan to run your instances exclusively on private subnets, set the follow
{% include_cached copy-clipboard.html %}
~~~ shell
$ curl -OOOOOOOOO \
- https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/{README.md,client-secure.yaml,cluster-init-secure.yaml,cockroachdb-statefulset-secure.yaml,dns-lb.yaml,example-app-secure.yaml,external-name-svc.yaml,setup.py,teardown.py}
+ https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/{README.md,client-secure.yaml,cluster-init-secure.yaml,cockroachdb-statefulset-secure.yaml,dns-lb.yaml,example-app-secure.yaml,external-name-svc.yaml,setup.py,teardown.py}
~~~
1. Retrieve the `kubectl` "contexts" for your clusters:
@@ -685,11 +685,11 @@ The below steps use [`cockroach cert` commands]({% link {{ page.version.version
### Create StatefulSets
-1. Download and open our [multi-region StatefulSet configuration](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml). You'll save three versions of this file locally, one for each set of 3 CockroachDB nodes per region.
+1. Download and open our [multi-region StatefulSet configuration](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml). You'll save three versions of this file locally, one for each set of 3 CockroachDB nodes per region.
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml
~~~
Look for **TODO** comments in the file. These highlight fields you need to define before deploying your StatefulSet.
@@ -814,7 +814,7 @@ The pod uses the `root` client certificate created earlier by the `setup.py` scr
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl create -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/client-secure.yaml --context --namespace
+ kubectl create -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/client-secure.yaml --context --namespace
~~~
~~~
diff --git a/src/current/v23.2/monitor-cockroachdb-kubernetes.md b/src/current/v23.2/monitor-cockroachdb-kubernetes.md
index d2ca66e397f..072c93c275f 100644
--- a/src/current/v23.2/monitor-cockroachdb-kubernetes.md
+++ b/src/current/v23.2/monitor-cockroachdb-kubernetes.md
@@ -128,7 +128,7 @@ If you're on Hosted GKE, before starting, make sure the email address associated
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/prometheus.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/prometheus.yaml
~~~
{{site.data.alerts.callout_info}}
@@ -177,14 +177,14 @@ If you're on Hosted GKE, before starting, make sure the email address associated
## Configure Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
-1. Download our alertmanager-config.yaml configuration file:
+1. Download our alertmanager-config.yaml configuration file:
{% include_cached copy-clipboard.html %}
~~~ shell
$ curl -O \
- https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager-config.yaml
+ https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager-config.yaml
~~~
1. Edit the `alertmanager-config.yaml` file to [specify the desired receivers for notifications](https://prometheus.io/docs/alerting/configuration/#receiver). Initially, the file contains a placeholder web hook.
@@ -214,12 +214,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
The name of the secret, `alertmanager-cockroachdb`, must match the name used in the `alertmanager.yaml` file. If they differ, the Alertmanager instance will start without configuration, and nothing will happen.
{{site.data.alerts.end}}
-1. Use our [`alertmanager.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
+1. Use our [`alertmanager.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml
~~~
~~~
@@ -244,12 +244,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
-1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml):
+1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alert-rules.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml
~~~
~~~
diff --git a/src/current/v23.2/monitor-cockroachdb-with-prometheus.md b/src/current/v23.2/monitor-cockroachdb-with-prometheus.md
index 7f4d671aa52..8b7f0876475 100644
--- a/src/current/v23.2/monitor-cockroachdb-with-prometheus.md
+++ b/src/current/v23.2/monitor-cockroachdb-with-prometheus.md
@@ -15,7 +15,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
- Make sure you have already started a CockroachDB cluster, either [locally]({% link {{ page.version.version }}/start-a-local-cluster.md %}) or in a [production environment]({% link {{ page.version.version }}/manual-deployment.md %}).
-- Note that all files used in this tutorial can be found in the [`monitoring`](https://github.com/cockroachdb/cockroach/tree/master/monitoring) directory of the CockroachDB repository.
+- Note that all files used in this tutorial can be found in the `monitoring` directory of the CockroachDB repository.
## Step 1. Install Prometheus
@@ -39,11 +39,11 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
## Step 2. Configure Prometheus
-1. Download the starter [Prometheus configuration file](https://github.com/cockroachdb/cockroach/blob/master/monitoring/prometheus.yml) for CockroachDB:
+1. Download the starter [Prometheus configuration file](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/prometheus.yml) for CockroachDB:
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o prometheus.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/prometheus.yml
+ curl -o prometheus.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/prometheus.yml
~~~
When you examine the configuration file, you'll see that it is set up to scrape the time series metrics of a single, insecure local node every 10 seconds:
@@ -60,7 +60,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
Production cluster | Change the `targets` field to include `':'` for each node in the cluster. Also, be sure your network configuration allows TCP communication on the specified ports.
Secure cluster | Uncomment `scheme: 'https'` and comment out `scheme: 'http'`.
-1. Create a `rules` directory and download the [aggregation rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/aggregation.rules.yml) and [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/alerts.rules.yml) for CockroachDB into it:
+1. Create a `rules` directory and download the [aggregation rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/aggregation.rules.yml) and [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml) for CockroachDB into it:
{% include_cached copy-clipboard.html %}
~~~ shell
@@ -74,12 +74,12 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o rules/aggregation.rules.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/rules/aggregation.rules.yml
+ curl -o rules/aggregation.rules.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/rules/aggregation.rules.yml
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o rules/alerts-rules.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/rules/alerts.rules.yml
+ curl -o rules/alerts-rules.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml
~~~
## Step 3. Start Prometheus
@@ -108,7 +108,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
## Step 4. Send notifications with Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. In step 2, you already downloaded CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/alerts.rules.yml). Now, download, configure, and start [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/).
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. In step 2, you already downloaded CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml). Now, download, configure, and start [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/).
1. Download the [latest Alertmanager tarball](https://prometheus.io/download/#alertmanager) for your OS.
@@ -173,29 +173,29 @@ Although Prometheus lets you graph metrics, [Grafana](https://grafana.com/) is a
Url | `http://:9090`
Access | Direct
-1. Download the starter [Grafana dashboards](https://github.com/cockroachdb/cockroach/tree/master/monitoring/grafana-dashboards/by-cluster) for CockroachDB:
+1. Download the starter Grafana dashboards for CockroachDB:
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o runtime.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/runtime.json
+ curl -o runtime.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/runtime.json
# runtime dashboard: node status, including uptime, memory, and cpu.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o storage.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/storage.json
+ curl -o storage.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/storage.json
# storage dashboard: storage availability.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o sql.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/sql.json
+ curl -o sql.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/sql.json
# sql dashboard: sql queries/transactions.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o replication.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/replication.json
+ curl -o replication.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/replication.json
# replicas dashboard: replica information and operations.
~~~
diff --git a/src/current/v23.2/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md b/src/current/v23.2/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
index 2ebfe678a0a..9d3334e3962 100644
--- a/src/current/v23.2/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
+++ b/src/current/v23.2/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
@@ -71,7 +71,7 @@ Instead of using this approach, you can now [enable global access](https://cloud
Our multi-region deployment approach relies on pod IP addresses being routable across three distinct Kubernetes clusters and regions. Both the hosted Google Kubernetes Engine (GKE) and Amazon Elastic Kubernetes Service (EKS) satisfy this requirement.
-If you want to run on another cloud or on-premises, use this [basic network test](https://github.com/cockroachdb/cockroach/tree/master/cloud/kubernetes/multiregion#pod-to-pod-connectivity) to see if it will work.
+If you want to run on another cloud or on-premises, use this basic network test to see if it will work.
1. Complete the **Before You Begin** steps described in the [Google Kubernetes Engine Quickstart](https://cloud.google.com/kubernetes-engine/docs/quickstart) documentation.
@@ -322,21 +322,21 @@ This important rule enables node communication between Kubernetes clusters in di
The Kubernetes cluster in each region needs to have a [Network Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html) pointed at its CoreDNS service, which you will configure in the next step.
-1. Upload our load balancer manifest [`dns-lb-eks.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml) to the Kubernetes clusters in all 3 regions:
+1. Upload our load balancer manifest [`dns-lb-eks.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml) to the Kubernetes clusters in all 3 regions:
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
You should see the load balancer appear in the Load Balancers section of the EC2 console in each region. This load balancer will route traffic to CoreDNS in the region.
@@ -369,11 +369,11 @@ Each Kubernetes cluster has a [CoreDNS](https://coredns.io/) service that respon
To enable traffic forwarding to CockroachDB pods in all 3 regions, you need to [modify the ConfigMap](https://kubernetes.io/docs/tasks/administer-cluster/dns-custom-nameservers/#coredns-configmap-options) for the CoreDNS Corefile in each region.
-1. Download and open our ConfigMap template [`configmap.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/configmap.yaml):
+1. Download and open our ConfigMap template [`configmap.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/configmap.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/configmap.yaml
+ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/configmap.yaml
~~~
1. After [obtaining the IP addresses of the Network Load Balancers](#set-up-load-balancing) in all 3 regions, you can use this information to define a **separate ConfigMap for each region**. Each unique ConfigMap lists the forwarding addresses for the pods in the 2 other regions.
@@ -467,7 +467,7 @@ If you plan to run your instances exclusively on private subnets, set the follow
{% include_cached copy-clipboard.html %}
~~~ shell
$ curl -OOOOOOOOO \
- https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/{README.md,client-secure.yaml,cluster-init-secure.yaml,cockroachdb-statefulset-secure.yaml,dns-lb.yaml,example-app-secure.yaml,external-name-svc.yaml,setup.py,teardown.py}
+ https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/{README.md,client-secure.yaml,cluster-init-secure.yaml,cockroachdb-statefulset-secure.yaml,dns-lb.yaml,example-app-secure.yaml,external-name-svc.yaml,setup.py,teardown.py}
~~~
1. Retrieve the `kubectl` "contexts" for your clusters:
@@ -685,11 +685,11 @@ The below steps use [`cockroach cert` commands]({% link {{ page.version.version
### Create StatefulSets
-1. Download and open our [multi-region StatefulSet configuration](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml). You'll save three versions of this file locally, one for each set of 3 CockroachDB nodes per region.
+1. Download and open our [multi-region StatefulSet configuration](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml). You'll save three versions of this file locally, one for each set of 3 CockroachDB nodes per region.
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml
~~~
Look for **TODO** comments in the file. These highlight fields you need to define before deploying your StatefulSet.
@@ -814,7 +814,7 @@ The pod uses the `root` client certificate created earlier by the `setup.py` scr
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl create -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/client-secure.yaml --context --namespace
+ kubectl create -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/client-secure.yaml --context --namespace
~~~
~~~
diff --git a/src/current/v24.1/monitor-cockroachdb-kubernetes.md b/src/current/v24.1/monitor-cockroachdb-kubernetes.md
index b6af4eab828..a175f0eefaa 100644
--- a/src/current/v24.1/monitor-cockroachdb-kubernetes.md
+++ b/src/current/v24.1/monitor-cockroachdb-kubernetes.md
@@ -128,7 +128,7 @@ If you're on Hosted GKE, before starting, make sure the email address associated
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/prometheus.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/prometheus.yaml
~~~
{{site.data.alerts.callout_info}}
@@ -177,14 +177,14 @@ If you're on Hosted GKE, before starting, make sure the email address associated
## Configure Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
-1. Download our alertmanager-config.yaml configuration file:
+1. Download our alertmanager-config.yaml configuration file:
{% include_cached copy-clipboard.html %}
~~~ shell
$ curl -O \
- https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager-config.yaml
+ https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager-config.yaml
~~~
1. Edit the `alertmanager-config.yaml` file to [specify the desired receivers for notifications](https://prometheus.io/docs/alerting/configuration/#receiver). Initially, the file contains a placeholder web hook.
@@ -214,12 +214,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
The name of the secret, `alertmanager-cockroachdb`, must match the name used in the `alertmanager.yaml` file. If they differ, the Alertmanager instance will start without configuration, and nothing will happen.
{{site.data.alerts.end}}
-1. Use our [`alertmanager.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
+1. Use our [`alertmanager.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml
~~~
~~~
@@ -244,12 +244,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
-1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml):
+1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alert-rules.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml
~~~
~~~
diff --git a/src/current/v24.1/monitor-cockroachdb-with-prometheus.md b/src/current/v24.1/monitor-cockroachdb-with-prometheus.md
index ad08d818b4e..2c661d20b66 100644
--- a/src/current/v24.1/monitor-cockroachdb-with-prometheus.md
+++ b/src/current/v24.1/monitor-cockroachdb-with-prometheus.md
@@ -15,7 +15,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
- Make sure you have already started a CockroachDB cluster, either [locally]({% link {{ page.version.version }}/start-a-local-cluster.md %}) or in a [production environment]({% link {{ page.version.version }}/manual-deployment.md %}).
-- Note that all files used in this tutorial can be found in the [`monitoring`](https://github.com/cockroachdb/cockroach/tree/master/monitoring) directory of the CockroachDB repository.
+- Note that all files used in this tutorial can be found in the `monitoring` directory of the CockroachDB repository.
## Step 1. Install Prometheus
@@ -39,11 +39,11 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
## Step 2. Configure Prometheus
-1. Download the starter [Prometheus configuration file](https://github.com/cockroachdb/cockroach/blob/master/monitoring/prometheus.yml) for CockroachDB:
+1. Download the starter [Prometheus configuration file](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/prometheus.yml) for CockroachDB:
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o prometheus.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/prometheus.yml
+ curl -o prometheus.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/prometheus.yml
~~~
When you examine the configuration file, you'll see that it is set up to scrape the time series metrics of a single, insecure local node every 10 seconds:
@@ -60,7 +60,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
Production cluster | Change the `targets` field to include `':'` for each node in the cluster. Also, be sure your network configuration allows TCP communication on the specified ports.
Secure cluster | Uncomment `scheme: 'https'` and comment out `scheme: 'http'`.
-1. Create a `rules` directory and download the [aggregation rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/aggregation.rules.yml) and [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/alerts.rules.yml) for CockroachDB into it:
+1. Create a `rules` directory and download the [aggregation rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/aggregation.rules.yml) and [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml) for CockroachDB into it:
{% include_cached copy-clipboard.html %}
~~~ shell
@@ -74,12 +74,12 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o rules/aggregation.rules.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/rules/aggregation.rules.yml
+ curl -o rules/aggregation.rules.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/rules/aggregation.rules.yml
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o rules/alerts.rules.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/rules/alerts.rules.yml
+ curl -o rules/alerts.rules.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml
~~~
## Step 3. Start Prometheus
@@ -108,7 +108,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
## Step 4. Send notifications with Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. In step 2, you already downloaded CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/alerts.rules.yml). Now, download, configure, and start [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/).
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. In step 2, you already downloaded CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml). Now, download, configure, and start [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/).
1. Download the [latest Alertmanager tarball](https://prometheus.io/download/#alertmanager) for your OS.
@@ -173,29 +173,29 @@ Although Prometheus lets you graph metrics, [Grafana](https://grafana.com/) is a
Url | `http://:9090`
Access | Direct
-1. Download the starter [Grafana dashboards](https://github.com/cockroachdb/cockroach/tree/master/monitoring/grafana-dashboards/by-cluster) for CockroachDB:
+1. Download the starter Grafana dashboards for CockroachDB:
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o runtime.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/runtime.json
+ curl -o runtime.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/runtime.json
# runtime dashboard: node status, including uptime, memory, and cpu.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o storage.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/storage.json
+ curl -o storage.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/storage.json
# storage dashboard: storage availability.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o sql.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/sql.json
+ curl -o sql.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/sql.json
# sql dashboard: sql queries/transactions.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o replication.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/replication.json
+ curl -o replication.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/replication.json
# replicas dashboard: replica information and operations.
~~~
diff --git a/src/current/v24.1/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md b/src/current/v24.1/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
index b47593bda96..09b361cd5ac 100644
--- a/src/current/v24.1/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
+++ b/src/current/v24.1/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
@@ -71,7 +71,7 @@ Instead of using this approach, you can now [enable global access](https://cloud
Our multi-region deployment approach relies on pod IP addresses being routable across three distinct Kubernetes clusters and regions. Both the hosted Google Kubernetes Engine (GKE) and Amazon Elastic Kubernetes Service (EKS) satisfy this requirement.
-If you want to run on another cloud or on-premises, use this [basic network test](https://github.com/cockroachdb/cockroach/tree/master/cloud/kubernetes/multiregion#pod-to-pod-connectivity) to see if it will work.
+If you want to run on another cloud or on-premises, use this basic network test to see if it will work.
1. Complete the **Before You Begin** steps described in the [Google Kubernetes Engine Quickstart](https://cloud.google.com/kubernetes-engine/docs/quickstart) documentation.
@@ -322,21 +322,21 @@ This important rule enables node communication between Kubernetes clusters in di
The Kubernetes cluster in each region needs to have a [Network Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html) pointed at its CoreDNS service, which you will configure in the next step.
-1. Upload our load balancer manifest [`dns-lb-eks.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml) to the Kubernetes clusters in all 3 regions:
+1. Upload our load balancer manifest [`dns-lb-eks.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml) to the Kubernetes clusters in all 3 regions:
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
You should see the load balancer appear in the Load Balancers section of the EC2 console in each region. This load balancer will route traffic to CoreDNS in the region.
@@ -369,11 +369,11 @@ Each Kubernetes cluster has a [CoreDNS](https://coredns.io/) service that respon
To enable traffic forwarding to CockroachDB pods in all 3 regions, you need to [modify the ConfigMap](https://kubernetes.io/docs/tasks/administer-cluster/dns-custom-nameservers/#coredns-configmap-options) for the CoreDNS Corefile in each region.
-1. Download and open our ConfigMap template [`configmap.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/configmap.yaml):
+1. Download and open our ConfigMap template [`configmap.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/configmap.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/configmap.yaml
+ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/configmap.yaml
~~~
1. After [obtaining the IP addresses of the Network Load Balancers](#set-up-load-balancing) in all 3 regions, you can use this information to define a **separate ConfigMap for each region**. Each unique ConfigMap lists the forwarding addresses for the pods in the 2 other regions.
@@ -467,7 +467,7 @@ If you plan to run your instances exclusively on private subnets, set the follow
{% include_cached copy-clipboard.html %}
~~~ shell
$ curl -OOOOOOOOO \
- https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/{README.md,client-secure.yaml,cluster-init-secure.yaml,cockroachdb-statefulset-secure.yaml,dns-lb.yaml,example-app-secure.yaml,external-name-svc.yaml,setup.py,teardown.py}
+ https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/{README.md,client-secure.yaml,cluster-init-secure.yaml,cockroachdb-statefulset-secure.yaml,dns-lb.yaml,example-app-secure.yaml,external-name-svc.yaml,setup.py,teardown.py}
~~~
1. Retrieve the `kubectl` "contexts" for your clusters:
@@ -685,11 +685,11 @@ The below steps use [`cockroach cert` commands]({% link {{ page.version.version
### Create StatefulSets
-1. Download and open our [multi-region StatefulSet configuration](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml). You'll save three versions of this file locally, one for each set of 3 CockroachDB nodes per region.
+1. Download and open our [multi-region StatefulSet configuration](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml). You'll save three versions of this file locally, one for each set of 3 CockroachDB nodes per region.
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml
~~~
Look for **TODO** comments in the file. These highlight fields you need to define before deploying your StatefulSet.
@@ -814,7 +814,7 @@ The pod uses the `root` client certificate created earlier by the `setup.py` scr
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl create -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/client-secure.yaml --context --namespace
+ kubectl create -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/client-secure.yaml --context --namespace
~~~
~~~
diff --git a/src/current/v24.2/monitor-cockroachdb-kubernetes.md b/src/current/v24.2/monitor-cockroachdb-kubernetes.md
index b800eb696dd..7aafea11e7e 100644
--- a/src/current/v24.2/monitor-cockroachdb-kubernetes.md
+++ b/src/current/v24.2/monitor-cockroachdb-kubernetes.md
@@ -128,7 +128,7 @@ If you're on Hosted GKE, before starting, make sure the email address associated
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/prometheus.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/prometheus.yaml
~~~
{{site.data.alerts.callout_info}}
@@ -177,14 +177,14 @@ If you're on Hosted GKE, before starting, make sure the email address associated
## Configure Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
-1. Download our alertmanager-config.yaml configuration file:
+1. Download our alertmanager-config.yaml configuration file:
{% include_cached copy-clipboard.html %}
~~~ shell
$ curl -O \
- https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager-config.yaml
+ https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager-config.yaml
~~~
1. Edit the `alertmanager-config.yaml` file to [specify the desired receivers for notifications](https://prometheus.io/docs/alerting/configuration/#receiver). Initially, the file contains a placeholder web hook.
@@ -214,12 +214,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
The name of the secret, `alertmanager-cockroachdb`, must match the name used in the `alertmanager.yaml` file. If they differ, the Alertmanager instance will start without configuration, and nothing will happen.
{{site.data.alerts.end}}
-1. Use our [`alertmanager.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
+1. Use our [`alertmanager.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml
~~~
~~~
@@ -244,12 +244,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
-1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml):
+1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alert-rules.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml
~~~
~~~
diff --git a/src/current/v24.2/monitor-cockroachdb-with-prometheus.md b/src/current/v24.2/monitor-cockroachdb-with-prometheus.md
index 4d391044080..b8d89a519c2 100644
--- a/src/current/v24.2/monitor-cockroachdb-with-prometheus.md
+++ b/src/current/v24.2/monitor-cockroachdb-with-prometheus.md
@@ -15,7 +15,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
- Make sure you have already started a CockroachDB cluster, either [locally]({% link {{ page.version.version }}/start-a-local-cluster.md %}) or in a [production environment]({% link {{ page.version.version }}/manual-deployment.md %}).
-- Note that all files used in this tutorial can be found in the [`monitoring`](https://github.com/cockroachdb/cockroach/tree/master/monitoring) directory of the CockroachDB repository.
+- Note that all files used in this tutorial can be found in the `monitoring` directory of the CockroachDB repository.
## Step 1. Install Prometheus
@@ -39,11 +39,11 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
## Step 2. Configure Prometheus
-1. Download the starter [Prometheus configuration file](https://github.com/cockroachdb/cockroach/blob/master/monitoring/prometheus.yml) for CockroachDB:
+1. Download the starter [Prometheus configuration file](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/prometheus.yml) for CockroachDB:
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o prometheus.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/prometheus.yml
+ curl -o prometheus.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/prometheus.yml
~~~
When you examine the configuration file, you'll see that it is set up to scrape the time series metrics of a single, insecure local node every 10 seconds:
@@ -60,7 +60,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
Production cluster | Change the `targets` field to include `':'` for each node in the cluster. Also, be sure your network configuration allows TCP communication on the specified ports.
Secure cluster | Uncomment `scheme: 'https'` and comment out `scheme: 'http'`.
-1. Create a `rules` directory and download the [aggregation rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/aggregation.rules.yml) and [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/alerts.rules.yml) for CockroachDB into it:
+1. Create a `rules` directory and download the [aggregation rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/aggregation.rules.yml) and [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml) for CockroachDB into it:
{% include_cached copy-clipboard.html %}
~~~ shell
@@ -74,12 +74,12 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o rules/aggregation.rules.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/rules/aggregation.rules.yml
+ curl -o rules/aggregation.rules.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/rules/aggregation.rules.yml
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o rules/alerts.rules.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/rules/alerts.rules.yml
+ curl -o rules/alerts.rules.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml
~~~
## Step 3. Start Prometheus
@@ -108,7 +108,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
## Step 4. Send notifications with Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. In step 2, you already downloaded CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/alerts.rules.yml). Now, download, configure, and start [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/).
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. In step 2, you already downloaded CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml). Now, download, configure, and start [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/).
1. Download the [latest Alertmanager tarball](https://prometheus.io/download/#alertmanager) for your OS.
@@ -173,29 +173,29 @@ Although Prometheus lets you graph metrics, [Grafana](https://grafana.com/) is a
Url | `http://:9090`
Access | Direct
-1. Download the starter [Grafana dashboards](https://github.com/cockroachdb/cockroach/tree/master/monitoring/grafana-dashboards/by-cluster) for CockroachDB:
+1. Download the starter Grafana dashboards for CockroachDB:
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o runtime.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/runtime.json
+ curl -o runtime.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/runtime.json
# runtime dashboard: node status, including uptime, memory, and cpu.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o storage.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/storage.json
+ curl -o storage.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/storage.json
# storage dashboard: storage availability.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o sql.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/sql.json
+ curl -o sql.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/sql.json
# sql dashboard: sql queries/transactions.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o replication.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/replication.json
+ curl -o replication.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/replication.json
# replicas dashboard: replica information and operations.
~~~
diff --git a/src/current/v24.2/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md b/src/current/v24.2/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
index b47593bda96..09b361cd5ac 100644
--- a/src/current/v24.2/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
+++ b/src/current/v24.2/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
@@ -71,7 +71,7 @@ Instead of using this approach, you can now [enable global access](https://cloud
Our multi-region deployment approach relies on pod IP addresses being routable across three distinct Kubernetes clusters and regions. Both the hosted Google Kubernetes Engine (GKE) and Amazon Elastic Kubernetes Service (EKS) satisfy this requirement.
-If you want to run on another cloud or on-premises, use this [basic network test](https://github.com/cockroachdb/cockroach/tree/master/cloud/kubernetes/multiregion#pod-to-pod-connectivity) to see if it will work.
+If you want to run on another cloud or on-premises, use this basic network test to see if it will work.
1. Complete the **Before You Begin** steps described in the [Google Kubernetes Engine Quickstart](https://cloud.google.com/kubernetes-engine/docs/quickstart) documentation.
@@ -322,21 +322,21 @@ This important rule enables node communication between Kubernetes clusters in di
The Kubernetes cluster in each region needs to have a [Network Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html) pointed at its CoreDNS service, which you will configure in the next step.
-1. Upload our load balancer manifest [`dns-lb-eks.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml) to the Kubernetes clusters in all 3 regions:
+1. Upload our load balancer manifest [`dns-lb-eks.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml) to the Kubernetes clusters in all 3 regions:
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
You should see the load balancer appear in the Load Balancers section of the EC2 console in each region. This load balancer will route traffic to CoreDNS in the region.
@@ -369,11 +369,11 @@ Each Kubernetes cluster has a [CoreDNS](https://coredns.io/) service that respon
To enable traffic forwarding to CockroachDB pods in all 3 regions, you need to [modify the ConfigMap](https://kubernetes.io/docs/tasks/administer-cluster/dns-custom-nameservers/#coredns-configmap-options) for the CoreDNS Corefile in each region.
-1. Download and open our ConfigMap template [`configmap.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/configmap.yaml):
+1. Download and open our ConfigMap template [`configmap.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/configmap.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/configmap.yaml
+ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/configmap.yaml
~~~
1. After [obtaining the IP addresses of the Network Load Balancers](#set-up-load-balancing) in all 3 regions, you can use this information to define a **separate ConfigMap for each region**. Each unique ConfigMap lists the forwarding addresses for the pods in the 2 other regions.
@@ -467,7 +467,7 @@ If you plan to run your instances exclusively on private subnets, set the follow
{% include_cached copy-clipboard.html %}
~~~ shell
$ curl -OOOOOOOOO \
- https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/{README.md,client-secure.yaml,cluster-init-secure.yaml,cockroachdb-statefulset-secure.yaml,dns-lb.yaml,example-app-secure.yaml,external-name-svc.yaml,setup.py,teardown.py}
+ https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/{README.md,client-secure.yaml,cluster-init-secure.yaml,cockroachdb-statefulset-secure.yaml,dns-lb.yaml,example-app-secure.yaml,external-name-svc.yaml,setup.py,teardown.py}
~~~
1. Retrieve the `kubectl` "contexts" for your clusters:
@@ -685,11 +685,11 @@ The below steps use [`cockroach cert` commands]({% link {{ page.version.version
### Create StatefulSets
-1. Download and open our [multi-region StatefulSet configuration](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml). You'll save three versions of this file locally, one for each set of 3 CockroachDB nodes per region.
+1. Download and open our [multi-region StatefulSet configuration](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml). You'll save three versions of this file locally, one for each set of 3 CockroachDB nodes per region.
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml
~~~
Look for **TODO** comments in the file. These highlight fields you need to define before deploying your StatefulSet.
@@ -814,7 +814,7 @@ The pod uses the `root` client certificate created earlier by the `setup.py` scr
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl create -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/client-secure.yaml --context --namespace
+ kubectl create -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/client-secure.yaml --context --namespace
~~~
~~~
diff --git a/src/current/v24.3/monitor-cockroachdb-kubernetes.md b/src/current/v24.3/monitor-cockroachdb-kubernetes.md
index 44a686fce22..322c288d073 100644
--- a/src/current/v24.3/monitor-cockroachdb-kubernetes.md
+++ b/src/current/v24.3/monitor-cockroachdb-kubernetes.md
@@ -128,7 +128,7 @@ If you're on Hosted GKE, before starting, make sure the email address associated
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/prometheus.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/prometheus.yaml
~~~
{{site.data.alerts.callout_info}}
@@ -177,14 +177,14 @@ If you're on Hosted GKE, before starting, make sure the email address associated
## Configure Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
-1. Download our alertmanager-config.yaml configuration file:
+1. Download our alertmanager-config.yaml configuration file:
{% include_cached copy-clipboard.html %}
~~~ shell
$ curl -O \
- https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager-config.yaml
+ https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager-config.yaml
~~~
1. Edit the `alertmanager-config.yaml` file to [specify the desired receivers for notifications](https://prometheus.io/docs/alerting/configuration/#receiver). Initially, the file contains a placeholder web hook.
@@ -214,12 +214,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
The name of the secret, `alertmanager-cockroachdb`, must match the name used in the `alertmanager.yaml` file. If they differ, the Alertmanager instance will start without configuration, and nothing will happen.
{{site.data.alerts.end}}
-1. Use our [`alertmanager.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
+1. Use our [`alertmanager.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml
~~~
~~~
@@ -244,12 +244,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
-1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml):
+1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alert-rules.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml
~~~
~~~
diff --git a/src/current/v24.3/monitor-cockroachdb-with-prometheus.md b/src/current/v24.3/monitor-cockroachdb-with-prometheus.md
index ad08d818b4e..2c661d20b66 100644
--- a/src/current/v24.3/monitor-cockroachdb-with-prometheus.md
+++ b/src/current/v24.3/monitor-cockroachdb-with-prometheus.md
@@ -15,7 +15,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
- Make sure you have already started a CockroachDB cluster, either [locally]({% link {{ page.version.version }}/start-a-local-cluster.md %}) or in a [production environment]({% link {{ page.version.version }}/manual-deployment.md %}).
-- Note that all files used in this tutorial can be found in the [`monitoring`](https://github.com/cockroachdb/cockroach/tree/master/monitoring) directory of the CockroachDB repository.
+- Note that all files used in this tutorial can be found in the `monitoring` directory of the CockroachDB repository.
## Step 1. Install Prometheus
@@ -39,11 +39,11 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
## Step 2. Configure Prometheus
-1. Download the starter [Prometheus configuration file](https://github.com/cockroachdb/cockroach/blob/master/monitoring/prometheus.yml) for CockroachDB:
+1. Download the starter [Prometheus configuration file](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/prometheus.yml) for CockroachDB:
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o prometheus.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/prometheus.yml
+ curl -o prometheus.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/prometheus.yml
~~~
When you examine the configuration file, you'll see that it is set up to scrape the time series metrics of a single, insecure local node every 10 seconds:
@@ -60,7 +60,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
Production cluster | Change the `targets` field to include `':'` for each node in the cluster. Also, be sure your network configuration allows TCP communication on the specified ports.
Secure cluster | Uncomment `scheme: 'https'` and comment out `scheme: 'http'`.
-1. Create a `rules` directory and download the [aggregation rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/aggregation.rules.yml) and [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/alerts.rules.yml) for CockroachDB into it:
+1. Create a `rules` directory and download the [aggregation rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/aggregation.rules.yml) and [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml) for CockroachDB into it:
{% include_cached copy-clipboard.html %}
~~~ shell
@@ -74,12 +74,12 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o rules/aggregation.rules.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/rules/aggregation.rules.yml
+ curl -o rules/aggregation.rules.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/rules/aggregation.rules.yml
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o rules/alerts.rules.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/rules/alerts.rules.yml
+ curl -o rules/alerts.rules.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml
~~~
## Step 3. Start Prometheus
@@ -108,7 +108,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
## Step 4. Send notifications with Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. In step 2, you already downloaded CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/alerts.rules.yml). Now, download, configure, and start [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/).
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. In step 2, you already downloaded CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml). Now, download, configure, and start [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/).
1. Download the [latest Alertmanager tarball](https://prometheus.io/download/#alertmanager) for your OS.
@@ -173,29 +173,29 @@ Although Prometheus lets you graph metrics, [Grafana](https://grafana.com/) is a
Url | `http://:9090`
Access | Direct
-1. Download the starter [Grafana dashboards](https://github.com/cockroachdb/cockroach/tree/master/monitoring/grafana-dashboards/by-cluster) for CockroachDB:
+1. Download the starter Grafana dashboards for CockroachDB:
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o runtime.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/runtime.json
+ curl -o runtime.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/runtime.json
# runtime dashboard: node status, including uptime, memory, and cpu.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o storage.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/storage.json
+ curl -o storage.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/storage.json
# storage dashboard: storage availability.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o sql.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/sql.json
+ curl -o sql.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/sql.json
# sql dashboard: sql queries/transactions.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o replication.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/replication.json
+ curl -o replication.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/replication.json
# replicas dashboard: replica information and operations.
~~~
diff --git a/src/current/v24.3/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md b/src/current/v24.3/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
index b47593bda96..09b361cd5ac 100644
--- a/src/current/v24.3/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
+++ b/src/current/v24.3/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
@@ -71,7 +71,7 @@ Instead of using this approach, you can now [enable global access](https://cloud
Our multi-region deployment approach relies on pod IP addresses being routable across three distinct Kubernetes clusters and regions. Both the hosted Google Kubernetes Engine (GKE) and Amazon Elastic Kubernetes Service (EKS) satisfy this requirement.
-If you want to run on another cloud or on-premises, use this [basic network test](https://github.com/cockroachdb/cockroach/tree/master/cloud/kubernetes/multiregion#pod-to-pod-connectivity) to see if it will work.
+If you want to run on another cloud or on-premises, use this basic network test to see if it will work.
1. Complete the **Before You Begin** steps described in the [Google Kubernetes Engine Quickstart](https://cloud.google.com/kubernetes-engine/docs/quickstart) documentation.
@@ -322,21 +322,21 @@ This important rule enables node communication between Kubernetes clusters in di
The Kubernetes cluster in each region needs to have a [Network Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html) pointed at its CoreDNS service, which you will configure in the next step.
-1. Upload our load balancer manifest [`dns-lb-eks.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml) to the Kubernetes clusters in all 3 regions:
+1. Upload our load balancer manifest [`dns-lb-eks.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml) to the Kubernetes clusters in all 3 regions:
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
You should see the load balancer appear in the Load Balancers section of the EC2 console in each region. This load balancer will route traffic to CoreDNS in the region.
@@ -369,11 +369,11 @@ Each Kubernetes cluster has a [CoreDNS](https://coredns.io/) service that respon
To enable traffic forwarding to CockroachDB pods in all 3 regions, you need to [modify the ConfigMap](https://kubernetes.io/docs/tasks/administer-cluster/dns-custom-nameservers/#coredns-configmap-options) for the CoreDNS Corefile in each region.
-1. Download and open our ConfigMap template [`configmap.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/configmap.yaml):
+1. Download and open our ConfigMap template [`configmap.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/configmap.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/configmap.yaml
+ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/configmap.yaml
~~~
1. After [obtaining the IP addresses of the Network Load Balancers](#set-up-load-balancing) in all 3 regions, you can use this information to define a **separate ConfigMap for each region**. Each unique ConfigMap lists the forwarding addresses for the pods in the 2 other regions.
@@ -467,7 +467,7 @@ If you plan to run your instances exclusively on private subnets, set the follow
{% include_cached copy-clipboard.html %}
~~~ shell
$ curl -OOOOOOOOO \
- https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/{README.md,client-secure.yaml,cluster-init-secure.yaml,cockroachdb-statefulset-secure.yaml,dns-lb.yaml,example-app-secure.yaml,external-name-svc.yaml,setup.py,teardown.py}
+ https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/{README.md,client-secure.yaml,cluster-init-secure.yaml,cockroachdb-statefulset-secure.yaml,dns-lb.yaml,example-app-secure.yaml,external-name-svc.yaml,setup.py,teardown.py}
~~~
1. Retrieve the `kubectl` "contexts" for your clusters:
@@ -685,11 +685,11 @@ The below steps use [`cockroach cert` commands]({% link {{ page.version.version
### Create StatefulSets
-1. Download and open our [multi-region StatefulSet configuration](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml). You'll save three versions of this file locally, one for each set of 3 CockroachDB nodes per region.
+1. Download and open our [multi-region StatefulSet configuration](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml). You'll save three versions of this file locally, one for each set of 3 CockroachDB nodes per region.
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml
~~~
Look for **TODO** comments in the file. These highlight fields you need to define before deploying your StatefulSet.
@@ -814,7 +814,7 @@ The pod uses the `root` client certificate created earlier by the `setup.py` scr
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl create -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/client-secure.yaml --context --namespace
+ kubectl create -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/client-secure.yaml --context --namespace
~~~
~~~
diff --git a/src/current/v25.1/monitor-cockroachdb-kubernetes.md b/src/current/v25.1/monitor-cockroachdb-kubernetes.md
index cf1ba80217b..322c288d073 100644
--- a/src/current/v25.1/monitor-cockroachdb-kubernetes.md
+++ b/src/current/v25.1/monitor-cockroachdb-kubernetes.md
@@ -128,7 +128,7 @@ If you're on Hosted GKE, before starting, make sure the email address associated
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/prometheus.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/prometheus.yaml
~~~
{{site.data.alerts.callout_info}}
@@ -163,11 +163,11 @@ If you're on Hosted GKE, before starting, make sure the email address associated
1. To verify that each CockroachDB node is connected to Prometheus, go to **Status > Targets**. The screen should look like this:
-
+
1. To verify that data is being collected, go to **Graph**, enter the `sys_uptime` variable in the field, click **Execute**, and then click the **Graph** tab. The screen should like this:
-
+
{{site.data.alerts.callout_success}}
Prometheus auto-completes CockroachDB time series metrics for you, but if you want to see a full listing, with descriptions, port-forward as described in {% if page.secure == true %}[Access the DB Console]({% link {{ page.version.version }}/deploy-cockroachdb-with-kubernetes.md %}#step-4-access-the-db-console){% else %}[Access the DB Console]({% link {{ page.version.version }}/deploy-cockroachdb-with-kubernetes.md %}#step-4-access-the-db-console){% endif %} and then point your browser to http://localhost:8080/_status/vars.
@@ -177,14 +177,14 @@ If you're on Hosted GKE, before starting, make sure the email address associated
## Configure Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
-1. Download our alertmanager-config.yaml configuration file:
+1. Download our alertmanager-config.yaml configuration file:
{% include_cached copy-clipboard.html %}
~~~ shell
$ curl -O \
- https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager-config.yaml
+ https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager-config.yaml
~~~
1. Edit the `alertmanager-config.yaml` file to [specify the desired receivers for notifications](https://prometheus.io/docs/alerting/configuration/#receiver). Initially, the file contains a placeholder web hook.
@@ -214,12 +214,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
The name of the secret, `alertmanager-cockroachdb`, must match the name used in the `alertmanager.yaml` file. If they differ, the Alertmanager instance will start without configuration, and nothing will happen.
{{site.data.alerts.end}}
-1. Use our [`alertmanager.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
+1. Use our [`alertmanager.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml
~~~
~~~
@@ -238,18 +238,18 @@ Active monitoring helps you spot problems early, but it is also essential to sen
1. Go to http://localhost:9093 in your browser. The screen should look like this:
-
+
1. Ensure that the Alertmanagers are visible to Prometheus by opening http://localhost:9090/status. The screen should look like this:
-
+
-1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml):
+1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alert-rules.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml
~~~
~~~
@@ -258,11 +258,11 @@ Active monitoring helps you spot problems early, but it is also essential to sen
1. Ensure that the rules are visible to Prometheus by opening http://localhost:9090/rules. The screen should look like this:
-
+
1. Verify that the `TestAlertManager` example alert is firing by opening http://localhost:9090/alerts. The screen should look like this:
-
+
1. To remove the example alert:
diff --git a/src/current/v25.1/monitor-cockroachdb-with-prometheus.md b/src/current/v25.1/monitor-cockroachdb-with-prometheus.md
index eceee578492..bde659b39e4 100644
--- a/src/current/v25.1/monitor-cockroachdb-with-prometheus.md
+++ b/src/current/v25.1/monitor-cockroachdb-with-prometheus.md
@@ -15,7 +15,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
- Make sure you have already started a CockroachDB cluster, either [locally]({% link {{ page.version.version }}/start-a-local-cluster.md %}) or in a [production environment]({% link {{ page.version.version }}/manual-deployment.md %}).
-- Note that all files used in this tutorial can be found in the [`monitoring`](https://github.com/cockroachdb/cockroach/tree/master/monitoring) directory of the CockroachDB repository.
+- Note that all files used in this tutorial can be found in the `monitoring` directory of the CockroachDB repository.
## Step 1. Install Prometheus
@@ -39,11 +39,11 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
## Step 2. Configure Prometheus
-1. Download the starter [Prometheus configuration file](https://github.com/cockroachdb/cockroach/blob/master/monitoring/prometheus.yml) for CockroachDB:
+1. Download the starter [Prometheus configuration file](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/prometheus.yml) for CockroachDB:
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o prometheus.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/prometheus.yml
+ curl -o prometheus.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/prometheus.yml
~~~
When you examine the configuration file, you'll see that it is set up to scrape the time series metrics of a single, insecure local node every 10 seconds:
@@ -60,7 +60,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
Production cluster | Change the `targets` field to include `':'` for each node in the cluster. Also, be sure your network configuration allows TCP communication on the specified ports.
Secure cluster | Uncomment `scheme: 'https'` and comment out `scheme: 'http'`.
-1. Create a `rules` directory and download the [aggregation rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/aggregation.rules.yml) and [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/alerts.rules.yml) for CockroachDB into it:
+1. Create a `rules` directory and download the [aggregation rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/aggregation.rules.yml) and [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml) for CockroachDB into it:
{% include_cached copy-clipboard.html %}
~~~ shell
@@ -74,12 +74,12 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o rules/aggregation.rules.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/rules/aggregation.rules.yml
+ curl -o rules/aggregation.rules.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/rules/aggregation.rules.yml
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o rules/alerts.rules.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/rules/alerts.rules.yml
+ curl -o rules/alerts.rules.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml
~~~
## Step 3. Start Prometheus
@@ -108,7 +108,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
## Step 4. Send notifications with Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. In step 2, you already downloaded CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/alerts.rules.yml). Now, download, configure, and start [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/).
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. In step 2, you already downloaded CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml). Now, download, configure, and start [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/).
1. Download the [latest Alertmanager tarball](https://prometheus.io/download/#alertmanager) for your OS.
@@ -173,29 +173,29 @@ Although Prometheus lets you graph metrics, [Grafana](https://grafana.com/) is a
Url | `http://:9090`
Access | Direct
-1. Download the starter [Grafana dashboards](https://github.com/cockroachdb/cockroach/tree/master/monitoring/grafana-dashboards/by-cluster) for CockroachDB:
+1. Download the starter Grafana dashboards for CockroachDB:
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o runtime.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/runtime.json
+ curl -o runtime.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/runtime.json
# runtime dashboard: node status, including uptime, memory, and cpu.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o storage.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/storage.json
+ curl -o storage.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/storage.json
# storage dashboard: storage availability.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o sql.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/sql.json
+ curl -o sql.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/sql.json
# sql dashboard: sql queries/transactions.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o replication.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/replication.json
+ curl -o replication.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/replication.json
# replicas dashboard: replica information and operations.
~~~
diff --git a/src/current/v25.1/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md b/src/current/v25.1/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
index b47593bda96..09b361cd5ac 100644
--- a/src/current/v25.1/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
+++ b/src/current/v25.1/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
@@ -71,7 +71,7 @@ Instead of using this approach, you can now [enable global access](https://cloud
Our multi-region deployment approach relies on pod IP addresses being routable across three distinct Kubernetes clusters and regions. Both the hosted Google Kubernetes Engine (GKE) and Amazon Elastic Kubernetes Service (EKS) satisfy this requirement.
-If you want to run on another cloud or on-premises, use this [basic network test](https://github.com/cockroachdb/cockroach/tree/master/cloud/kubernetes/multiregion#pod-to-pod-connectivity) to see if it will work.
+If you want to run on another cloud or on-premises, use this basic network test to see if it will work.
1. Complete the **Before You Begin** steps described in the [Google Kubernetes Engine Quickstart](https://cloud.google.com/kubernetes-engine/docs/quickstart) documentation.
@@ -322,21 +322,21 @@ This important rule enables node communication between Kubernetes clusters in di
The Kubernetes cluster in each region needs to have a [Network Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html) pointed at its CoreDNS service, which you will configure in the next step.
-1. Upload our load balancer manifest [`dns-lb-eks.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml) to the Kubernetes clusters in all 3 regions:
+1. Upload our load balancer manifest [`dns-lb-eks.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml) to the Kubernetes clusters in all 3 regions:
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
You should see the load balancer appear in the Load Balancers section of the EC2 console in each region. This load balancer will route traffic to CoreDNS in the region.
@@ -369,11 +369,11 @@ Each Kubernetes cluster has a [CoreDNS](https://coredns.io/) service that respon
To enable traffic forwarding to CockroachDB pods in all 3 regions, you need to [modify the ConfigMap](https://kubernetes.io/docs/tasks/administer-cluster/dns-custom-nameservers/#coredns-configmap-options) for the CoreDNS Corefile in each region.
-1. Download and open our ConfigMap template [`configmap.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/configmap.yaml):
+1. Download and open our ConfigMap template [`configmap.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/configmap.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/configmap.yaml
+ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/configmap.yaml
~~~
1. After [obtaining the IP addresses of the Network Load Balancers](#set-up-load-balancing) in all 3 regions, you can use this information to define a **separate ConfigMap for each region**. Each unique ConfigMap lists the forwarding addresses for the pods in the 2 other regions.
@@ -467,7 +467,7 @@ If you plan to run your instances exclusively on private subnets, set the follow
{% include_cached copy-clipboard.html %}
~~~ shell
$ curl -OOOOOOOOO \
- https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/{README.md,client-secure.yaml,cluster-init-secure.yaml,cockroachdb-statefulset-secure.yaml,dns-lb.yaml,example-app-secure.yaml,external-name-svc.yaml,setup.py,teardown.py}
+ https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/{README.md,client-secure.yaml,cluster-init-secure.yaml,cockroachdb-statefulset-secure.yaml,dns-lb.yaml,example-app-secure.yaml,external-name-svc.yaml,setup.py,teardown.py}
~~~
1. Retrieve the `kubectl` "contexts" for your clusters:
@@ -685,11 +685,11 @@ The below steps use [`cockroach cert` commands]({% link {{ page.version.version
### Create StatefulSets
-1. Download and open our [multi-region StatefulSet configuration](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml). You'll save three versions of this file locally, one for each set of 3 CockroachDB nodes per region.
+1. Download and open our [multi-region StatefulSet configuration](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml). You'll save three versions of this file locally, one for each set of 3 CockroachDB nodes per region.
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml
~~~
Look for **TODO** comments in the file. These highlight fields you need to define before deploying your StatefulSet.
@@ -814,7 +814,7 @@ The pod uses the `root` client certificate created earlier by the `setup.py` scr
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl create -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/client-secure.yaml --context --namespace
+ kubectl create -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/client-secure.yaml --context --namespace
~~~
~~~
diff --git a/src/current/v25.2/monitor-cockroachdb-kubernetes.md b/src/current/v25.2/monitor-cockroachdb-kubernetes.md
index da454617282..2f6365cf34c 100644
--- a/src/current/v25.2/monitor-cockroachdb-kubernetes.md
+++ b/src/current/v25.2/monitor-cockroachdb-kubernetes.md
@@ -132,7 +132,7 @@ If you're on Hosted GKE, before starting, make sure the email address associated
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/prometheus.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/prometheus.yaml
~~~
{{site.data.alerts.callout_info}}
@@ -167,11 +167,11 @@ If you're on Hosted GKE, before starting, make sure the email address associated
1. To verify that each CockroachDB node is connected to Prometheus, go to **Status > Targets**. The screen should look like this:
-
+
1. To verify that data is being collected, go to **Graph**, enter the `sys_uptime` variable in the field, click **Execute**, and then click the **Graph** tab. The screen should like this:
-
+
{{site.data.alerts.callout_success}}
Prometheus auto-completes CockroachDB time series metrics for you, but if you want to see a full listing, with descriptions, port-forward as described in {% if page.secure == true %}[Access the DB Console]({% link {{ page.version.version }}/deploy-cockroachdb-with-kubernetes.md %}#step-4-access-the-db-console){% else %}[Access the DB Console]({% link {{ page.version.version }}/deploy-cockroachdb-with-kubernetes.md %}#step-4-access-the-db-console){% endif %} and then point your browser to http://localhost:8080/_status/vars.
@@ -181,14 +181,14 @@ If you're on Hosted GKE, before starting, make sure the email address associated
## Configure Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
-1. Download our alertmanager-config.yaml configuration file:
+1. Download our alertmanager-config.yaml configuration file:
{% include_cached copy-clipboard.html %}
~~~ shell
$ curl -O \
- https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager-config.yaml
+ https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager-config.yaml
~~~
1. Edit the `alertmanager-config.yaml` file to [specify the desired receivers for notifications](https://prometheus.io/docs/alerting/configuration/#receiver). Initially, the file contains a placeholder web hook.
@@ -218,12 +218,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
The name of the secret, `alertmanager-cockroachdb`, must match the name used in the `alertmanager.yaml` file. If they differ, the Alertmanager instance will start without configuration, and nothing will happen.
{{site.data.alerts.end}}
-1. Use our [`alertmanager.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
+1. Use our [`alertmanager.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml
~~~
~~~
@@ -242,18 +242,18 @@ Active monitoring helps you spot problems early, but it is also essential to sen
1. Go to http://localhost:9093 in your browser. The screen should look like this:
-
+
1. Ensure that the Alertmanagers are visible to Prometheus by opening http://localhost:9090/status. The screen should look like this:
-
+
-1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml):
+1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alert-rules.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml
~~~
~~~
@@ -262,11 +262,11 @@ Active monitoring helps you spot problems early, but it is also essential to sen
1. Ensure that the rules are visible to Prometheus by opening http://localhost:9090/rules. The screen should look like this:
-
+
1. Verify that the `TestAlertManager` example alert is firing by opening http://localhost:9090/alerts. The screen should look like this:
-
+
1. To remove the example alert:
diff --git a/src/current/v25.2/monitor-cockroachdb-operator.md b/src/current/v25.2/monitor-cockroachdb-operator.md
index 86cd625d29f..a929a833631 100644
--- a/src/current/v25.2/monitor-cockroachdb-operator.md
+++ b/src/current/v25.2/monitor-cockroachdb-operator.md
@@ -76,7 +76,7 @@ If you're on Hosted GKE, before starting, make sure the email address associated
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/prometheus.yaml
+ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/prometheus.yaml
~~~
1. Apply the Prometheus manifest. This creates the various objects necessary to run a Prometheus instance:
@@ -119,13 +119,13 @@ For more details on using the Prometheus UI, see their [official documentation](
## Configure Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
-1. Download our [alertmanager-config.yaml](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager-config.yaml) configuration file:
+1. Download our [alertmanager-config.yaml](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager-config.yaml) configuration file:
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager-config.yaml
+ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager-config.yaml
~~~
1. Edit the `alertmanager-config.yaml` file to [specify the desired receivers for notifications](https://prometheus.io/docs/alerting/configuration/#receiver). Initially, the file contains a placeholder web hook.
@@ -152,12 +152,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
The name of the secret, `alertmanager-cockroachdb`, must match the name used in the `alertmanager.yaml` file. If they differ, the Alertmanager instance will start without configuration, and nothing will happen.
{{site.data.alerts.end}}
-1. Use our [alertmanager.yaml](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
+1. Use our [alertmanager.yaml](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
{% include_cached copy-clipboard.html %}
~~~ shell
kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml
~~~
~~~ shell
alertmanager.monitoring.coreos.com/cockroachdb created
@@ -180,12 +180,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
-1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml):
+1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alert-rules.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml
~~~
~~~ shell
prometheusrule.monitoring.coreos.com/prometheus-cockroachdb-rules created
diff --git a/src/current/v25.2/monitor-cockroachdb-with-prometheus.md b/src/current/v25.2/monitor-cockroachdb-with-prometheus.md
index 573576f0097..f5e07f80181 100644
--- a/src/current/v25.2/monitor-cockroachdb-with-prometheus.md
+++ b/src/current/v25.2/monitor-cockroachdb-with-prometheus.md
@@ -15,7 +15,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
- Make sure you have already started a CockroachDB cluster, either [locally]({% link {{ page.version.version }}/start-a-local-cluster.md %}) or in a [production environment]({% link {{ page.version.version }}/manual-deployment.md %}).
-- Note that all files used in this tutorial can be found in the [`monitoring`](https://github.com/cockroachdb/cockroach/tree/master/monitoring) directory of the CockroachDB repository.
+- Note that all files used in this tutorial can be found in the `monitoring` directory of the CockroachDB repository.
## Step 1. Install Prometheus
@@ -39,11 +39,11 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
## Step 2. Configure Prometheus
-1. Download the starter [Prometheus configuration file](https://github.com/cockroachdb/cockroach/blob/master/monitoring/prometheus.yml) for CockroachDB:
+1. Download the starter [Prometheus configuration file](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/prometheus.yml) for CockroachDB:
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o prometheus.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/prometheus.yml
+ curl -o prometheus.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/prometheus.yml
~~~
When you examine the configuration file, you'll see that it is set up to scrape the time series metrics of a single, insecure local node every 10 seconds:
@@ -60,7 +60,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
Production cluster | Change the `targets` field to include `':'` for each node in the cluster. Also, be sure your network configuration allows TCP communication on the specified ports.
Secure cluster | Uncomment `scheme: 'https'` and comment out `scheme: 'http'`.
-1. Create a `rules` directory and download the [aggregation rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/aggregation.rules.yml) and [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/alerts.rules.yml) for CockroachDB into it:
+1. Create a `rules` directory and download the [aggregation rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/aggregation.rules.yml) and [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml) for CockroachDB into it:
{% include_cached copy-clipboard.html %}
~~~ shell
@@ -74,12 +74,12 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o rules/aggregation.rules.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/rules/aggregation.rules.yml
+ curl -o rules/aggregation.rules.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/rules/aggregation.rules.yml
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o rules/alerts.rules.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/rules/alerts.rules.yml
+ curl -o rules/alerts.rules.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml
~~~
## Step 3. Start Prometheus
@@ -108,7 +108,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
## Step 4. Send notifications with Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. In step 2, you already downloaded CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/alerts.rules.yml). Now, download, configure, and start [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/).
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. In step 2, you already downloaded CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml). Now, download, configure, and start [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/).
1. Download the [latest Alertmanager tarball](https://prometheus.io/download/#alertmanager) for your OS.
@@ -173,29 +173,29 @@ Although Prometheus lets you graph metrics, [Grafana](https://grafana.com/) is a
Url | `http://:9090`
Access | Direct
-1. Download the starter [Grafana dashboards](https://github.com/cockroachdb/cockroach/tree/master/monitoring/grafana-dashboards/by-cluster) for CockroachDB:
+1. Download the starter Grafana dashboards for CockroachDB:
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o runtime.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/runtime.json
+ curl -o runtime.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/runtime.json
# runtime dashboard: node status, including uptime, memory, and cpu.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o storage.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/storage.json
+ curl -o storage.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/storage.json
# storage dashboard: storage availability.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o sql.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/sql.json
+ curl -o sql.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/sql.json
# sql dashboard: sql queries/transactions.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o replication.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/replication.json
+ curl -o replication.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/replication.json
# replicas dashboard: replica information and operations.
~~~
diff --git a/src/current/v25.2/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md b/src/current/v25.2/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
index 55f144f4d74..34fcca4fb4a 100644
--- a/src/current/v25.2/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
+++ b/src/current/v25.2/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
@@ -71,7 +71,7 @@ Instead of using this approach, you can now [enable global access](https://cloud
Our multi-region deployment approach relies on pod IP addresses being routable across three distinct Kubernetes clusters and regions. Both the hosted Google Kubernetes Engine (GKE) and Amazon Elastic Kubernetes Service (EKS) satisfy this requirement.
-If you want to run on another cloud or on-premises, use this [basic network test](https://github.com/cockroachdb/cockroach/tree/master/cloud/kubernetes/multiregion#pod-to-pod-connectivity) to see if it will work.
+If you want to run on another cloud or on-premises, use this basic network test to see if it will work.
1. Complete the **Before You Begin** steps described in the [Google Kubernetes Engine Quickstart](https://cloud.google.com/kubernetes-engine/docs/quickstart) documentation.
@@ -322,21 +322,21 @@ This important rule enables node communication between Kubernetes clusters in di
The Kubernetes cluster in each region needs to have a [Network Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html) pointed at its CoreDNS service, which you will configure in the next step.
-1. Upload our load balancer manifest [`dns-lb-eks.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml) to the Kubernetes clusters in all 3 regions:
+1. Upload our load balancer manifest [`dns-lb-eks.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml) to the Kubernetes clusters in all 3 regions:
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
You should see the load balancer appear in the Load Balancers section of the EC2 console in each region. This load balancer will route traffic to CoreDNS in the region.
@@ -369,11 +369,11 @@ Each Kubernetes cluster has a [CoreDNS](https://coredns.io/) service that respon
To enable traffic forwarding to CockroachDB pods in all 3 regions, you need to [modify the ConfigMap](https://kubernetes.io/docs/tasks/administer-cluster/dns-custom-nameservers/#coredns-configmap-options) for the CoreDNS Corefile in each region.
-1. Download and open our ConfigMap template [`configmap.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/configmap.yaml):
+1. Download and open our ConfigMap template [`configmap.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/configmap.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/configmap.yaml
+ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/configmap.yaml
~~~
1. After [obtaining the IP addresses of the Network Load Balancers](#set-up-load-balancing) in all 3 regions, you can use this information to define a **separate ConfigMap for each region**. Each unique ConfigMap lists the forwarding addresses for the pods in the 2 other regions.
@@ -467,7 +467,7 @@ If you plan to run your instances exclusively on private subnets, set the follow
{% include_cached copy-clipboard.html %}
~~~ shell
$ curl -OOOOOOOOO \
- https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/{README.md,client-secure.yaml,cluster-init-secure.yaml,cockroachdb-statefulset-secure.yaml,dns-lb.yaml,example-app-secure.yaml,external-name-svc.yaml,setup.py,teardown.py}
+ https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/{README.md,client-secure.yaml,cluster-init-secure.yaml,cockroachdb-statefulset-secure.yaml,dns-lb.yaml,example-app-secure.yaml,external-name-svc.yaml,setup.py,teardown.py}
~~~
1. Retrieve the `kubectl` "contexts" for your clusters:
@@ -685,11 +685,11 @@ The below steps use [`cockroach cert` commands]({% link {{ page.version.version
### Create StatefulSets
-1. Download and open our [multi-region StatefulSet configuration](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml). You'll save three versions of this file locally, one for each set of 3 CockroachDB nodes per region.
+1. Download and open our [multi-region StatefulSet configuration](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml). You'll save three versions of this file locally, one for each set of 3 CockroachDB nodes per region.
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml
~~~
Look for **TODO** comments in the file. These highlight fields you need to define before deploying your StatefulSet.
@@ -814,7 +814,7 @@ The pod uses the `root` client certificate created earlier by the `setup.py` scr
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl create -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/client-secure.yaml --context --namespace
+ kubectl create -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/client-secure.yaml --context --namespace
~~~
~~~
diff --git a/src/current/v25.3/monitor-cockroachdb-kubernetes.md b/src/current/v25.3/monitor-cockroachdb-kubernetes.md
index e53bde93041..ed16b0be75d 100644
--- a/src/current/v25.3/monitor-cockroachdb-kubernetes.md
+++ b/src/current/v25.3/monitor-cockroachdb-kubernetes.md
@@ -132,7 +132,7 @@ If you're on Hosted GKE, before starting, make sure the email address associated
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/prometheus.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/prometheus.yaml
~~~
{{site.data.alerts.callout_info}}
@@ -167,11 +167,11 @@ If you're on Hosted GKE, before starting, make sure the email address associated
1. To verify that each CockroachDB node is connected to Prometheus, go to **Status > Targets**. The screen should look like this:
-
+
1. To verify that data is being collected, go to **Graph**, enter the `sys_uptime` variable in the field, click **Execute**, and then click the **Graph** tab. The screen should like this:
-
+
{{site.data.alerts.callout_success}}
Prometheus auto-completes CockroachDB time series metrics for you, but if you want to see a full listing, with descriptions, port-forward as described in {% if page.secure == true %}[Access the DB Console]({% link {{ page.version.version }}/deploy-cockroachdb-with-kubernetes.md %}#step-4-access-the-db-console){% else %}[Access the DB Console]({% link {{ page.version.version }}/deploy-cockroachdb-with-kubernetes.md %}#step-4-access-the-db-console){% endif %} and then point your browser to the [Prometheus endpoint]({% link {{ page.version.version }}/prometheus-endpoint.md %}).
@@ -181,14 +181,14 @@ If you're on Hosted GKE, before starting, make sure the email address associated
## Configure Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
-1. Download our alertmanager-config.yaml configuration file:
+1. Download our alertmanager-config.yaml configuration file:
{% include_cached copy-clipboard.html %}
~~~ shell
$ curl -O \
- https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager-config.yaml
+ https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager-config.yaml
~~~
1. Edit the `alertmanager-config.yaml` file to [specify the desired receivers for notifications](https://prometheus.io/docs/alerting/configuration/#receiver). Initially, the file contains a placeholder web hook.
@@ -218,12 +218,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
The name of the secret, `alertmanager-cockroachdb`, must match the name used in the `alertmanager.yaml` file. If they differ, the Alertmanager instance will start without configuration, and nothing will happen.
{{site.data.alerts.end}}
-1. Use our [`alertmanager.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
+1. Use our [`alertmanager.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml
~~~
~~~
@@ -242,18 +242,18 @@ Active monitoring helps you spot problems early, but it is also essential to sen
1. Go to http://localhost:9093 in your browser. The screen should look like this:
-
+
1. Ensure that the Alertmanagers are visible to Prometheus by opening http://localhost:9090/status. The screen should look like this:
-
+
-1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml):
+1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alert-rules.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml
~~~
~~~
@@ -262,11 +262,11 @@ Active monitoring helps you spot problems early, but it is also essential to sen
1. Ensure that the rules are visible to Prometheus by opening http://localhost:9090/rules. The screen should look like this:
-
+
1. Verify that the `TestAlertManager` example alert is firing by opening http://localhost:9090/alerts. The screen should look like this:
-
+
1. To remove the example alert:
diff --git a/src/current/v25.3/monitor-cockroachdb-operator.md b/src/current/v25.3/monitor-cockroachdb-operator.md
index e44c73c252d..7555d79b787 100644
--- a/src/current/v25.3/monitor-cockroachdb-operator.md
+++ b/src/current/v25.3/monitor-cockroachdb-operator.md
@@ -76,7 +76,7 @@ If you're on Hosted GKE, before starting, make sure the email address associated
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/prometheus.yaml
+ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/prometheus.yaml
~~~
1. Apply the Prometheus manifest. This creates the various objects necessary to run a Prometheus instance:
@@ -119,13 +119,13 @@ For more details on using the Prometheus UI, see their [official documentation](
## Configure Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
-1. Download our [alertmanager-config.yaml](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager-config.yaml) configuration file:
+1. Download our [alertmanager-config.yaml](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager-config.yaml) configuration file:
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager-config.yaml
+ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager-config.yaml
~~~
1. Edit the `alertmanager-config.yaml` file to [specify the desired receivers for notifications](https://prometheus.io/docs/alerting/configuration/#receiver). Initially, the file contains a placeholder web hook.
@@ -152,12 +152,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
The name of the secret, `alertmanager-cockroachdb`, must match the name used in the `alertmanager.yaml` file. If they differ, the Alertmanager instance will start without configuration, and nothing will happen.
{{site.data.alerts.end}}
-1. Use our [alertmanager.yaml](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
+1. Use our [alertmanager.yaml](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
{% include_cached copy-clipboard.html %}
~~~ shell
kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml
~~~
~~~ shell
alertmanager.monitoring.coreos.com/cockroachdb created
@@ -180,12 +180,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
-1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml):
+1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alert-rules.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml
~~~
~~~ shell
prometheusrule.monitoring.coreos.com/prometheus-cockroachdb-rules created
diff --git a/src/current/v25.3/monitor-cockroachdb-with-prometheus.md b/src/current/v25.3/monitor-cockroachdb-with-prometheus.md
index 2a9ff6f04ae..f2ca3db97c8 100644
--- a/src/current/v25.3/monitor-cockroachdb-with-prometheus.md
+++ b/src/current/v25.3/monitor-cockroachdb-with-prometheus.md
@@ -15,7 +15,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
- Make sure you have already started a CockroachDB cluster, either [locally]({% link {{ page.version.version }}/start-a-local-cluster.md %}) or in a [production environment]({% link {{ page.version.version }}/manual-deployment.md %}).
-- Note that all files used in this tutorial can be found in the [`monitoring`](https://github.com/cockroachdb/cockroach/tree/master/monitoring) directory of the CockroachDB repository.
+- Note that all files used in this tutorial can be found in the `monitoring` directory of the CockroachDB repository.
## Step 1. Install Prometheus
@@ -39,11 +39,11 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
## Step 2. Configure Prometheus
-1. Download the starter [Prometheus configuration file](https://github.com/cockroachdb/cockroach/blob/master/monitoring/prometheus.yml) for CockroachDB:
+1. Download the starter [Prometheus configuration file](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/prometheus.yml) for CockroachDB:
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o prometheus.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/prometheus.yml
+ curl -o prometheus.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/prometheus.yml
~~~
When you examine the configuration file, you'll see that it is set up to scrape the time series metrics of a single, insecure local node every 10 seconds:
@@ -60,7 +60,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
Production cluster | Change the `targets` field to include `':'` for each node in the cluster. Also, be sure your network configuration allows TCP communication on the specified ports.
Secure cluster | Uncomment `scheme: 'https'` and comment out `scheme: 'http'`.
-1. Create a `rules` directory and download the [aggregation rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/aggregation.rules.yml) and [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/alerts.rules.yml) for CockroachDB into it:
+1. Create a `rules` directory and download the [aggregation rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/aggregation.rules.yml) and [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml) for CockroachDB into it:
{% include_cached copy-clipboard.html %}
~~~ shell
@@ -74,12 +74,12 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o rules/aggregation.rules.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/rules/aggregation.rules.yml
+ curl -o rules/aggregation.rules.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/rules/aggregation.rules.yml
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o rules/alerts.rules.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/rules/alerts.rules.yml
+ curl -o rules/alerts.rules.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml
~~~
## Step 3. Start Prometheus
@@ -108,7 +108,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
## Step 4. Send notifications with Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. In step 2, you already downloaded CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/alerts.rules.yml). Now, download, configure, and start [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/).
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. In step 2, you already downloaded CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml). Now, download, configure, and start [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/).
1. Download the [latest Alertmanager tarball](https://prometheus.io/download/#alertmanager) for your OS.
@@ -173,29 +173,29 @@ Although Prometheus lets you graph metrics, [Grafana](https://grafana.com/) is a
Url | `http://:9090`
Access | Direct
-1. Download the starter [Grafana dashboards](https://github.com/cockroachdb/cockroach/tree/master/monitoring/grafana-dashboards/by-cluster) for CockroachDB:
+1. Download the starter Grafana dashboards for CockroachDB:
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o runtime.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/runtime.json
+ curl -o runtime.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/runtime.json
# runtime dashboard: node status, including uptime, memory, and cpu.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o storage.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/storage.json
+ curl -o storage.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/storage.json
# storage dashboard: storage availability.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o sql.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/sql.json
+ curl -o sql.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/sql.json
# sql dashboard: sql queries/transactions.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o replication.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/replication.json
+ curl -o replication.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/replication.json
# replicas dashboard: replica information and operations.
~~~
diff --git a/src/current/v25.3/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md b/src/current/v25.3/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
index 55f144f4d74..34fcca4fb4a 100644
--- a/src/current/v25.3/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
+++ b/src/current/v25.3/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
@@ -71,7 +71,7 @@ Instead of using this approach, you can now [enable global access](https://cloud
Our multi-region deployment approach relies on pod IP addresses being routable across three distinct Kubernetes clusters and regions. Both the hosted Google Kubernetes Engine (GKE) and Amazon Elastic Kubernetes Service (EKS) satisfy this requirement.
-If you want to run on another cloud or on-premises, use this [basic network test](https://github.com/cockroachdb/cockroach/tree/master/cloud/kubernetes/multiregion#pod-to-pod-connectivity) to see if it will work.
+If you want to run on another cloud or on-premises, use this basic network test to see if it will work.
1. Complete the **Before You Begin** steps described in the [Google Kubernetes Engine Quickstart](https://cloud.google.com/kubernetes-engine/docs/quickstart) documentation.
@@ -322,21 +322,21 @@ This important rule enables node communication between Kubernetes clusters in di
The Kubernetes cluster in each region needs to have a [Network Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html) pointed at its CoreDNS service, which you will configure in the next step.
-1. Upload our load balancer manifest [`dns-lb-eks.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml) to the Kubernetes clusters in all 3 regions:
+1. Upload our load balancer manifest [`dns-lb-eks.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml) to the Kubernetes clusters in all 3 regions:
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
You should see the load balancer appear in the Load Balancers section of the EC2 console in each region. This load balancer will route traffic to CoreDNS in the region.
@@ -369,11 +369,11 @@ Each Kubernetes cluster has a [CoreDNS](https://coredns.io/) service that respon
To enable traffic forwarding to CockroachDB pods in all 3 regions, you need to [modify the ConfigMap](https://kubernetes.io/docs/tasks/administer-cluster/dns-custom-nameservers/#coredns-configmap-options) for the CoreDNS Corefile in each region.
-1. Download and open our ConfigMap template [`configmap.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/configmap.yaml):
+1. Download and open our ConfigMap template [`configmap.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/configmap.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/configmap.yaml
+ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/configmap.yaml
~~~
1. After [obtaining the IP addresses of the Network Load Balancers](#set-up-load-balancing) in all 3 regions, you can use this information to define a **separate ConfigMap for each region**. Each unique ConfigMap lists the forwarding addresses for the pods in the 2 other regions.
@@ -467,7 +467,7 @@ If you plan to run your instances exclusively on private subnets, set the follow
{% include_cached copy-clipboard.html %}
~~~ shell
$ curl -OOOOOOOOO \
- https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/{README.md,client-secure.yaml,cluster-init-secure.yaml,cockroachdb-statefulset-secure.yaml,dns-lb.yaml,example-app-secure.yaml,external-name-svc.yaml,setup.py,teardown.py}
+ https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/{README.md,client-secure.yaml,cluster-init-secure.yaml,cockroachdb-statefulset-secure.yaml,dns-lb.yaml,example-app-secure.yaml,external-name-svc.yaml,setup.py,teardown.py}
~~~
1. Retrieve the `kubectl` "contexts" for your clusters:
@@ -685,11 +685,11 @@ The below steps use [`cockroach cert` commands]({% link {{ page.version.version
### Create StatefulSets
-1. Download and open our [multi-region StatefulSet configuration](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml). You'll save three versions of this file locally, one for each set of 3 CockroachDB nodes per region.
+1. Download and open our [multi-region StatefulSet configuration](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml). You'll save three versions of this file locally, one for each set of 3 CockroachDB nodes per region.
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml
~~~
Look for **TODO** comments in the file. These highlight fields you need to define before deploying your StatefulSet.
@@ -814,7 +814,7 @@ The pod uses the `root` client certificate created earlier by the `setup.py` scr
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl create -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/client-secure.yaml --context --namespace
+ kubectl create -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/client-secure.yaml --context --namespace
~~~
~~~
diff --git a/src/current/v25.4/monitor-cockroachdb-kubernetes.md b/src/current/v25.4/monitor-cockroachdb-kubernetes.md
index b8ab1dafdac..39e0956e1d6 100644
--- a/src/current/v25.4/monitor-cockroachdb-kubernetes.md
+++ b/src/current/v25.4/monitor-cockroachdb-kubernetes.md
@@ -132,7 +132,7 @@ If you're on Hosted GKE, before starting, make sure the email address associated
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/prometheus.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/prometheus.yaml
~~~
{{site.data.alerts.callout_info}}
@@ -181,14 +181,14 @@ If you're on Hosted GKE, before starting, make sure the email address associated
## Configure Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
-1. Download our alertmanager-config.yaml configuration file:
+1. Download our alertmanager-config.yaml configuration file:
{% include_cached copy-clipboard.html %}
~~~ shell
$ curl -O \
- https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager-config.yaml
+ https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager-config.yaml
~~~
1. Edit the `alertmanager-config.yaml` file to [specify the desired receivers for notifications](https://prometheus.io/docs/alerting/configuration/#receiver). Initially, the file contains a placeholder web hook.
@@ -218,12 +218,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
The name of the secret, `alertmanager-cockroachdb`, must match the name used in the `alertmanager.yaml` file. If they differ, the Alertmanager instance will start without configuration, and nothing will happen.
{{site.data.alerts.end}}
-1. Use our [`alertmanager.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
+1. Use our [`alertmanager.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml
~~~
~~~
@@ -248,12 +248,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
-1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml):
+1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alert-rules.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml
~~~
~~~
diff --git a/src/current/v25.4/monitor-cockroachdb-operator.md b/src/current/v25.4/monitor-cockroachdb-operator.md
index 5bf93ce305d..3ccfb20a588 100644
--- a/src/current/v25.4/monitor-cockroachdb-operator.md
+++ b/src/current/v25.4/monitor-cockroachdb-operator.md
@@ -76,7 +76,7 @@ If you're on Hosted GKE, before starting, make sure the email address associated
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/prometheus.yaml
+ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/prometheus.yaml
~~~
1. Apply the Prometheus manifest. This creates the various objects necessary to run a Prometheus instance:
@@ -119,13 +119,13 @@ For more details on using the Prometheus UI, see their [official documentation](
## Configure Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
-1. Download our [alertmanager-config.yaml](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager-config.yaml) configuration file:
+1. Download our [alertmanager-config.yaml](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager-config.yaml) configuration file:
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager-config.yaml
+ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager-config.yaml
~~~
1. Edit the `alertmanager-config.yaml` file to [specify the desired receivers for notifications](https://prometheus.io/docs/alerting/configuration/#receiver). Initially, the file contains a placeholder web hook.
@@ -152,12 +152,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
The name of the secret, `alertmanager-cockroachdb`, must match the name used in the `alertmanager.yaml` file. If they differ, the Alertmanager instance will start without configuration, and nothing will happen.
{{site.data.alerts.end}}
-1. Use our [alertmanager.yaml](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
+1. Use our [alertmanager.yaml](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
{% include_cached copy-clipboard.html %}
~~~ shell
kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml
~~~
~~~ shell
alertmanager.monitoring.coreos.com/cockroachdb created
@@ -180,12 +180,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
-1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml):
+1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alert-rules.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml
~~~
~~~ shell
prometheusrule.monitoring.coreos.com/prometheus-cockroachdb-rules created
diff --git a/src/current/v25.4/monitor-cockroachdb-with-prometheus.md b/src/current/v25.4/monitor-cockroachdb-with-prometheus.md
index 2a9ff6f04ae..f2ca3db97c8 100644
--- a/src/current/v25.4/monitor-cockroachdb-with-prometheus.md
+++ b/src/current/v25.4/monitor-cockroachdb-with-prometheus.md
@@ -15,7 +15,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
- Make sure you have already started a CockroachDB cluster, either [locally]({% link {{ page.version.version }}/start-a-local-cluster.md %}) or in a [production environment]({% link {{ page.version.version }}/manual-deployment.md %}).
-- Note that all files used in this tutorial can be found in the [`monitoring`](https://github.com/cockroachdb/cockroach/tree/master/monitoring) directory of the CockroachDB repository.
+- Note that all files used in this tutorial can be found in the `monitoring` directory of the CockroachDB repository.
## Step 1. Install Prometheus
@@ -39,11 +39,11 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
## Step 2. Configure Prometheus
-1. Download the starter [Prometheus configuration file](https://github.com/cockroachdb/cockroach/blob/master/monitoring/prometheus.yml) for CockroachDB:
+1. Download the starter [Prometheus configuration file](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/prometheus.yml) for CockroachDB:
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o prometheus.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/prometheus.yml
+ curl -o prometheus.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/prometheus.yml
~~~
When you examine the configuration file, you'll see that it is set up to scrape the time series metrics of a single, insecure local node every 10 seconds:
@@ -60,7 +60,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
Production cluster | Change the `targets` field to include `':'` for each node in the cluster. Also, be sure your network configuration allows TCP communication on the specified ports.
Secure cluster | Uncomment `scheme: 'https'` and comment out `scheme: 'http'`.
-1. Create a `rules` directory and download the [aggregation rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/aggregation.rules.yml) and [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/alerts.rules.yml) for CockroachDB into it:
+1. Create a `rules` directory and download the [aggregation rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/aggregation.rules.yml) and [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml) for CockroachDB into it:
{% include_cached copy-clipboard.html %}
~~~ shell
@@ -74,12 +74,12 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o rules/aggregation.rules.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/rules/aggregation.rules.yml
+ curl -o rules/aggregation.rules.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/rules/aggregation.rules.yml
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o rules/alerts.rules.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/rules/alerts.rules.yml
+ curl -o rules/alerts.rules.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml
~~~
## Step 3. Start Prometheus
@@ -108,7 +108,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
## Step 4. Send notifications with Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. In step 2, you already downloaded CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/alerts.rules.yml). Now, download, configure, and start [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/).
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. In step 2, you already downloaded CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml). Now, download, configure, and start [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/).
1. Download the [latest Alertmanager tarball](https://prometheus.io/download/#alertmanager) for your OS.
@@ -173,29 +173,29 @@ Although Prometheus lets you graph metrics, [Grafana](https://grafana.com/) is a
Url | `http://:9090`
Access | Direct
-1. Download the starter [Grafana dashboards](https://github.com/cockroachdb/cockroach/tree/master/monitoring/grafana-dashboards/by-cluster) for CockroachDB:
+1. Download the starter Grafana dashboards for CockroachDB:
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o runtime.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/runtime.json
+ curl -o runtime.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/runtime.json
# runtime dashboard: node status, including uptime, memory, and cpu.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o storage.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/storage.json
+ curl -o storage.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/storage.json
# storage dashboard: storage availability.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o sql.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/sql.json
+ curl -o sql.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/sql.json
# sql dashboard: sql queries/transactions.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o replication.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/replication.json
+ curl -o replication.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/replication.json
# replicas dashboard: replica information and operations.
~~~
diff --git a/src/current/v25.4/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md b/src/current/v25.4/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
index 55f144f4d74..34fcca4fb4a 100644
--- a/src/current/v25.4/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
+++ b/src/current/v25.4/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
@@ -71,7 +71,7 @@ Instead of using this approach, you can now [enable global access](https://cloud
Our multi-region deployment approach relies on pod IP addresses being routable across three distinct Kubernetes clusters and regions. Both the hosted Google Kubernetes Engine (GKE) and Amazon Elastic Kubernetes Service (EKS) satisfy this requirement.
-If you want to run on another cloud or on-premises, use this [basic network test](https://github.com/cockroachdb/cockroach/tree/master/cloud/kubernetes/multiregion#pod-to-pod-connectivity) to see if it will work.
+If you want to run on another cloud or on-premises, use this basic network test to see if it will work.
1. Complete the **Before You Begin** steps described in the [Google Kubernetes Engine Quickstart](https://cloud.google.com/kubernetes-engine/docs/quickstart) documentation.
@@ -322,21 +322,21 @@ This important rule enables node communication between Kubernetes clusters in di
The Kubernetes cluster in each region needs to have a [Network Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html) pointed at its CoreDNS service, which you will configure in the next step.
-1. Upload our load balancer manifest [`dns-lb-eks.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml) to the Kubernetes clusters in all 3 regions:
+1. Upload our load balancer manifest [`dns-lb-eks.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml) to the Kubernetes clusters in all 3 regions:
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
You should see the load balancer appear in the Load Balancers section of the EC2 console in each region. This load balancer will route traffic to CoreDNS in the region.
@@ -369,11 +369,11 @@ Each Kubernetes cluster has a [CoreDNS](https://coredns.io/) service that respon
To enable traffic forwarding to CockroachDB pods in all 3 regions, you need to [modify the ConfigMap](https://kubernetes.io/docs/tasks/administer-cluster/dns-custom-nameservers/#coredns-configmap-options) for the CoreDNS Corefile in each region.
-1. Download and open our ConfigMap template [`configmap.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/configmap.yaml):
+1. Download and open our ConfigMap template [`configmap.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/configmap.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/configmap.yaml
+ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/configmap.yaml
~~~
1. After [obtaining the IP addresses of the Network Load Balancers](#set-up-load-balancing) in all 3 regions, you can use this information to define a **separate ConfigMap for each region**. Each unique ConfigMap lists the forwarding addresses for the pods in the 2 other regions.
@@ -467,7 +467,7 @@ If you plan to run your instances exclusively on private subnets, set the follow
{% include_cached copy-clipboard.html %}
~~~ shell
$ curl -OOOOOOOOO \
- https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/{README.md,client-secure.yaml,cluster-init-secure.yaml,cockroachdb-statefulset-secure.yaml,dns-lb.yaml,example-app-secure.yaml,external-name-svc.yaml,setup.py,teardown.py}
+ https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/{README.md,client-secure.yaml,cluster-init-secure.yaml,cockroachdb-statefulset-secure.yaml,dns-lb.yaml,example-app-secure.yaml,external-name-svc.yaml,setup.py,teardown.py}
~~~
1. Retrieve the `kubectl` "contexts" for your clusters:
@@ -685,11 +685,11 @@ The below steps use [`cockroach cert` commands]({% link {{ page.version.version
### Create StatefulSets
-1. Download and open our [multi-region StatefulSet configuration](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml). You'll save three versions of this file locally, one for each set of 3 CockroachDB nodes per region.
+1. Download and open our [multi-region StatefulSet configuration](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml). You'll save three versions of this file locally, one for each set of 3 CockroachDB nodes per region.
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml
~~~
Look for **TODO** comments in the file. These highlight fields you need to define before deploying your StatefulSet.
@@ -814,7 +814,7 @@ The pod uses the `root` client certificate created earlier by the `setup.py` scr
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl create -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/client-secure.yaml --context --namespace
+ kubectl create -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/client-secure.yaml --context --namespace
~~~
~~~
diff --git a/src/current/v26.1/monitor-cockroachdb-kubernetes.md b/src/current/v26.1/monitor-cockroachdb-kubernetes.md
index 581d156b4fd..760facc2268 100644
--- a/src/current/v26.1/monitor-cockroachdb-kubernetes.md
+++ b/src/current/v26.1/monitor-cockroachdb-kubernetes.md
@@ -132,7 +132,7 @@ If you're on Hosted GKE, before starting, make sure the email address associated
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/prometheus.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/prometheus.yaml
~~~
{{site.data.alerts.callout_info}}
@@ -181,14 +181,14 @@ If you're on Hosted GKE, before starting, make sure the email address associated
## Configure Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
-1. Download our alertmanager-config.yaml configuration file:
+1. Download our alertmanager-config.yaml configuration file:
{% include_cached copy-clipboard.html %}
~~~ shell
$ curl -O \
- https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager-config.yaml
+ https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager-config.yaml
~~~
1. Edit the `alertmanager-config.yaml` file to [specify the desired receivers for notifications](https://prometheus.io/docs/alerting/configuration/#receiver). Initially, the file contains a placeholder web hook.
@@ -218,12 +218,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
The name of the secret, `alertmanager-cockroachdb`, must match the name used in the `alertmanager.yaml` file. If they differ, the Alertmanager instance will start without configuration, and nothing will happen.
{{site.data.alerts.end}}
-1. Use our [`alertmanager.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
+1. Use our [`alertmanager.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml
~~~
~~~
@@ -248,12 +248,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
-1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml):
+1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alert-rules.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml
~~~
~~~
diff --git a/src/current/v26.1/monitor-cockroachdb-operator.md b/src/current/v26.1/monitor-cockroachdb-operator.md
index daf5e678e7b..8b3c613a40c 100644
--- a/src/current/v26.1/monitor-cockroachdb-operator.md
+++ b/src/current/v26.1/monitor-cockroachdb-operator.md
@@ -76,7 +76,7 @@ If you're on Hosted GKE, before starting, make sure the email address associated
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/prometheus.yaml
+ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/prometheus.yaml
~~~
1. Apply the Prometheus manifest. This creates the various objects necessary to run a Prometheus instance:
@@ -119,13 +119,13 @@ For more details on using the Prometheus UI, see their [official documentation](
## Configure Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
-1. Download our [alertmanager-config.yaml](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager-config.yaml) configuration file:
+1. Download our [alertmanager-config.yaml](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager-config.yaml) configuration file:
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager-config.yaml
+ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager-config.yaml
~~~
1. Edit the `alertmanager-config.yaml` file to [specify the desired receivers for notifications](https://prometheus.io/docs/alerting/configuration/#receiver). Initially, the file contains a placeholder web hook.
@@ -152,12 +152,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
The name of the secret, `alertmanager-cockroachdb`, must match the name used in the `alertmanager.yaml` file. If they differ, the Alertmanager instance will start without configuration, and nothing will happen.
{{site.data.alerts.end}}
-1. Use our [alertmanager.yaml](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
+1. Use our [alertmanager.yaml](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
{% include_cached copy-clipboard.html %}
~~~ shell
kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml
~~~
~~~ shell
alertmanager.monitoring.coreos.com/cockroachdb created
@@ -180,12 +180,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
-1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml):
+1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alert-rules.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml
~~~
~~~ shell
prometheusrule.monitoring.coreos.com/prometheus-cockroachdb-rules created
diff --git a/src/current/v26.1/monitor-cockroachdb-with-prometheus.md b/src/current/v26.1/monitor-cockroachdb-with-prometheus.md
index 2a9ff6f04ae..f2ca3db97c8 100644
--- a/src/current/v26.1/monitor-cockroachdb-with-prometheus.md
+++ b/src/current/v26.1/monitor-cockroachdb-with-prometheus.md
@@ -15,7 +15,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
- Make sure you have already started a CockroachDB cluster, either [locally]({% link {{ page.version.version }}/start-a-local-cluster.md %}) or in a [production environment]({% link {{ page.version.version }}/manual-deployment.md %}).
-- Note that all files used in this tutorial can be found in the [`monitoring`](https://github.com/cockroachdb/cockroach/tree/master/monitoring) directory of the CockroachDB repository.
+- Note that all files used in this tutorial can be found in the `monitoring` directory of the CockroachDB repository.
## Step 1. Install Prometheus
@@ -39,11 +39,11 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
## Step 2. Configure Prometheus
-1. Download the starter [Prometheus configuration file](https://github.com/cockroachdb/cockroach/blob/master/monitoring/prometheus.yml) for CockroachDB:
+1. Download the starter [Prometheus configuration file](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/prometheus.yml) for CockroachDB:
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o prometheus.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/prometheus.yml
+ curl -o prometheus.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/prometheus.yml
~~~
When you examine the configuration file, you'll see that it is set up to scrape the time series metrics of a single, insecure local node every 10 seconds:
@@ -60,7 +60,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
Production cluster | Change the `targets` field to include `':'` for each node in the cluster. Also, be sure your network configuration allows TCP communication on the specified ports.
Secure cluster | Uncomment `scheme: 'https'` and comment out `scheme: 'http'`.
-1. Create a `rules` directory and download the [aggregation rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/aggregation.rules.yml) and [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/alerts.rules.yml) for CockroachDB into it:
+1. Create a `rules` directory and download the [aggregation rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/aggregation.rules.yml) and [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml) for CockroachDB into it:
{% include_cached copy-clipboard.html %}
~~~ shell
@@ -74,12 +74,12 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o rules/aggregation.rules.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/rules/aggregation.rules.yml
+ curl -o rules/aggregation.rules.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/rules/aggregation.rules.yml
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o rules/alerts.rules.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/rules/alerts.rules.yml
+ curl -o rules/alerts.rules.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml
~~~
## Step 3. Start Prometheus
@@ -108,7 +108,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
## Step 4. Send notifications with Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. In step 2, you already downloaded CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/alerts.rules.yml). Now, download, configure, and start [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/).
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. In step 2, you already downloaded CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml). Now, download, configure, and start [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/).
1. Download the [latest Alertmanager tarball](https://prometheus.io/download/#alertmanager) for your OS.
@@ -173,29 +173,29 @@ Although Prometheus lets you graph metrics, [Grafana](https://grafana.com/) is a
Url | `http://:9090`
Access | Direct
-1. Download the starter [Grafana dashboards](https://github.com/cockroachdb/cockroach/tree/master/monitoring/grafana-dashboards/by-cluster) for CockroachDB:
+1. Download the starter Grafana dashboards for CockroachDB:
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o runtime.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/runtime.json
+ curl -o runtime.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/runtime.json
# runtime dashboard: node status, including uptime, memory, and cpu.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o storage.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/storage.json
+ curl -o storage.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/storage.json
# storage dashboard: storage availability.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o sql.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/sql.json
+ curl -o sql.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/sql.json
# sql dashboard: sql queries/transactions.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o replication.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/replication.json
+ curl -o replication.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/replication.json
# replicas dashboard: replica information and operations.
~~~
diff --git a/src/current/v26.1/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md b/src/current/v26.1/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
index 55f144f4d74..34fcca4fb4a 100644
--- a/src/current/v26.1/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
+++ b/src/current/v26.1/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
@@ -71,7 +71,7 @@ Instead of using this approach, you can now [enable global access](https://cloud
Our multi-region deployment approach relies on pod IP addresses being routable across three distinct Kubernetes clusters and regions. Both the hosted Google Kubernetes Engine (GKE) and Amazon Elastic Kubernetes Service (EKS) satisfy this requirement.
-If you want to run on another cloud or on-premises, use this [basic network test](https://github.com/cockroachdb/cockroach/tree/master/cloud/kubernetes/multiregion#pod-to-pod-connectivity) to see if it will work.
+If you want to run on another cloud or on-premises, use this basic network test to see if it will work.
1. Complete the **Before You Begin** steps described in the [Google Kubernetes Engine Quickstart](https://cloud.google.com/kubernetes-engine/docs/quickstart) documentation.
@@ -322,21 +322,21 @@ This important rule enables node communication between Kubernetes clusters in di
The Kubernetes cluster in each region needs to have a [Network Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html) pointed at its CoreDNS service, which you will configure in the next step.
-1. Upload our load balancer manifest [`dns-lb-eks.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml) to the Kubernetes clusters in all 3 regions:
+1. Upload our load balancer manifest [`dns-lb-eks.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml) to the Kubernetes clusters in all 3 regions:
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
You should see the load balancer appear in the Load Balancers section of the EC2 console in each region. This load balancer will route traffic to CoreDNS in the region.
@@ -369,11 +369,11 @@ Each Kubernetes cluster has a [CoreDNS](https://coredns.io/) service that respon
To enable traffic forwarding to CockroachDB pods in all 3 regions, you need to [modify the ConfigMap](https://kubernetes.io/docs/tasks/administer-cluster/dns-custom-nameservers/#coredns-configmap-options) for the CoreDNS Corefile in each region.
-1. Download and open our ConfigMap template [`configmap.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/configmap.yaml):
+1. Download and open our ConfigMap template [`configmap.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/configmap.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/configmap.yaml
+ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/configmap.yaml
~~~
1. After [obtaining the IP addresses of the Network Load Balancers](#set-up-load-balancing) in all 3 regions, you can use this information to define a **separate ConfigMap for each region**. Each unique ConfigMap lists the forwarding addresses for the pods in the 2 other regions.
@@ -467,7 +467,7 @@ If you plan to run your instances exclusively on private subnets, set the follow
{% include_cached copy-clipboard.html %}
~~~ shell
$ curl -OOOOOOOOO \
- https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/{README.md,client-secure.yaml,cluster-init-secure.yaml,cockroachdb-statefulset-secure.yaml,dns-lb.yaml,example-app-secure.yaml,external-name-svc.yaml,setup.py,teardown.py}
+ https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/{README.md,client-secure.yaml,cluster-init-secure.yaml,cockroachdb-statefulset-secure.yaml,dns-lb.yaml,example-app-secure.yaml,external-name-svc.yaml,setup.py,teardown.py}
~~~
1. Retrieve the `kubectl` "contexts" for your clusters:
@@ -685,11 +685,11 @@ The below steps use [`cockroach cert` commands]({% link {{ page.version.version
### Create StatefulSets
-1. Download and open our [multi-region StatefulSet configuration](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml). You'll save three versions of this file locally, one for each set of 3 CockroachDB nodes per region.
+1. Download and open our [multi-region StatefulSet configuration](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml). You'll save three versions of this file locally, one for each set of 3 CockroachDB nodes per region.
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml
~~~
Look for **TODO** comments in the file. These highlight fields you need to define before deploying your StatefulSet.
@@ -814,7 +814,7 @@ The pod uses the `root` client certificate created earlier by the `setup.py` scr
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl create -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/client-secure.yaml --context --namespace
+ kubectl create -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/client-secure.yaml --context --namespace
~~~
~~~
diff --git a/src/current/v26.2/monitor-cockroachdb-kubernetes.md b/src/current/v26.2/monitor-cockroachdb-kubernetes.md
index 0d4fe02ebc5..3af904b5d15 100644
--- a/src/current/v26.2/monitor-cockroachdb-kubernetes.md
+++ b/src/current/v26.2/monitor-cockroachdb-kubernetes.md
@@ -132,7 +132,7 @@ If you're on Hosted GKE, before starting, make sure the email address associated
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/prometheus.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/prometheus.yaml
~~~
{{site.data.alerts.callout_info}}
@@ -181,14 +181,14 @@ If you're on Hosted GKE, before starting, make sure the email address associated
## Configure Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
-1. Download our alertmanager-config.yaml configuration file:
+1. Download our alertmanager-config.yaml configuration file:
{% include_cached copy-clipboard.html %}
~~~ shell
$ curl -O \
- https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager-config.yaml
+ https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager-config.yaml
~~~
1. Edit the `alertmanager-config.yaml` file to [specify the desired receivers for notifications](https://prometheus.io/docs/alerting/configuration/#receiver). Initially, the file contains a placeholder web hook.
@@ -218,12 +218,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
The name of the secret, `alertmanager-cockroachdb`, must match the name used in the `alertmanager.yaml` file. If they differ, the Alertmanager instance will start without configuration, and nothing will happen.
{{site.data.alerts.end}}
-1. Use our [`alertmanager.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
+1. Use our [`alertmanager.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml
~~~
~~~
@@ -248,12 +248,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
-1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml):
+1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
$ kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alert-rules.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml
~~~
~~~
diff --git a/src/current/v26.2/monitor-cockroachdb-operator.md b/src/current/v26.2/monitor-cockroachdb-operator.md
index c47ec443b79..d49baf76d7e 100644
--- a/src/current/v26.2/monitor-cockroachdb-operator.md
+++ b/src/current/v26.2/monitor-cockroachdb-operator.md
@@ -76,7 +76,7 @@ If you're on Hosted GKE, before starting, make sure the email address associated
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/prometheus.yaml
+ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/prometheus.yaml
~~~
1. Apply the Prometheus manifest. This creates the various objects necessary to run a Prometheus instance:
@@ -119,13 +119,13 @@ For more details on using the Prometheus UI, see their [official documentation](
## Configure Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. This section shows you how to use [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) and CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml) to do this.
-1. Download our [alertmanager-config.yaml](https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager-config.yaml) configuration file:
+1. Download our [alertmanager-config.yaml](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager-config.yaml) configuration file:
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager-config.yaml
+ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager-config.yaml
~~~
1. Edit the `alertmanager-config.yaml` file to [specify the desired receivers for notifications](https://prometheus.io/docs/alerting/configuration/#receiver). Initially, the file contains a placeholder web hook.
@@ -152,12 +152,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
The name of the secret, `alertmanager-cockroachdb`, must match the name used in the `alertmanager.yaml` file. If they differ, the Alertmanager instance will start without configuration, and nothing will happen.
{{site.data.alerts.end}}
-1. Use our [alertmanager.yaml](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
+1. Use our [alertmanager.yaml](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml) file to create the various objects necessary to run an Alertmanager instance, including a ClusterIP service so that Prometheus can forward alerts:
{% include_cached copy-clipboard.html %}
~~~ shell
kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alertmanager.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alertmanager.yaml
~~~
~~~ shell
alertmanager.monitoring.coreos.com/cockroachdb created
@@ -180,12 +180,12 @@ Active monitoring helps you spot problems early, but it is also essential to sen
-1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/prometheus/alert-rules.yaml):
+1. Add CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
kubectl apply \
- -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/prometheus/alert-rules.yaml
+ -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/prometheus/alert-rules.yaml
~~~
~~~ shell
prometheusrule.monitoring.coreos.com/prometheus-cockroachdb-rules created
diff --git a/src/current/v26.2/monitor-cockroachdb-with-prometheus.md b/src/current/v26.2/monitor-cockroachdb-with-prometheus.md
index 3d03a38c9d3..966168d0534 100644
--- a/src/current/v26.2/monitor-cockroachdb-with-prometheus.md
+++ b/src/current/v26.2/monitor-cockroachdb-with-prometheus.md
@@ -15,7 +15,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
- Make sure you have already started a CockroachDB cluster, either [locally]({% link {{ page.version.version }}/start-a-local-cluster.md %}) or in a [production environment]({% link {{ page.version.version }}/manual-deployment.md %}).
-- Note that all files used in this tutorial can be found in the [`monitoring`](https://github.com/cockroachdb/cockroach/tree/master/monitoring) directory of the CockroachDB repository.
+- Note that all files used in this tutorial can be found in the `monitoring` directory of the CockroachDB repository.
## Step 1. Install Prometheus
@@ -39,11 +39,11 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
## Step 2. Configure Prometheus
-1. Download the starter [Prometheus configuration file](https://github.com/cockroachdb/cockroach/blob/master/monitoring/prometheus.yml) for CockroachDB:
+1. Download the starter [Prometheus configuration file](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/prometheus.yml) for CockroachDB:
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o prometheus.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/prometheus.yml
+ curl -o prometheus.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/prometheus.yml
~~~
When you examine the configuration file, you'll see that it is set up to scrape the time series metrics of a single, insecure local node every 10 seconds:
@@ -60,7 +60,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
Production cluster | Change the `targets` field to include `':'` for each node in the cluster. Also, be sure your network configuration allows TCP communication on the specified ports.
Secure cluster | Uncomment `scheme: 'https'` and comment out `scheme: 'http'`.
-1. Create a `rules` directory and download the [aggregation rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/aggregation.rules.yml) and [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/alerts.rules.yml) for CockroachDB into it:
+1. Create a `rules` directory and download the [aggregation rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/aggregation.rules.yml) and [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml) for CockroachDB into it:
{% include_cached copy-clipboard.html %}
~~~ shell
@@ -74,12 +74,12 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o rules/aggregation.rules.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/rules/aggregation.rules.yml
+ curl -o rules/aggregation.rules.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/rules/aggregation.rules.yml
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o rules/alerts.rules.yml https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/rules/alerts.rules.yml
+ curl -o rules/alerts.rules.yml https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml
~~~
## Step 3. Start Prometheus
@@ -108,7 +108,7 @@ This tutorial explores the CockroachDB {{ site.data.products.core }} integration
## Step 4. Send notifications with Alertmanager
-Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. In step 2, you already downloaded CockroachDB's starter [alerting rules](https://github.com/cockroachdb/cockroach/blob/master/monitoring/rules/alerts.rules.yml). Now, download, configure, and start [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/).
+Active monitoring helps you spot problems early, but it is also essential to send notifications when there are events that require investigation or intervention. In step 2, you already downloaded CockroachDB's starter [alerting rules](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/monitoring/rules/alerts.rules.yml). Now, download, configure, and start [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/).
1. Download the [latest Alertmanager tarball](https://prometheus.io/download/#alertmanager) for your OS.
@@ -173,29 +173,29 @@ Although Prometheus lets you graph metrics, [Grafana](https://grafana.com/) is a
Url | `http://:9090`
Access | Direct
-1. Generate a standardized Grafana dashboard using [`cockroach gen dashboard --tool=grafana`]({% link {{ page.version.version }}/cockroach-gen.md %}#generate-a-dashboard). Alternatively, download one or more of the starter [Grafana dashboards](https://github.com/cockroachdb/cockroach/tree/master/monitoring/grafana-dashboards/by-cluster) for CockroachDB to focus on specific metrics:
+1. Generate a standardized Grafana dashboard using [`cockroach gen dashboard --tool=grafana`]({% link {{ page.version.version }}/cockroach-gen.md %}#generate-a-dashboard). Alternatively, download one or more of the starter Grafana dashboards for CockroachDB to focus on specific metrics:
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o runtime.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/runtime.json
+ curl -o runtime.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/runtime.json
# runtime dashboard: node status, including uptime, memory, and cpu.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o storage.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/storage.json
+ curl -o storage.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/storage.json
# storage dashboard: storage availability.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o sql.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/sql.json
+ curl -o sql.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/sql.json
# sql dashboard: sql queries/transactions.
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -o replication.json https://raw.githubusercontent.com/cockroachdb/cockroach/master/monitoring/grafana-dashboards/by-cluster/replication.json
+ curl -o replication.json https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/monitoring/grafana-dashboards/by-cluster/replication.json
# replicas dashboard: replica information and operations.
~~~
diff --git a/src/current/v26.2/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md b/src/current/v26.2/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
index 55f144f4d74..34fcca4fb4a 100644
--- a/src/current/v26.2/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
+++ b/src/current/v26.2/orchestrate-cockroachdb-with-kubernetes-multi-cluster.md
@@ -71,7 +71,7 @@ Instead of using this approach, you can now [enable global access](https://cloud
Our multi-region deployment approach relies on pod IP addresses being routable across three distinct Kubernetes clusters and regions. Both the hosted Google Kubernetes Engine (GKE) and Amazon Elastic Kubernetes Service (EKS) satisfy this requirement.
-If you want to run on another cloud or on-premises, use this [basic network test](https://github.com/cockroachdb/cockroach/tree/master/cloud/kubernetes/multiregion#pod-to-pod-connectivity) to see if it will work.
+If you want to run on another cloud or on-premises, use this basic network test to see if it will work.
1. Complete the **Before You Begin** steps described in the [Google Kubernetes Engine Quickstart](https://cloud.google.com/kubernetes-engine/docs/quickstart) documentation.
@@ -322,21 +322,21 @@ This important rule enables node communication between Kubernetes clusters in di
The Kubernetes cluster in each region needs to have a [Network Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html) pointed at its CoreDNS service, which you will configure in the next step.
-1. Upload our load balancer manifest [`dns-lb-eks.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml) to the Kubernetes clusters in all 3 regions:
+1. Upload our load balancer manifest [`dns-lb-eks.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml) to the Kubernetes clusters in all 3 regions:
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
+ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/dns-lb-eks.yaml --context
~~~
You should see the load balancer appear in the Load Balancers section of the EC2 console in each region. This load balancer will route traffic to CoreDNS in the region.
@@ -369,11 +369,11 @@ Each Kubernetes cluster has a [CoreDNS](https://coredns.io/) service that respon
To enable traffic forwarding to CockroachDB pods in all 3 regions, you need to [modify the ConfigMap](https://kubernetes.io/docs/tasks/administer-cluster/dns-custom-nameservers/#coredns-configmap-options) for the CoreDNS Corefile in each region.
-1. Download and open our ConfigMap template [`configmap.yaml`](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/configmap.yaml):
+1. Download and open our ConfigMap template [`configmap.yaml`](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/configmap.yaml):
{% include_cached copy-clipboard.html %}
~~~ shell
- curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/configmap.yaml
+ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/configmap.yaml
~~~
1. After [obtaining the IP addresses of the Network Load Balancers](#set-up-load-balancing) in all 3 regions, you can use this information to define a **separate ConfigMap for each region**. Each unique ConfigMap lists the forwarding addresses for the pods in the 2 other regions.
@@ -467,7 +467,7 @@ If you plan to run your instances exclusively on private subnets, set the follow
{% include_cached copy-clipboard.html %}
~~~ shell
$ curl -OOOOOOOOO \
- https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/{README.md,client-secure.yaml,cluster-init-secure.yaml,cockroachdb-statefulset-secure.yaml,dns-lb.yaml,example-app-secure.yaml,external-name-svc.yaml,setup.py,teardown.py}
+ https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/{README.md,client-secure.yaml,cluster-init-secure.yaml,cockroachdb-statefulset-secure.yaml,dns-lb.yaml,example-app-secure.yaml,external-name-svc.yaml,setup.py,teardown.py}
~~~
1. Retrieve the `kubectl` "contexts" for your clusters:
@@ -685,11 +685,11 @@ The below steps use [`cockroach cert` commands]({% link {{ page.version.version
### Create StatefulSets
-1. Download and open our [multi-region StatefulSet configuration](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml). You'll save three versions of this file locally, one for each set of 3 CockroachDB nodes per region.
+1. Download and open our [multi-region StatefulSet configuration](https://github.com/cockroachdb/docs/blob/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml). You'll save three versions of this file locally, one for each set of 3 CockroachDB nodes per region.
{% include_cached copy-clipboard.html %}
~~~ shell
- $ curl -O https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml
+ $ curl -O https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/eks/cockroachdb-statefulset-secure-eks.yaml
~~~
Look for **TODO** comments in the file. These highlight fields you need to define before deploying your StatefulSet.
@@ -814,7 +814,7 @@ The pod uses the `root` client certificate created earlier by the `setup.py` scr
{% include_cached copy-clipboard.html %}
~~~ shell
- kubectl create -f https://raw.githubusercontent.com/cockroachdb/cockroach/master/cloud/kubernetes/multiregion/client-secure.yaml --context --namespace
+ kubectl create -f https://raw.githubusercontent.com/cockroachdb/docs/main/src/current/files/cockroach/cloud/kubernetes/multiregion/client-secure.yaml --context --namespace
~~~
~~~