Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 3 additions & 6 deletions docs/administration/.pages
Original file line number Diff line number Diff line change
@@ -1,7 +1,4 @@
icon: material/shield-account
nav:
- 'index.md'
- 'clients.md'
- 'config.md'
- 'pools.md'
- 'servers.md'
- '...'
- index.md
- ... | flat
4 changes: 4 additions & 0 deletions docs/administration/clients.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
---
icon: material/account-multiple
---

# Client connections

PgDog provides real time statistics and information on all client connections. They can be accessed by connecting to the [admin database](index.md) and running the `SHOW CLIENTS` command:
Expand Down
6 changes: 4 additions & 2 deletions docs/administration/config.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
---
icon: material/cog
---

# Configuration

PgDog provides real time access to its current configuration values. They can be accessed by connecting to the [admin database](index.md) and running the `SHOW CONFIG` command:
Expand All @@ -12,8 +16,6 @@ The following configuration parameters are available and their current values:
|------|---------|
| `auth_type` [:material-link:](../configuration/pgdog.toml/general.md#auth_type) | `scram` |
| `ban_timeout` [:material-link:](../configuration/pgdog.toml/general.md#ban_timeout) | `5m` |
| `broadcast_address` [:material-link:](../configuration/pgdog.toml/general.md#broadcast_address) | `default` |
| `broadcast_port` [:material-link:](../configuration/pgdog.toml/general.md#broadcast_port) | `6433` |
| `checkout_timeout` [:material-link:](../configuration/pgdog.toml/general.md#checkout_timeout) | `1s` |
| `client_idle_timeout` [:material-link:](../configuration/pgdog.toml/general.md#client_idle_timeout) | `18446744073709551615ms` |
| `connect_attempt_delay` [:material-link:](../configuration/pgdog.toml/general.md#connect_attempt_delay) | `0ms` |
Expand Down
4 changes: 4 additions & 0 deletions docs/administration/index.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
---
icon: material/monitor-dashboard
---

# Administration overview

PgDog keeps track of clients, servers and connection pools. It provides real time statistics on its internal operations for system
Expand Down
4 changes: 4 additions & 0 deletions docs/administration/maintenance_mode.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
---
icon: material/wrench
---

# Maintenance mode

Maintenance mode pauses queries from all clients so you can synchronize configuration changes across multiple instances of PgDog. This is useful if you're changing the [sharding](../features/sharding/index.md) options.
Expand Down
4 changes: 4 additions & 0 deletions docs/administration/pools.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
---
icon: material/pool
---

# Connection pools

PgDog provides real time statistics and information on its connection pools. You can view them by connecting to the [admin database](index.md) and running the `SHOW POOLS` command:
Expand Down
4 changes: 4 additions & 0 deletions docs/administration/replication.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
---
icon: material/content-copy
---

# Replication

PgDog provides a real time view into PostgreSQL replication for the purposes of monitoring [replication delay](../features/load-balancer/replication-failover.md#replication) and performing query traffic [failover](../features/load-balancer/replication-failover.md#failover).
Expand Down
4 changes: 4 additions & 0 deletions docs/administration/servers.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
---
icon: material/server
---

# Server connections

PgDog provides real time statistics and information on all connections to PostgreSQL databases. They can be accessed by connecting to the [admin database](index.md) and running the `SHOW SERVERS` command:
Expand Down
4 changes: 4 additions & 0 deletions docs/architecture/.pages
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
icon: material/sitemap
nav:
- index.md
- ... | flat
4 changes: 4 additions & 0 deletions docs/architecture/benchmarks.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
---
icon: material/chart-bar
---

# Benchmarks

PgDog does its best to minimize its impact on database performance. Great care is taken to make sure as few operations as possible are performed
Expand Down
4 changes: 4 additions & 0 deletions docs/architecture/comparison.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
---
icon: material/scale-balance
---

# Comparison to other poolers

PgDog aims to be the de facto PostgreSQL proxy and pooler. Below is a feature comparison between PgDog and a few popular alternatives.
Expand Down
4 changes: 4 additions & 0 deletions docs/architecture/index.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
---
icon: material/cube-outline
---

# Architecture overview

PgDog is written in the [Rust](https://rust-lang.org) programming language. It is also asynchronous, powered by the [Tokio](https://tokio.rs) runtime. This allows PgDog to serve hundreds of thousands of connections on one machine and to take advantage of multiple CPUs.
Expand Down
13 changes: 12 additions & 1 deletion docs/configuration/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,18 @@ By default, PgDog looks for both configuration files in the current working dire

## Hot reload

Most settings can be reloaded without restarting PgDog. This allows you to tweak them at runtime without breaking client or server connections. For settings that require a restart, a note is added to the documentation.
Most settings can be reloaded without restarting PgDog. This allows to tweak PgDog at runtime without breaking client or server connections. For settings that require a restart, a note is added to the documentation.

Hot reload can be triggered by sending `SIGHUP` to the `pgdog` process or by connecting to the [admin database](../administration/index.md) and running the `RELOAD` command.

=== "Process signal"
```bash
kill -HUP pgdog
```
=== "Admin database"
```bash
PGPASSWORD=admin psql -h 127.0.0.1 -p 6432 -U admin admin -c 'RELOAD'
```

## Units

Expand Down
18 changes: 3 additions & 15 deletions docs/configuration/pgdog.toml/general.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ General settings are relevant to the operations of the pooler itself, or apply t

The IP address of the local network interface PgDog will bind to listen for connections.

!!! note
!!! note "Requires restart"
This setting cannot be changed at runtime.

Default: **`0.0.0.0`** (all interfaces)
Expand All @@ -21,7 +21,7 @@ The TCP port PgDog will bind to listen for connections.

Default: **`6432`**

!!! note
!!! note "Requires restart"
This setting cannot be changed at runtime.

### `workers`
Expand All @@ -31,7 +31,7 @@ virtual CPU. The value `0` means to spawn no threads and use the current thread

Default: **`2`**

!!! note
!!! note "Requires restart"
This setting cannot be changed at runtime.

### `default_pool_size`
Expand Down Expand Up @@ -74,18 +74,12 @@ Path to the TLS certificate PgDog will use to setup TLS connections with clients

Default: **none**

!!! note
This setting cannot be changed at runtime.

### `tls_private_key`

Path to the TLS private key PgDog will use to setup TLS connections with clients. If none is provided, TLS will be disabled.

Default: **none**

!!! note
This setting cannot be changed at runtime.

### `tls_client_required`

Reject clients that connect without TLS. Consider setting this to `true` when using the `enabled_plain` mode of [`passthrough_auth`](#passthrough_auth).
Expand All @@ -105,16 +99,10 @@ Available options are:
* `verify_ca` (validate certificate only)
* `verify_full` (validate certificate _and_ matching hostname)

!!! note
This setting cannot be changed at runtime.

### `tls_server_ca_certificate`

Path to a certificate bundle used to validate the server certificate on TLS connection creation. Used in conjunction with `verify_ca` or `verify_full` in [`tls_verify`](#tls_verify).

!!! note
This setting cannot be changed at runtime.


## Healthchecks

Expand Down
6 changes: 4 additions & 2 deletions docs/configuration/pgdog.toml/rewrite.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,5 +39,7 @@ The setting changes are applied immediately. These overrides allow canary testin

### Read more

- [Cross-shard INSERT](../../features/sharding/cross-shard-queries/insert.md#multiple-tuples)
- [Cross-shard UPDATE](../../features/sharding/cross-shard-queries/update.md#sharding-key-updates)
{{ next_steps_links([
("Cross-shard INSERT", "../../features/sharding/cross-shard-queries/insert.md#multiple-tuples", "Insert rows with multiple tuples across shards."),
("Cross-shard UPDATE", "../../features/sharding/cross-shard-queries/update.md#sharding-key-updates", "Update rows that require sharding key changes."),
]) }}
2 changes: 1 addition & 1 deletion docs/features/load-balancer/healthchecks.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ All databases load balanced by PgDog are regularly checked with health checks. A
If a replica database fails a health check, it's temporarily removed from the load balancer, preventing it from serving queries for a configurable period of time.

<center>
<img src="/images/healthchecks.png" width="95%" alt="Healthchecks"/>
<img src="/images/healthchecks.png" width="80%" alt="Healthchecks"/>
</center>

## How it works
Expand Down
2 changes: 1 addition & 1 deletion docs/features/load-balancer/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Applications can connect to a single PgDog [endpoint](#single-endpoint), without
When a query is received by PgDog, it will inspect it using the native Postgres SQL parser. If the query is a `SELECT` and the [configuration](../../configuration/pgdog.toml/databases.md) contains both primary and replica databases, PgDog will send it to one of the replicas. For all other queries, PgDog will send them to the primary.

<center>
<img src="/images/replicas.png" width="95%" alt="Load balancer" />
<img src="/images/replicas.png" width="80%" alt="Load balancer" />
</center>

Applications don't have to manually route queries between databases or maintain several connection pools internally.
Expand Down
4 changes: 2 additions & 2 deletions docs/features/load-balancer/manual-routing.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ The PostgreSQL protocol supports changing connection parameters using the `SET`
For example, to make sure all subsequent queries are sent to the primary, you can execute the following statement:

```postgresql
SET pgdog.role TO "primary";
SET "pgdog"."role" TO "primary";
```

The parameter is persisted on the connection until it's closed or the value is changed with another `SET` statement. Before routing a query, the load balancer will check the value of this parameter, so setting it early on during connection creation ensures all transactions are executed on the right database.
Expand All @@ -134,7 +134,7 @@ It's possible to set routing hints for the lifetime of a single transaction, by

```postgresql
BEGIN;
SET LOCAL pgdog.role TO "primary";
SET LOCAL "pgdog"."role" TO "primary";
```

In this example, all transaction statements (including the `BEGIN` statement) will be sent to the primary database. Whether the transaction is committed or reverted, the value of `pgdog.role` will be reset to its previous value.
Expand Down
2 changes: 1 addition & 1 deletion docs/features/mirroring.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Database mirroring replicates traffic, byte for byte, from one database to anoth
Mirroring in PgDog is asynchronous and should have minimal impact on production databases: transactions are sent to a background task, which in turn forwards them to one or more mirror databases. If any statement fails, the error is ignored and the next one is executed.

<center>
<img src="/images/mirroring.png" width="90%" height="auto" alt="Mirroring">
<img src="/images/mirroring.png" width="80%" height="auto" alt="Mirroring">
</center>

## Configuration
Expand Down
2 changes: 1 addition & 1 deletion docs/features/session-mode.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,6 @@ when a client disconnects, the PostgreSQL server connection remains intact and c

#### Lazy connections
Until a client issues their first query, PgDog doesn't attach it to a server connection. This allows one set of clients to connect before the previous set disconnects,
which is common when using zero-downtime deployment strategies like blue-green [^1].
which is common when using zero-downtime deployment strategies like blue/green [^1].

[^1]: [https://docs.aws.amazon.com/whitepapers/latest/overview-deployment-options/bluegreen-deployments.html](https://docs.aws.amazon.com/whitepapers/latest/overview-deployment-options/bluegreen-deployments.html)
6 changes: 4 additions & 2 deletions docs/features/sharding/2pc.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,5 +103,7 @@ Two-phase commit is used for writes only. Read transactions are finished using n

## Read more

- [Omnisharded tables](omnishards.md)
- [Cross-shard queries](cross-shard-queries/index.md)
{{ next_steps_links([
("Omnisharded tables", "omnishards.md", "Tables replicated to every shard for fast local joins."),
("Cross-shard queries", "cross-shard-queries/index.md", "Run queries that span multiple shards transparently."),
]) }}
6 changes: 4 additions & 2 deletions docs/features/sharding/basics.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,5 +36,7 @@ Alternatively, a sharding function can map the sharding keys directly to a shard

## Read more

- [Sharding functions](sharding-functions.md)
- [Direct-to-shard queries](query-routing.md)
{{ next_steps_links([
("Sharding functions", "sharding-functions.md", "Control how rows are distributed across shards."),
("Direct-to-shard queries", "query-routing.md", "Route queries directly to the correct shard."),
]) }}
6 changes: 4 additions & 2 deletions docs/features/sharding/cross-shard-queries/copy.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,5 +50,7 @@ If the query fetches rows from more than one shard, PgDog will also ignore any `

## Read more

- [Two-phase commit](../2pc.md)
- [Omnisharded tables](../omnishards.md)
{{ next_steps_links([
("Two-phase commit", "../2pc.md", "Atomic transactions spanning multiple shards."),
("Omnisharded tables", "../omnishards.md", "Tables replicated to every shard for fast local joins."),
]) }}
16 changes: 9 additions & 7 deletions docs/features/sharding/cross-shard-queries/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ icon: material/multicast
If a client can't or doesn't specify a sharding key in the query, PgDog will send that query to all shards in parallel, and combine the results automatically. To the client, this looks like the query was executed by a single database.

<center style="margin-top: 2rem;">
<img src="/images/cross-shard.png" width="95%" alt="Cross-shard queries" />
<img src="/images/cross-shard.png" width="80%" alt="Cross-shard queries" />
</center>

## How it works
Expand Down Expand Up @@ -66,9 +66,11 @@ When this setting is enabled and a query doesn't have a sharding key, instead of

## Read more

- [Sharding functions](../sharding-functions.md)
- [Cross-shard `SELECT`](select.md)
- [Cross-shard `INSERT`](insert.md)
- [Cross-shard `UPDATE` and `DELETE`](update.md)
- [DDL, e.g., `CREATE TABLE`](ddl.md)
- [`COPY` command](copy.md)
{{ next_steps_links([
("Sharding functions", "../sharding-functions.md", "Control how rows are distributed across shards."),
("Cross-shard SELECT", "select.md", "Query data across all shards with automatic merging."),
("Cross-shard INSERT", "insert.md", "Insert rows that get routed to the correct shard."),
("Cross-shard UPDATE and DELETE", "update.md", "Modify or remove rows in tables spanning multiple shards."),
("DDL, e.g. CREATE TABLE", "ddl.md", "Run schema changes across all shards at once."),
("COPY command", "copy.md", "Bulk load data across shards with the COPY protocol."),
]) }}
2 changes: 2 additions & 0 deletions docs/features/sharding/cross-shard-queries/select.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ The SQL language allows for powerful data filtration and manipulation. While we
| `ORDER BY` | :material-check: | Columns must be part of the returned tuples. See [sorting](#sorting). |
| `DISTINCT` / `DISTINCT BY`| :material-check: | Columns must be part of the returned tuples. |
| `GROUP BY` | :material-wrench: | Columns must be part of the returned tuples. See [aggregates](#aggregates). |
| `LIMIT` | :material-check: | None. |
| `OFFSET` | :material-check: | Rows are filtered in-memory, so paginating becomes linearly more expensive with the number of pages. |
| CTEs | :material-wrench: | CTE must refer to data located on the same shard. |
| Window functions | :material-close: | Not currently supported. |
| Subqueries | :material-wrench: | Subqueries must refer to data located on the same shard. |
Expand Down
19 changes: 7 additions & 12 deletions docs/features/sharding/manual-routing.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ The `SET` command comes from the PostgreSQL query language and is used to change

```postgresql
BEGIN;
SET pgdog.shard TO 0;
SET LOCAL pgdog.shard TO 0;
CREATE INDEX users_id_idx USING btree(id);
COMMIT;
```
Expand All @@ -78,21 +78,14 @@ The `SET` command comes from the PostgreSQL query language and is used to change

```postgresql
BEGIN;
SET pgdog.sharding_key TO 'us-east-1';
SET LOCAL pgdog.sharding_key TO 'us-east-1';
SELECT * FROM users WHERE is_admin = true;
COMMIT;
```

### Limitations

Since `SET` changes session variables, we want to avoid leaking this state between transactions. For this reason, routing hints provided using this method are only supported inside transactions. For example:

```postgresql
BEGIN;
SET pgdog.sharding_key TO 'us-east-1';
SELECT * FROM users WHERE is_admin = true;
COMMIT;
```
Sharding hints provided using `SET` follow the same semantics as regular `SET` variables. For this reason, it's always best to use them inside transactions with `SET LOCAL`. Using `SET` will persist the query routing hint until the parameter is set again.

### Latency

Expand Down Expand Up @@ -140,5 +133,7 @@ You can read more about this in [our blog](https://pgdog.dev/blog/sharding-a-rea

## Read more

- [Cross-shard queries](cross-shard-queries/index.md)
- [Sharding functions](sharding-functions.md)
{{ next_steps_links([
("Cross-shard queries", "cross-shard-queries/index.md", "Run queries that span multiple shards transparently."),
("Sharding functions", "sharding-functions.md", "Control how rows are distributed across shards."),
]) }}
2 changes: 1 addition & 1 deletion docs/features/sharding/omnishards.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ tables = [
]
```

This is configurable with the `system_catalogs` setting in [`pgdog.toml`](../../configuration/pgdog.toml/general.md#system_catalogs_omnisharded):
This is configurable with the `system_catalogs` setting in [`pgdog.toml`](../../configuration/pgdog.toml/general.md#system_catalogs):

```toml
[general]
Expand Down
Loading
Loading