You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/configuration/pgdog.toml/general.md
+6Lines changed: 6 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -410,6 +410,12 @@ Available options:
410
410
411
411
Default: **`auto`**
412
412
413
+
### `system_catalogs_omnisharded`
414
+
415
+
Enables sticky routing for system catalog tables and treats them as [omnisharded](../../features/sharding/omnishards.md) tables. This makes tools like `psql` work out of the box.
Copy file name to clipboardExpand all lines: docs/configuration/pgdog.toml/sharded_tables.md
+7-1Lines changed: 7 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -83,10 +83,16 @@ The data type of the column. Currently supported options are:
83
83
84
84
[Omnisharded](../../features/sharding/omnishards.md) tables are tables that have the same data on all shards. They typically are small and contain metadata, e.g., list of countries, cities, etc., and are used in joins. PgDog allows to read from these tables directly and load balances traffic evenly across all shards.
85
85
86
-
#### Example
86
+
By default, all tables unless otherwise configured as sharded, are considered omnisharded.
87
+
88
+
#### Sticky routing
89
+
90
+
Sticky routing disables round robin for omnisharded tables and sends the queries touching those tables to the same shard, guaranteeing consistent results for the duration of a client's connection:
Copy file name to clipboardExpand all lines: docs/features/sharding/omnishards.md
+59-16Lines changed: 59 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,47 +9,83 @@ Other names for these tables include **mirrored tables** and **replicated tables
9
9
10
10
## Configuration
11
11
12
-
Omnisharded tables are configured in [`pgdog.toml`](../../configuration/pgdog.toml/sharded_tables.md#omnisharded-tables):
12
+
Unless otherwise specified as a [sharded table](../../configuration/pgdog.toml/sharded_tables.md), all tables are omnisharded by default. This makes configuration simpler, and doesn't require explicitly enumerating all tables in `pgdog.toml`. For example:
13
13
14
14
```toml
15
-
[[omnisharded_tables]]
15
+
[[sharded_tables]]
16
16
database = "prod"
17
-
tables = [
18
-
"settings",
19
-
"cities",
20
-
"terms_of_service",
21
-
"ip_blocks",
22
-
]
17
+
column = "user_id"
23
18
```
24
19
25
-
## Query routing
20
+
This will configure all tables that have the `user_id` as sharded and all others as omnisharded.
21
+
22
+
### Query routing
26
23
27
24
Omnisharded tables are treated differently by the query router. Write queries are sent to all shards concurrently, while read queries are distributed evenly between shards using round robin.
28
25
29
-
If the query contains a sharding key, it will be used instead, and omnisharded tables in that query will be ignored.
26
+
For example, the following `INSERT` query will be sent to all shards concurrently:
27
+
28
+
```postgresql
29
+
INSERT INTO omnisharded_table (id, value) VALUES ($1, $2);
30
+
```
31
+
32
+
All configured shards will receive and store the same row. When reading that row, PgDog will choose one of the shards using the round robin algorithm, to distribute read load evenly.
33
+
34
+
#### Sharded and omnisharded tables
35
+
36
+
If a query references both sharded and omnisharded tables, the **sharded** table routing will take priority. Omnisharded tables are assumed to contain the same data on all shards, so joins referencing omnisharded tables will work as expected.
37
+
38
+
For example, assuming `users` table is sharded on the `id` column and `global_settings` table is omnisharded, the following query will be sent to the shard corresponding to the value of the `users.id` filter:
39
+
40
+
```postgresql
41
+
SELECT * FROM users
42
+
INNER JOIN global_settings ON global_settings.active = true
43
+
WHERE users.id = $1;
44
+
```
30
45
31
46
### Consistency
32
47
33
48
Writing data to omnisharded tables is atomic if you enable [two-phase commit](2pc.md).
34
49
35
50
If you can't or choose not to use 2pc, make sure writes to omnisharded tables can be repeated in case of failure. This can be achieved by using unique indexes and `INSERT ... ON CONFLICT ... DO UPDATE` queries.
36
51
37
-
Since reads from omnisharded tables are routed to individual shards, while a two-phase commit takes place, queries to these tables may return different results for a brief period of time.
52
+
Since data in all omnisharded tables is identical, no cross-shard indexes are necessary to achieve data integrity. You can use regular PostgreSQL `UNIQUE` indexes on individual shards.
53
+
54
+
!!! note "Eventual consistency"
55
+
Reads from omnisharded tables are routed to individual shards using round robin. While a two-phase commit takes place, different transactions may return different results for a brief period of time (usually less than a millisecond).
56
+
38
57
39
58
### Sticky routing
40
59
41
60
While most omnisharded tables should be identical on all shards, others could differ in subtle ways.
42
61
43
-
For example, if you configure system catalogs as omnisharded, e.g. to make Rails or other ORMs work out of the box, round robin query routing will return different results for each query.
62
+
For example, system catalogs (e.g. `pg_database`, `pg_class`, etc.) could have different OIDs for custom data types (e.g. `VECTOR`, `CREATE TYPE`) on different shards. To make Rails and some other ORMs work out of the box, you can enable sticky routing, which disables round robin and sends omnisharded queries to one shard for the duration of a client's connection.
63
+
64
+
For example:
44
65
45
-
When enabled, sticky routing will ensure that queries sent by a client to omnisharded tables will be consistently routed to the same shard, for the duration of the client connection.
66
+
```toml
67
+
[[omnisharded_tables]]
68
+
database = "prod"
69
+
sticky = true
70
+
tables = [
71
+
"pg_class",
72
+
"pg_database"
73
+
]
74
+
```
46
75
47
-
To enable it, configure your omnisharded tables as follows:
76
+
You can enable sticky routing for all omnisharded tables in [`pgdog.toml`](../../configuration/pgdog.toml/general.md#omnisharded_sticky):
77
+
78
+
```toml
79
+
[general]
80
+
omnisharded_sticky = true
81
+
```
82
+
83
+
The following system catalogs are using sticky routing by default:
48
84
49
85
```toml
50
86
[[omnisharded_tables]]
51
87
database = "prod"
52
-
sticky = true# Enable sticky routing for the following tables.
88
+
sticky = true
53
89
tables = [
54
90
"pg_class",
55
91
"pg_attribute",
@@ -70,4 +106,11 @@ tables = [
70
106
]
71
107
```
72
108
73
-
Once configured, commands like `\d`, `\d+` and others sent from `psql` will start to return correct results as well.
109
+
This is configurable with the `system_catalogs_omnisharded` setting in [`pgdog.toml`](../../configuration/pgdog.toml/general.md#system_catalogs_omnisharded):
110
+
111
+
```toml
112
+
[general]
113
+
system_catalogs_omnisharded = true
114
+
```
115
+
116
+
If enabled (it is by default), commands like `\d`, `\d+` and others sent from `psql` will start to return correct results.
Copy file name to clipboardExpand all lines: docs/roadmap.md
+7-5Lines changed: 7 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -44,14 +44,14 @@ Query engine provides a uniform view over multiple shards. Clients can use regul
44
44
| Feature | Status | Notes |
45
45
|----------|--------|-------|
46
46
|[Direct-to-shard reads](features/sharding/query-routing.md#select)| :material-check-circle-outline: | Sharding key must be specified in the query. |
47
-
|[Direct-to-shard writes](features/sharding/query-routing.md#insert)| :material-wrench: | Sharding key must be specified in the query. Multi-tuple `INSERT`s not supported yet. |
47
+
|[Direct-to-shard writes](features/sharding/query-routing.md#insert)| :material-check-circle-outline: | Sharding key must be specified in the query. Multi-tuple `INSERT`s are supported and sent to their respective shards automatically with a cross-shard query. Sharding key updates are supported for one row at a time. |
48
48
|[Cross-shard queries](features/sharding/cross-shard-queries/index.md)| :material-wrench: | Partial [aggregates](#aggregates) and [sorting](#sorting) support. CTEs & subqueries not supported yet. |
|[Omnisharded tables](features/sharding/omnishards.md)| :material-wrench: | Unsharded tables with identical data on all shards. |
54
-
| Rewrite queries | :material-calendar-check: | Alter queries to support aggregate/sorting by rows not returned in result set. |
54
+
| Rewrite queries | :material-wrench: | Alter queries to support aggregate/sorting by rows not returned in result set. |
55
55
|[`COPY`](features/sharding/cross-shard-queries/copy.md)| :material-check-circle-outline: | Sharding key must be specified in the statement and the data. Supports text, CSV, and binary formats only. |
56
56
| Multi-statement queries | :material-calendar-check: | e.g.: `SELECT 1; SELECT 2;`. First query is used for routing only, entire request sent to the same shard(s). [#395](https://github.com/pgdogdev/pgdog/issues/395). |
57
57
@@ -66,7 +66,9 @@ Support for aggregate functions in [cross-shard](features/sharding/cross-shard-q
0 commit comments