diff --git a/analytics/README.md b/analytics/README.md
index 86607cf..df53a12 100644
--- a/analytics/README.md
+++ b/analytics/README.md
@@ -1,2 +1,5 @@
Placeholder for:
-https://github.com/dwyl/learn-devops/issues/91
\ No newline at end of file
+https://github.com/dwyl/learn-devops/issues/91
+
+Meanwhile, see:
+https://github.com/dwyl/learn-analytics/tree/main/plausible
\ No newline at end of file
diff --git a/cheatsheet.md b/cheatsheet.md
new file mode 100644
index 0000000..7b54364
--- /dev/null
+++ b/cheatsheet.md
@@ -0,0 +1,12 @@
+# Cheat Sheet
+
+This [cheat sheet](https://en.wikipedia.org/wiki/Cheat_sheet)
+helps us manage our `DevOps` _fast_.
+
+## Update & Reboot Ubuntu Server
+
+```sh
+sudo apt update -y && sudo apt full-upgrade -y && sudo apt autoremove -y && sudo apt clean -y && sudo apt autoclean -y && sudo reboot
+```
+
+The `alias` for this command on our servers is `upr`.
\ No newline at end of file
diff --git a/hetzner/README.md b/hetzner/README.md
new file mode 100644
index 0000000..27bd7be
--- /dev/null
+++ b/hetzner/README.md
@@ -0,0 +1,56 @@
+
+
+
+
+We've made the switch to `Hetzner`,
+we think you should too.
+
+
+
+# Why?
+
+After more than a decade using various "Cloud" providers
+including all the _Big_ Tech (`AWS`, `Azure`, `GCP`, `DigitalOcean`, `Linode`)
+and many PaaS such as Lambda, Fly.io, Heroku, Vercel, etc.
+we've finally bitten the bullet and gone _back_ to our roots; Servers!
+
+## Brief Aside on Motivation
+
+There two types of motivation: towards and away.
+**Toward** is the **_positive_** motivation such as getting fit/healthy.
+**Away** motivation is when we want to _avoid_ something, like being unfit.
+Both types of motivation have their place.
+Some people are exclusively motivated by loss/pain/risk aversion,
+while others are driven by gain/reward/returns and downplay the downside.
+I go through phases of being super risk averse
+and others when I'm happy to take calculated risks
+that others who haven't done the math think are _crazy_.
+
+In the case of self-hosting our web apps on **barebones servers**,
+we have _plenty_ of experience from the _pre-AWS_ days.
+Yes, this ages us, but the experience was formative.
+And means I'm not afraid to dive in.
+
+I'm motivated _away_ from the
+[data loss](https://github.com/dwyl/auth/issues/325#issuecomment-1792297886)
+we experienced on `Fly.io`
+and _toward_ the high availability/affordability of `Hetzner`.
+I know this will require some setup/config work,
+but am undeterred;
+that's why we write systematic & meticulous notes!
+
+> “_Notes aren’t a **record** of my thinking process.
+> They **are** my thinking process_.”
+~ Richard Feynman
+
+
+## Recommended Reading
+
++ Good summary including "incidents":
+[wikipedia.org/wiki/Hetzner](https://en.wikipedia.org/wiki/Hetzner)
++ `Hetzner` "about" page:
+[hetzner.com/unternehmen/ueber-uns](https://www.hetzner.com/unternehmen/ueber-uns/)
++ Sustainability report:
+[hetzner.com/unternehmen/nachhaltigkeit](https://www.hetzner.com/unternehmen/nachhaltigkeit)
++ Finances:
+[northdata.com/Hetzner](https://www.northdata.com/Hetzner%20Online%20GmbH,%20Gunzenhausen/Amtsgericht%20Ansbach%20HRB%206089)
\ No newline at end of file
diff --git a/nginx/README.md b/nginx/README.md
new file mode 100644
index 0000000..1b3f026
--- /dev/null
+++ b/nginx/README.md
@@ -0,0 +1,319 @@
+
+
+# `nginx` _Speedy_ Setup
+
+This is a speed run of using `nginx`
+to proxy an app running on a `Hetzner` server.
+
+
+
+## 1. Install `nginx` on `Ubuntu`
+
+`SSH` into the virtual machine, e.g:
+
+```sh
+ssh root@88.99.81.115
+```
+
+Ensure that everything is up-to-date on the VM:
+
+```sh
+sudo apt update -y && sudo apt full-upgrade -y && sudo apt autoremove -y && sudo apt clean -y && sudo apt autoclean -y
+```
+
+Followed by:
+
+```sh
+sudo reboot
+```
+
+Now we can proceed with installing `nginx`.
+
+Official instructions:
+https://ubuntu.com/tutorials/install-and-configure-nginx#1-overview
+
+```sh
+sudo apt install nginx
+```
+
+That installs and automatically starts the `nginx` server.
+
+Check the status:
+
+```sh
+service nginx status
+```
+
+Output:
+
+```sh
+● nginx.service - A high performance web server and a reverse proxy server
+ Loaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; preset: enabled)
+ Active: active (running) since Fri 2025-03-21 10:49:46 UTC; 56s ago
+ Docs: man:nginx(8)
+ Process: 754 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
+ Process: 778 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
+ Main PID: 803 (nginx)
+ Tasks: 3 (limit: 4540)
+ Memory: 3.7M (peak: 3.8M)
+ CPU: 34ms
+ CGroup: /system.slice/nginx.service
+ ├─803 "nginx: master process /usr/sbin/nginx -g daemon on; master_process on;"
+ ├─804 "nginx: worker process"
+ └─805 "nginx: worker process"
+```
+
+Visit:
+http://88.99.81.115
+
+
+
+
+## 2. Certbot
+
+Instructions:
+https://certbot.eff.org/instructions?ws=nginx&os=ubuntufocal
+
+```sh
+sudo snap install --classic certbot
+```
+
+Output:
+
+```sh
+2025-03-21T11:48:11Z INFO Waiting for automatic snapd restart...
+certbot 3.3.0 from Certbot Project (certbot-eff✓) installed
+```
+
+Link the command:
+
+```sh
+sudo ln -s /snap/bin/certbot /usr/bin/certbot
+```
+
+Basic `certbot` setup for an `nginx` server:
+
+```sh
+sudo certbot --nginx
+```
+
+Output:
+
+```sh
+Requesting a certificate for dwy.is
+
+Successfully received certificate.
+Certificate is saved at: /etc/letsencrypt/live/dwy.is/fullchain.pem
+Key is saved at: /etc/letsencrypt/live/dwy.is/privkey.pem
+This certificate expires on 2025-06-19.
+These files will be updated when the certificate renews.
+Certbot has set up a scheduled task to automatically renew this certificate in the background.
+
+Deploying certificate
+Successfully deployed certificate for dwy.is to /etc/nginx/sites-enabled/default
+Congratulations! You have successfully enabled HTTPS on https://dwy.is
+
+- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
+If you like Certbot, please consider supporting our work by:
+ * Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate
+ * Donating to EFF: https://eff.org/donate-le
+- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
+```
+
+Dry run renewal:
+
+```sh
+sudo certbot renew --dry-run
+```
+
+```sh
+Saving debug log to /var/log/letsencrypt/letsencrypt.log
+
+- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
+Processing /etc/letsencrypt/renewal/dwy.is.conf
+- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
+Account registered.
+Simulating renewal of an existing certificate for dwy.is
+
+- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
+Congratulations, all simulated renewals succeeded:
+ /etc/letsencrypt/live/dwy.is/fullchain.pem (success)
+- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
+```
+
+The full config including the TLS is in
+`/etc/nginx/sites-available/default`
+
+Now create a new config file
+_just_ for the subdomain.
+
+## 3. Configure `nginx` Subdomain
+
+```sh
+vi /etc/nginx/sites-enabled/autobase
+```
+
+Paste the contents from this file:
+`nginx/sites-enabled/autobase`
+
+Test `nginx` config:
+
+```sh
+nginx -t
+```
+
+You should see output similar to the following:
+
+```sh
+nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
+nginx: configuration file /etc/nginx/nginx.conf test is successful
+```
+
+Test a specific configuration file:
+
+```sh
+nginx -t -c /path/to/conf
+```
+
+In our case:
+
+```sh
+nginx -t -c /etc/nginx/sites-enabled/autobase
+```
+
+If your config fails the test for any reason,
+try checking it online:
+[google.com/search?q=nginx+syntax+check+online](https://www.google.com/search?q=nginx+syntax+check+online)
+e.g:
+[getpagespeed.com/check-nginx-config](https://www.getpagespeed.com/check-nginx-config)
+
+Restart `nginx`:
+
+```sh
+sudo service nginx restart
+```
+
+## 4. Wildcard Certificate
+
+In our case, I actually wanted a wildcard certificate
+so that I can add any subdomain I want later.
+
+Wildcard Certificate instructions:
+https://www.baeldung.com/linux/letsencrypt-certbot-add-subdomains
+
+Sample command:
+
+```sh
+sudo certbot certonly -i nginx -d example.com -d *.example.com
+```
+
+In our case:
+
+```sh
+sudo certbot certonly -i nginx -d dwy.is -d *.dwy.is -v
+```
+
+Output:
+
+```sh
+Saving debug log to /var/log/letsencrypt/letsencrypt.log
+Plugins selected: Authenticator manual, Installer None
+
+- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
+You have an existing certificate that contains a portion of the domains you
+requested (ref: /etc/letsencrypt/renewal/dwy.is.conf)
+
+It contains these names: dwy.is
+
+You requested these names for the new certificate: dwy.is, *.dwy.is.
+
+Do you want to expand and replace this existing certificate with the new
+certificate?
+- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
+(E)xpand/(C)ancel: E
+Renewing an existing certificate for dwy.is and *.dwy.is
+Performing the following challenges:
+dns-01 challenge for dwy.is
+
+- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
+Please deploy a DNS TXT record under the name:
+
+_acme-challenge.dwy.is.
+
+with the following value:
+
+8GC-85xs1BGQDlU7YKpxA5fyHBV20PqBU8aMA9lAN10
+
+Before continuing, verify the TXT record has been deployed. Depending on the DNS
+provider, this may take some time, from a few seconds to multiple minutes. You can
+check if it has finished deploying with aid of online tools, such as the Google
+Admin Toolbox: https://toolbox.googleapps.com/apps/dig/#TXT/_acme-challenge.dwy.is.
+Look for one or more bolded line(s) below the line ';ANSWER'. It should show the
+value(s) you've just added.
+
+- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
+Press Enter to Continue
+```
+
+I created the `TXT` record:
+
+https://ap.www.namecheap.com/domains/domaincontrolpanel/dwy.is/advancedns
+
+
+
+But this was incorrect!
+The host needed to be `_acme-challenge`
+***NOT*** `_acme-challenge.dwy.is`
+as was implied by `certbot`.
+i.e. the domain `dwy.is` should not be in the host!
+
+https://toolbox.googleapps.com/apps/dig/#TXT/_acme-challenge.dwy.is
+
+
+
+I refreshed this like a million times over `48h`
+but it never updated.
+
+```sh
+dig -t txt _acme-challenge.dwy.is
+```
+
+I decided to contact `NameCheap` support via live chat:
+https://www.namecheap.com/help-center/live-chat
+
+They were helpful and together we determined that _I_ had misconfigured the `TXT` record ... 🤦
+
+Updated config:
+
+https://ap.www.namecheap.com/domains/domaincontrolpanel/dwy.is/advancedns
+
+
+
+Full transcript: [Chat_Transcript_23_Mar_2025.pdf](https://github.com/user-attachments/files/19410954/Chat_Transcript_23_Mar_2025.pdf)
+
+Final output:
+
+```sh
+Renewing an existing certificate for dwy.is and *.dwy.is
+Reloading nginx server after certificate issuance
+
+Successfully received certificate.
+Certificate is saved at: /etc/letsencrypt/live/dwy.is/fullchain.pem
+Key is saved at: /etc/letsencrypt/live/dwy.is/privkey.pem
+This certificate expires on 2025-06-21.
+These files will be updated when the certificate renews.
+Certbot has set up a scheduled task to automatically renew this certificate in the background.
+
+- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
+If you like Certbot, please consider supporting our work by:
+ * Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate
+ * Donating to EFF: https://eff.org/donate-le
+- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
+```
+
+Working!
+
+
+
+Also used:
+https://dnschecker.org/#TXT/_acme-challenge.dwy.is
\ No newline at end of file
diff --git a/nginx/nginx.conf b/nginx/nginx.conf
new file mode 100644
index 0000000..0388fd0
--- /dev/null
+++ b/nginx/nginx.conf
@@ -0,0 +1,84 @@
+# /etc/nginx/nginx.conf
+user www-data;
+worker_processes auto;
+pid /run/nginx.pid;
+error_log /var/log/nginx/error.log;
+include /etc/nginx/modules-enabled/*.conf;
+
+events {
+ worker_connections 768;
+ # multi_accept on;
+}
+
+http {
+
+ ##
+ # Basic Settings
+ ##
+
+ sendfile on;
+ tcp_nopush on;
+ types_hash_max_size 2048;
+ # server_tokens off;
+
+ # server_names_hash_bucket_size 64;
+ # server_name_in_redirect off;
+
+ include /etc/nginx/mime.types;
+ default_type application/octet-stream;
+
+ ##
+ # SSL Settings
+ ##
+
+ ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
+ ssl_prefer_server_ciphers on;
+
+ ##
+ # Logging Settings
+ ##
+
+ access_log /var/log/nginx/access.log;
+
+ ##
+ # Gzip Settings
+ ##
+
+ gzip on;
+
+ # gzip_vary on;
+ # gzip_proxied any;
+ # gzip_comp_level 6;
+ # gzip_buffers 16 8k;
+ # gzip_http_version 1.1;
+ # gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;
+
+ ##
+ # Virtual Host Configs
+ ##
+
+ include /etc/nginx/conf.d/*.conf;
+ include /etc/nginx/sites-enabled/*;
+}
+
+
+#mail {
+# # See sample authentication script at:
+# # http://wiki.nginx.org/ImapAuthenticateWithApachePhpScript
+#
+# # auth_http localhost/auth.php;
+# # pop3_capabilities "TOP" "USER";
+# # imap_capabilities "IMAP4rev1" "UIDPLUS";
+#
+# server {
+# listen localhost:110;
+# protocol pop3;
+# proxy on;
+# }
+#
+# server {
+# listen localhost:143;
+# protocol imap;
+# proxy on;
+# }
+#}
\ No newline at end of file
diff --git a/nginx/sites-enabled/autobase b/nginx/sites-enabled/autobase
new file mode 100644
index 0000000..057c1cd
--- /dev/null
+++ b/nginx/sites-enabled/autobase
@@ -0,0 +1,43 @@
+server {
+ listen 80;
+ server_name autobase.dwy.is;
+
+ return 301 https://$host$request_uri;
+}
+
+server {
+ server_name autobase.dwy.is;
+
+ location / {
+ # stackoverflow.com/questions/14501047/add-response-header-nginx-proxy-pass
+ # 1. hide the Access-Control-Allow-Origin from the server response
+ proxy_hide_header Access-Control-Allow-Origin;
+ # 2. add a new custom header that allows all * origins instead
+ add_header 'Access-Control-Allow-Origin' '*';
+ add_header 'Access-Control-Allow-Credentials' 'true';
+ add_header 'Access-Control-Allow-Methods' '*';
+ add_header 'Access-Control-Allow-Headers' '*';
+
+ proxy_pass http://172.17.0.1:82;
+ }
+
+ listen 443 ssl; # managed by Certbot
+ ssl_certificate /etc/letsencrypt/live/dwy.is/fullchain.pem; # managed by Certbot
+ ssl_certificate_key /etc/letsencrypt/live/dwy.is/privkey.pem; # managed by Certbot
+ include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
+ ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
+}
+
+server {
+ listen 8080 ssl;
+ ssl_certificate /etc/letsencrypt/live/dwy.is/fullchain.pem; # managed by Certbot
+ ssl_certificate_key /etc/letsencrypt/live/dwy.is/privkey.pem; # managed by Certbot
+ include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
+ ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
+
+ server_name autobase.dwy.is;
+
+ location / {
+ proxy_pass http://172.17.0.1:8082/;
+ }
+}
\ No newline at end of file
diff --git a/nginx/sites-enabled/default b/nginx/sites-enabled/default
new file mode 100644
index 0000000..ebc6122
--- /dev/null
+++ b/nginx/sites-enabled/default
@@ -0,0 +1,161 @@
+##
+# You should look at the following URL's in order to grasp a solid understanding
+# of Nginx configuration files in order to fully unleash the power of Nginx.
+# https://www.nginx.com/resources/wiki/start/
+# https://www.nginx.com/resources/wiki/start/topics/tutorials/config_pitfalls/
+# https://wiki.debian.org/Nginx/DirectoryStructure
+#
+# In most cases, administrators will remove this file from sites-enabled/ and
+# leave it as reference inside of sites-available where it will continue to be
+# updated by the nginx packaging team.
+#
+# This file will automatically load configuration files provided by other
+# applications, such as Drupal or Wordpress. These applications will be made
+# available underneath a path with that package name, such as /drupal8.
+#
+# Please see /usr/share/doc/nginx-doc/examples/ for more detailed examples.
+##
+
+# Default server configuration
+#
+server {
+ listen 80 default_server;
+ listen [::]:80 default_server;
+
+ # SSL configuration
+ #
+ # listen 443 ssl default_server;
+ # listen [::]:443 ssl default_server;
+ #
+ # Note: You should disable gzip for SSL traffic.
+ # See: https://bugs.debian.org/773332
+ #
+ # Read up on ssl_ciphers to ensure a secure configuration.
+ # See: https://bugs.debian.org/765782
+ #
+ # Self signed certs generated by the ssl-cert package
+ # Don't use them in a production server!
+ #
+ # include snippets/snakeoil.conf;
+
+ root /var/www/html;
+
+ # Add index.php to the list if you are using PHP
+ index index.html index.htm index.nginx-debian.html;
+
+ server_name _;
+
+ location / {
+ # First attempt to serve request as file, then
+ # as directory, then fall back to displaying a 404.
+ try_files $uri $uri/ =404;
+ }
+
+ # pass PHP scripts to FastCGI server
+ #
+ #location ~ \.php$ {
+ # include snippets/fastcgi-php.conf;
+ #
+ # # With php-fpm (or other unix sockets):
+ # fastcgi_pass unix:/run/php/php7.4-fpm.sock;
+ # # With php-cgi (or other tcp sockets):
+ # fastcgi_pass 127.0.0.1:9000;
+ #}
+
+ # deny access to .htaccess files, if Apache's document root
+ # concurs with nginx's one
+ #
+ #location ~ /\.ht {
+ # deny all;
+ #}
+}
+
+
+# Virtual Host configuration for example.com
+#
+# You can move that to a different file under sites-available/ and symlink that
+# to sites-enabled/ to enable it.
+#
+#server {
+# listen 80;
+# listen [::]:80;
+#
+# server_name example.com;
+#
+# root /var/www/example.com;
+# index index.html;
+#
+# location / {
+# try_files $uri $uri/ =404;
+# }
+#}
+
+server {
+
+ # SSL configuration
+ #
+ # listen 443 ssl default_server;
+ # listen [::]:443 ssl default_server;
+ #
+ # Note: You should disable gzip for SSL traffic.
+ # See: https://bugs.debian.org/773332
+ #
+ # Read up on ssl_ciphers to ensure a secure configuration.
+ # See: https://bugs.debian.org/765782
+ #
+ # Self signed certs generated by the ssl-cert package
+ # Don't use them in a production server!
+ #
+ # include snippets/snakeoil.conf;
+
+ root /var/www/html;
+
+ # Add index.php to the list if you are using PHP
+ index index.html index.htm index.nginx-debian.html;
+ server_name dwy.is; # managed by Certbot
+
+
+ location / {
+ # First attempt to serve request as file, then
+ # as directory, then fall back to displaying a 404.
+ try_files $uri $uri/ =404;
+ }
+
+ # pass PHP scripts to FastCGI server
+ #
+ #location ~ \.php$ {
+ # include snippets/fastcgi-php.conf;
+ #
+ # # With php-fpm (or other unix sockets):
+ # fastcgi_pass unix:/run/php/php7.4-fpm.sock;
+ # # With php-cgi (or other tcp sockets):
+ # fastcgi_pass 127.0.0.1:9000;
+ #}
+
+ # deny access to .htaccess files, if Apache's document root
+ # concurs with nginx's one
+ #
+ #location ~ /\.ht {
+ # deny all;
+ #}
+
+
+ listen [::]:443 ssl ipv6only=on; # managed by Certbot
+ listen 443 ssl; # managed by Certbot
+ ssl_certificate /etc/letsencrypt/live/dwy.is/fullchain.pem; # managed by Certbot
+ ssl_certificate_key /etc/letsencrypt/live/dwy.is/privkey.pem; # managed by Certbot
+ include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
+ ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
+
+}
+server {
+ if ($host = dwy.is) {
+ return 301 https://$host$request_uri;
+ } # managed by Certbot
+
+
+ listen 80 ;
+ listen [::]:80 ;
+ server_name dwy.is;
+ return 404; # managed by Certbot
+}
\ No newline at end of file
diff --git a/postgres/README.md b/postgres/README.md
index cc360fa..de9290e 100644
--- a/postgres/README.md
+++ b/postgres/README.md
@@ -1,23 +1,26 @@
-

# Deployment
-`Postgres` deployment is divided into two options:
+`Postgres` deployment is split into two options:
1. Managed - the infrastructure provider manages the instances for you
including data backups and failover in the event of a crash/corruption.
+
2. Unmanaged - we the engineers or operations team need to manage it
including data integrity and availability.
Both have their place.
But if you are not an _experienced_
-Database Administrator (DBA)
-or System Administrator (SysAdmin),
+Database Administrator
+([DBA](https://en.wikipedia.org/wiki/Database_administration))
+or System Administrator
+([SysAdmin](https://en.wikipedia.org/wiki/System_administrator)),
you should seriously consider a _managed_ service.
The risk of data loss greatly outweighs
the cost of a _managed_ service.
@@ -35,10 +38,12 @@ There are _many_ other options for "cloud" providers
for managed `Postgres` and other `SQL` databases.
We looked at all the major ones including
`AWS`, `GCP`, `Azure`.
-Sadly, `AWS Aurara` while appealing,
-has _deliberately_ confusing pricing:
-https://aws.amazon.com/rds/aurora/pricing
+Sadly, `AWS Aurara` while appealing,
+has **_deliberately_ confusing pricing**:
+[aws.amazon.com/rds/aurora/pricing](https://aws.amazon.com/rds/aurora/pricing)
They have costs for I/O requests, storage,
backtrack (backup) and data transfer.
By contrast `DigitalOcean` has _transparent_
pricing based on the VPS (Memory, CPU and SSD) used.
+But `DigitalOcean` also gets _very_ expensive ...
+So we decided to invest the time to _self-manage_.
diff --git a/postgres/autobase-ha-cluster.md b/postgres/autobase-ha-cluster.md
new file mode 100644
index 0000000..57ede7c
--- /dev/null
+++ b/postgres/autobase-ha-cluster.md
@@ -0,0 +1,387 @@
+
+
+# High Availability `Postgres` Cluster With `Autobase`
+
+
+
+
+
+
+Deploy a
+[high availability](https://en.wikipedia.org/wiki/High_availability)
+`Postgres` database cluster
+(on
+[`Hetzner`](../hetzner))
+using [**`autobase`**](https://autobase.tech).
+
+
+
+Along the way we will clarify as many of the steps as possible.
+But please keep in mind it's _not possible_
+to cover everything in a **7 minute video**.
+
+If you have questions, suggestions or just want to say hi,
+**please comment on YouTube**;
+thanks.
+
+With all that out of the way, lets dive in!
+
+## 1. Login to `Hetzner` Cloud
+
+When you _first_ login to `Hetzner`,
+you will see the message:
+
+"You don't have any servers yet."
+
+
+
+Click the "**Add Server**" button to begin your quest!
+
+> If you don't yet have a `Hezner` account,
+> please consider using our **referral link**:
+> [hetzner.cloud/?ref=ahpZuUB3t0XI](https://hetzner.cloud/?ref=ahpZuUB3t0XI) 🔗
+> to get **`$20`** in credit. 💵
+> Helps us do what we love too. Thanks. ❤
+
+## 2. Create a New "Cloud" Virtual Private Server (VPS)
+
+Select all the default options,
+add your `ssh` `public` key
+and create your server.
+
+
+
+## 3. `SSH` into the `Hetzner` Server
+
+Use your `Terminal` to login to the newly created `Hetzner`server, e.g:
+
+```sh
+ssh root@116.202.31.52
+```
+
+Once you have successfully connected via `ssh`,
+a best practice we recommend is to run a quick update.
+
+### Update The Server
+
+Run the following command chain:
+
+```sh
+sudo apt update -y && sudo apt full-upgrade -y && sudo apt autoremove -y && sudo apt clean -y && sudo apt autoclean -y
+```
+
+> Updates and installs usually take a couple of minutes.
+> We speed installs up for brevity.
+
+With everything up-to-date, install the necessary dependencies.
+
+
+## 4. Install Dependencies
+
+As per the `autobase` getting started guide:
+[autobase.tech/docs#getting-started](https://autobase.tech/docs#getting-started)
+run the following command to get the necessary dependencies:
+
+```sh
+sudo apt update && sudo apt install -y python3-pip sshpass git
+pip3 install ansible
+```
+
+### Install `Docker`
+
+Install `Docker` on the `Ubuntu` server
+following the installation instructions
+in the **official `Docker` docs**:
+https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository
+
+Run:
+
+```sh
+# Add Docker's official GPG key:
+sudo apt-get update
+sudo apt-get install ca-certificates curl
+sudo install -m 0755 -d /etc/apt/keyrings
+sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
+sudo chmod a+r /etc/apt/keyrings/docker.asc
+
+# Add the repository to Apt sources:
+echo \
+ "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
+ $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
+ sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
+sudo apt-get update
+```
+
+Followed by:
+
+```sh
+sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
+```
+
+Verify that the installation is successful by running the `hello-world` image:
+
+```sh
+$ sudo docker run hello-world
+```
+
+With that confirmed working,
+go back to the previous step and run the `autobase`command.
+
+## 5. Run `autobase` Console Boot Script
+
+Install and run the `latest` version of `Autobase` console.
+
+Sample:
+
+```sh
+docker run -d --name autobase-console \
+ --publish 80:80 \
+ --publish 8080:8080 \
+ --env PG_CONSOLE_API_URL=http://localhost:8080/api/v1 \
+ --env PG_CONSOLE_AUTHORIZATION_TOKEN=secret_token \
+ --env PG_CONSOLE_DOCKER_IMAGE=autobase/automation:latest \
+ --volume console_postgres:/var/lib/postgresql \
+ --volume /var/run/docker.sock:/var/run/docker.sock \
+ --volume /tmp/ansible:/tmp/ansible \
+ --restart=unless-stopped \
+ autobase/console:latest
+```
+
+You will nee to replace the `localhost` in the `PG_CONSOLE_API_URL`
+with the IP (v4) address of your server
+and `secret_token` for the `PG_CONSOLE_AUTHORIZATION_TOKEN`
+
++ IP: 116.202.31.52 (yours will be different!)
++ Token: 5b0b6259-a7d4-4435-947dba (create your own!)
+
+Actual:
+
+```sh
+docker run -d --name autobase-console \
+ --publish 80:80 \
+ --publish 8080:8080 \
+ --env PG_CONSOLE_API_URL=http://116.202.31.52:8080/api/v1 \
+ --env PG_CONSOLE_AUTHORIZATION_TOKEN=5b0b6259-a7d4-4435-947dba \
+ --env PG_CONSOLE_DOCKER_IMAGE=autobase/automation:latest \
+ --volume console_postgres:/var/lib/postgresql \
+ --volume /var/run/docker.sock:/var/run/docker.sock \
+ --volume /tmp/ansible:/tmp/ansible \
+ --restart=unless-stopped \
+ autobase/console:latest
+```
+
+Confirm it worked with the `docker ps` command.
+You should see something similar to the following:
+
+```sh
+CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
+
+9740dfd66c42 autobase/console:latest "/usr/bin/supervisor…" About a minute ago Up About a minute 0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp, 5432/tcp autobase-console
+```
+
+## 6. Login To `autobase` Console Web UI
+
+Visit the IP Address of your server in you web browser e.g:
+http://116.202.31.52
+
+You should see a login screen:
+
+
+
+Copy-paste the Token you defined in step 5 above.
+
+When you first login you should see that there are **No Postgres Clusters**:
+
+
+
+## 7. Create Postgres Cluster
+
+Click the "**CREATE CLUSTER**" button:
+
+
+
+Select `hetzner`and the datacenter region you prefer, in our case Europe:
+
+
+
+The default disk storage is **`100Gb`**;
+
+
+
+this is _way_ too much for most simple projects.
+lower it to **`10Gb`** for each instance to instantly save **50%** of the cost!
+(you're welcome!)
+
+
+
+> **Note**: all values for `DISK`storage, `RAM`, and `CPU`can easily be scaled later.
+
+Finally, you'll need to add your `public` SSH key.
+
+### Copy Your SSH Public Key
+
+```sh
+cat ~/.ssh/id_ed25519.pub | pbcopy
+```
+
+Paste it into the **`SSH public key`** field:
+
+
+
+Then scroll down and click the "**CREATE CLUSTER**" button.
+
+You will see a modal window appear prompting you to input a `Hetzner`API Key:
+
+
+
+### 8. Generate an API token
+
+Follow the instructions in the official `Hetzner` docs:
+https://docs.hetzner.com/cloud/api/getting-started/generating-api-token/
+
+In the `Hetzner`console, navigate to **Security** > **API tokens**.
+You should see the message:
+"**You haven't generated an API token yet.**"
+
+
+
+Click on the "**Generate API token**" button:
+
+
+
+That will open _another_ modal window,
+input the description for your key,
+e.g:
+"postgres-cluster-api-key"
+and select "**Read and Write**":
+
+
+
+Finally, click on the "**Generate API token**" button.
+You should see a confirmation message:
+
+
+
+Click to reveal the token you created:
+
+
+
+Copy the token to your clipboard, e.g:
+
+```sh
+zH2qdgCeogrKjVKgV7sngMRxCfewgSdDARUBr8yqcjuHhGzlNdY72H13Sjh1il2D
+```
+
+> **Note**: for security reasons, this API key is no longer valid.
+
+Paste it into the Cluster creation window:
+
+
+
+_Optionally_ save the API Key to the console and then
+click "**CREATE CLUSTER**":
+
+
+
+created:
+
+
+
+Cluster details:
+
+
+
+The `Postgres` cluster _appears_ to be deployed,
+but how do we _know_ that it worked?
+
+## 9. Test The Cluster! 👩🔬
+
+First: _connect_ to the **primary** `Postgres` instance.
+In our case this is: `10.0.1.4`
+
+
+
+Sample:
+
+```sh
+export PGPASSWORD='password';
+psql -h 127.0.0.1 -p 5432 -U postgres -d postgres
+```
+
+Get the **Password** and **Port** from the **Connection info** panel:
+
+
+
+Actual:
+
+```sh
+export PGPASSWORD='9Djw2LNRMWwaDS1F9TlxeXiGj4dV3zNk';
+psql -h 88.99.81.115 -p 5432 -U postgres -d postgres
+```
+
+```sh
+psql -h 10.0.1.4 -p 6432 -U postgres -d postgres
+```
+
+```sh
+psql -h 10.0.1.4 -p 6432 -U postgres -d postgres -c "select version()"
+```
+
+Got the following error:
+
+```sh
+Command 'psql' not found, but can be installed with:
+
+apt install postgresql-client-common
+```
+
+This is a barebones `Ubuntu` instance, remember, so it's not surprising that it doesn't have `psql` installed. So follow the instruction and install it:
+
+```sh
+sudo apt install postgresql-client-common
+```
+
+The output is:
+
+But when trying to run `psql` again, we still get an error:
+
+```sh
+Error: You must install at least one postgresql-client- package
+```
+
+## Outro:
+
+Given that this is a technical guide for an evolving system,
+it may need to be enhanced/extended or updated in future,
+that will be done on GitHub;
+_everyone_ is welcome to and _encouraged_ to contribute!
+Again, link in the description.
+
+Thanks for watching/listening.
+If you found it useful and want to see more,
+please subscribe.
+
+## Privacy Disclaimer
+
+By the time you read/watch this,
+all of the sensitive data such as passwords, IP addresses,
+public keys and auth tokens will have been updated.
+This avoids anyone getting ideas about accessing backend systems.
+
+We publish our notes and videos on how we do things
+so that we can be as transparent as possible.
+We have a strong security & privacy focus for all our systems
+so all private backend systems like databases are always locked down.
+
+As always, if you have a security question or concern,
+please contact us responsibly.
+
diff --git a/postgres/backup-fly-postgres.md b/postgres/backup-fly-postgres.md
index e1eea0c..ca59f33 100644
--- a/postgres/backup-fly-postgres.md
+++ b/postgres/backup-fly-postgres.md
@@ -1,9 +1,9 @@
-# How to Backup Fly.io Postgres Database
+# How to Backup `Fly.io` `Postgres` Database
A comprehensive step-by-step guide
-to backing-up your Fly.io `Postgres` database
+to backing-up your `Fly.io` `Postgres` database
on your `localhost`.
@@ -12,11 +12,22 @@ on your `localhost`.
You need to get the data
from a Fly.io `Postgres` instance.
+_Your_ reason may be different,
+see our [context](#context-) below.
+
+## What? 🤔
+
+Backup your `Postgres` DB running on `Fly.io`
+and use the data somewhere `else`;
+in our case we are migrating our DBs to `Hetzner`
+where we have a
+[high availability](https://en.wikipedia.org/wiki/High_availability)
+cluster.
## How? 👩💻
### 0. Before You Start
-n
+
Before you attempt to access the `Postgres` database on `Fly.io`,
ensure you are authenticated with your `Fly.io` account;
run the command:
@@ -79,29 +90,39 @@ pg_dump -h localhost -U hits_e2k5m6j4k46d0v7p -d hits --verbose > backup.sql
> **Note**: If you need to get the password for the
`Postgres` instance, use the following command:
+
```sh
flyctl ssh console -a hits -C "printenv DATABASE_URL"
```
+
> in our case it was:
+
```sh
flyctl ssh console -a hits -C "printenv DATABASE_URL"
```
+
We saw:
+
```sh
postgres://hits_e2k5m6j4k46d0v7p:baf3d9f0bdf155bfetc@hits-db.internal:5432/hits?sslmode=disable
```
+
Where the first section `postgres://` is the protocol,
the `hits_e2k5m6j4k46d0v7p` is the DB username,
`baf3d9f0bdf155bfetc` is the password
and `hits` is the name of the database.
Ref:
https://community.fly.io/t/how-to-view-environment-variables-in-a-fly-machine/10830/2
+
Once you have the password,
export it as an environtment variable:
+
```sh
export PGPASSWORD="$put_here_the_password"
```
-> in our case it was:
+
+in our case it was:
+
```sh
export PGPASSWORD="baf3d9f0bdf155bfetc"
```
@@ -114,7 +135,7 @@ Once the `pg_dump` command finishes, proceed to the next step.
### 3. Close your port forwarding
-Kill the connection to the `Fly.io` instance
+Kill the connection to the `Fly.io` instance
using keyboard shortcut: `Ctrl` + `C` (twice).
### 4. Restore your local database
@@ -123,7 +144,6 @@ To restore the database you just backed up to `Postgres`
running on your `localhost`,
you _first_ need to ensure that `Postgres` is indeed running!
-
With the `backup.sql` on your `localhost`,
run the following command in the working directory:
@@ -158,7 +178,7 @@ e.g: http://localhost:8081/#
-i.e. there have been 3 page views on https://github.com/dwyl/start-here since we did the SQL dump a few mins ago.
+i.e. there have been 3 page views on https://github.com/dwyl/start-here since we did the SQL dump a few mins ago.
We can check the "live" count at: https://hits.dwyl.com/dwyl/start-here.svg
e.g: 
diff --git a/postgres/migrate-db.md b/postgres/migrate-db.md
new file mode 100644
index 0000000..5301991
--- /dev/null
+++ b/postgres/migrate-db.md
@@ -0,0 +1,56 @@
+
+
+# Migrate `Postgres` DB to `Hetzner` Cluster
+
+
+
+Migrate/restore a snapshot of `Postgres` Database
+from `Fly.io` (unreliable) to a
+[high availability](https://en.wikipedia.org/wiki/High_availability)
+cluster
+running on `Hetzner`.
+
+## 0. Before You Start: Get the Snapshot
+
+We wrote _detailed_ instructions for backing up
+a `Postgres` DB running on `Fly.io`,
+see:
+[postgres/backup-fly-postgres.md]
+
+
+
+With the `backup.sql` on your `localhost`,
+you can start.
+
+## 1. Connect To `Hezner` VPS Using `Cyberduck`
+
+There are several ways to upload large files to a remote server,
+we've been using
+[`Cyberduck`](https://en.wikipedia.org/wiki/Cyberduck)
+for the past few decades and it works very well.
+It uses
+[`SFTP`](https://en.wikipedia.org/wiki/SSH_File_Transfer_Protocol)
+to securely transfer files
+and is Open Source:
+[github.com/iterate-ch/cyberduck](https://github.com/iterate-ch/cyberduck)
+
+> The Official Docs are great:
+[docs.cyberduck.io](https://docs.cyberduck.io/cyberduck/)
+and if you get stuck,
+just Google:
+[google.com/search?q=cyberduck+tutorial](https://www.google.com/search?q=cyberduck+tutorial)
+
+Open `Cyberduck`
+and navigate to the `/tmp` directory of the VPS:
+
+
+
+Drag the `backup.sql` file from the `finder` window on `localhost`
+to the `Cyberduc` window to start the upload.
+
+
+
+Take a screen break and refill your water bottle
+while you wait for upload to complete.
+
+
\ No newline at end of file