diff --git a/src/how-to/administrate/README.md b/src/how-to/administrate/README.md index fba57e5..83b60d4 100644 --- a/src/how-to/administrate/README.md +++ b/src/how-to/administrate/README.md @@ -68,46 +68,63 @@ ## 9. Restund (TURN) - [Restund (TURN)](restund.md) - - [Wire-Server Configuration](restund.md#wire-server-configuration) - - [How to see how many people are currently connected to the restund server](restund.md#how-to-see-how-many-people-are-currently-connected-to-the-restund-server) - - [How to restart restund (with downtime)](restund.md#how-to-restart-restund-with-downtime) - - [Rebooting a Restund node](restund.md#rebooting-a-restund-node) - - [How to restart restund without having downtime](restund.md#how-to-restart-restund-without-having-downtime) - - [How to renew a certificate for restund](restund.md#how-to-renew-a-certificate-for-restund) - - [How to check which restund/TURN servers will be used by clients](restund.md#how-to-check-which-restund-turn-servers-will-be-used-by-clients) - -## 10. Investigative Tasks + - [Wire-Server Configuration](restund.md#wire-server-configuration) + - [How to see how many people are currently connected to the restund server](restund.md#how-to-see-how-many-people-are-currently-connected-to-the-restund-server) + - [How to restart restund (with downtime)](restund.md#how-to-restart-restund-with-downtime) + - [Rebooting a Restund node](restund.md#rebooting-a-restund-node) + - [How to restart restund without having downtime](restund.md#how-to-restart-restund-without-having-downtime) + - [How to renew a certificate for restund](restund.md#how-to-renew-a-certificate-for-restund) + - [How to check which restund/TURN servers will be used by clients](restund.md#how-to-check-which-restund-turn-servers-will-be-used-by-clients) + +## 10. Coturn Client Certificate with Extended Key Usage (EKU) + +- [Coturn Client Certificate with Extended Key Usage (EKU)](coturn-client-certificate.md) + - [Overview](coturn-client-certificate.md#overview) + - [Prerequisites](coturn-client-certificate.md#prerequisites) + - [Step 1: Create a Self-Signed CA](coturn-client-certificate.md#step-1-create-a-self-signed-ca-skip-if-you-already-have-one) + - [Step 2: Create a Certificate Signing Request (CSR)](coturn-client-certificate.md#step-2-create-a-certificate-signing-request-csr) + - [Step 3: Sign the Certificate with Your CA](coturn-client-certificate.md#step-3-sign-the-certificate-with-your-ca) + - [Step 4: Verify Certificate Has Correct EKU](coturn-client-certificate.md#step-4-verify-certificate-has-correct-eku) + - [Step 5: Prepare Certificate Files](coturn-client-certificate.md#step-5-prepare-certificate-files) + - [Step 6: Update Coturn Helm Values](coturn-client-certificate.md#step-6-update-coturn-helm-values) + - [Step 7: Deploy Coturn with Updated Configuration](coturn-client-certificate.md#step-7-deploy-coturn-with-updated-configuration) + - [Step 8: Verify Certificate is Deployed](coturn-client-certificate.md#step-8-verify-certificate-is-deployed) + - [Step 9: Test Federation DTLS Connection](coturn-client-certificate.md#step-9-test-federation-dtls-connection) + - [Troubleshooting](coturn-client-certificate.md#troubleshooting) + - [Certificate Renewal](coturn-client-certificate.md#certificate-renewal) + +## 11. Investigative Tasks - [Investigative tasks (e.g. searching for users as server admin)](users.md) - - [Manually searching for users in Cassandra](users.md#manually-searching-for-users-in-cassandra) - - [Deleting a user which is not a team user](users.md#deleting-a-user-which-is-not-a-team-user) - - [Searching and deleting users with no team](users.md#searching-and-deleting-users-with-no-team) - - [Manual search on Elasticsearch (via brig, recommended)](users.md#manual-search-on-elasticsearch-via-brig-recommended) - - [How to manually search for a user on Elasticsearch directly (not recommended)](users.md#how-to-manually-search-for-a-user-on-elasticsearch-directly-not-recommended) - - [How to manually delete a user from Elasticsearch only](users.md#how-to-manually-delete-a-user-from-elasticsearch-only) - - [Mass-invite users to a team](users.md#mass-invite-users-to-a-team) - - [How to obtain logs from an Android client to investigate issues](users.md#how-to-obtain-logs-from-an-android-client-to-investigate-issues) - - [How to obtain logs from an iOS client to investigate issues](users.md#how-to-obtain-logs-from-an-ios-client-to-investigate-issues) - - [How to retrieve metric values manually](users.md#how-to-retrieve-metric-values-manually) - - [Reset session cookies](users.md#reset-session-cookies) - - [Identify all users using SSO](users.md#identify-sso-users) - - [Create a team using the SCIM API](users.md#create-a-team-using-the-scim-api) - -## 11. Manuals + - [Manually searching for users in Cassandra](users.md#manually-searching-for-users-in-cassandra) + - [Deleting a user which is not a team user](users.md#deleting-a-user-which-is-not-a-team-user) + - [Searching and deleting users with no team](users.md#searching-and-deleting-users-with-no-team) + - [Manual search on Elasticsearch (via brig, recommended)](users.md#manual-search-on-elasticsearch-via-brig-recommended) + - [How to manually search for a user on Elasticsearch directly (not recommended)](users.md#how-to-manually-search-for-a-user-on-elasticsearch-directly-not-recommended) + - [How to manually delete a user from Elasticsearch only](users.md#how-to-manually-delete-a-user-from-elasticsearch-only) + - [Mass-invite users to a team](users.md#mass-invite-users-to-a-team) + - [How to obtain logs from an Android client to investigate issues](users.md#how-to-obtain-logs-from-an-android-client-to-investigate-issues) + - [How to obtain logs from an iOS client to investigate issues](users.md#how-to-obtain-logs-from-an-ios-client-to-investigate-issues) + - [How to retrieve metric values manually](users.md#how-to-retrieve-metric-values-manually) + - [Reset session cookies](users.md#reset-session-cookies) + - [Identify all users using SSO](users.md#identify-sso-users) + - [Create a team using the SCIM API](users.md#create-a-team-using-the-scim-api) + +## 12. Manuals - [Test an ingress is working from inside the cluster](manuals.md#test-an-ingress-is-working-from-inside-the-cluster) - [Load an image into containerd in an offline/airgapped environment](manuals.md#load-an-image-into-containerd-in-an-offlineairgapped-environment) -## 12. Migration +## 13. Migration - [Migrate team features](migrate-team-features.md) - [Migrate to postgresql](migrate-to-postgresql.md) -## 13. Wire-utility +## 14. Wire-utility - [Wire utility tool](wire-utility-tool.md) -## 14. PostgreSQL +## 15. PostgreSQL - [PostgreSQL](postgresql.md) - [PostgreSQL Connection Budget](postgresql.md#postgresql-connection-budget) diff --git a/src/how-to/administrate/coturn-client-certificate.md b/src/how-to/administrate/coturn-client-certificate.md new file mode 100644 index 0000000..76d9b62 --- /dev/null +++ b/src/how-to/administrate/coturn-client-certificate.md @@ -0,0 +1,367 @@ +# Coturn Client Certificate with Extended Key Usage (EKU) + +This section is about **how to perform a specific task**. If you want to **understand how a certain component works, please see** [Reference](../../understand/README.md#understand) + +This guide explains how to create and deploy a coturn client certificate with Extended Key Usage (EKU) support for federation DTLS connections. When coturn needs to authenticate with federation partner servers over DTLS on port 9191, the certificate must include both server and client authentication capabilities. + +## Overview + +Coturn federation DTLS connections (port 9191) require mutual TLS authentication. Your coturn certificate must be signed by a Certificate Authority (CA) that Wire Cloud trusts. The certificate must include both serverAuth and clientAuth Extended Key Usage (EKU) extensions. + +### Prerequisites + +- Coturn deployment via Helm in Kubernetes +- Access to the coturn Helm values configuration +- `openssl` command-line tool installed locally +- `kubectl` access to your cluster +- FQDN for your coturn deployment (e.g., `coturn.example.com`) +- **CA certificate** and **CA private key** — create one in Step 1 if you don't have one yet; once your federation partner (e.g. Wire Cloud) trusts your CA you can reuse it for all future renewals + +### Deployment Model + +The typical workflow for a self-managed coturn deployment: + +1. **Select issuing CA** (create self-signed or use existing private CA) +2. **Send your CA** to Wire Cloud to add to their federation trust store +3. **Generate a coturn certificate** signed by that CA with serverAuth+clientAuth EKU +4. **Deploy coturn** with the certificate in Helm values +5. **Renew annually** by signing a new certificate with the same CA — no need to contact your federation partner again + +## Step 1: Create a Self-Signed CA (skip if you already have one) + +If your organization already has a CA and your federation partners already trusts it, skip to Step 2. + +Otherwise, create a self-signed CA. You do this **once** — the CA is long-lived (10 years) and is reused for all future coturn certificate renewals without needing to contact your federation partners again. + +```bash +# Generate CA private key (2048-bit RSA) +openssl genrsa -out my-ca-key.pem 2048 + +# Create CA certificate signing request +openssl req -new \ + -key my-ca-key.pem \ + -out my-ca.csr \ + -subj "/C=XX/O=Your Organization/CN=Your Organization CA" + +# Self-sign the CA certificate (valid 10 years = 3650 days) +openssl x509 -req \ + -in my-ca.csr \ + -signkey my-ca-key.pem \ + -out my-ca.pem \ + -days 3650 \ + -extfile <(printf "basicConstraints=critical,CA:TRUE\nkeyUsage=critical,keyCertSign,cRLSign\nsubjectKeyIdentifier=hash\nauthorityKeyIdentifier=keyid:always,issuer:always") +``` + +**Result**: +- `my-ca.pem` — CA certificate; send this to Wire Cloud to add to their federation trust store +- `my-ca-key.pem` — CA private key; keep this secure, it is needed for all future renewals +- `my-ca.csr` — can be deleted + +**Important**: Wait until your federation partner confirms your CA is in their trust store before proceeding. + +## Step 2: Create a Certificate Signing Request (CSR) + +First, generate a private key and create a signing request for your coturn certificate. + +```bash +# Create a coturn private key (2048-bit RSA) +openssl genrsa -out coturn-key.pem 2048 + +# OR use ECDSA if your organization prefers (recommended for modern deployments) +openssl ecparam -name secp256r1 -genkey -noout -out coturn-key.pem + +# Create a certificate signing request with both server and client authentication EKU +# Replace coturn.example.com with your actual coturn FQDN +openssl req -new \ + -key coturn-key.pem \ + -out coturn.csr \ + -subj "/C=US/ST=State/L=City/O=Your Organization/CN=coturn.example.com" \ + -addext "subjectAltName=DNS:coturn.example.com,DNS:coturn-0.example.com,DNS:coturn-1.example.com,DNS:coturn-2.example.com" \ + -addext "extendedKeyUsage=serverAuth,clientAuth" \ + -addext "keyUsage=digitalSignature" +``` + +Verify the CSR: + +```bash +openssl req -in coturn.csr -text -noout +``` + +## Step 3: Sign the Certificate with Your CA + +Using your CA certificate and private key, sign the coturn certificate. This certificate will be valid for 365 days. + +```bash +# Sign the CSR with your CA certificate and key +openssl x509 -req -days 365 \ + -in coturn.csr \ + -CA my-ca.pem \ + -CAkey my-ca-key.pem \ + -CAcreateserial \ + -out coturn-cert.pem \ + -extfile <(printf "subjectAltName=DNS:coturn.example.com,DNS:coturn-0.example.com,DNS:coturn-1.example.com,DNS:coturn-2.example.com\nkeyUsage=digitalSignature\nextendedKeyUsage=serverAuth,clientAuth") +``` + +Verify the signed certificate: + +```bash +openssl x509 -in coturn-cert.pem -text -noout +``` + +## Step 4: Verify Certificate Has Correct EKU + +Before deploying, verify that your certificate includes both serverAuth and clientAuth extensions, and is signed by your CA: + +```bash +# Check Extended Key Usage +openssl x509 -in coturn-cert.pem -text -noout | grep -A 3 "Extended Key Usage" +``` + +Expected output: + +``` +X509v3 Extended Key Usage: + TLS Web Server Authentication, TLS Web Client Authentication +``` + +Verify the certificate issuer: + +```bash +openssl x509 -in coturn-cert.pem -noout -issuer +# Output should show your CA's CN, e.g.: issuer=C=XX, O=Your Organization, CN=Your Organization CA +``` + +Verify the validity dates: + +```bash +openssl x509 -in coturn-cert.pem -noout -dates +# Output should show: notBefore=... and notAfter=... (365 days from now) +``` + +## Step 5: Prepare Certificate Files + +Display the PEM content of your certificate and private key, ready to copy into the Helm values file. The Helm chart encodes them automatically — paste the raw PEM including the `-----BEGIN`/`-----END` headers. + +```bash +echo "=== Coturn Certificate ===" +cat coturn-cert.pem +echo "" +echo "=== Coturn Private Key ===" +cat coturn-key.pem +``` + +Copy both PEM strings for the next step. + +## Step 6: Update Coturn Helm Values + +Update your coturn Helm values file (`values/coturn/values.yaml`) with the PEM certificate and key. Use YAML block scalars (`|`) to preserve the multi-line PEM format: + +```yaml +# Existing coturn configuration +# DTLS Federation certificate configuration +federate: + dtls: + tls: + key: | + -----BEGIN PRIVATE KEY----- + + -----END PRIVATE KEY----- + crt: | + -----BEGIN CERTIFICATE----- + + -----END CERTIFICATE----- + + +## Step 7: Deploy Coturn with Updated Configuration + +**Prerequisite**: Requires coturn chart version `4.6.2-federation-wireapp.44` or later. +Apply the updated Helm values: + +```bash +# Deploy or upgrade the coturn Helm chart with the updated values +helm upgrade --install coturn ./charts/coturn \ + -n default \ + -f values/coturn/values.yaml \ + --wait \ + --timeout 5m +``` + + +Monitor the rollout: + +```bash +kubectl rollout status statefulset/coturn -n default +``` + +The Helm chart will automatically: +1. Create a Kubernetes Secret with the certificate and key data +2. Mount the certificate in each coturn pod +3. Restart all coturn pods to pick up the new certificate +4. Update Helm release history for tracking and potential rollbacks + +## Step 8: Verify Certificate is Deployed + +After the Helm upgrade completes, verify that the certificate is properly mounted in the coturn pods: + +```bash +# Get the coturn pod name +COTURN_POD=$(kubectl get pods -l app=coturn -n -o jsonpath='{.items[0].metadata.name}') + +# Verify the Kubernetes Secret was created +kubectl get secret coturn-dtls-certificate -n -o yaml + +# Check if certificate files exist in the pod +kubectl exec -it $COTURN_POD -n -- ls -la /coturn-dtls-certificate/ + +# Verify the certificate content and EKU from the pod +kubectl exec -it $COTURN_POD -n -- openssl x509 -in /coturn-dtls-certificate/tls.crt -text -noout | grep -A 3 "Extended Key Usage" +``` + +Expected output showing both serverAuth and clientAuth: + +``` +X509v3 Extended Key Usage: + TLS Web Server Authentication, TLS Web Client Authentication +``` + +## Step 9: Test Federation DTLS Connection + +To verify that coturn can now authenticate with federation partners, check the coturn logs: + +```bash +# Get the coturn pod name (if not already set) +COTURN_POD=$(kubectl get pods -l app=coturn -n -o jsonpath='{.items[0].metadata.name}') + +# Stream coturn logs to watch for DTLS connections +kubectl logs -f $COTURN_POD -n + +# Or retrieve recent logs and filter for DTLS-related messages +kubectl logs $COTURN_POD -n --tail=100 | grep -iE 'dtls|tls|certificate|federation' +``` + +Look for successful DTLS connection messages or client authentication confirmations in the logs. + +### Check Helm Release History + +To verify the chart was updated with your changes: + +```bash +# View Helm release history +helm history coturn -n default + +# Get current Helm values +helm get values coturn -n default | grep -A 10 "federate" +``` + +## Troubleshooting + +### Common Diagnostics + +Commands referenced by multiple sections below: + +```bash +# Get coturn pod name +COTURN_POD=$(kubectl get pods -l app=coturn -n -o jsonpath='{.items[0].metadata.name}') + +# Check pod events and logs +kubectl describe pod $COTURN_POD -n +kubectl logs $COTURN_POD -n --previous + +# Check Helm secret +kubectl get secret coturn-dtls-certificate -n -o yaml + +# Check rollout status +kubectl rollout status statefulset/coturn -n +``` + +### Certificate Not Loaded in Pod + +**Symptom**: Pod starts but certificate is missing or stale. + +1. Check the Helm secret exists and has data (see Common Diagnostics). +2. Inspect the certificate in the pod: + ```bash + kubectl exec $COTURN_POD -- openssl x509 -in /coturn-dtls-certificate/tls.crt -noout -dates + ``` +3. If the pod still has the old certificate, restart it: + ```bash + kubectl delete pod $COTURN_POD -n + kubectl wait --for=condition=ready pod -l app=coturn -n --timeout=300s + ``` + +### Certificate Missing Client EKU + +**Symptom**: DTLS authentication fails with federation partners. + +Verify both EKU values are present: + +```bash +openssl x509 -in coturn-cert.pem -text -noout | grep -A 3 "Extended Key Usage" +``` + +Output must show `TLS Web Server Authentication` and `TLS Web Client Authentication`. If not, regenerate following Steps 2–3 and redeploy via Steps 5–7. + +### Pod Fails to Start (CrashLoopBackOff) + +**Symptom**: Pod enters `CrashLoopBackOff` after Helm upgrade. + +1. Check pod events and logs (see Common Diagnostics). +2. Check for YAML syntax errors: + ```bash + helm lint ./charts/coturn -f values/coturn/values.yaml + ``` +3. If issues persist, rollback: + ```bash + helm rollback coturn -n + ``` + +### Helm Upgrade Stuck or Slow + +**Symptom**: `helm upgrade` hangs. + +1. Check pod status: + ```bash + kubectl get pods -n | grep coturn + ``` +2. Re-run with a longer timeout: + ```bash + helm upgrade coturn ./charts/coturn -f values/coturn/values.yaml -n --wait --timeout 10m + ``` + +### Certificate Signed by Wrong CA + +**Symptom**: Certificate looks valid locally but federation partners can't authenticate. + +Check the issuer: + +```bash +openssl x509 -in coturn-cert.pem -noout -issuer +``` + +If it doesn't match your CA, regenerate following Steps 2–3 with the correct CA and redeploy via Steps 5–7. + +## Certificate Renewal + +Renewal follows the same process as initial deployment (Steps 2–8). Your CA is already trusted by Wire Cloud — no need to notify them again. Plan renewal at least 30 days before expiry: + +```bash +openssl x509 -in coturn-cert.pem -noout -dates +``` + +Then follow Steps 2–8 using your existing `my-ca.pem` and `my-ca-key.pem`. + +### Rollback + +If something goes wrong, rollback to the previous Helm release: + +```bash +helm history coturn -n default +helm rollback coturn -n default +kubectl rollout status statefulset/coturn -n default +``` + +## Related Documentation + +- [Coturn Installation](../install/) - General coturn setup in wire-server-deploy +- [Wire-Server TLS Configuration](../install/tls.md) - TLS certificate management +- [Federation Configuration](../../understand/federation/README.md) - Understanding federation in Wire