Skip to content

Commit e5eb836

Browse files
LakshanSSLakshanSS
committed
docs: add tutorial for db provisioning workflow
Signed-off-by: LakshanSS <lakshan230897@gmail.com>
1 parent f2a7373 commit e5eb836

2 files changed

Lines changed: 387 additions & 0 deletions

File tree

‎docs/use-cases/self-service-database-provisioning.mdx‎

Lines changed: 386 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,386 @@
1+
---
2+
title: Self-Service Database Provisioning with Workflows
3+
description: Let developers spin up throwaway AWS RDS PostgreSQL instances on demand using an OpenChoreo Generic Workflow — no platform team involvement required.
4+
sidebar_position: 6
5+
---
6+
7+
# Self-Service Database Provisioning with Workflows
8+
9+
This guide walks you through using an OpenChoreo Generic Workflow to provision an AWS RDS PostgreSQL instance on demand. A developer can fill in a few parameters and trigger the workflow to get a ready-to-use connection string — without waiting on the platform team to manually create infrastructure.
10+
11+
## Overview
12+
13+
A common pain point in development is waiting for a database instance to be provisioned for feature work or testing. With OpenChoreo's Workflow Plane, you can package the full infrastructure provisioning lifecycle into a workflow that any developer can self-service trigger.
14+
15+
This use case uses the [`aws-rds-postgres-create`](https://github.com/openchoreo/openchoreo/tree/main/samples/workflows/aws-rds-postgres-create) sample, which:
16+
17+
1. Clones the repository to retrieve the Terraform configuration
18+
2. Creates an S3 bucket for Terraform state (if it does not already exist)
19+
3. Runs `terraform init`, `plan`, and `apply` to provision the instance
20+
4. Prints the connection string at the end
21+
22+
The companion `aws-rds-postgres-delete` workflow tears everything down using the same Terraform state, so cleanup is just as self-service as provisioning.
23+
24+
### What gets provisioned
25+
26+
| Resource | Details |
27+
|---|---|
28+
| `aws_db_instance` | PostgreSQL `db.t3.micro` (free-tier eligible), 20 GiB gp2 storage, single-AZ |
29+
| `aws_db_subnet_group` | Uses subnets from the default VPC |
30+
| `aws_security_group` | Opens port 5432 so you can connect from your workstation |
31+
32+
:::note
33+
This sample is sized for development and testing. The instance is publicly accessible and not encrypted. For production workloads, review the Terraform configuration and restrict the security group and access settings accordingly.
34+
:::
35+
36+
---
37+
38+
## Prerequisites
39+
40+
Before you begin, ensure you have:
41+
42+
- **OpenChoreo installed** in your Kubernetes cluster with the Workflow Plane enabled
43+
- **kubectl** configured to access your cluster
44+
- **An AWS account** with an IAM user that has the required permissions (see below)
45+
46+
### IAM permissions
47+
48+
The IAM user needs permission to manage RDS instances, related EC2 networking resources, and S3 for Terraform state. The S3 bucket is created automatically by the workflow on first run — no manual bucket creation needed.
49+
50+
```json
51+
{
52+
"Version": "2012-10-17",
53+
"Statement": [
54+
{
55+
"Effect": "Allow",
56+
"Action": [
57+
"rds:CreateDBInstance",
58+
"rds:DeleteDBInstance",
59+
"rds:DescribeDBInstances",
60+
"rds:AddTagsToResource",
61+
"rds:ListTagsForResource",
62+
"rds:CreateDBSubnetGroup",
63+
"rds:DescribeDBSubnetGroups",
64+
"rds:DeleteDBSubnetGroup",
65+
"ec2:DescribeVpcs",
66+
"ec2:DescribeVpcAttribute",
67+
"ec2:DescribeSubnets",
68+
"ec2:DescribeSecurityGroups",
69+
"ec2:DescribeNetworkInterfaces",
70+
"ec2:CreateSecurityGroup",
71+
"ec2:AuthorizeSecurityGroupIngress",
72+
"ec2:AuthorizeSecurityGroupEgress",
73+
"ec2:RevokeSecurityGroupEgress",
74+
"ec2:DeleteSecurityGroup",
75+
"ec2:CreateTags",
76+
"s3:CreateBucket",
77+
"s3:GetObject",
78+
"s3:PutObject",
79+
"s3:ListBucket",
80+
"s3:DeleteObject"
81+
],
82+
"Resource": "*"
83+
}
84+
]
85+
}
86+
```
87+
88+
---
89+
90+
## Step 1 — Store your credentials as a Kubernetes Secret
91+
92+
The workflow reads AWS credentials and the database password from a Kubernetes Secret. Create the secret in the workflow execution namespace.
93+
94+
:::info Workflow execution namespace
95+
Workflows execute in a namespace named `workflows-<namespace>`, where `<namespace>` is the namespace your `WorkflowRun` is applied in. If you apply the `WorkflowRun` to the `default` namespace, the execution namespace is `workflows-default`.
96+
:::
97+
98+
```bash
99+
kubectl create secret generic aws-rds-credentials \
100+
--from-literal=accessKeyId=<your-aws-access-key-id> \
101+
--from-literal=secretAccessKey=<your-aws-secret-access-key> \
102+
--from-literal=dbPassword=<your-chosen-db-password> \
103+
--namespace=workflows-default
104+
```
105+
106+
Replace the placeholder values:
107+
108+
- `<your-aws-access-key-id>` — AWS access key ID for the IAM user above
109+
- `<your-aws-secret-access-key>` — Corresponding secret access key
110+
- `<your-chosen-db-password>` — Master password you want to set on the PostgreSQL instance
111+
112+
The database password is stored in the Secret and injected into the workflow as an environment variable. It is never passed as a plain workflow parameter.
113+
114+
---
115+
116+
## Step 2 — Deploy the Workflow definitions
117+
118+
The workflow is defined across two Kubernetes resources that need to be applied to different clusters:
119+
120+
- **`ClusterWorkflowTemplate`** — the Argo Workflows execution template, applied to the **Workflow Plane** cluster
121+
- **`Workflow`** — the OpenChoreo CR that defines the parameter schema, applied to the **Control Plane** cluster
122+
123+
Both resources are bundled in the same YAML file. Apply it with:
124+
125+
```bash
126+
kubectl apply -f https://raw.githubusercontent.com/openchoreo/openchoreo/main/samples/workflows/aws-rds-postgres-create/aws-rds-postgres-create.yaml
127+
```
128+
129+
Or if you have the repository cloned locally:
130+
131+
```bash
132+
kubectl apply -f samples/workflows/aws-rds-postgres-create/aws-rds-postgres-create.yaml
133+
```
134+
135+
Verify the `Workflow` CR was created:
136+
137+
```bash
138+
kubectl get workflow aws-rds-postgres-create -n default
139+
```
140+
141+
---
142+
143+
## Step 3 — Trigger a WorkflowRun
144+
145+
A `WorkflowRun` is what actually kicks off an execution. Create one with your specific values:
146+
147+
```bash
148+
kubectl apply -f - <<EOF
149+
apiVersion: openchoreo.dev/v1alpha1
150+
kind: WorkflowRun
151+
metadata:
152+
name: my-app-db-run
153+
namespace: default
154+
spec:
155+
workflow:
156+
name: aws-rds-postgres-create
157+
parameters:
158+
git:
159+
repoUrl: "https://github.com/openchoreo/openchoreo.git"
160+
branch: "main"
161+
tfPath: "samples/workflows/aws-rds-postgres-create/terraform"
162+
aws:
163+
region: "us-east-1"
164+
credentialsSecret: "aws-rds-credentials"
165+
tfState:
166+
s3Bucket: "my-org-rds-tfstate"
167+
db:
168+
identifier: "my-app-db"
169+
name: "myappdb"
170+
username: "dbadmin"
171+
engineVersion: "16"
172+
EOF
173+
```
174+
175+
Replace the following with your own values:
176+
177+
| Field | Description |
178+
|---|---|
179+
| `aws.region` | AWS region to create the RDS instance in (e.g. `us-east-1`, `ap-southeast-1`) |
180+
| `aws.credentialsSecret` | Name of the Secret you created in Step 1 |
181+
| `tfState.s3Bucket` | A globally unique S3 bucket name for Terraform state storage. The workflow creates this bucket if it doesn't exist. |
182+
| `db.identifier` | A unique RDS instance identifier (e.g. `my-app-db`). Also used as the Terraform state key prefix, so different identifiers get isolated state. |
183+
| `db.name` | The initial database name to create inside the instance |
184+
| `db.username` | Master username for the database |
185+
186+
---
187+
188+
## Step 4 — Monitor the workflow
189+
190+
Check that the `WorkflowRun` was accepted:
191+
192+
```bash
193+
kubectl get workflowrun my-app-db-run -n default
194+
```
195+
196+
The workflow runs six sequential steps. You can follow the progress by watching the underlying Argo Workflow in the execution namespace:
197+
198+
```bash
199+
kubectl get workflow -n workflows-default
200+
```
201+
202+
To stream logs from a specific step (for example the apply step):
203+
204+
```bash
205+
kubectl logs -n workflows-default -l workflows.argoproj.io/workflow=my-app-db-run --follow
206+
```
207+
208+
The pipeline steps in order:
209+
210+
| Step | What it does |
211+
|---|---|
212+
| `clone` | Clones the repository to get the Terraform files |
213+
| `setup` | Creates the S3 state bucket if it doesn't exist yet |
214+
| `init` | Runs `terraform init` with the S3 backend configured |
215+
| `plan` | Runs `terraform plan` — output is visible in logs for review |
216+
| `apply` | Runs `terraform apply` to provision the RDS instance |
217+
| `report` | Prints the connection details |
218+
219+
:::tip
220+
RDS instance creation typically takes 5–10 minutes. The `apply` step will appear to hang during this time — this is expected while AWS provisions the instance.
221+
:::
222+
223+
---
224+
225+
## Step 5 — Get the connection details
226+
227+
Once the workflow completes, the `report` step prints the connection summary. View it in the logs:
228+
229+
```bash
230+
kubectl logs -n workflows-default \
231+
-l workflows.argoproj.io/workflow=my-app-db-run \
232+
-c main --tail=30
233+
```
234+
235+
The output looks like this:
236+
237+
```
238+
=================================================
239+
AWS RDS PostgreSQL Instance Created
240+
=================================================
241+
Host: my-app-db.xxxxxxxxxxxx.us-east-1.rds.amazonaws.com
242+
Port: 5432
243+
Database: myappdb
244+
Username: dbadmin
245+
ARN: arn:aws:rds:us-east-1:111122223333:db:my-app-db
246+
-------------------------------------------------
247+
Connection String (template):
248+
postgresql://dbadmin:<password>@my-app-db.xxxxxxxxxxxx.us-east-1.rds.amazonaws.com:5432/myappdb
249+
250+
NOTE: Password is stored in the 'aws-rds-credentials'
251+
Kubernetes Secret under the 'dbPassword' key.
252+
Retrieve it with:
253+
kubectl get secret aws-rds-credentials \
254+
-o jsonpath='{.data.dbPassword}' | base64 -d
255+
=================================================
256+
```
257+
258+
Retrieve the password and build the full connection string:
259+
260+
```bash
261+
DB_PASSWORD=$(kubectl get secret aws-rds-credentials \
262+
-n workflows-default \
263+
-o jsonpath='{.data.dbPassword}' | base64 -d)
264+
265+
echo "postgresql://dbadmin:${DB_PASSWORD}@<host>:5432/myappdb"
266+
```
267+
268+
Replace `<host>` with the hostname from the report output.
269+
270+
### Test the connection
271+
272+
You can verify connectivity using `psql`:
273+
274+
```bash
275+
psql "postgresql://dbadmin:${DB_PASSWORD}@<host>:5432/myappdb"
276+
```
277+
278+
Or using any PostgreSQL client (DBeaver, DataGrip, TablePlus, etc.) with the host, port, database, username, and password from the report.
279+
280+
---
281+
282+
## Cleaning up — delete the instance
283+
284+
When you no longer need the database, use the companion `aws-rds-postgres-delete` workflow to destroy the RDS instance and all associated AWS resources (subnet group, security group). It reuses the same Terraform state stored in S3, so no manual AWS cleanup is needed.
285+
286+
First, apply the delete workflow definition:
287+
288+
```bash
289+
kubectl apply -f https://raw.githubusercontent.com/openchoreo/openchoreo/main/samples/workflows/aws-rds-postgres-create/aws-rds-postgres-delete.yaml
290+
```
291+
292+
Then trigger it with the **same parameters** you used when creating the instance:
293+
294+
```bash
295+
kubectl apply -f - <<EOF
296+
apiVersion: openchoreo.dev/v1alpha1
297+
kind: WorkflowRun
298+
metadata:
299+
name: my-app-db-delete-run
300+
namespace: default
301+
spec:
302+
workflow:
303+
name: aws-rds-postgres-delete
304+
parameters:
305+
git:
306+
repoUrl: "https://github.com/openchoreo/openchoreo.git"
307+
branch: "main"
308+
tfPath: "samples/workflows/aws-rds-postgres-create/terraform"
309+
aws:
310+
region: "us-east-1"
311+
credentialsSecret: "aws-rds-credentials"
312+
tfState:
313+
s3Bucket: "my-org-rds-tfstate"
314+
db:
315+
identifier: "my-app-db"
316+
name: "myappdb"
317+
username: "dbadmin"
318+
engineVersion: "16"
319+
EOF
320+
```
321+
322+
:::warning
323+
The `db.identifier`, `tfState.s3Bucket`, and `aws.region` values must exactly match the values used when creating the instance. Terraform uses these to locate the correct state file.
324+
:::
325+
326+
Monitor the delete workflow and confirm the instance is gone:
327+
328+
```bash
329+
kubectl logs -n workflows-default \
330+
-l workflows.argoproj.io/workflow=my-app-db-delete-run \
331+
--follow
332+
```
333+
334+
---
335+
336+
## How it works under the hood
337+
338+
Understanding the three OpenChoreo resources helps if you want to adapt this workflow for your own infrastructure:
339+
340+
```
341+
ClusterWorkflowTemplate ← Argo Workflows execution logic (Build/Workflow Plane)
342+
│
343+
│ referenced by
344+
▼
345+
Workflow ← OpenChoreo parameter schema and run template (Control Plane)
346+
│
347+
│ triggered by
348+
▼
349+
WorkflowRun ← A single execution with concrete parameter values
350+
```
351+
352+
**`ClusterWorkflowTemplate`** contains the actual step-by-step logic: git clone, S3 bucket setup, and the three Terraform steps. Each step runs in its own container image (`alpine/git`, `amazon/aws-cli`, `hashicorp/terraform`). A shared `PersistentVolume` at `/mnt/data` passes artifacts (the cloned repo, Terraform outputs) between steps.
353+
354+
**`Workflow`** is the OpenChoreo abstraction. It defines the parameter schema using OpenAPI v3, sets default values, maps parameters to the Argo template arguments, and defines how runs are named.
355+
356+
**`WorkflowRun`** provides the concrete parameter values for one execution and triggers the pipeline.
357+
358+
Terraform state is stored at:
359+
```
360+
s3://<tfState.s3Bucket>/rds/<db.identifier>/terraform.tfstate
361+
```
362+
363+
Each `db.identifier` gets its own isolated state key, so multiple developers can provision independent instances simultaneously without interfering with each other.
364+
365+
---
366+
367+
## Customizing for your own Terraform
368+
369+
The workflow is designed to work with any Terraform configuration, not just the bundled one. To use your own:
370+
371+
1. Fork or host your own repository with Terraform files
372+
2. Set `git.repoUrl` and `git.branch` to point at your repo
373+
3. Set `git.tfPath` to the directory within the repo that contains your `.tf` files
374+
375+
The workflow will clone your repo and run Terraform from that directory at runtime.
376+
377+
---
378+
379+
## Summary
380+
381+
You've learned how to use an OpenChoreo Generic Workflow to let developers self-service provision AWS RDS PostgreSQL instances without involving the platform team. The workflow handles S3 state bucket creation, Terraform init/plan/apply, and outputs the connection string — all triggered by a single `WorkflowRun` CR.
382+
383+
## Next Steps
384+
385+
- Explore the other workflow samples in [Workflow Samples](https://github.com/openchoreo/openchoreo/tree/main/samples/workflows)
386+
- [Deploy a Prebuilt Container Image](./deploy-prebuilt-image.mdx) to connect an application to your new database

‎sidebars.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -197,6 +197,7 @@ const sidebars: SidebarsConfig = {
197197
label: 'Tutorials',
198198
items: [
199199
'use-cases/deploy-prebuilt-image',
200+
'use-cases/self-service-database-provisioning',
200201
]
201202
},
202203
{

0 commit comments

Comments
 (0)