diff --git a/docs/ddg.md b/docs/ddg.md index 36e2b3a34..ad59090eb 100644 --- a/docs/ddg.md +++ b/docs/ddg.md @@ -4,9 +4,9 @@ Cloud Foundation Fabric Detailed Deployment Guide | Created: | June 04, 2023 | | :------------------------------------ | :------------ | -| Updated: | Mar 06, 2025 | -| Version: | v2.7.1 | -| Most recent changes: | N/A | +| Updated: | May 11, 2027 | +| Version: | v2.9.1 | +| Most recent changes: | Refresh to v2.9.1| ## @@ -67,14 +67,14 @@ To make using this deployment guide easier, the variables described below need t | :------------------------------------ | :------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Billing Account | `billing_account.id` | The billing account to use for the deployment of the environments. Console Link | | Bootstrap Project ID | `bootstrap_project` | The bootstrap project id (created below) | -| Compliance Regime | `assured_workloads.regime` | The compliance regime for this environment, (confirmed working in IL4, IL5, FEDRAMP_HIGH, and NO_COMPLIANCE_REGIME) | +| Compliance Regime | `assured_workloads.regime` | The compliance regime for this environment, (confirmed working in IL4, IL5, FEDRAMP_HIGH, and COMPLIANCE_REGIME_UNSPECIFIED) | | Customer ID | `organization.customer_id` | The Google Workspace Directory Customer ID.
Run gcloud organizations list to view. | | Domain Name | `organization.domain` | The primary Fully Qualified Domain Name (FQDN). Run gcloud organizations list to view (make sure you have authorized as per prerequisites below) | | Alert Email | `alert_email` | The email address used for logging alerts notifications. | | Organization ID | `organization.id` | The Organization ID for the GCP Organization. Run gcloud organizations list to view. | -| Prefix | `prefix` | This is the prefix appended to the beginning of projects and resources deployed selected by your or your organization. Full project names must be globally unique and the prefix must use a maximum of 7 characters. A 409 error will occur if a globally unique project name is not created. | +| Prefix | `prefix` | This is the prefix appended to the beginning of projects and resources deployed selected by your or your organization. Full project names must be globally unique and the prefix must use a maximum of 6 characters. A 409 error will occur if a globally unique project name is not created. | | Region | `assured_workloads.location` | This is the (US) based region that we are deploying resources into (Dual regions like “NAM9” or continents are currently not supported) | -| Tenant Name | `tenants` (Stage 1) | The name for the first tenant that will be deployed via this document. Full project names must be globally unique and the tenant-name must use a maximum of 7 characters. | +| Tenant Name | `tenants` (Stage 1) | The name for the first tenant that will be deployed via this document. Full project names must be globally unique and the tenant-name must use a maximum of 6 characters. | ## Prerequisites @@ -86,10 +86,16 @@ yourself on the organization node or have your Administrator grant them for you. A hard refresh of the cloud console may be required to be able to use the active permissions.** -- A Google Cloud Organization - - If creating a new organization, see Appendix below - - Login it at least once admin.google.com +### Local Environment Setup: - Clone [Stellar Engine Github](https://github.com/google/stellar-engine/) +- Install [Google Cloud SDK](https://cloud.google.com/sdk/docs/install) +- Update local [Terraform](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli) to version >= 1.8.1 +- Install [jq binary](https://jqlang.github.io/jq/download/) + +### Google Cloud Setup: +- A Google Cloud Organization + - If creating a new organization, see [Appendix](#appendices) below + - Login into admin.google.com at least once - [Create a bootstrap project manually in Google Cloud](https://console.cloud.google.com/projectcreate) if you do not already have one @@ -98,11 +104,11 @@ permissions.** - [Enable The Cloud Monitoring API](https://console.developers.google.com/apis/api/monitoring.googleapis.com/overview) in the bootstrap project -- Edit Variables Section above -- Install [Google Cloud SDK](https://cloud.google.com/sdk/docs/install) -- **gcloud auth login** -- **gcloud config set project ``** -- **gcloud auth application-default login** +- Edit [Variables](#variables) Section above +- Authenticate and set the active project + - **gcloud auth login** + - **gcloud config set project ``** + - **gcloud auth application-default login** - Navigate to [IAM & Admin](https://console.cloud.google.com/iam-admin/iam) at the Organization level **_(not project-specific)_** in the GCP Console and assign the following IAM roles for the deploying user. See the note at the @@ -126,16 +132,11 @@ permissions.** automated by running the following script: - **Warning: You will lose all current permissions for your user besides Super User** - - **./setIam.sh \ ``** in the - fast/stages-aw/0-bootstrap folder. + - **./setIam.sh \ ``** from within the fast/stages-aw/0-bootstrap folder - Navigate to the [Super Admin](https://admin.google.com/ac/roles) roles section in Google Workspace to ensure that the deploying user is a Super Admin - -- Update local - [Terraform](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli) - to version \>= 1.8.1 - Follow the [Initial Groups and Administrative Access in Cloud Setup Steps 2 and 3](https://console.cloud.google.com/cloud-setup/overview) instructions adding all the below groups. @@ -154,18 +155,15 @@ permissions.** - gcp-vpc-network-admins@`` - gcp-security-admins@`` - We need to enable these Google Cloud Services by running the following - command: - - `gcloud services enable {iam,cloudkms,pubsub,serviceusage,cloudresourcemanager,bigquery,assuredworkloads,cloudbilling,logging,iamcredentials,orgpolicy}.googleapis.com` + script: + - fast/stages-aw/0-bootstrap/enable_services.sh + - If you run into issues with the above command, you can simply run the following deprecated command (on MacOS, works on other *nix variants) + - `echo "iam cloudkms pubsub serviceusage cloudresourcemanager bigquery assuredworkloads cloudbilling logging iamcredentials orgpolicy" | xargs -n1 -I {} gcloud services enable "{}.googleapis.com”` - [Enable Access Transparency](https://console.cloud.google.com/iam-admin/settings) for your organization - Note: If this is unavailable, make sure you have the Access Transparency Admin role and try again -- Install [jq binary](https://jqlang.github.io/jq/download/) -- (Optional) Install OpenSSL 3.3.1 (The macOS built-in LibreSSL version will - not work. Install via homebrew for macOS) - - For MacOS, see the steps to install OpenSSL - - Follow instruction at https://formulae.brew.sh/formula/openssl@3 - Request “13 projects” here if your quota is below 13 @@ -206,7 +204,7 @@ locations = { bq = "``" gcs = "``" logging = "``" - pubsub = ["``"] + pubsub = "``" kms = "``" } # use `gcloud organizations list` @@ -216,7 +214,7 @@ organization = { customer_id = "``" } outputs_location = "~/fast-config" -# use something unique and no longer than 9 characters +# use something unique and no longer than 6 characters prefix = "``" # full project names must be globally unique log_sinks = { audit-logs = { @@ -237,14 +235,16 @@ log_sinks = { } } org_policies_config = { - import_defaults = false # handled via import script -} + constraints = { + allowed_policy_member_domains = [] #Update with additional customer IDs if needed + } + } fast_features = { envs = true } assured_workloads = { regime = "``" - location = "us-east4" + location = "``" } bootstrap_project = "``" alert_email = "``" @@ -255,27 +255,29 @@ alert_email = "``" 'value(core.account)')** - Type **yes** when prompted - **Note:** You may receive an error in this stage where it reports that - ‘bigquery.googleapis.com\` is not usable in the Assured Workloads. If - you see this error, go to the [Assured Workloads - ](https://console.cloud.google.com/compliance/assuredworkload)page and - for the StellarEngine-`` folder (and Networking folder, if applicable), - click “Review Available Updates”, go to “Allowed Services”, and click - “Allow services” to bring in the BigQuery family of APIs. If prompted, - say yes to the additional dialog confirming your choice. After - making this change, you should wait \~2 minutes and then re-run - **terraform apply -var bootstrap_user=$(gcloud config list --format + ‘bigquery.googleapis.com\` is not usable in the Assured Workloads. + - If you see this error, go to the [Assured Workloads + ](https://console.cloud.google.com/compliance/assuredworkload)page + - Click the StellarEngine-`` folder (and Networking folder, if applicable) + - Click “Review Available Updates”, + - Go to “Allowed Services” + - Click “Allow services” to bring in the BigQuery family of APIs. + - If prompted, say yes to the additional dialog confirming your choice. + - After making this change, you should wait \~2 minutes and then re-run: + - **terraform apply -var bootstrap_user=$(gcloud config list --format 'value(core.account)')** - Type **yes** when prompted - **Note:** You may encounter a bug where your bootstrap project loses access to your billing account. If so [re-enable billing for your bootstrap project](https://console.cloud.google.com/billing/projects) - Switch project to your new project - - **gcloud config set project ``-prod-iac-core-0** + - **gcloud config set project ``-prod-iac-core-0** - Copy the new providers local - **gcloud alpha storage cp gs://``-prod-iac-core-outputs-0/providers/0-bootstrap-providers.tf - .** -- Migrate the state from local to remote **terraform init --migrate-state** + ./** +- Migrate the state from local to remote using + - **terraform init --migrate-state** - Type **yes** when prompted - Run ./**import.sh** - Apply Terraform one more time before moving on to the next stage via @@ -295,10 +297,10 @@ variable as seen below. ### Steps - **Note:** If you are using an external billing account, you have to add the - Billing Account Administrator to - **``-prod-resman-0@``-prod-iac-core-0.iam.gserviceaccount.com**. + Billing Account Administrator for the following service account to the external billing account: + - **``-prod-resman-0@``-prod-iac-core-0.iam.gserviceaccount.com** - **Steps to add the external billing account:** + **Steps to add the external billing account (if applicable):** - In the Google Cloud console (External billing Account), go to the Account management page for the Cloud Billing account, select the Organization level and Go to Account management in Cloud Billing @@ -307,12 +309,11 @@ variable as seen below. do the following: - Click Add principal. - In the New principals field, enter the email address for the principals - you want to add for example - ``-prod-resman-0@``-prod-iac-core-0.iam.gserviceaccount.com + you want to add for example: + - ``-prod-resman-0@``-prod-iac-core-0.iam.gserviceaccount.com - Select a permission for the principal(s) from Select a role as “Billing Account Administrator”. - When done, click Save. - - Change directory into **fast/stages-aw/1-resman** - Copy file **terraform.tfvars.sample** to **terraform.tfvars ** - **cp terraform.tfvars.sample terraform.tfvars** @@ -323,21 +324,21 @@ variable as seen below. ```hcl tenants = { - tenant_name = { ## Change tenant_name here - 7 or less characters - admin_principal = "group:gcp-devops@``" - descriptive_name = "``" ## Change descriptive_name here - locations = { - gcs = "us-east4" - kms = "us-east4" - } +`` = { ## Change tenant_name here - 6 or less characters + admin_principal = "group:gcp-devops@``" + descriptive_name = `` ## Change descriptive_name here + locations = { + gcs = `` + kms = `` + } }, - tenant_name-2 = { ## Change tenant_name-2 here - 7 or less characters - admin_principal = "group:gcp-devops@``" - descriptive_name = "tenant-name-2" ## Change descriptive_name here - locations = { - gcs = "us-east4" - kms = "us-east4" - } + tenant_name-2 = { ## Change tenant_name-2 here - 6 or less characters + admin_principal = "group:gcp-devops@``" + descriptive_name = "tenant-name-2" ## Change descriptive_name here + locations = { + gcs = `` + kms = `` + } } ## You can have “n” number of tenants } @@ -346,19 +347,19 @@ fast_features = { } envs_folders = { Prod = { - admin = "gcp-organization-admins@``" + admin = "gcp-organization-admins@``" }, Int = { - admin = "gcp-organization-admins@``" + admin = "gcp-organization-admins@``" }, Test = { - admin = "gcp-organization-admins@``" + admin = "gcp-organization-admins@``" } } ``` -- Copy the tfvars files from the GCS - - **gcloud storage[^1] cp +- Copy the tfvars files from GCS + - **gcloud storage cp gs://``-prod-iac-core-outputs-0/providers/1-resman-providers.tf ./** - **gcloud storage cp gs://``-prod-iac-core-outputs-0/tfvars/0-globals.auto.tfvars.json @@ -376,10 +377,11 @@ envs_folders = { ### Steps -- **Note: If you are using an external billing account, you have to add the - Billing Account Administrator to - ``-prod-resman-net-0@``-prod-iac-core-0.iam.gserviceaccount.com from - the external account. Steps to add the external billing account:** +**Note:** If you are using an external billing account, you have to add the + Billing Account Administrator for the following service account to the external billing account: + - **``-prod-resman-net-0@``-prod-iac-core-0.iam.gserviceaccount.com** + + **Steps to add the external billing account (if applicable):** - In the Google Cloud console (External billing Account), go to the Account management page for the Cloud Billing account, select the Organization level and Go to Account management in Cloud Billing @@ -388,8 +390,8 @@ envs_folders = { do the following: - Click Add principal. - In the New principals field, enter the email address for the principals - you want to add for example - ``-prod-resman-0@``-prod-iac-core-0.iam.gserviceaccount.com + you want to add for example: + - ``-prod-resman-net-0@``-prod-iac-core-0.iam.gserviceaccount.com - Select a permission for the principal(s) from Select a role as “Billing Account Administrator”. - When done, click Save. @@ -411,7 +413,7 @@ envs_folders = { - **terraform apply** - Type **yes** when prompted -## IL4/IL5 Stage 2.1 - Networking - WIP +## IL4/IL5 Stage 2.1 - Networking ### Description @@ -423,10 +425,11 @@ a VM code and register them. For more instructions, see the README in the the ### Steps -- **Note: If you are using an external billing account, you have to add the - Billing Account Administrator to - ``-prod-resman-net-0@``-prod-iac-core-0.iam.gserviceaccount.com from - the external account. Steps to add the external billing account:** +**Note:** If you are using an external billing account, you have to add the + Billing Account Administrator for the following service account to the external billing account: + - **``-prod-resman-net-0@``-prod-iac-core-0.iam.gserviceaccount.com** + + **Steps to add the external billing account (if applicable):** - In the Google Cloud console (External billing Account), go to the Account management page for the Cloud Billing account, select the Organization level and Go to Account management in Cloud Billing @@ -435,8 +438,8 @@ a VM code and register them. For more instructions, see the README in the the do the following: - Click Add principal. - In the New principals field, enter the email address for the principals - you want to add for example - ``-prod-resman-net-0@``-prod-iac-core-0.iam.gserviceaccount.com + you want to add for example: + - ``-prod-resman-net-0@``-prod-iac-core-0.iam.gserviceaccount.com - Select a permission for the principal(s) from Select a role as “Billing Account Administrator”. - When done, click Save. @@ -455,16 +458,75 @@ a VM code and register them. For more instructions, see the README in the the gs://``-prod-iac-core-outputs-0/tfvars/1-resman.auto.tfvars.json ./** - Run **terraform init** -- Run **terraform apply -target - google_project_iam_custom_role.ngfw-custom-role** - - Type **yes** when prompted -- **terraform apply** +- Run **terraform apply -target google_project_iam_custom_role.ngfw-custom-role** - Type **yes** when prompted - **Note: **If you receive an error relating to a service account and/or - KMS not existing, please click “Settings” in the ``-prod-net-landing-0 + KMS not existing, please click “Settings” in the ``-net-vdss-host storage account on the console, and it will generate the service account for you +## IL2/FedRAMP Mod Stage 2.1 - Networking + +### Steps + +**Note:** If you are using an external billing account, you have to add the + Billing Account Administrator for the following service account to the external billing account: + - **``-prod-resman-net-0@``-prod-iac-core-0.iam.gserviceaccount.com** + + **Steps to add the external billing account (if applicable):** + - In the Google Cloud console (External billing Account), go to the + Account management page for the Cloud Billing account, select the + Organization level and Go to Account management in Cloud Billing + - At the prompt, choose the Cloud Billing account you want to view. + - In the Permissions panel, To add new principals and assign permissions, + do the following: + - Click Add principal. + - In the New principals field, enter the email address for the principals + you want to add for example: + - ``-prod-resman-net-0@``-prod-iac-core-0.iam.gserviceaccount.com + - Select a permission for the principal(s) from Select a role as “Billing + Account Administrator”. + - When done, click Save. + +- Change directory into **fast/stages-aw/2-networking-c-fedramp-mod** +- Copy file terraform.tfvars.sample to terraform.tfvars + - `cp terraform.tfvars.sample terraform.tfvars` +- Update information in terraform.tfvars as follows + - The `ngfw` variable determines whether or not firewall endpoints will be created for your network. + - Available options for `min_tls_version`: TLS_VERSION_UNSPECIFIED, TLS_1_0, TLS_1_1, TLS_1_2, TLS_1_3. + - Available options for `tls_feature_profile`: PROFILE_UNSPECIFIED, PROFILE_COMPATIBLE, PROFILE_MODERN, PROFILE_RESTRICTED, PROFILE_CUSTOM. + - You may also delete the `tls` variable if you wish to have no TLS inspection policy associated with your firewall endpoints. + +**`fast/stages-aw/2-networking-c-fedramp-mod/terraform.tfvars`** +```hcl +ngfw = true +tls = { + min_tls_version = "TLS_1_2", + tls_feature_profile = "PROFILE_MODERN" +} +``` + +- Copy the **terraform.tfvars.tf** files from the GCS buckets + - **gcloud storage cp + gs://``-prod-iac-core-outputs-0/providers/2-networking-providers.tf + ./** + - **gcloud storage cp + gs://``-prod-iac-core-outputs-0/tfvars/0-globals.auto.tfvars.json + ./** + - **gcloud storage cp + gs://``-prod-iac-core-outputs-0/tfvars/0-bootstrap.auto.tfvars.json + ./** + - **gcloud storage cp + gs://``-prod-iac-core-outputs-0/tfvars/1-resman.auto.tfvars.json + ./** +- Run **terraform init** +- Run **terraform apply -target google_project_iam_custom_role.ngfw-custom-role** + - Type **yes** when prompted + - **Note: **If you receive an error relating to a service account and/or + KMS not existing, please click “Settings” in the ``-net-vdss-host + storage account on the console, and it will generate the service account + for you + ## Stage 3 - Security and Audit Account Configuration ### Description @@ -497,9 +559,9 @@ are responsible for the audit project. ### Steps - **Note: If you are using an external billing account, you have to add the - Billing Account Administrator to - ``-security-0@``-prod-iac-core-0.iam.gserviceaccount.com from the - external account.** + Billing Account Administrator for the following service account to the external billing account:** + - ``-security-0@``-prod-iac-core-0.iam.gserviceaccount.com + **Steps to add the external billing account:** - In the Google Cloud console (External billing Account), go to the Account management page for the Cloud Billing account, select the @@ -509,8 +571,8 @@ are responsible for the audit project. do the following: - Click Add principal. - In the New principals field, enter the email address for the principals - you want to add for example - ``-security-0@``-prod-iac-core-0.iam.gserviceaccount.com + you want to add for example: + - ``-security-0@``-prod-iac-core-0.iam.gserviceaccount.com - Select a permission for the principal(s) from Select a role as “Billing Account Administrator”. - When done, click Save. @@ -535,9 +597,7 @@ are responsible for the audit project. **terraform apply** - Run **./sa_lockdown.sh** to disable the Service Accounts used during the deployment -- Delete the `` project by running the following command: - - **./delete_gcp_project.sh --project-id=``** - - Confirm by re-entering the project-id **``** when prompted + - **Note:** You may be unable to deploy certain blueprints until the Service Accounts are re-enabled. If you are deploying additional services/blueprints into the Stellar Engine environment, do not run this script until deployments are complete. **Congratulations, you have successfully deployed Stellar Engine\! For further securing of the environment, please see the** [**Stellar Engine Security Best @@ -549,26 +609,57 @@ Guide**](security-best-practices.md). ### Creating a new Google Cloud Org 1. Create Basic Cloud Identity Account - 1. - 1. You must first log into the Google Admin console, and then cloud + - + - You must first log into the Google Admin console, and then cloud console, and wait approximately 2 minutes to provision the org 2. Complete Domain Name verification - 1. This depends on your DNS provider + - This depends on your DNS provider 3. Enable the account in GCP - 1. + - ### Billing Accounts 1. Create a billing account + - + +### Modifying Tenant Projects + +Perform the following steps when adding or removing tenants projects for an existing Stellar Engine deployment. + +#### Authenticate and Set Active Project +- `gcloud auth login ` +- `gcloud config set project xxx-prod-iac-core-0` +- `gcloud auth application-default login ` + +#### Enable FAST Stages Service Accounts +- Change directory into fast/stages-aw/3-security +- `./sa_lockdown.sh --enable` + +#### Apply FAST Stage: 01-resman +- Change directory into fast/stages-aw/1-resman +- Update information in terraform.tfvars to your new requirements +- Run terraform init +- Run terraform apply + - Type yes when prompted + +#### Apply FAST Stage: 02-networking +- Change directory into appropriate network folder for your Stellar Engine deployment: + - fast/stages-aw/2-networking-a-fedramp-high + - fast/stages-aw/2-networking-b-il5-ngfw + - fast/stages-aw/2-networking-c-fedramp-mod +- Copy the 1-resman 1-resman tfvars file from the GCS bucket + - `gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/1-resman.auto.tfvars.json ./` +- Run terraform init +- Run terraform apply + - Type yes when prompted - +#### Disable FAST Stages Service Accounts +- Change directory into fast/stages-aw/3-security +- `./sa_lockdown.sh` -1. ### Additional Notes -- To add or modify tenants, rerun stage 1 with the updated tenant info - - You may need to re-pull the variables files - When modifying modules is necessary, please copy the entire module over, and use the naming convention \ to avoid merge conflicts when periodic updates are pulled in from the CFF @@ -577,5 +668,7 @@ Guide**](security-best-practices.md). Management](https://console.cloud.google.com/security/kms/keyrings). If you receive these additional errors, please wait \~1 minute and rerun **terraform apply** +- On a Windows Machine, symlinks may not work, and specific files may need to be copied over manually, specifically psc.tf and log-metric-alerts.tf during the “2-network” stages +- If you run into billing/quota issues, make sure your quota project is set. You can set it by running `gcloud auth application-default set-quota-project xxx-prod-iac-core-0`, or change it to a project of your choice. ####