Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
name: Deploy jth sandbox to AWS
name: Deploy HyP3 EDC Sandbox to AWS

permissions:
contents: read

on:
push:
branches:
- hyp3-jth-sandbox
- hyp3-edc-sandbox

concurrency: ${{ github.workflow }}-${{ github.ref }}

Expand All @@ -14,13 +17,13 @@ jobs:
fail-fast: false
matrix:
include:
- environment: hyp3-jth-sandbox
- environment: hyp3-edc-sandbox
template_bucket: cf-templates-bywc0durdnqy-us-west-2
image_tag: test
product_lifetime_in_days: 14
default_credits_per_user: 0
default_application_status: APPROVED
cost_profile: DEFAULT
cost_profile: EDC
opera_rtc_s1_end_date: Default
job_files: >-
job_spec/AUTORIFT.yml
Expand All @@ -36,7 +39,7 @@ jobs:
required_surplus: 0
security_environment: EDC
ami_id: /ngap/amis/image_id_ecs_al2023_x86
distribution_url: ''
distribution_url: 'https://d3bvvghf83wjqc.cloudfront.net'

environment:
name: ${{ matrix.environment }}
Expand Down
79 changes: 18 additions & 61 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,10 @@ also referred to as "security environments" throughout our code and docs
- JPL
- JPL-public

For EDC, you will also need to refer to our
[Deploy HyP3 to Earthdata Cloud](https://github.com/ASFHyP3/.github-private/blob/main/docs/Deploy-HyP3-to-Earthdata-Cloud.md)
internal docs article (only accessible to members of ASF).

> [!IMPORTANT]
> JPL deployments _must_ start with the JPL security environment, but can be migrated to `JPL-public`
> after they are fully deployed and approved to have a public bucket.
Expand All @@ -67,19 +71,6 @@ For JPL, these deployment docs assume that:
- the JPL account was set up in the "default" manner by the JPL cloud team
- the developer deploying the account is able to log in with the `power_user` role

For a new EDC deployment, you need the following items (not necessarily a comprehensive list):
- SSL certificate in AWS Certificate Manager for custom CloudFront domain name
- ID of the CloudFront Origin Access Identity used to access data in S3

EDC UAT/prod deployment steps are not fully documented here.
When deploying HyP3 to a new EDC account for the first time, you should also refer to the
[SOP for deploying HyP3 to EDC](https://asfdaac.atlassian.net/wiki/spaces/ST/pages/2290319361/SOP-ASF-DAAC-EDC-011).
You should then be able to deploy additional copies of HyP3 to an EDC Sandbox account
by following this README alone.

After deploying HyP3 to an EDC Sandbox account, you'll need to follow our documentation on
[Accessing Private API Gateways in Earthdata Cloud](https://github.com/ASFHyP3/.github-private/blob/main/docs/Accessing-Private-API-Gateways-in-Earthdata-Cloud.md).

> [!TIP]
> You can expand and collapse details specific to a security environment as you go through this README.
> Make sure you're looking at the details for the security environment you're deploying into!
Expand Down Expand Up @@ -240,52 +231,21 @@ Go to AWS console -> Secrets Manager, then:
1. Click the orange "Next" button (we won't configure rotation)
1. Click the orange "Store" button to save the Secret

#### Upload SSL cert

> [!WARNING]
> This step must be done by an ASF employee.

To allow HTTPS connections, HyP3 needs an SSL certificate that is valid for its deployment domain name (URL).

If HyP3 is being deployed to an ASF-managed AWS account, we can use the master certificate that covers all
`*.asf.alaska.edu` domains. Otherwise, we'll need a deployment specific certificate.

*Important: Skip this step for EDC Sandbox deployments.*

<details>
<summary>ASF-managed AWS account: Upload the ASF master SSL certificate</summary>
<br />

Upload the `*.asf.alaska.edu` SSL certificate to AWS Certificate Manager (ACM):
#### Request SSL cert

1. AWS console -> Certificate Manager (ACM) -> import certificate
1. Open https://gitlab.asf.alaska.edu/operations/puppet/-/tree/production/modules/certificates/files
1. The contents of the `asf.alaska.edu.cer` file go in "Certificate body"
1. The contents of the `asf.alaska.edu.key` file go in "Certificate private key"
1. The contents of the `intermediates.pem` file go in "Certificate chain"
</details>
*Note: For EDC accounts, you should create the cert in the `us-east-1` region
for use with the CloudFront distribution that you will create later,
even if you're deploying HyP3 to `us-west-2`.*

<details>
<summary>Externally-managed AWS account (e.g., JPL, EDC, CloudBank): Request and upload deployment specific SSL certificate</summary>
<br />
To allow HTTPS connections, HyP3 needs an SSL certificate that is valid for its deployment domain name (URL):

Submit a Platform request in ASF JIRA for a new certificate, including the domain name
(e.g. `hyp3-foobar.asf.alaska.edu`).
AWS console -> AWS Certificate Manager -> Request a public certificate:
- The domain name should be something like `hyp3-foobar.asf.alaska.edu`
- Choose "DNS validation"

Once you receive the certificate's private key and links to download the certificate in various formats,
download these files:
1. Certificate Body (the "as Certificate Only, PEM encoded" link from the email)
2. Certificate Private Key (from the Platform team; typically in MatterMost)
3. Certificate Chain (the "as Root/Intermediate(s) only, PEM encoded" link from the email)

and then upload them to AWS Certificate Manager (ACM):

1. AWS console -> Certificate Manager (ACM) -> import certificate
1. The contents of (1) above goes in Certificate body
1. The contents of (2) above goes in Certificate private key
1. The contents of (3) above goes in Certificate chain

</details>
Then add the validation record to
https://gitlab.asf.alaska.edu/operations/puppet/-/edit/production/modules/legacy_dns/files/asf.alaska.edu.db
(see previous records for examples).

### Create the GitHub environment

Expand All @@ -301,8 +261,7 @@ and then upload them to AWS Certificate Manager (ACM):
(This is typically `main` for prod deployments, `develop` for test deployments, or a feature branch name for sandbox deployments.)
4. Add the following environment secrets:
- `AWS_REGION` - e.g. `us-west-2`
- `BUCKET_READ_PRINCIPALS` (EDC only) - List of AWS IAM principals granted read access to data in S3 for Earthdata Cloud deployments. For EDC Sandbox deployments, if you don't know what to put here, you can simply set it to `arn:aws:iam::<edc-sandbox-account-id>:root`, where `<edc-sandbox-account-id>` is the AWS account ID for the EDC Sandbox account.
- `CERTIFICATE_ARN` (ASF and JPL only) - ARN of the AWS Certificate Manager certificate that you imported manually (aws console -> certificate manager -> list certificates, e.g. `arn:aws:acm:us-west-2:xxxxxxxxxxxx:certificate/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx`)
- `CERTIFICATE_ARN` (ASF and JPL only) - ARN of the AWS Certificate Manager certificate that you created manually
- `CLOUDFORMATION_ROLE_ARN` (ASF only) - part of the `hyp3-ci` stack that you deployed, e.g. `arn:aws:iam::xxxxxxxxxxxx:role/hyp3-ci-CloudformationDeploymentRole-XXXXXXXXXXXXX`
- `SECRET_ARN` - ARN for the AWS Secrets Manager Secret that you created manually
- `V2_AWS_ACCESS_KEY_ID` - AWS access key ID:
Expand All @@ -317,7 +276,7 @@ and then upload them to AWS Certificate Manager (ACM):

You will need to add the deployment to the matrix in an existing GitHub Actions `deploy-*.yml` workflow or create
a new one for the deployment. If you need to create a new one, we recommend copying one of the
`deploy-*-sandbox.yml` workflows, and then updating all of the fields (`environment`, `domain`, `template_bucket`, etc.)
existing workflows, and then updating all of the fields
as appropriate for your deployment. Also make sure to update the top-level `name` of the workflow and the name
of the branch to deploy from. (This is typically `main` for prod deployments, `develop` for test deployments, or a feature branch name for sandbox deployments.)

Expand All @@ -336,12 +295,10 @@ Once HyP3 is deployed, there are a few follow on tasks you may need to do for a
> [!WARNING]
> This step must be done by an ASF employee.

*Important: Skip this step for EDC Sandbox deployments.*

Open a PR adding a line to https://gitlab.asf.alaska.edu/operations/puppet/-/blob/production/modules/legacy_dns/files/asf.alaska.edu.db
for the new custom domain name (AWS console -> api gateway -> custom domain names -> "API Gateway domain name").

Ask the Platform team in the `~development-support` channel in Mattermost to review/merge the PR.
Ask someone from ASF support to review/merge the PR.

Changes should take effect within 15-60 minutes after merging.
Confirm that a Swagger UI is available at your chosen API URL.
Expand Down