openid-client spike

README.md

@@ -153,7 +153,11 @@ npm run test
 ```
 
 ### Run UI driven tests
-- in headless mode
+- make sure to build and run the Next.js app
+  ```
+  npm run app
+  ```
+- run tests in headless mode
   ```
   npm run e2e
   ```

docs/RUNBOOK.md

@@ -0,0 +1,22 @@
+# Runbook for solving production issues
+
+## Manual vaccine content cache refresh
+
+### Triggers
+- When the periodic cache refresh has failed for some reason
+- When nhs.uk tells us that the content needs to be refreshed asap
+
+### Action
+1. Login to the AWS environment and ensure you are in London **eu-west-2** region.
+2. Navigate to **Lambda** service.
+3. Click on the lambda with prefix `gh-main-vita` and suffix `nextjs-warmer-function`
+4. Select **Test** tab to create a new test event
+5. Give the new event a  meaningful name
+6. Keep the event sharing settings to **Private**
+7. Use the following template as the event JSON (feel free to add other non-PII fields as necessary for audit)
+```json
+{
+  "who": "<name and email of the person triggering this action>",
+  "why": "<reason for this manual cache refresh>"
+}
+```

docs/adr/ADR-004_Content_caching_architecture.md

@@ -0,0 +1,97 @@
+# ADR-004: Content caching architecture
+
+>|              |                                                        |
+>| ------------ |--------------------------------------------------------|
+>| Date         | `22/04/2025`                                           |
+>| Status       | `Accepted`                                             |
+>| Deciders     | `Engineering, Architecture, `                          |
+>| Significance | `Structure, Nonfunctional characteristics, Interfaces` |
+>| Owners       | Ankur Jain, Elena Oanea                                |
+
+---
+
+- [ADR-004: Content caching architecture](#adr-004-content-caching-architecture)
+  - [Context](#context)
+  - [Decision](#decision)
+    - [Assumptions](#assumptions)
+    - [Drivers](#drivers)
+    - [Options](#options)
+    - [Outcome](#outcome)
+    - [Rationale](#rationale)
+  - [Consequences](#consequences)
+  - [Compliance](#compliance)
+  - [Notes](#notes)
+  - [Actions](#actions)
+  - [Tags](#tags)
+
+## Context
+According to the NHS.UK website content API guidelines for caching, we require a minimum of 7 days before calling the
+endpoint again to fetch updated content. This requires us to keep a cache in place. There are two problems to design for.
+Firstly, how do we keep the cache updated - meaning cache inserts, updates and deletes. Secondly, how and when does the
+app read/write from/to cache that is the latest content.
+
+## Decision
+We decided to keep the design simple for maintainability purposes by separating the reading and writing to two separate
+and independent processes. The writing part ensures to keep the cache updated as per the caching guidelines. The reading
+part just reads from the cache assuming it is the correct and most up-to-date. This architecture is optimised for write
+few and read many operations, which is our use case.
+
+To ensure that the cache has the most up-to-date data for all vaccines, we have three mechanisms: -
+1. a post deployment trigger that updates the cache each time a change is made to the writer or vaccines list.
+2. a periodic cron job that updates the cache at a predefined frequency.
+3. on demand trigger that allows updating when requested by nhs.uk.
+
+```mermaid
+---
+title: Content cache architecture
+---
+graph TD;
+   A(Show vaccine content - client side);
+   B(AWS Lambda - nextjs server action);
+   C[(AWS S3 Cache)];
+   D(Content cache hydrator);
+   E(Cron);
+   A -- triggers --> B;
+   B -- reads --> C;
+   E -- triggers --> D;
+   D -- writes --> C;
+```
+
+### Assumptions
+- The data itself is not updated very frequently because if it did, then the reads could suffer latency due to locking for updates.
+- When we roll out the VitA app for the first time, there is a flag in NHS app that needs to be turned on for us to go live.
+  This gives sufficient time for the cache writer to have pulled in all vaccine content.
+- When we roll out new vaccines, there is a slight delay (seconds) between deployment finish and cache writer trigger.
+  During this time, if a user visits the new vaccine page, they will see an error page. This should go away as soon as
+  the new content is available.
+
+### Drivers
+Mostly simplifying the read process, so that it does not get mixed with writing logic.
+
+### Options
+Alternative design was to fetch the content on a cache miss and update the cache. After which, the subsequent reads
+would succeed. This makes the read complex, and we really wanted to make read simple as that is going to be frequent.
+
+### Outcome
+The decision is reversible, if it turns out that people are seeing error pages frequently. This would be monitored.
+
+### Rationale
+Design that optimises for multiple reads and very infrequent writes.
+Design that is simple to debug and separating the concerns makes it easier.
+
+## Consequences
+As outlined above in assumptions, there might be intermittent error pages shown when the vaccine content is being pulled.
+
+## Compliance
+The errors would be monitored so that service level objectives are met.
+
+## Notes
+None
+
+## Actions
+
+- [x] Ankur Jain, 22/04/2025, created the ADR
+
+## Tags
+
+`#performance, #maintainability, #testability, #deployability, #modularity, #simplicity`

esbuild.config.mjs

@@ -10,20 +10,33 @@ const removeDistDirectory = async () => {
   });
 };
 
+const copyExtraFiles = async () => {
+  // there is an issue with esbuild putting a warning for xhr-sync-worker.js file being external
+  // at runtime, lambda complains that the file is not present
+  // DOMPurify apparently depends on this file
+  fs.copyFile("./node_modules/jsdom/lib/jsdom/living/xhr/xhr-sync-worker.js", `${OUTPUT_DIR}/xhr-sync-worker.js`, (err) => {
+    if (err) throw err;
+    console.log("Copied xhr-sync-worker.js (workaround)");
+  });
+};
+
 const buildLambda = async () => {
   await esbuild.build({
     entryPoints: ["src/_lambda/content-cache-hydrator/handler.ts"],
     bundle: true,
     minify: true,
     platform: "node",
+    jsx: "automatic",
     target: "node22",
+    external: ["./xhr-sync-worker.js"],
     outfile: `${OUTPUT_DIR}/lambda.js`
   });
 };
 
 try {
   await removeDistDirectory();
   await buildLambda();
+  await copyExtraFiles();
   console.log(`Built lambda successfully -> ${OUTPUT_DIR}/lambda.js`);
 } catch (e) {
   console.error("Building lambda failed: ", e);

infrastructure/environments/dev/main.tf

@@ -11,3 +11,12 @@ module "deploy" {
   log_retention_in_days = local.log_retention_in_days
   pino_log_level        = local.pino_log_level
 }
+
+module "post_deploy" {
+  source = "../../modules/post_deploy"
+
+  default_tags = local.default_tags
+  prefix       = local.prefix
+
+  depends_on = [module.deploy]
+}

infrastructure/github-iam-role-policy.json

@@ -28,15 +28,28 @@
         "dynamodb:DescribeTimeToLive",
         "dynamodb:ListTagsOfResource",
         "dynamodb:TagResource",
+        "events:DeleteRule",
+        "events:DescribeRule",
+        "events:ListTagsForResource",
+        "events:ListTargetsByRule",
+        "events:PutRule",
+        "events:PutTargets",
+        "events:RemoveTargets",
+        "events:TagResource",
         "iam:AttachRolePolicy",
         "iam:CreatePolicy",
         "iam:CreateRole",
+        "iam:DeletePolicy",
+        "iam:DeleteRole",
         "iam:DeleteRolePolicy",
+        "iam:DetachRolePolicy",
         "iam:GetPolicy",
         "iam:GetPolicyVersion",
         "iam:GetRole",
         "iam:GetRolePolicy",
         "iam:ListAttachedRolePolicies",
+        "iam:ListInstanceProfilesForRole",
+        "iam:ListPolicyVersions",
         "iam:ListRolePolicies",
         "iam:PassRole",
         "iam:PutRolePolicy",
@@ -47,16 +60,19 @@
         "lambda:CreateEventSourceMapping",
         "lambda:CreateFunction",
         "lambda:CreateFunctionUrlConfig",
+        "lambda:DeleteFunction",
         "lambda:GetAlias",
         "lambda:GetEventSourceMapping",
         "lambda:GetFunction",
         "lambda:GetFunctionCodeSigningConfig",
         "lambda:GetFunctionConfiguration",
         "lambda:GetFunctionUrlConfig",
         "lambda:GetPolicy",
+        "lambda:ListFunctions",
         "lambda:ListTags",
         "lambda:ListVersionsByFunction",
         "lambda:PublishVersion",
+        "lambda:RemovePermission",
         "lambda:TagResource",
         "lambda:UpdateAlias",
         "lambda:UpdateFunctionCode",

infrastructure/modules/deploy_app/main.tf

@@ -29,7 +29,7 @@ module "deploy_app" {
     additional_iam_policies = [aws_iam_policy.cache_lambda_additional_policy]
     runtime = var.nodejs_version
     concurrency = 1
-    schedule = "rate(2 days)"
+    schedule = "rate(7 days)"
     additional_environment_variables = {
       SSM_PREFIX          = var.ssm_prefix
       CONTENT_CACHE_PATH  = var.content_cache_path

infrastructure/modules/post_deploy/event_bridge.tf

@@ -0,0 +1,30 @@
+resource "aws_cloudwatch_event_rule" "warmer_lambda_deployment_event_rule" {
+  name        = "${var.prefix}-post-deployment-event-rule"
+  description = "Triggers Warmer Lambda on deployment"
+  tags        = var.default_tags
+  event_pattern = jsonencode({
+    "source" : ["aws.lambda"],
+    "detail-type" : ["AWS API Call via CloudTrail"],
+    "detail" : {
+      "eventSource" : ["lambda.amazonaws.com"],
+      "eventName" : ["CreateFunction20150331", "UpdateFunctionCode20150331v2", "UpdateFunctionConfiguration20150331v2"],
+      "requestParameters" : {
+        "functionName" : [local.warmer_function_name]
+      }
+    }
+  })
+}
+
+resource "aws_cloudwatch_event_target" "lambda_target" {
+  target_id = "lambda"
+  rule      = aws_cloudwatch_event_rule.warmer_lambda_deployment_event_rule.name
+  arn       = local.warmer_function_arn
+}
+
+resource "aws_lambda_permission" "allow_event_bridge_to_invoke_lambda" {
+  statement_id  = "AllowExecutionFromEventBridge"
+  action        = "lambda:InvokeFunction"
+  function_name = local.warmer_function_name
+  principal     = "events.amazonaws.com"
+  source_arn    = aws_cloudwatch_event_rule.warmer_lambda_deployment_event_rule.arn
+}

infrastructure/modules/post_deploy/locals.tf

@@ -0,0 +1,13 @@
+data "aws_lambda_functions" "all_lambda_functions" {}
+
+locals {
+  warmer_function_name = one([
+    for name in data.aws_lambda_functions.all_lambda_functions.function_names :
+    name if length(regexall("^${var.prefix}.*warmer-function$", name)) == 1
+  ])
+
+  warmer_function_arn = one([
+    for arn in data.aws_lambda_functions.all_lambda_functions.function_arns :
+    arn if length(regexall("^${var.prefix}.*warmer-function$", split(":", arn)[6])) == 1
+  ])
+}

infrastructure/modules/post_deploy/variables.tf

@@ -0,0 +1,9 @@
+variable "prefix" {
+  type        = string
+  description = "Prefix to be applied to resources created"
+}
+
+variable "default_tags" {
+  type = map(string)
+  description = "Map of default key-value pair of tags to add to resources"
+}

jest.config.ts

@@ -19,7 +19,7 @@ const config: Config = {
     "^@src/(.*)$": "<rootDir>/src/$1",
     "^@test-data/(.*)$": "<rootDir>/mocks/$1"
   },
-  testPathIgnorePatterns: ["<rootDir>/e2e/"],
+  testPathIgnorePatterns: ["<rootDir>/e2e/", "<rootDir>/.open-next/"],
 };
 
 // createJestConfig is exported this way to ensure that next/jest can load the Next.js config which is async