From c1467331d8c213bc8312ba414bf977d5799cb9dd Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 16 May 2026 13:59:16 -0400 Subject: [PATCH 1/2] solutions-engineering: simplify migration lab README Consolidate candidate-facing setup guidance into the lab README and keep prep focused on installing Blacksmith before the live migration exercise. Co-authored-by: Cursor --- solutions-engineering/migration-lab/README.md | 161 +++++++----------- .../migration-lab/docs/candidate-prep.md | 34 ---- .../migration-lab/docs/customer-brief.md | 19 --- 3 files changed, 64 insertions(+), 150 deletions(-) delete mode 100644 solutions-engineering/migration-lab/docs/candidate-prep.md delete mode 100644 solutions-engineering/migration-lab/docs/customer-brief.md diff --git a/solutions-engineering/migration-lab/README.md b/solutions-engineering/migration-lab/README.md index bc110ca..920cc06 100644 --- a/solutions-engineering/migration-lab/README.md +++ b/solutions-engineering/migration-lab/README.md @@ -1,97 +1,64 @@ -# Solutions Engineering Migration Lab - -Thank you for interviewing with Blacksmith. - -This exercise is meant to give you a practical feel for the kind of work Solutions Engineers do with customers. You will help a fictional company migrate a GitHub Actions repository to Blacksmith, inspect CI behavior, and prepare recommendations. - -You can spend as much time as you would like, but the expected preparation time is about 60 to 90 minutes. If setup takes longer than that, stop and write down where you got stuck. - -## Scenario - -Acme Payments is migrating from GitHub-hosted runners to Blacksmith. They care about faster CI, lower spend, and a smoother debugging workflow when jobs fail. - -They have a mixed CI pipeline with: - -- Standard Linux jobs. -- Larger GitHub-hosted runner labels for heavier jobs. -- An Arm Docker build path. -- Unit tests split across shards. -- Integration tests that depend on Postgres. - -Your goal is to get the repository running on Blacksmith and come prepared to explain what you observed. - -## Prerequisites - -- A GitHub account. -- A GitHub organization where you can install GitHub Apps. -- An SSH key configured on your GitHub account. -- Access to [app.blacksmith.sh](https://app.blacksmith.sh/). -- Node.js 20 or newer if you want to run parts of the app locally. - -Blacksmith supports GitHub organizations, not personal repositories. If you do not already have a GitHub organization for testing, create one before starting. - -## Preparation - -1. Read the prep guide in [docs/candidate-prep.md](docs/candidate-prep.md). -2. Fork this repository into a GitHub organization that you control. -3. Install Blacksmith for the forked repository. -4. Use the Blacksmith migration wizard to migrate the GitHub Actions workflow. -5. Enable SSH access for the organization in Blacksmith. -6. If GitHub prompts you to approve Actions for the fork, approve it from the repository's Actions tab. -7. Run the workflow at least once on Blacksmith. -8. Review the run in Blacksmith Run History and CI Analytics. -9. Write short notes about what changed, what worked, what failed, and what you would investigate next. - -## Repository Layout - -```text -solutions-engineering/migration-lab/ - app/ - src/ - scripts/ - fixtures/ - docker/ - docs/ -``` - -The workflow for this lab is located at the repository root: - -```text -.github/workflows/acme-ci.yml -``` - -The workflow starts with GitHub-hosted runner labels on purpose. Use the migration wizard rather than manually editing every `runs-on` field. - -## Deliverable - -Prepare a short summary covering: - -- How you migrated the workflow to Blacksmith. -- Which Blacksmith runner types you selected and why. -- What you observed in Run History and CI Analytics. -- What you would tell Acme Payments as next steps. - -You do not need to fully optimize the repository. We care more about how you investigate and explain tradeoffs than about perfect final YAML. - -## Useful Commands - -From this directory: - -```bash -cd solutions-engineering/migration-lab/app -npm ci -npm run lint -npm run build -npm run test:unit -- --shard=1/8 -``` - -The integration test expects Postgres to be running. The workflow starts Postgres through Docker Compose. - -## Useful Links - -- [Blacksmith Quickstart](https://docs.blacksmith.sh/introduction/quickstart) -- [Runner Instance Types](https://docs.blacksmith.sh/blacksmith-runners/overview) -- [CI Analytics](https://docs.blacksmith.sh/blacksmith-observability/dashboard) -- [Run History](https://docs.blacksmith.sh/blacksmith-observability/history) -- [Monitors](https://docs.blacksmith.sh/blacksmith-observability/monitors) -- [SSH Access](https://docs.blacksmith.sh/blacksmith-observability/ssh-access) +# Solutions Engineering Migration Lab + +Thank you for interviewing with Blacksmith. + +This exercise is meant to give you a practical feel for the kind of work Solutions Engineers do with customers. You will help a fictional company migrate a GitHub Actions repository to Blacksmith, inspect CI behavior, make workflow changes, and explain recommendations. + +## Scenario + +Acme Payments is evaluating Blacksmith for its GitHub Actions workloads. The team cares about faster CI, lower spend, and a smoother debugging workflow when jobs fail. + +The repository contains a representative CI pipeline with: + +- Runner labels that should be migrated to Blacksmith. +- Jobs with different compute requirements. +- Matrix jobs. +- Docker builds for multiple architectures. +- Tests split across shards. +- Integration tests that depend on Postgres. + +Your goal during the interview is to help Acme get started on Blacksmith, inspect the resulting CI behavior, and recommend changes. + +## Preparation + +Please complete these steps before the interview. Do not run the migration wizard before the interview. + +1. Create or use a GitHub organization where you can install GitHub Apps. This can be a personal organization, a project organization, or any other organization you control. +2. Fork this repository into that organization. +3. Go to [app.blacksmith.sh](https://app.blacksmith.sh/) and install the Blacksmith GitHub App for the forked repository. +4. Confirm that you can open the Blacksmith dashboard and see your organization. +5. Enable SSH access for the organization in Blacksmith settings. +6. If GitHub prompts you to approve Actions for the fork, approve it from the repository's Actions tab. + +Success criteria for preparation: you can log into Blacksmith, select the organization that contains your fork, and see that organization in the dashboard. + +Again, do not migrate the workflow before the interview. We will do that together as part of the exercise. + +## CI Workflow + +The GitHub Actions workflow for this lab is: + +```text +.github/workflows/acme-ci.yml +``` + +The workflow is intentionally the main surface area for the exercise. It contains jobs named: + +- `Background report profile` +- `Fraud model benchmark` +- `Unit tests shard */8` +- `Integration tests` +- `Docker build (...)` + +You should be comfortable opening this workflow, reading the job definitions, and making changes during the interview. + +## What To Bring + +Come prepared to discuss: + +- How you would start onboarding a customer to Blacksmith. +- Where you would look in the Blacksmith dashboard to confirm jobs are running. +- How you would approach debugging a CI job that is slow or failing. +- How you would communicate findings and tradeoffs to a customer. + +You do not need to solve anything before the interview. diff --git a/solutions-engineering/migration-lab/docs/candidate-prep.md b/solutions-engineering/migration-lab/docs/candidate-prep.md deleted file mode 100644 index 054175d..0000000 --- a/solutions-engineering/migration-lab/docs/candidate-prep.md +++ /dev/null @@ -1,34 +0,0 @@ -# Candidate Prep - -Please complete these steps before discussing the exercise. - -## Read - -- Blacksmith Quickstart: https://docs.blacksmith.sh/introduction/quickstart -- Runner instance types: https://docs.blacksmith.sh/blacksmith-runners/overview -- CI Analytics: https://docs.blacksmith.sh/blacksmith-observability/dashboard -- Run History: https://docs.blacksmith.sh/blacksmith-observability/history -- Monitors: https://docs.blacksmith.sh/blacksmith-observability/monitors -- SSH Access: https://docs.blacksmith.sh/blacksmith-observability/ssh-access - -## Set Up - -1. Create or use a GitHub organization. -2. Fork https://github.com/useblacksmith/interview-exercises into that organization. -3. Install Blacksmith for the forked repository. -4. Use the Blacksmith migration wizard to migrate `.github/workflows/acme-ci.yml`. -5. Enable SSH access in Blacksmith settings. -6. If GitHub prompts you to approve Actions for the fork, approve it from the repository's Actions tab. -7. Run the workflow at least once on Blacksmith. -8. Review the run in Blacksmith Run History and CI Analytics. - -## Write Notes - -Write brief notes on: - -- What changed during migration. -- Which runner labels you selected. -- Any failures you saw. -- What you would investigate next for the customer. - -It is fine if you do not resolve every issue. diff --git a/solutions-engineering/migration-lab/docs/customer-brief.md b/solutions-engineering/migration-lab/docs/customer-brief.md deleted file mode 100644 index 10fff23..0000000 --- a/solutions-engineering/migration-lab/docs/customer-brief.md +++ /dev/null @@ -1,19 +0,0 @@ -# Customer Brief - -Acme Payments is moving its GitHub Actions workloads to Blacksmith. - -The team has three goals: - -1. Reduce CI spend. -2. Improve end-to-end CI time. -3. Make failed job debugging easier. - -The repository contains a representative CI pipeline with linting, builds, unit tests, integration tests, and Docker builds. - -Acme has asked for help answering these questions: - -- Are we using the right runner sizes? -- Are our test shards balanced? -- Are there jobs that should stay on x64 or move to Arm? -- How should we investigate intermittent CI failures? -- What changes would you recommend before rolling this out to more repositories? From 39283b9dee560460b400fadd3b6e165e5be6ac8b Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 16 May 2026 14:13:53 -0400 Subject: [PATCH 2/2] solutions-engineering: refine migration lab prep guidance Clarify dashboard verification, local AI coding tool setup, and external prep resources while keeping the README focused on pre-interview setup. Co-authored-by: Cursor --- solutions-engineering/migration-lab/README.md | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-) diff --git a/solutions-engineering/migration-lab/README.md b/solutions-engineering/migration-lab/README.md index 920cc06..519e4a1 100644 --- a/solutions-engineering/migration-lab/README.md +++ b/solutions-engineering/migration-lab/README.md @@ -12,7 +12,6 @@ The repository contains a representative CI pipeline with: - Runner labels that should be migrated to Blacksmith. - Jobs with different compute requirements. -- Matrix jobs. - Docker builds for multiple architectures. - Tests split across shards. - Integration tests that depend on Postgres. @@ -26,11 +25,14 @@ Please complete these steps before the interview. Do not run the migration wizar 1. Create or use a GitHub organization where you can install GitHub Apps. This can be a personal organization, a project organization, or any other organization you control. 2. Fork this repository into that organization. 3. Go to [app.blacksmith.sh](https://app.blacksmith.sh/) and install the Blacksmith GitHub App for the forked repository. -4. Confirm that you can open the Blacksmith dashboard and see your organization. +4. Confirm that you can open the Blacksmith dashboard and see your organization in the top-left organization dropdown. 5. Enable SSH access for the organization in Blacksmith settings. 6. If GitHub prompts you to approve Actions for the fork, approve it from the repository's Actions tab. +7. Check out the repository locally and make sure Claude Code or Codex is working in that checkout. -Success criteria for preparation: you can log into Blacksmith, select the organization that contains your fork, and see that organization in the dashboard. +If your Blacksmith account needs additional verification, reach out by email before the interview. + +Success criteria for preparation: you can log into Blacksmith, select the organization that contains your fork from the top-left dropdown, and see that organization in the dashboard. Again, do not migrate the workflow before the interview. We will do that together as part of the exercise. @@ -52,13 +54,8 @@ The workflow is intentionally the main surface area for the exercise. It contain You should be comfortable opening this workflow, reading the job definitions, and making changes during the interview. -## What To Bring - -Come prepared to discuss: +## Resources -- How you would start onboarding a customer to Blacksmith. -- Where you would look in the Blacksmith dashboard to confirm jobs are running. -- How you would approach debugging a CI job that is slow or failing. -- How you would communicate findings and tradeoffs to a customer. +A Notion prep doc will be shared with you as part of the interview invite. Use that document for the customer brief and recommended reading before the interview. You do not need to solve anything before the interview.