Backstage Catalog Components

This repository holds sample catalog entities used when testing Backstage, RHDH, and related plugins. It includes small curated sets and a generator for larger synthetic organizations.

Layout

Use Marvel for realistic curated demos (Keycloak alignment, templates, annotations). Use small-org for a single tiny component. Use large-org when you need volume (search, graph, ingestion stress); run the generator locally first.

• catalog-entities/marvel/ — curated Marvel fixtures (components, systems, domain/API/resources, scaffolder templates, optional static users). Start here: catalog-entities/marvel/README.md for layout, Guest vs Keycloak vs static users, ${MARVEL_*} rendering, and RBAC probe notes.
• catalog-entities/small-org/ — minimal examples.
• catalog-entities/large-org/ — bulk catalog output. Generated files use the *.local.yaml suffix and are gitignored so you can regenerate freely. The committed catalog-entities/large-org/all.yaml Location points at generated-locations-by-folder.local.yaml, which pulls in per-folder entity files after you run the generator.

Large-org catalog generator (TypeScript)

Implementation lives under scripts/catalog-generator/: constants.ts (word pools and presets), rng.ts, fs-utils.ts (YAML + disk), builders.ts (entity documents), create-resources.ts, cli.ts, main.ts. The thin CLI entry is scripts/generate-large-org-catalog.ts. Requires Node and repo dependencies (yarn install). No Python.

--seed uses a deterministic PRNG in TypeScript. The same integer as the old Python generator will not reproduce identical entity names or YAML; use yarn validate:catalog:generated after changes to confirm entities still validate.

Generate a catalog (default output: catalog-entities/large-org/):

yarn generate:catalog

Or run the script directly:

yarn tsx scripts/generate-large-org-catalog.ts

Presets and flags

Flag	Purpose
--preset small | medium | large	Bundled counts (overrides -g/-u/… when set).
--out-dir PATH	Output directory (absolute or relative).
--seed N	Reproducible RNG for this generator (default 42).
--dry-run	Print a short summary; do not write files.
--no-clean	Skip deleting existing *.local.yaml under --out-dir before writing (default is to clean).

Example:

yarn generate:catalog --preset medium --seed 7 --out-dir catalog-entities/large-org

With Yarn 1, flags after the script name are forwarded to tsx. If your package manager behaves differently, run yarn tsx scripts/generate-large-org-catalog.ts … directly.

What gets written

All generated filenames end with .local.yaml (ignored by git via .gitignore).

1. Per-entity files — one entity per file under groups/, domains/, users/, systems/, apis/, resources/, and components/ (no duplicate copies at the repo root).
2. Location indexes at the output root:
   • generated-locations-<kind>.local.yaml — lists ./<kind>/….local.yaml targets for that kind.
   • generated-locations-by-folder.local.yaml — points at each of those per-kind Location files (this is what catalog-entities/large-org/all.yaml references).

Ingestion shape: the generator emits only the per-file tree plus those Location indexes. It does not write root-level aggregate bundles (for example a single users.local.yaml listing every user entity in one file). If you need that style for a file-provider experiment, you would maintain it separately or extend the script; most setups load the Location chain above.

Generated richness (synthetic, for plugin smoke tests): each slice gets a Domain owned by the team Group; Systems set spec.domain to that domain name. OpenAPI-typed APIs include a minimal inline spec.definition. Some Components randomly get backstage.io/techdocs-ref: dir:. and/or github.com/project-slug: catalog-gen/<component-name> (not real repos).

Re-running and cleanup

By default, the generator deletes every *.local.yaml under --out-dir (recursive) before writing, so shrinking counts or changing seeds does not leave orphan entities on disk. That walk is only under --out-dir (default catalog-entities/large-org); other directories in the repo are never touched, so you can safely add *.local.yaml files elsewhere. Anything you place under the default output tree is treated as disposable generator output unless you use --no-clean.

Registering in Backstage

Register the committed Location catalog-entities/large-org/all.yaml after running yarn generate:catalog so generated-locations-by-folder.local.yaml and the folder tree exist. You can instead point the catalog only at generated-locations-by-folder.local.yaml if you prefer not to use all.yaml.

Entities use the default namespace in compound refs (for example group:default/tomatoes-0, resource:default/halibut-g0-r0). Systems, APIs, resources, and components created for a group reference only that group’s systems and owners so the graph stays locally coherent.

Example catalog.locations snippets

Paths below use <PATH_TO_THIS_REPO> as a placeholder for the directory where you cloned this repository (absolute path, or a path relative to your Backstage backend app-config.yaml). Substitute it literally; plain app-config.yaml does not expand environment variables inside strings unless your deployment layer templates them.

Curated Marvel fixtures (no generator; good default for plugin demos):

catalog:
  locations:
    - type: file
      target: <PATH_TO_THIS_REPO>/catalog-entities/marvel/all.yaml

Minimal small-org (single sample component):

catalog:
  locations:
    - type: file
      target: <PATH_TO_THIS_REPO>/catalog-entities/small-org/components/test.yaml

Bulk large-org via committed entry Location (run yarn generate:catalog first so *.local.yaml exists under large-org/):

catalog:
  locations:
    - type: file
      target: <PATH_TO_THIS_REPO>/catalog-entities/large-org/all.yaml

Bulk large-org without all.yaml — register only the generated folder index (same prerequisite: generator has been run):

catalog:
  locations:
    - type: file
      target: <PATH_TO_THIS_REPO>/catalog-entities/large-org/generated-locations-by-folder.local.yaml

Combining Marvel and large-org — both trees load; ensure you do not register overlapping entity files twice:

catalog:
  locations:
    - type: file
      target: <PATH_TO_THIS_REPO>/catalog-entities/marvel/all.yaml
    - type: file
      target: <PATH_TO_THIS_REPO>/catalog-entities/large-org/all.yaml

RHDH and Backstage use the same catalog.locations shape under app-config.yaml (or Helm values that render into it).

Catalog validation (Yarn + TypeScript)

Committed YAML under catalog-entities/ (skipping *.local.yaml) is validated with @backstage/catalog-model. The checker lives in scripts/validate-catalog.ts and runs via tsx.

Marvel ${MARVEL_*} placeholders: validated locally via substitution (see catalog-entities/marvel/README.md and scripts/catalog-env-substitute.ts). Backstage does not substitute these at ingest—render before the catalog loads YAML from Git or disk.

yarn install --frozen-lockfile
yarn typecheck
yarn validate:catalog

After generating large-org locally, you can validate those files too:

yarn generate:catalog --preset small
yarn validate:catalog:generated

Or run generator + validation in one step: yarn validate:catalog:with-generator.

Keycloak provisioning (Marvel realm)

scripts/provision-keycloak.ts pushes keycloak/groups.json and keycloak/users.json into a Keycloak realm via the Admin REST API (no secrets stored in JSON). Optional: import keycloak/marvel-client.json for an OIDC client aligned with the marvel realm (clientId: marvel-client).

Password safety: KEYCLOAK_DEFAULT_USER_PASSWORD applies a shared dev password to all realm users when set; the script also requires KEYCLOAK_ALLOW_INSECURE_PASSWORDS=true so you do not apply that by accident. Use only on isolated machines. Existing users keep passwords unless you pass --reset-passwords. Prefer KEYCLOAK_PASSWORD_COMMAND for one-off secrets when you can.

Typical flow: (1) Copy .env.example → .env and set admin URL/user/password. (2) Run Keycloak (Docker one-shot or Compose below). (3) yarn provision:keycloak (add --import-client / --smoke as needed). (4) Point Backstage at the marvel realm and enable the Keycloak catalog provider so Users/Groups sync; align Marvel YAML spec.owner with realm group names.

Prerequisites: Node 18+ (global fetch), a reachable Keycloak, and an admin user in the admin realm (often admin in realm master). Copy .env.example to .env, add only locally the secrets and passwords your environment needs (never commit them).

The provisioner reads .env from the repository root if that file exists (KEY=value lines, # comments; existing shell exports are not overwritten). You do not need source .env or set -a for yarn provision:keycloak. Other tools behave differently: Docker Compose reads .env for compose substitution, and plain bash does not load .env unless you source it—the “always picked up” feeling usually comes from frameworks or Compose, not from Node itself.

yarn install --frozen-lockfile
yarn provision:keycloak --smoke

Flags: --dry-run, --import-client, --update-client, --reset-passwords, --smoke, --password-command '…'. Run yarn provision:keycloak --help for the full list.

Idempotency

Resource	Behavior on re-run
Realm	Created if missing; 409 treated as already present.
Groups	keycloak/groups.json is a tree: top-level nodes are realm groups; children become Keycloak sub-groups. Names must stay unique across the tree so users.json groups entries resolve to one id. Missing nodes are created (children via POST .../groups/{parentId}/children).
Users	Matched by username; profile fields updated from JSON.
Group membership	Reconciled to match each user’s groups array (adds and removes).
Passwords	Set on create when a password is configured (KEYCLOAK_DEFAULT_USER_PASSWORD + KEYCLOAK_ALLOW_INSECURE_PASSWORDS=true, or KEYCLOAK_PASSWORD_COMMAND). Existing users only get a new password when you pass --reset-passwords.
OIDC client	With --import-client, creates if missing. If the client exists, --update-client is required to PUT merged settings; otherwise the client is left unchanged.

Run Keycloak in Docker (version in one variable)

Pick an image tag once; Keycloak publishes tags such as 26.0, 24.0.5, 23.0.7 on quay.io/keycloak/keycloak.

One-shot container (dev mode):

Export Keycloak’s bootstrap settings in your shell first: KEYCLOAK_ADMIN (the bootstrap username Keycloak expects, often admin) and KEYCLOAK_ADMIN_PASSWORD (a value you choose locally — never commit it). Then:

export KEYCLOAK_IMAGE_TAG=26.0
docker pull "quay.io/keycloak/keycloak:${KEYCLOAK_IMAGE_TAG}"
docker run --rm --name keycloak -p 8080:8080 \
  -e KEYCLOAK_ADMIN \
  -e KEYCLOAK_ADMIN_PASSWORD \
  "quay.io/keycloak/keycloak:${KEYCLOAK_IMAGE_TAG}" \
  start-dev

-e VAR (no =) passes the variable from your current environment into the container. Change only KEYCLOAK_IMAGE_TAG (or inline the tag in the image string) to try another version.

Compose / Podman Compose — same idea with a default image tag you can override from the shell (still no committed passwords):

services:
  keycloak:
    image: quay.io/keycloak/keycloak:${KEYCLOAK_IMAGE_TAG:-26.0}
    command: start-dev
    environment:
      KEYCLOAK_ADMIN: admin
      KEYCLOAK_ADMIN_PASSWORD: ${KEYCLOAK_ADMIN_PASSWORD}
    ports:
      - '8080:8080'

Set KEYCLOAK_ADMIN_PASSWORD in a local .env file consumed by Compose, or export it before docker compose up / podman-compose up. Optionally export KEYCLOAK_IMAGE_TAG=24.0.5 to pin a different Keycloak build.

Provision the marvel realm (after Keycloak is up): put KEYCLOAK_URL, KEYCLOAK_ADMIN_USER, KEYCLOAK_ADMIN_PASSWORD, and any optional overrides in a local repo-root .env (see .env.example), or export them in the shell. Shared realm user passwords follow the Password safety rules above.

yarn provision:keycloak --import-client --smoke

Tested Keycloak versions

The provisioner uses straightforward Admin REST calls (/admin/realms, /groups, /users, /clients). It does not branch on server version today; set KEYCLOAK_VERSION in .env only as a local reminder of what you pointed at.

Major line	Notes
18.x	Legacy Wildfly-era distributions; Admin REST paths under /auth/admin/... in older configs—confirm your base URL matches your distribution.
24.x	Quarkus distribution; typical KEYCLOAK_URL like http://localhost:8080 with admin API under /admin/....
26.x	Current Quarkus tags (for example 26.0, 26.0.5 on quay.io/keycloak/keycloak).

Keycloak 26.4+ and serverinfo: From 26.4.0, the Admin serverinfo response may omit systemInfo.version unless the caller is an administrator in the administrator (master) realm—see Keycloak’s upgrade note: The serverinfo endpoint only returns the system info for administrators in the administrator realm.

The Backstage Keycloak catalog provider previously relied on that version field for sub-group loading (issue #6065, fixed in PR #8406). This repo’s provision script only prints version during --smoke when systemInfo.version is present.

Security and repo hygiene

• Secrets: Never commit repo-root .env, Keycloak admin passwords, OIDC client secrets, or Sonar/ServiceNow tokens. .gitignore ignores .env and common local secret file patterns; *.local.yaml is ignored repo-wide (large-org generator output and Backstage’s common suffix for locally rendered catalog files).
• Keycloak JSON: keycloak/users.json has no passwords; keycloak/marvel-client.json has no embedded client secret—set KEYCLOAK_CLIENT_SECRET locally if you manage one out-of-band (.env.example).
• Scaffolder: Templates that ask for tokens pass them to actions; treat task logs and support bundles as sensitive if debug logging is enabled.
• Marvel client fixture: marvel-client.json enables implicit and direct access grants for local OIDC convenience—tighten for anything beyond isolated dev.

Legacy-style counts (no preset)

yarn generate:catalog --groups 2 --users 3 --apis 3 --components 3 --resources 3 --systems 3

This creates two groups; each group gets the requested numbers of users, APIs, components, resources, and systems.