Add Vault secrets provider with hash-separated keys and automatic JWT generation

[Copilot]

Adds HashiCorp Vault as a read-only secrets provider to support environments where the Vault Secrets Operator cannot inject values (e.g., templated config files, Helm chart limitations). Enables secure multi-tenant deployments with flexible JWT authentication options.

Implementation

VaultSecretProvider (src/nyl/secrets/vault.py)

• Dual JWT authentication modes:
  • kubernetes (default): Kubernetes service account token for simple deployments
  • nyl: Nyl automatically generates ArgoCD application-specific JWT tokens for multi-tenant environments
• Token auth via VAULT_TOKEN env var or ~/.vault-token (local dev)
• Automatic context detection based on ARGOCD_APP_NAME presence
• Hash-separated key format: path/to/secret#field.nested.path supports secrets with dots in paths
• KV v2 API with efficient caching
• Read-only: set() and unset() raise NotImplementedError

JWT Token Generation (src/nyl/tools/jwt.py)

• Nyl generates JWT tokens internally using ArgoCD environment variables
• Tokens signed with HS256 algorithm using shared secret (jwt_signing_key)
• Token payload includes: iss, aud, sub, argocd_project, argocd_app, repository
• Automatic generation in template command when in ArgoCD context
• Tokens valid for 1 hour

Configuration

default:
  type: vault
  url: https://vault.example.com:8200
  mount_point: secret
  path: myapp/
  jwt_role: nyl-argocd-role
  jwt_auth_method: nyl  # or "kubernetes" (default)
  jwt_signing_key: ${NYL_VAULT_JWT_SIGNING_KEY}  # shared secret with Vault

Usage

apiVersion: v1
kind: Secret
metadata:
  name: db-credentials
stringData:
  # Access specific field with hash separator
  password: ${{ secrets.get("database#password") }}
  # Access nested field
  username: ${{ secrets.get("database#credentials.username") }}
  # Access secret with dots in path
  api-key: ${{ secrets.get("my-app.v2#api_key") }}

Design Notes

• Hash separator (#): Separates secret path from field access, allowing dots in secret paths
• Read-only provider: Vault secrets must be managed through Vault's interface or CLI
• Nyl JWT mode: Tokens are automatically generated by Nyl based on ArgoCD environment variables, not read from external sources
• JWT generation: Happens in template command entrypoint before secrets are accessed
• Sensitive paths logged at debug level to prevent exposure
• 26 unit tests covering both auth methods, key parsing, JWT generation, and error handling

Dependencies

• Added hvac>=2.3.0 (no known vulnerabilities)
• Added pyjwt>=2.8.0 for JWT token generation (no known vulnerabilities)

---

• Fixes Support Vault as secrets provider #108

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Draft: Support Vault as secrets provider</issue_title>
<issue_description>When using Vault for managing secrets in a Kubernetes cluster, one would usually want to use the Vault secrets operator. However, there are sometimes limitations in Helm charts that require injecting secret values in places that the Vault secrets operator could not (e.g. as an input that is used to template a longer secret value such as a configuration file), or when no secret reference can be specified.

For this purpose, it should be possible for Nyl to provide secrets via a Vault instance instead of a SOPS file. We should aim to design this in a secure fashion to be compatible in a multi-tenant environment where a single ArgoCD repo-server (and thus Nyl plugin container) is used to deploy applications owned by many teams.

Assuming we solve the arbitrary code execution that still needs to be addressed aside (see #48), we can trust the user input to Nyl's templating engine to the extend that they can not elevate their privileges and in return trust in Nyl to issue a workload identity according to the application that is rendered when invoked via ArgoCD.

This workload identity (asserted via a JWT) can then be configured in Vault to be trusted and assigned permissions as required to successfully render the manifests of a template, specifically for access to Vault secrets.

When running outside of ArgoCD, it would use the Vault identity obtained locally via vault login instead.</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Support Vault as secrets provider #108

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[NiklasRosenstein]

@copilot Using dot notation for nested access makes it impossible to use secrets with dots in the path. We should separate the secret path with fields for nested access with a hash character followed by dot notation, e.g. path/to/secret#database.password.

[Copilot]

Done in c2eea33. Changed key format to use hash separator (#) between path and field access. Now supports secrets with dots in paths: "db.prod#password" or "my-app.v2#field". Updated all tests and documentation.

[NiklasRosenstein]

@copilot The Vault implementation should not support setting/unsetting. Let's simply raise a NotImplementedError and document this behavior. Make sure to cleanup unused code after removing these implementations.

[Copilot]

Done in c2eea33. Both set() and unset() now raise NotImplementedError with clear messages directing users to manage secrets through Vault's interface. Removed ~120 lines of unused implementation code and updated tests.

[NiklasRosenstein]

@copilot The workload identity method described in #108 actually does not refer to the Kubernetes token, but instead through a token issued by Nyl to assert the ArgoCD application that is being templated.

I think we should consider allowing both options: Using the Kubernetes service account for simple usage, and Nyl-issuer ArgoCD-application specific workload identities for mutli-tenant deployments.

The payload for a token could be e.g.

{
  "iss": "https://my-argocd.example.com/#nyl-v1",
  "aud": "https://my-vault.example.com:8200",
  "sub": "project:default:application:my-argo-app",
  "argocd_project": "default",
  "argocd_app": "my-argo-app",
  "repository": "git@github.com:example-org/manifests.git"
}

[Copilot]

Done in c2eea33. Added support for both authentication methods via jwt_auth_method config:

• "kubernetes" (default): Uses Kubernetes service account token
• "nyl": Uses Nyl-issued JWT from NYL_VAULT_JWT env var with ArgoCD app claims

The Nyl JWT mode expects tokens with payload including argocd_project, argocd_app, and repository fields as described. Both methods coexist and are selected via configuration.

[NiklasRosenstein]

@copilot But this code is Nyl, we need to generate a token here (or use one we previously generated). I think a good place for issuing a new token would be in the main entrypoint when we detect that we're running in ArgoCD with the Nyl JWT auth method enabled.

[Copilot]

Done in f399007. Nyl now generates JWT tokens internally:

• Added src/nyl/tools/jwt.py module with generate_nyl_jwt_from_argocd_env() function
• Reads ArgoCD environment variables (ARGOCD_APP_NAME, ARGOCD_APP_PROJECT_NAME, ARGOCD_APP_SOURCE_REPO_URL)
• Generates HS256-signed JWT tokens with claims: iss, aud, sub, argocd_project, argocd_app, repository
• Token generation happens in template command entrypoint (after loading secrets config, before use)
• Added jwt_signing_key config option (or NYL_VAULT_JWT_SIGNING_KEY env var) as shared secret
• VaultSecretProvider receives generated token via set_nyl_jwt_token() method
• All tests passing (26 total: 18 Vault + 8 JWT)