feat: add built-in prompts and prompt templates for agents (#1401)

docs/architecture/prompt-templates.md

@@ -0,0 +1,290 @@
+# Prompt Templates
+
+This document explains the prompt templates feature — how it works, the design decisions behind it, and how to use it.
+
+## Overview
+
+Prompt templates allow agents to compose their system messages from reusable fragments and dynamic variables, instead of writing everything from scratch. The system message field on the Agent CRD becomes a Go [text/template](https://pkg.go.dev/text/template) when a prompt template is configured, supporting two capabilities:
+
+1. **Include directives** — `{{include "source/key"}}` inserts a prompt fragment from a ConfigMap
+2. **Variable interpolation** — `{{.AgentName}}`, `{{.ToolNames}}`, etc. inject agent metadata
+
+Templates are resolved at **reconciliation time** by the controller. The final, fully-resolved prompt is baked into the agent's config Secret, so the Python ADK runtime receives a plain string with no template syntax.
+
+```text
+┌──────────────────────────────────────────────────────────┐
+│                   Agent CRD                              │
+│                                                          │
+│  systemMessage: |                                        │
+│    {{include "builtin/skills-usage"}}                    │
+│    You are {{.AgentName}}.                               │
+│                                                          │
+│  promptTemplate:                                         │
+│    dataSources:                                          │
+│      - kind: ConfigMap                                   │
+│        name: kagent-builtin-prompts                      │
+│        alias: builtin                                    │
+└──────────────────────┬───────────────────────────────────┘
+                       │ reconciliation
+                       ▼
+┌──────────────────────────────────────────────────────────┐
+│              Controller (Go)                             │
+│                                                          │
+│  1. Resolve raw system message                           │
+│  2. Translate tools (MCP servers, agents)                │
+│  3. Fetch all data from referenced ConfigMaps            │
+│  4. Build template context from agent + translated config│
+│  5. Execute Go text/template with include + variables    │
+│  6. Store resolved string in config Secret               │
+└──────────────────────┬───────────────────────────────────┘
+                       │
+                       ▼
+┌──────────────────────────────────────────────────────────┐
+│           Config Secret → Python ADK                     │
+│                                                          │
+│  instruction: |                                          │
+│    ## Skills                                             │
+│    You have access to skills — pre-built capabilities... │
+│    You are my-agent.                                     │
+└──────────────────────────────────────────────────────────┘
+```
+
+## Design Decisions
+
+### Why `source/key` syntax?
+
+ConfigMaps naturally contain multiple keys. Rather than requiring users to declare each key individually as a separate template reference, the `source/key` syntax lets you reference an entire ConfigMap once and access any of its keys directly. This significantly reduces boilerplate when using multiple prompts from the same source.
+
+The `/` separator (rather than `.`) avoids ambiguity since dots are valid in Kubernetes resource names but slashes are not.
+
+### Why aliases?
+
+ConfigMap names can be long (e.g., `kagent-builtin-prompts`). The optional `alias` field lets users define a shorter identifier for use in include directives, keeping the system message readable: `{{include "builtin/skills-usage"}}` instead of `{{include "kagent-builtin-prompts/skills-usage"}}`.
+
+### Why Go text/template?
+
+Prompt engineering is a text-authoring activity. Users need fine-grained control over where fragments are placed within their prose — a structured ordered-parts list would force artificial boundaries. Since the controller is written in Go and already uses `text/template` for other purposes (skills init scripts), it was a natural choice.
+
+### Why resolve at reconciliation time?
+
+Resolving templates in the controller (rather than at runtime in the Python ADK) gives several benefits:
+
+- **Predictability** — the resolved prompt is visible in the config Secret, making debugging straightforward
+- **No Python-side changes** — the ADK runtime receives a plain string, keeping the runtime simple
+- **Validation** — template errors are caught immediately at reconciliation time and surfaced via the Agent's `Accepted` status condition
+
+### Why no nested includes?
+
+Content pulled from ConfigMaps via `{{include "source/key"}}` is treated as **plain text**, not as a template. This means included fragments cannot themselves use `{{include}}` or `{{.Variable}}` directives. This keeps the system simple, predictable, and avoids potential circular reference issues.
+
+### Why `TypedLocalReference` for prompt sources?
+
+Each prompt source uses an inlined `TypedLocalReference` (`kind`, `apiGroup`, `name`) rather than a fixed enum. This makes the API extensible — today it supports ConfigMaps, but a future `PromptLibrary` CRD (`kind: PromptLibrary, apiGroup: kagent.dev`) could be added without changing the schema. The reference is local (same namespace as the agent) for simplicity and performance.
+
+### Why ConfigMaps only (no Secrets)?
+
+Prompt templates contain prompt text, not sensitive credentials. Supporting Secrets would introduce unnecessary security risk — users might accidentally expose sensitive data in system prompts. ConfigMaps are the right primitive for non-sensitive configuration data.
+
+### Backwards compatibility
+
+To avoid breaking existing agents that may have literal `{{` characters in their system messages, the `systemMessage` field is **only** treated as a template when `promptTemplate` is set. If no prompt template is configured, the system message is passed through as a plain string, preserving existing behavior.
+
+## Usage
+
+### Basic example
+
+Create a ConfigMap with your prompt fragments, then reference it from your agent:
+
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: my-prompts
+  namespace: default
+data:
+  preamble: |
+    You are a helpful assistant specialized in Kubernetes operations.
+    Always explain your reasoning before taking action.
+  safety: |
+    Never delete resources without explicit user confirmation.
+    Never expose secrets or credentials in your responses.
+---
+apiVersion: kagent.dev/v1alpha2
+kind: Agent
+metadata:
+  name: k8s-helper
+  namespace: default
+spec:
+  description: "Kubernetes troubleshooting agent"
+  declarative:
+    modelConfig: default-model-config
+    systemMessage: |
+      {{include "my-prompts/preamble"}}
+
+      Your name is {{.AgentName}} and you operate in the {{.AgentNamespace}} namespace.
+      Your purpose: {{.Description}}
+
+      {{include "my-prompts/safety"}}
+    promptTemplate:
+      dataSources:
+        - kind: ConfigMap
+          name: my-prompts
+    tools:
+      - type: McpServer
+        mcpServer:
+          name: k8s-tools
+          kind: RemoteMCPServer
+          apiGroup: kagent.dev
+          toolNames: ["get-pods", "describe-pod"]
+```
+
+This resolves to:
+
+```
+You are a helpful assistant specialized in Kubernetes operations.
+Always explain your reasoning before taking action.
+
+Your name is k8s-helper and you operate in the default namespace.
+Your purpose: Kubernetes troubleshooting agent
+
+Never delete resources without explicit user confirmation.
+Never expose secrets or credentials in your responses.
+```
+
+### Using aliases
+
+Use the `alias` field to shorten include paths:
+
+```yaml
+promptTemplate:
+  dataSources:
+    - kind: ConfigMap
+      name: kagent-builtin-prompts
+      alias: builtin
+    - kind: ConfigMap
+      name: my-custom-prompts
+      alias: custom
+```
+
+Then in your system message:
+
+```yaml
+systemMessage: |
+  {{include "builtin/skills-usage"}}
+  {{include "custom/my-rules"}}
+```
+
+Without aliases, you'd write `{{include "kagent-builtin-prompts/skills-usage"}}`.
+
+### Using built-in prompts
+
+Kagent ships a `kagent-builtin-prompts` ConfigMap (deployed via Helm) with the following keys:
+
+| Key | Description |
+|-----|-------------|
+| `skills-usage` | Instructions for agents that use skills from `/skills` |
+| `tool-usage-best-practices` | Guidelines for effective tool usage (read before write, explain before acting, verify after changes) |
+| `safety-guardrails` | Safety rules: no destructive ops without confirmation, least privilege, rollback planning, protect sensitive data |
+| `kubernetes-context` | Kubernetes operational methodology: investigation protocol, problem-solving framework, key principles |
+| `a2a-communication` | Guidelines for agent-to-agent communication |
+
+Example using built-in prompts:
+
+```yaml
+apiVersion: kagent.dev/v1alpha2
+kind: Agent
+metadata:
+  name: my-agent
+  namespace: kagent  # same namespace where kagent is installed
+spec:
+  description: "A safe Kubernetes agent with skills"
+  declarative:
+    systemMessage: |
+      {{include "builtin/skills-usage"}}
+      {{include "builtin/tool-usage-best-practices"}}
+
+      You are {{.AgentName}}. {{.Description}}.
+
+      {{include "builtin/safety-guardrails"}}
+      {{include "builtin/kubernetes-context"}}
+    promptTemplate:
+      dataSources:
+        - kind: ConfigMap
+          name: kagent-builtin-prompts
+          alias: builtin
+```
+
+All 10 shipped agent Helm charts (k8s, helm, cilium-debug, cilium-manager, cilium-policy, istio, kgateway, observability, promql, argo-rollouts) use the built-in prompts to replace their repeated safety, operational, and tool-usage sections.
+
+### Using multiple sources
+
+You can reference multiple ConfigMaps simultaneously:
+
+```yaml
+promptTemplate:
+  dataSources:
+    - kind: ConfigMap
+      name: kagent-builtin-prompts
+      alias: builtin
+    - kind: ConfigMap
+      name: team-prompts
+      alias: team
+```
+
+Then use any key from any source:
+
+```yaml
+systemMessage: |
+  {{include "builtin/safety-guardrails"}}
+  {{include "team/coding-standards"}}
+```
+
+### Available template variables
+
+When `promptTemplate` is set, the following variables are available in `systemMessage`:
+
+| Variable | Type | Source |
+|----------|------|--------|
+| `{{.AgentName}}` | `string` | `metadata.name` |
+| `{{.AgentNamespace}}` | `string` | `metadata.namespace` |
+| `{{.Description}}` | `string` | `spec.description` |
+| `{{.ToolNames}}` | `[]string` | Collected from the translated agent config (HTTP and SSE MCP tools) |
+| `{{.SkillNames}}` | `[]string` | Derived from `skills.refs` and `skills.gitRefs` using shared OCI/Git name helpers |
+
+Template resolution happens **after** tools are translated, so `.ToolNames` reflects the actual tool names from the fully resolved MCP server configurations.
+
+You can use standard Go template constructs to work with these variables:
+
+```yaml
+# List tools
+Available tools: {{range .ToolNames}}- {{.}}
+{{end}}
+
+# Conditional
+{{if .SkillNames}}You have skills available.{{end}}
+
+# Count
+You have {{len .ToolNames}} tools configured.
+```
+
+## ConfigMap change detection
+
+The agent controller watches ConfigMaps referenced by agents via `promptTemplate.dataSources` or `systemMessageFrom`. When a ConfigMap's content changes, all agents referencing it are automatically re-reconciled, and their resolved system messages are updated.
+
+## Error handling
+
+If template resolution fails (missing ConfigMap, invalid template syntax, unknown path in `{{include}}`), the agent's `Accepted` status condition is set to `False` with reason `ReconcileFailed` and a message describing the error. The agent's deployment is not updated until the error is resolved.
+
+```bash
+# Check for template errors
+kubectl get agent my-agent -o jsonpath='{.status.conditions[?(@.type=="Accepted")]}'
+```
+
+## Related Files
+
+- [agent_types.go](../../go/api/v1alpha2/agent_types.go) — `PromptTemplateSpec`, `PromptSource` types
+- [template.go](../../go/internal/controller/translator/agent/template.go) — Template resolution logic
+- [template_test.go](../../go/internal/controller/translator/agent/template_test.go) — Unit tests
+- [adk_api_translator.go](../../go/internal/controller/translator/agent/adk_api_translator.go) — Template integration in `translateInlineAgent()`
+- [agent_controller.go](../../go/internal/controller/agent_controller.go) — ConfigMap watch setup
+- [builtin-prompts-configmap.yaml](../../helm/kagent/templates/builtin-prompts-configmap.yaml) — Built-in prompt templates

go/.gitignore

@@ -25,3 +25,6 @@ go.work
 *.swp
 *.swo
 *~
+
+# turso
+file::*

go/Makefile

@@ -137,7 +137,7 @@ CONTROLLER_TOOLS_VERSION ?= v0.19.0
 ENVTEST_VERSION ?= $(shell go list -m -f "{{ .Version }}" sigs.k8s.io/controller-runtime | awk -F'[v.]' '{printf "release-%d.%d", $$2, $$3}')
 #ENVTEST_K8S_VERSION is the version of Kubernetes to use for setting up ENVTEST binaries (i.e. 1.31)
 ENVTEST_K8S_VERSION ?= $(shell go list -m -f "{{ .Version }}" k8s.io/api | awk -F'[v.]' '{printf "1.%d", $$3}')
-GOLANGCI_LINT_VERSION ?= v2.7.2
+GOLANGCI_LINT_VERSION ?= v2.10.1
 
 .PHONY: controller-gen
 controller-gen: $(CONTROLLER_GEN) ## Download controller-gen locally if necessary.