Skip to content

feat: embedded MCP server exposing the admin API as tools#44

Open
dfradehubs wants to merge 4 commits into
achetronic:masterfrom
dfradehubs:feature/admin-mcp-server
Open

feat: embedded MCP server exposing the admin API as tools#44
dfradehubs wants to merge 4 commits into
achetronic:masterfrom
dfradehubs:feature/admin-mcp-server

Conversation

@dfradehubs
Copy link
Copy Markdown

Closes #43.

Summary

  • New opt-in MCP server embedded in magec-server (server.mcp.enabled, default port 8082).
  • ~61 MCP tools covering full CRUD across backends, memory providers, MCP servers, agents (with link/unlink MCP), clients (with token regeneration), commands, flows, secrets (value redacted on reads), settings, conversations, plus type catalogues.
  • Streamable HTTP transport. Auth reuses server.adminPassword as bearer via a new middleware.BearerAuth that mirrors AdminAuth (constant-time compare + per-IP rate limit, 5 failures/min).
  • Tools call *store.Store directly. No HTTP roundtrip back to admin port.

Design notes

  • Skills upload/download and backup/restore stay on admin REST. Binary archives do not map cleanly to MCP tool inputs and outputs.
  • Validators are shared, not duplicated: admin.ValidateFlowStep, admin.ValidateClientConfig, admin.SecretToResponse are exported and reused.
  • Flow tools use an explicit *jsonschema.Schema{Type:"object"} because store.FlowStep is self-referential and the SDK's reflection-based schema generator does not support cycles. Runtime behaviour is unchanged.
  • omitempty added to id tags and client.token in store/types.go so the MCP SDK does not mark server-assigned fields as required on create_* inputs. Wire shape on responses is unchanged because those fields are always populated at write time.
  • Empty server.adminPassword keeps both admin REST and MCP open with a clear startup WARN per surface (admin and MCP independently).

Docs

  • New page: website/content/docs/admin-mcp-server.md with enablement, auth, Claude Code / mcp-cli connection examples, full tool catalogue and troubleshooting.
  • Sidebar entry weight 6 under Core Concepts (website/hugo.toml).
  • .agents/AGENTS.md, .agents/MULTI_AGENT_ADMIN_API.md updated with pointers.
  • .agents/DECISIONS.md adds decision Chore/complete code audit #30 covering motivation, trade-offs and the explicit do not list.
  • .agents/TODO.md "Recently Completed" entry.

Test plan

  • go test -race ./api/mcp/... ./api/admin/... ./middleware/... ./store/... — green.
  • Per-resource table-driven tests under server/api/mcp/tools_*_test.go.
  • In-memory transport smoke (server_test.go): server connects, lists ≥50 tools, all representative tool names present.
  • HTTP-level bearer smoke (server_http_test.go): 401 without/with wrong bearer, anything-but-401 with correct bearer, open mode bypasses auth when password is empty.
  • Local end-to-end run with server.mcp.enabled: true. Verified initializenotifications/initializedtools/list (61 tools) → tools/call for create/update/delete across every resource group. Cross-surface verification: writes via MCP appear in admin REST immediately and vice versa.
  • Secret redaction verified: grepping the plaintext value across MCP responses returns zero matches in create/list/get paths.

Out of scope

  • MCP resources/prompts. Tools-only keeps every CLI client compatible. Open question on the linked issue.
  • Skill upload/download and backup/restore through MCP. Admin REST remains authoritative.
  • Refactoring AdminAuth to call BearerAuth internally. The two middlewares share the rate limiter and bearer extractor but differ in their /api/ carve-out; consolidating is a follow-up.

Add an opt-in MCP server (server.mcp.enabled, default port 8082) that
exposes the entire admin API as ~61 MCP tools over Streamable HTTP.
Authentication reuses server.adminPassword as a bearer token via a new
middleware.BearerAuth that mirrors AdminAuth's constant-time compare
and per-IP rate limiter.

Tools call *store.Store directly. There is no HTTP roundtrip back to
the admin port, following the spirit of decision achetronic#6 ("Admin UI never
accesses the User API"). Skills upload/download and backup/restore
remain on admin REST because binary archives do not map cleanly to
MCP tool inputs.

Implementation lives in server/api/mcp/, one tools_<resource>.go per
admin resource group. Existing admin validators (ValidateFlowStep,
ValidateClientConfig, SecretToResponse) are exported and reused so
both surfaces enforce the same rules.

Secret values are never returned by GET tools (magec_get_secret,
magec_list_secrets), matching the admin REST redaction policy.

Adds 'omitempty' to id and client.token tags in store.types so the
SDK's JSON schema reflection does not mark server-assigned fields as
required on the inputs of create_* tools.

Docs: website/content/docs/admin-mcp-server.md plus sidebar entry.
Decision record: .agents/DECISIONS.md achetronic#30.
Replace the hand-written tools_*.go files with runtime discovery via
github.com/achetronic/openapi2tools. The library parses the admin
swagger spec (converted from 2.0 to 3.0 with kin-openapi), builds JSON
Schema inputs from path, query, header and body parameters, and
produces one MCP tool per filtered operation. A thin adapter wires the
resulting descriptors onto the go-sdk v1.4.1 server (the library ships
its own adapter for an older sdk version, hence the local one).

Tool calls reach the admin REST API over the loopback interface via
mcptools.HTTPExecutor with the bearer token forwarded as a default
header. Validation, secret redaction and conversation logging stay in
the admin handlers and apply uniformly whether the caller is the admin
UI, a REST client, or an MCP tool. Adding a new admin endpoint
propagates to the MCP catalogue on the next build with no changes to
this package.

Effects:

- Removes the 12 tools_*.go files and their per-resource tests
  (~1.7k LOC).
- Removes the admin handler exports that were only used by the old
  MCP package: ValidateFlowStep, ValidateClientConfig,
  SecretToResponse, SessionService accessor, Store accessor.
- Reverts the omitempty workaround on id/token tags in store types.
  The new path takes its schema from swagger, not from struct
  reflection, so server-assigned fields are no longer marked as
  required on create_* inputs.
- Adds dependencies: github.com/achetronic/openapi2tools,
  github.com/getkin/kin-openapi.

Tests update: handler_test.go drives the server through the in-memory
transport, exercises the dispatcher against a stub admin (recording
bearer, method, path and body), and re-verifies the bearer middleware
on both 401 and authenticated paths.
@dfradehubs
Copy link
Copy Markdown
Author

Pivot in 5911ca6: the catalogue is no longer hand-written, it is discovered at startup from the admin OpenAPI spec via openapi2tools + kin-openapi for the swagger 2.0 → 3.0 conversion.

What changed:

  • Deleted the 12 tools_*.go files (~1.7k LOC) and their per-resource tests.
  • New server/api/mcp/handler.go (156 LOC): spec load → convert to 3.0 → openapi2tools loader → route filter → describe → register.
  • New server/api/mcp/adapter.go (125 LOC): thin go-sdk v1.4.1 adapter, because the library's bundled adapter targets an older sdk version.
  • New server/api/mcp/handler_test.go: smoke via in-memory transport, dispatcher test against a stub admin (asserts bearer/method/path/body), bearer middleware test.
  • Reverted the omitempty workaround on id/token tags in store/types.go — the schemas now come from swagger, not struct reflection.
  • Reverted the admin handler exports that were only used by MCP (ValidateFlowStep, ValidateClientConfig, SecretToResponse, SessionService, Store).
  • Tool calls go to the admin port on the loopback interface with the bearer forwarded as a default header. Admin REST stays the single layer for validation and redaction; nothing is duplicated inside MCP.
  • Default route filter excludes /skills/upload, /skills/{id}/download, /settings/backup, /settings/restore, /auth/check, /overview (binary streams + admin-UI helpers).

Net diff in this commit: +550 / -2056 (24 files removed, 2 new).

Verified locally with make build + make build-admin + a config with server.mcp.enabled: true: tools=60 registered, magec_post_backends / magec_get_backends / magec_post_agents round-trip cross-surface (admin REST sees writes immediately and vice versa), validation surfaces correctly ({"error":"name is required"} propagates from admin REST as a tool error), bearer auth still gates the transport (401 without / 401 with wrong / 200 with correct), and secret values stay redacted on the wire.

The decision record (.agents/DECISIONS.md #30) has been rewritten to reflect the new approach.

middleware:
- Extract a shared passwordCheck primitive (matches/allow/writeJSONError)
  used by both AdminAuth and BearerAuth, replacing 40 LOC of copy-paste.
- Carve-out logic in AdminAuth is now obvious (static SPA path, auth/check
  endpoint that skips the rate limiter). BearerAuth is six lines.

api/mcp/adapter.go:
- Use http.MethodGet/Put/Delete instead of bare string literals.
- Extract makeToolHandler(td) so the registration loop stays readable and
  the dispatch path can be followed in isolation.
- Move the destructive=true sentinel inside the DELETE branch — the only
  one that uses it.
- Wrap json marshal/unmarshal errors with context to match the project's
  error-handling convention.

api/mcp/handler.go:
- Rename opaque v2/v3 locals to rawSwagger/swaggerSpec/openAPISpec/openAPIBytes
  in loadAdminSpec.
- Document HandlerConfig fields individually instead of cramming the
  config and middleware names into a single paragraph.

api/mcp/handler_test.go:
- Replace the manual Read/buf loop in adminStub.ServeHTTP with io.ReadAll.
- Factor connectInMemory(t, h) so the two MCP tests do not duplicate the
  in-memory transport dance.
- Use http.MethodPost and the package-level initFrame constant in the
  bearer-required test.

main.go:
- Drop a stale comment about BearerAuth's internals; the middleware
  package now documents them.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Proposal: embedded MCP server exposing the admin API as tools

1 participant