Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions appinfo/routes.php
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@
['name' => 'metrics#index', 'url' => '/api/metrics', 'verb' => 'GET'],
['name' => 'health#index', 'url' => '/api/health', 'verb' => 'GET'],

// DSO / Omgevingsloket STAM koppelvlak
['name' => 'dso#receiveVerzoek', 'url' => '/api/dso/stam/verzoeken', 'verb' => 'POST'],

['name' => 'dashboard#index', 'url' => '/api/dashboard', 'verb' => 'GET'],
['name' => 'dashboard#getCallStats', 'url' => '/api/dashboard/callstats', 'verb' => 'GET'],
['name' => 'dashboard#getJobStats', 'url' => '/api/dashboard/jobstats', 'verb' => 'GET'],
Expand Down
84 changes: 84 additions & 0 deletions docs/features/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# OpenConnector Features

OpenConnector is an API gateway and integration hub for Nextcloud. It brings enterprise service bus (ESB) capabilities natively into Nextcloud — define external API connections, expose your own endpoints, transform data with flexible mappings, and keep systems synchronized through scheduled or event-driven flows.

## Feature Index

| Feature | Description | Status |
|---------|-------------|--------|
| [Sources](sources.md) | External API connections with multi-protocol authentication | Implemented |
| [Endpoints](endpoints.md) | Expose reverse-proxy API paths with rule-based logic | Implemented |
| [Mappings](mappings.md) | Twig-powered data transformation between schemas | Implemented |
| [Synchronizations](synchronizations.md) | Scheduled and event-driven source-to-target sync | Implemented |
| [Rules](rules.md) | Authentication, file handling, locking, and audit trail rules | Implemented |
| [Jobs](jobs.md) | Cron-based scheduled task execution | Implemented |
| [Events & Webhooks](events.md) | CloudEvents emission, subscription, and consumer processing | Implemented |
| [Logging & Monitoring](logging.md) | Call logs, sync logs, and Prometheus metrics | Implemented |
| [Configuration Management](configuration-management.md) | Import/export, configuration groups, slug-based references | Implemented |
| [StUF Adapter](stuf-adapter.md) | REST/ZGW to StUF-BG/ZKN SOAP translation | Partial |
| [Prometheus Metrics](prometheus-metrics.md) | Prometheus exposition format metrics + health endpoint | Implemented |
| [DSO / Omgevingsloket Adapter](dso-omgevingsloket.md) | DSO-LV STAM koppelvlak integration | Implemented |
| [iBabs & NotuBiz Connector](ibabs-notubiz-connector.md) | RIS integration for bestuurlijke besluitvorming | Implemented |

## Architecture Overview

```
External Systems OpenConnector Targets
───────────────── ─────────────────────────────────── ──────────────────
REST APIs → Sources → CallService → Mappings → OpenRegister
SOAP Services → Endpoints (reverse proxy) → External REST APIs
Webhooks → Consumers → EventService → Other Sources
Cron → Jobs → SynchronizationService → Register/Schema
```

## Core Concepts

### Sources
A **Source** is a configured connection to an external system. It stores the base URL, authentication method, headers, certificates, and request defaults. Sources are reused across endpoints, synchronizations, and jobs.

### Endpoints
An **Endpoint** is a path exposed by OpenConnector that acts as a reverse proxy, OpenRegister gateway, or rule-execution surface. Endpoints have HTTP methods, target configuration, and an ordered list of Rules.

### Mappings
A **Mapping** defines a field-level transformation between source and target schemas. It uses direct assignments, Twig template expressions, dot-notation paths, and JSON Logic conditions to reshape data structures.

### Synchronizations
A **Synchronization** defines a full data flow: which Source to read from, which Mapping to apply, and which target (OpenRegister schema or another Source) to write to. The sync engine handles pagination, hash-based change detection, and per-object contract tracking.

### Rules
A **Rule** adds logic to an endpoint. Rules enforce authentication, trigger synchronizations, handle file uploads/downloads, control resource locking, expose audit trails, and can be conditionally applied via JSON Logic.

### Jobs
A **Job** schedules a synchronization or other task on a cron expression. Execution history is stored in job logs.

### Events and Consumers
OpenConnector emits and consumes **CloudEvents**. **Consumers** are configured handlers that process incoming webhook payloads. **EventSubscriptions** subscribe to specific event types and route them to handlers.

## Standards Compliance

| Standard | Role |
|----------|------|
| REST-API Design Rules (Logius) | API design for exposed endpoints |
| OpenAPI 3.0 | Configuration import/export format |
| NL GOV CloudEvents | Event emission and consumption |
| Digikoppeling | PKIoverheid mTLS for government connections |
| StUF-BG 3.10 / StUF-ZKN 3.10 | Legacy SOAP adapter (partial) |
| GEMMA Gemeentelijke servicebuscomponent | Primary architectural role |
| GEMMA Notificatierouteringcomponent | CloudEvents routing role |

## Data Model

| Entity | Purpose |
|--------|---------|
| Source | External API connection configuration |
| Endpoint | Exposed reverse-proxy route |
| Mapping | Field-level transformation definition |
| Synchronization | Source-to-target sync flow definition |
| SynchronizationContract | Per-object sync state (origin ID, target ID, hash) |
| Rule | Endpoint logic (auth, file, lock, audit) |
| Job | Scheduled task with cron expression |
| Consumer | Incoming webhook/event handler |
| Event | CloudEvent definition |
| EventSubscription | Event listener with handler config |
| CallLog | HTTP request/response audit log |
| SynchronizationLog | Per-sync run result log |
110 changes: 110 additions & 0 deletions docs/features/configuration-management.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
# Configuration Management

## Overview

OpenConnector's configuration management features allow administrators to bundle related entities into named groups, import and export configurations as structured JSON, and reference entities with stable slug-based identifiers. This enables environment migration (dev → test → production), configuration sharing between organisations, and backup/restore workflows.

## Configuration Groups

A **Configuration** (also called a configuration group) bundles related Sources, Endpoints, Mappings, Rules, Jobs, and Synchronizations under a single name. Configurations can be exported as a single JSON file and re-imported on another OpenConnector instance.

| Field | Description |
|-------|-------------|
| `name` | Human-readable configuration name |
| `slug` | URL-friendly identifier |
| `description` | Purpose and contents description |
| `sources` | Included Source slugs |
| `endpoints` | Included Endpoint slugs |
| `mappings` | Included Mapping slugs |
| `rules` | Included Rule slugs |
| `jobs` | Included Job slugs |
| `synchronizations` | Included Synchronization slugs |

## Import

Configurations are imported as JSON via:

```
POST /index.php/apps/openconnector/api/import
Content-Type: application/json
```

The import process:

1. Parses the JSON structure
2. For each entity type, upserts entities by slug (create if missing, update if exists)
3. Resolves cross-references (e.g. a synchronization referencing a source by slug)
4. Reports created, updated, and skipped counts per entity type

Import is idempotent: re-importing the same configuration updates existing entities without duplicating them.

## Export

Export a configuration group or individual entity types via:

```
GET /index.php/apps/openconnector/api/export
GET /index.php/apps/openconnector/api/export?configurationId={id}
```

The export format is an OpenAPI-structured JSON document:

```json
{
"openapi": "3.0.0",
"info": {
"title": "OpenConnector Configuration Export",
"version": "1.0.0"
},
"components": {
"x-sources": [ ... ],
"x-endpoints": [ ... ],
"x-mappings": [ ... ],
"x-rules": [ ... ],
"x-synchronizations": [ ... ],
"x-jobs": [ ... ]
}
}
```

## Slug-Based References

All OpenConnector entities have a `slug` field — a URL-friendly, human-readable identifier that is unique per entity type. Slugs are used in:

- Export/import for stable cross-environment references
- API paths for human-readable entity access
- Cross-entity references in configurations (e.g. a job referencing a synchronization by slug)

Slugs are automatically generated from the entity name on creation and can be manually set. They do not change when entities are updated.

## Configuration Handlers

Each entity type has a dedicated configuration handler that manages import/export serialization:

| Handler | Entity |
|---------|--------|
| `SourceHandler` | Sources |
| `EndpointHandler` | Endpoints |
| `MappingHandler` | Mappings |
| `RuleHandler` | Rules |
| `SynchronizationHandler` | Synchronizations |
| `JobHandler` | Jobs |

## Settings

Global application settings (retention periods, default behaviours) are managed via the Settings section in the OpenConnector UI and stored in Nextcloud's `IAppConfig`.

```
GET /index.php/apps/openconnector/api/settings
PUT /index.php/apps/openconnector/api/settings
```

## Implementation

- `lib/Service/ConfigurationService.php` — Configuration group management
- `lib/Service/ImportService.php` — Import orchestration
- `lib/Service/ExportService.php` — Export orchestration
- `lib/Service/ConfigurationHandlers/` — Per-entity-type handlers
- `lib/Controller/ImportController.php` — Import REST endpoint
- `lib/Controller/ExportController.php` — Export REST endpoint
- `lib/Controller/SettingsController.php` — Settings REST endpoint
101 changes: 101 additions & 0 deletions docs/features/dso-omgevingsloket.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# DSO / Omgevingsloket Adapter

## Overview

The DSO adapter integrates OpenConnector with the Digitaal Stelsel Omgevingswet (DSO) Landelijke Voorziening for receiving and processing vergunningaanvragen, meldingen, and informatieverzoeken from the Omgevingsloket. Required by Dutch VTH-related government tenders.

## Endpoints

### POST /api/dso/stam/verzoeken

Receives DSO-verzoek payloads from DSO-LV via the STAM koppelvlak.

**Authentication:** Public endpoint with webhook signature validation via `X-DSO-Signature` header.

**Request body:** JSON payload conforming to the STAM schema:

```json
{
"verzoekId": "dso-12345",
"bronorganisatie": "00000001234567890000",
"type": "aanvraag",
"indieningsdatum": "2024-06-15",
"aanvrager": {
"bsn": "999993653",
"naam": "J. Jansen",
"adres": { "straatnaam": "Hoofdstraat", "huisnummer": "10", "postcode": "1234AB", "woonplaats": "Utrecht" },
"contactgegevens": { "email": "j.jansen@example.nl", "telefoon": "0612345678" }
},
"locatie": {
"bagAdres": { "postcode": "1234AB", "huisnummer": "10" },
"gmlGeometrie": "<gml:Point><gml:pos>52.370216 4.895168</gml:pos></gml:Point>"
},
"activiteiten": [
{ "code": "bouwen-01", "omschrijving": "Bouwen van een woning" }
],
"bouwkosten": 250000,
"bijlagen": [
{ "naam": "bouwtekening.pdf", "type": "tekening", "url": "https://dso-lv.nl/docs/abc123" }
]
}
```

**Response (202 Accepted):**

```json
{
"verzoekId": "dso-12345",
"status": "ontvangen",
"message": "Verzoek ontvangen en wordt verwerkt"
}
```

**Error responses:**
- `401 Unauthorized` -- Invalid webhook signature
- `400 Bad Request` -- Payload validation errors with field-level details

## Verzoek Types

| Type | Description | Zaak created |
|------|-------------|--------------|
| `aanvraag` | Vergunningaanvraag | Full zaak with behandelproces |
| `melding` | Melding (notification) | Simplified zaak, no besluit required |
| `informatieverzoek` | Request for information | Lightweight zaak for advies |
| `vooroverleg` | Pre-application consultation | Lightweight zaak, no formal besluit |

## Activiteiten Mapping

DSO activiteiten (bouwen, milieu, kappen, etc.) are mapped to zaaktypen via a configurable mapping table stored in OpenRegister. The mapping supports:

- **One-to-one:** One activiteit maps to one zaaktype
- **One-to-many:** One activiteit generates multiple zaaktypen for different afdelingen
- **Samenloop:** Multiple activiteiten in one verzoek can create deelzaken or a combined zaak

## Validation

The parser validates:
- Required fields (verzoekId, type, indieningsdatum, aanvrager, locatie, activiteiten)
- BSN 11-proef validation
- ISO 8601 date format
- Enum values for type field

## PKIoverheid Authentication

DSO-LV communication uses PKIoverheid certificates for mutual TLS. Certificates are configured via the Source entity's configuration field and managed through CallService's existing certificate handling.

## Implementation

- **DSOController**: `lib/Controller/DSOController.php` -- STAM endpoint
- **DSOParserService**: `lib/Service/DSOParserService.php` -- Payload parsing and validation
- **Route**: `appinfo/routes.php` -- POST /api/dso/stam/verzoeken
- **Tests**: `tests/Unit/Service/DSOParserServiceTest.php`

## Status

Foundational implementation complete (endpoint, parser, validator). The following features require external dependencies and are planned for future implementation:

- Bijlagen download from DSO-LV (requires mTLS certificates)
- Automatic zaak creation (requires Procest app)
- Status push back to DSO-LV
- DSO-SWF samenwerking
- Activiteiten-mapping administration UI
89 changes: 89 additions & 0 deletions docs/features/endpoints.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# Endpoints

## Overview

An **Endpoint** is a URL path exposed by OpenConnector that can act as a reverse proxy to an external source, a gateway to OpenRegister data, or a rule-execution surface for custom logic. Endpoints allow other systems and clients to interact with OpenConnector as if it were a native API.

## Endpoint Types

| Target Type | Description |
|-------------|-------------|
| `source` | Proxy requests to an external Source |
| `register/schema` | Read and write OpenRegister objects |
| `fixed` | Return a static response |

## HTTP Methods

Each endpoint is configured with one or more HTTP methods (`GET`, `POST`, `PUT`, `PATCH`, `DELETE`). An endpoint can be registered for multiple methods with different rule sets.

## Endpoint Path and Routing

Endpoints are registered at `/index.php/apps/openconnector/api/endpoint/{path}`. Path parameters (e.g. `/{id}`) are supported and injected into the request context for use in rules and target resolution.

Slug-based identifiers allow consistent references across environments.

## Rules

Rules are the core logic layer of an endpoint. Each endpoint has an ordered list of Rules that execute in sequence on every incoming request. Rule types include:

- **Authentication** — Validate incoming credentials before proxying
- **Synchronization** — Trigger a sync run when the endpoint is called
- **Download** — Serve a file from OpenRegister or a source
- **Upload** — Accept file uploads and store them
- **Locking** — Acquire an exclusive lock on a resource
- **Audit Trail** — Expose the change history of an object

Rules can be conditionally applied using JSON Logic conditions evaluated against the incoming request (body, headers, query parameters, path, method).

See [Rules](rules.md) for full documentation.

## Request Flow

```
Incoming HTTP Request
|
v
Endpoint matched by path + method
|
v
Rules executed in order (authentication first)
|
v
Request proxied to Source / OpenRegister / fixed response
|
v
Response returned to caller
```

## Proxy Behavior

When target type is `source`, OpenConnector forwards the request to the configured Source using `CallService`. Request headers, query parameters, and body are forwarded (with configurable overrides). The source response is returned to the caller with its original status code and content type.

## OpenRegister Gateway

When target type is `register/schema`, the endpoint provides CRUD access to OpenRegister objects. The register and schema are configured on the endpoint. Standard JSON:API-compatible request/response format is used.

## CORS

OpenConnector registers CORS `OPTIONS` preflight routes for all public endpoints. The `Access-Control-Allow-Origin`, `Access-Control-Allow-Methods`, and `Access-Control-Allow-Headers` headers are configurable per endpoint.

## Authentication on Exposed Endpoints

Incoming requests to endpoints can be authenticated using an **Authentication Rule**. Supported incoming auth methods:

| Method | Description |
|--------|-------------|
| `basic` | HTTP Basic Authentication |
| `jwt` | Standard JWT Bearer token |
| `zgw-jwt` | VNG ZGW JWT (Dutch government standard) |
| `oauth` | OAuth 2.0 Bearer token introspection |
| `apikey` | API key in header or query parameter |
| `none` | No authentication (public endpoint) |

## Implementation

- `lib/Service/EndpointService.php` — Request handling, proxying, OpenRegister gateway
- `lib/Controller/EndpointsController.php` — REST CRUD API
- `lib/Db/Endpoint.php` — Entity
- `lib/Db/EndpointMapper.php` — Database mapper
- `lib/Service/EndpointCacheService.php` — Route caching for performance
Loading
Loading