Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
328 changes: 65 additions & 263 deletions .github/workflows/ci.yml

Large diffs are not rendered by default.

32 changes: 17 additions & 15 deletions .github/workflows/qodana.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,33 +16,35 @@ jobs:
runs-on: ubuntu-latest
env:
QODANA_TOKEN: ${{ secrets.QODANA_TOKEN }}

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: "10.0.102"
- name: Run Qodana
if: env.QODANA_TOKEN != ''
uses: JetBrains/qodana-action@v2025.3
with:
args: --linter,jetbrains/qodana-dotnet:2025.3
results-dir: qodana-results
results-dir: artifacts/ci/qodana
upload-result: false

- name: Skip Qodana (missing token)
if: env.QODANA_TOKEN == ''
run: echo "QODANA_TOKEN is not set; skipping Qodana scan."
- name: Run Entry Check
if: env.QODANA_TOKEN != ''
run: bash tools/ci/bin/run.sh qodana
- name: Upload SARIF To Code Scanning
if: env.QODANA_TOKEN != '' && github.event_name != 'pull_request'
if: github.event_name != 'pull_request' && env.QODANA_TOKEN != ''
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: qodana-results/qodana.sarif.json

- name: Upload SARIF Artifact
if: env.QODANA_TOKEN != ''
sarif_file: artifacts/ci/qodana/qodana.sarif.json
- name: Upload Artifact
if: always() && env.QODANA_TOKEN != ''
uses: actions/upload-artifact@v4
with:
name: qodana-sarif
path: qodana-results/qodana.sarif.json
name: ci-qodana
path: artifacts/ci/qodana/
if-no-files-found: error

- name: Skip Message (No Token)
if: env.QODANA_TOKEN == ''
run: echo "QODANA_TOKEN not set; qodana scan skipped."
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,10 @@ tools/*
!tools/check-quality.sh
!tools/run-coverage.sh
!tools/test-bdd-readable.sh
!tools/ci/
!tools/ci/**
tools/ci/checks/**/bin/
tools/ci/checks/**/obj/
!tools/versioning/
!tools/versioning/check-versioning.sh
!tools/versioning/labels.json
Expand Down
9 changes: 5 additions & 4 deletions Directory.Build.props
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
<Project>
<PropertyGroup>
<!-- Central, deterministic project versioning (SemVer baseline). -->
<Version>4.0.0</Version>
<AssemblyVersion>4.0.0.0</AssemblyVersion>
<FileVersion>4.0.0.0</FileVersion>
<InformationalVersion>4.0.0</InformationalVersion>
<Version>4.1.0</Version>
<AssemblyVersion>4.1.0.0</AssemblyVersion>
<FileVersion>4.1.0.0</FileVersion>
<InformationalVersion>4.1.0</InformationalVersion>
<Deterministic>true</Deterministic>
<ContinuousIntegrationBuild Condition="'$(CI)' == 'true'">true</ContinuousIntegrationBuild>
<RestorePackagesWithLockFile>true</RestorePackagesWithLockFile>
</PropertyGroup>
</Project>
5 changes: 3 additions & 2 deletions Directory.Packages.props
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,14 @@
<PackageVersion Include="coverlet.collector" Version="6.0.4" />
<PackageVersion Include="coverlet.msbuild" Version="6.0.4" />
<PackageVersion Include="Microsoft.IO.RecyclableMemoryStream" Version="3.0.1" />
<PackageVersion Include="Microsoft.NET.Test.Sdk" Version="17.14.1" />
<PackageVersion Include="Microsoft.NET.Test.Sdk" Version="18.0.1" />
<PackageVersion Include="Mime" Version="3.8.0" />
<PackageVersion Include="Reqnroll.Tools.MsBuild.Generation" Version="3.3.3" />
<PackageVersion Include="Reqnroll.xUnit" Version="3.3.3" />
<PackageVersion Include="SharpCompress" Version="0.39.0" />
<PackageVersion Include="System.IO.Hashing" Version="10.0.2" />
<PackageVersion Include="YamlDotNet" Version="16.3.0" />
<PackageVersion Include="xunit" Version="2.9.3" />
<PackageVersion Include="xunit.runner.visualstudio" Version="3.1.4" />
<PackageVersion Include="xunit.runner.visualstudio" Version="3.1.5" />
</ItemGroup>
</Project>
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ Die CI ist deterministisch und auditierbar aufgebaut. Alle Checks laufen in sepa
- Auto-Labeling & Auto-Versionierung: [docs/AUTO_LABELING_AND_VERSIONING.md](docs/AUTO_LABELING_AND_VERSIONING.md)
- BDD-Testkatalog und Testablauf: [docs/tests/README.md](docs/tests/README.md)
- Governance/Ownership: [docs/governance/LABELING_OWNERSHIP.md](docs/governance/LABELING_OWNERSHIP.md)
- Qodana läuft als separater Workflow, schreibt Ergebnisse nach `artifacts/ci/qodana/` und veröffentlicht diese als Artefakt.

## 7. Runbook (reproduzierbar)
```bash
Expand All @@ -50,9 +51,8 @@ TEST_BDD_OUTPUT_DIR=artifacts/tests bash tools/test-bdd-readable.sh -- \
/p:Threshold=85%2c69 \
/p:ThresholdType=line%2cbranch \
/p:ThresholdStat=total
bash tools/sync-portable-filetypedetection.sh
bash tools/check-portable-filetypedetection.sh --clean
```
Hinweis: Portable-Sync/Check-Tools sind aktuell nicht im Repository enthalten.

## 7.1 Versionierung (zentral)
- Zentrale Versionsquelle: `Directory.Build.props`.
Expand All @@ -66,7 +66,7 @@ bash tools/check-portable-filetypedetection.sh --clean
Im Root von `src/FileTypeDetection` liegen nur die Public APIs.

### 8.2 Portable
Die portable Spiegelstruktur wird lokal über die Tools erzeugt und ist nicht Teil des veröffentlichten Repository-Inhalts.
Die portable Spiegelstruktur ist nicht Teil des Repository-Inhalts; es gibt keine Sync/Check-Skripte im Repo.

### 8.3 Abstractions-Ordnerhierarchie
Die Modellschicht unter `src/FileTypeDetection/Abstractions` ist nach Verantwortlichkeiten getrennt:
Expand Down
12 changes: 12 additions & 0 deletions docs/CI_PIPELINE.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,18 @@ Für `pull_request` wird SARIF als Workflow-Artefakt veröffentlicht.
Code-Scanning-SARIF-Upload erfolgt nur auf non-PR-Runs, um PR-Noise zu vermeiden.
Profil-Hinweis: In `.qodana/profiles/fileclassifier.yaml` sind nur testpfad-spezifische Excludes für reine Redundanz-Inspections gesetzt (`tests/**`), Produktionscode bleibt unverändert streng.

### 7.1 Qodana Ergebnisse & Artefakte
- Wenn `QODANA_TOKEN` fehlt, wird Qodana im Workflow übersprungen.
- Qodana schreibt Ergebnisse in `artifacts/ci/qodana/` (inkl. `qodana.sarif.json`).
- Für `pull_request`-Runs wird das Verzeichnis als Artefakt veröffentlicht.
- Für non-PR-Runs wird `artifacts/ci/qodana/qodana.sarif.json` in GitHub Code Scanning hochgeladen.
- Artefakte:
- `artifacts/ci/qodana/qodana.sarif.json`
- Verzeichnis `artifacts/ci/qodana/`

Branch-Protection-Hinweis:
- Der Workflow-Status `qodana` muss in GitHub als Required Check konfiguriert werden, damit PR-Merges blockiert werden.

## 8. Lokale Reproduktion
```bash
node tools/versioning/test-compute-pr-labels.js
Expand Down
12 changes: 9 additions & 3 deletions docs/DIN_SPECIFICATION_DE.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,11 +48,17 @@ Die technische Detailbeschreibung der öffentlichen Schnittstellen ist in `01_FU
## 7. Verifikation
Pflichtlauf für Freigabe:
```bash
python3 tools/check-docs.py
dotnet restore FileClassifier.sln -v minimal
dotnet build FileClassifier.sln --no-restore -v minimal
dotnet test tests/FileTypeDetectionLib.Tests/FileTypeDetectionLib.Tests.csproj --no-build -v minimal
bash tools/sync-portable-filetypedetection.sh
bash tools/sync-doc-conventions.sh
TEST_BDD_OUTPUT_DIR=artifacts/tests bash tools/test-bdd-readable.sh -- \
/p:CollectCoverage=true \
/p:Include="[FileTypeDetectionLib]*" \
/p:CoverletOutputFormat=cobertura \
/p:CoverletOutput="$(pwd)/artifacts/coverage/coverage" \
/p:Threshold=85%2c69 \
/p:ThresholdType=line%2cbranch \
/p:ThresholdStat=total
```

## 8. Rückverfolgbarkeit
Expand Down
33 changes: 33 additions & 0 deletions docs/governance/CI_PIPELINE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# CI Pipeline (SSOT)

## Purpose
Deterministic and auditable CI with contract-first artifacts and strict fail-closed execution.

## Required Jobs
- `preflight`
- `build`
- `security-nuget`
- `tests-bdd-coverage`
- `summary`
- `qodana` (separate workflow)

## Artifact Root (single SSOT)
- `artifacts/ci/<check_id>/raw.log`
- `artifacts/ci/<check_id>/summary.md`
- `artifacts/ci/<check_id>/result.json`
- `artifacts/ci/qodana/*.sarif`

No alternative artifact roots are allowed.

## Stage Order
1. Preflight: governance-safe checks and policy guards.
2. Build: restore + build with warnings as errors.
3. Security: NuGet vulnerability/deprecation scan.
4. Tests: BDD + coverage gate.
5. Summary: aggregate and enforce artifact contract + schema validation.
6. Qodana workflow: token/sarif contract and dead-code gate.

## Workflow Constraints
- Workflow YAML contains entry-calls only.
- Check logic is implemented in `.NET` validators under `tools/ci/checks/`.
- Shell scripts in `tools/ci/` handle orchestration and artifact handling only.
43 changes: 43 additions & 0 deletions docs/governance/CI_POLICY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# CI Policy (SSOT)

## Scope
This document is the single source of truth for CI rule IDs, severity handling, and exit code policy.

## Global Rules
- Fail-closed: no silent bypass paths.
- No `continue-on-error: true` in workflow files.
- No `|| true` on critical workflow paths.
- No `set +e` without explicit allow-list entry.
- Workflow YAML only calls entry scripts under `tools/ci/bin/`.

## Result Contract
All required checks MUST write:
- `artifacts/ci/<check_id>/raw.log`
- `artifacts/ci/<check_id>/summary.md`
- `artifacts/ci/<check_id>/result.json`

`result.json` must comply with `tools/ci/schema/result.schema.json`.

## Rule Catalog
- `CI-ARTIFACT-001` fail: required artifact missing.
- `CI-SCHEMA-001` fail: `result.json` schema validation failed.
- `CI-SHELL-001` fail: found `continue-on-error: true`.
- `CI-SHELL-002` fail: found `|| true` in critical workflow path.
- `CI-SHELL-003` fail: found `set +e` outside allow-list.
- `CI-SHELL-004` fail: workflow `run: |` block exceeds configured max lines.
- `CI-GRAPH-001` fail: required CI graph edge or job constraint violated.
- `CI-QODANA-001` fail: `QODANA_TOKEN` missing.
- `CI-QODANA-002` fail: expected SARIF missing.
- `CI-QODANA-003` fail: SARIF invalid JSON.

## Severity Rules
- `warn`: visible, non-blocking.
- `fail`: blocking, exit code non-zero.

## Exit Code Matrix
- `0`: success (`pass` or `warn`)
- `1`: policy/contract/check failure (`fail`)
- `2`: invalid invocation or missing prerequisites

## set +e Allow-list
No allow-list entries in Phase 1.
2 changes: 1 addition & 1 deletion docs/versioning/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Alle Aenderungen werden hier technisch dokumentiert. Die Version selbst ist in
`Directory.Build.props` die SSOT.

## [Unreleased]
- BREAKING: Version-Baseline auf `4.0.0` angehoben (Major-Bump durch oeffentliche API-/Struktur-Aenderungen im Branch).
- Hinweis: Version-Baseline `4.0.0` ist bereits in `main` enthalten.
- Added:
- Changed:
- Fixed:
Expand Down
6 changes: 6 additions & 0 deletions global.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"sdk": {
"version": "10.0.102",
"rollForward": "latestPatch"
}
}
2 changes: 1 addition & 1 deletion src/FileClassifier.App/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,6 @@ dotnet run --project src/FileClassifier.App -- ./tests/FileTypeDetectionLib.Test
## Dokumentpflege-Checkliste

- [ ] Inhalt auf aktuellen Code-Stand geprüft.
- [ ] Links und Anker mit `python3 tools/check-markdown-links.py` geprüft.
- [ ] Links und Anker mit `python3 tools/check-docs.py` geprüft.
- [ ] Beispiele/Kommandos lokal verifiziert.
- [ ] Begriffe mit `docs/01_FUNCTIONS.md` abgeglichen.
56 changes: 56 additions & 0 deletions src/FileClassifier.App/packages.lock.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
{
"version": 2,
"dependencies": {
"net10.0": {
"MimeTypesMap": {
"type": "Transitive",
"resolved": "1.0.9",
"contentHash": "M0TuSCwL1a8QV0VKw8ysY4AIs6v/Aor3N7GXQeqgNlAvqjx9Kj9KxNd09Pg5RzpY1tCOU8mkrfYBi1Lxwj8quQ=="
},
"ZstdSharp.Port": {
"type": "Transitive",
"resolved": "0.8.4",
"contentHash": "eieSXq3kakCUXbgdxkKaRqWS6hF0KBJcqok9LlDCs60GOyrynLvPOcQ0pRw7shdPF7lh/VepJ9cP9n9HHc759g=="
},
"filetypedetectionlib": {
"type": "Project",
"dependencies": {
"Microsoft.IO.RecyclableMemoryStream": "[3.0.1, )",
"Mime": "[3.8.0, )",
"SharpCompress": "[0.39.0, )",
"System.IO.Hashing": "[10.0.2, )"
}
},
"Microsoft.IO.RecyclableMemoryStream": {
"type": "CentralTransitive",
"requested": "[3.0.1, )",
"resolved": "3.0.1",
"contentHash": "s/s20YTVY9r9TPfTrN5g8zPF1YhwxyqO6PxUkrYTGI2B+OGPe9AdajWZrLhFqXIvqIW23fnUE4+ztrUWNU1+9g=="
},
"Mime": {
"type": "CentralTransitive",
"requested": "[3.8.0, )",
"resolved": "3.8.0",
"contentHash": "SG8QHXjnyLoVeIOSw4ym7orS5LIRPBpzFQYfkgSqyAkeog+eZNMj32UOEO1SxLNBASxNPgVBIacxOOZsenBImg==",
"dependencies": {
"MimeTypesMap": "1.0.9"
}
},
"SharpCompress": {
"type": "CentralTransitive",
"requested": "[0.39.0, )",
"resolved": "0.39.0",
"contentHash": "0esqIUDlg68Z7+Weuge4QzEvNtawUO4obTJFL7xuf4DBHMxVRr+wbNgiX9arMrj3kGXQSvLe0zbZG3oxpkwJOA==",
"dependencies": {
"ZstdSharp.Port": "0.8.4"
}
},
"System.IO.Hashing": {
"type": "CentralTransitive",
"requested": "[10.0.2, )",
"resolved": "10.0.2",
"contentHash": "AKJknIFi9O3+rGExxTry188JPvUoZAPcCtS2qdqyFhIzsxQ1Ap94BeGDG0VzVEHakhmRxmJtVih6TsHoghIt/g=="
}
}
}
}
2 changes: 1 addition & 1 deletion src/FileTypeDetection/Abstractions/Archive/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,6 @@ Archiv-Eintragsmodell für sichere In-Memory-Extraktion.
## Dokumentpflege-Checkliste

- [ ] Inhalt auf aktuellen Code-Stand geprüft.
- [ ] Links und Anker mit `python3 tools/check-markdown-links.py` geprüft.
- [ ] Links und Anker mit `python3 tools/check-docs.py` geprüft.
- [ ] Beispiele/Kommandos lokal verifiziert.
- [ ] Begriffe mit `docs/01_FUNCTIONS.md` abgeglichen.
2 changes: 1 addition & 1 deletion src/FileTypeDetection/Abstractions/Detection/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,6 @@ Detektions-Rückgabemodelle der Public API.
## Dokumentpflege-Checkliste

- [ ] Inhalt auf aktuellen Code-Stand geprüft.
- [ ] Links und Anker mit `python3 tools/check-markdown-links.py` geprüft.
- [ ] Links und Anker mit `python3 tools/check-docs.py` geprüft.
- [ ] Beispiele/Kommandos lokal verifiziert.
- [ ] Begriffe mit `docs/01_FUNCTIONS.md` abgeglichen.
2 changes: 1 addition & 1 deletion src/FileTypeDetection/Abstractions/Hashing/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,6 @@ Deterministische Hash-Evidence-Modelle für Physical/Logical-Nachweise und Round
## Dokumentpflege-Checkliste

- [ ] Inhalt auf aktuellen Code-Stand geprüft.
- [ ] Links und Anker mit `python3 tools/check-markdown-links.py` geprüft.
- [ ] Links und Anker mit `python3 tools/check-docs.py` geprüft.
- [ ] Beispiele/Kommandos lokal verifiziert.
- [ ] Begriffe mit `docs/01_FUNCTIONS.md` abgeglichen.
2 changes: 1 addition & 1 deletion src/FileTypeDetection/Abstractions/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,6 @@ Immutable Rückgabemodelle für stabile API-Verträge.
## Dokumentpflege-Checkliste

- [ ] Inhalt auf aktuellen Code-Stand geprüft.
- [ ] Links und Anker mit `python3 tools/check-markdown-links.py` geprüft.
- [ ] Links und Anker mit `python3 tools/check-docs.py` geprüft.
- [ ] Beispiele/Kommandos lokal verifiziert.
- [ ] Begriffe mit `docs/01_FUNCTIONS.md` abgeglichen.
2 changes: 1 addition & 1 deletion src/FileTypeDetection/Configuration/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,6 @@ flowchart LR
## Dokumentpflege-Checkliste

- [ ] Inhalt auf aktuellen Code-Stand geprüft.
- [ ] Links und Anker mit `python3 tools/check-markdown-links.py` geprüft.
- [ ] Links und Anker mit `python3 tools/check-docs.py` geprüft.
- [ ] Beispiele/Kommandos lokal verifiziert.
- [ ] Begriffe mit `docs/01_FUNCTIONS.md` abgeglichen.
21 changes: 12 additions & 9 deletions src/FileTypeDetection/Detection/FileTypeRegistry.vb
Original file line number Diff line number Diff line change
Expand Up @@ -138,15 +138,18 @@ Namespace FileTypeDetection
Friend Shared Function DetectByMagic(header As Byte()) As FileKind
If header Is Nothing OrElse header.Length = 0 Then Return FileKind.Unknown

Dim match = MagicRules.
SelectMany(Function(rule) rule.Patterns.
Select(Function(pattern) New With {.Rule = rule, .Pattern = pattern})).
FirstOrDefault(Function(item)
Dim segments = item.Pattern.Segments
Return segments.All(Function(segment) HasSegment(header, segment))
End Function)

Return If(match Is Nothing, FileKind.Unknown, match.Rule.Kind)
For i = 0 To MagicRules.Length - 1
Dim rule = MagicRules(i)
Dim patterns = rule.Patterns
For j = 0 To patterns.Length - 1
Dim segments = patterns(j).Segments
If segments.All(Function(segment) HasSegment(header, segment)) Then
Return rule.Kind
End If
Next
Next

Return FileKind.Unknown
End Function

Friend Shared Function HasDirectHeaderDetection(kind As FileKind) As Boolean
Expand Down
2 changes: 1 addition & 1 deletion src/FileTypeDetection/Detection/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,6 @@ flowchart TD
## Dokumentpflege-Checkliste

- [ ] Inhalt auf aktuellen Code-Stand geprüft.
- [ ] Links und Anker mit `python3 tools/check-markdown-links.py` geprüft.
- [ ] Links und Anker mit `python3 tools/check-docs.py` geprüft.
- [ ] Beispiele/Kommandos lokal verifiziert.
- [ ] Begriffe mit `docs/01_FUNCTIONS.md` abgeglichen.
Loading
Loading