feat: Participant Audit Log Table

[Warren-Pitterson]

Description

Audit Log Table & Blob Storage

• Added ParticipantAuditLog EF model and a new EF Core migration (AddParticipantAuditLog) to create the audit log table in the database

• Added AuditSource enum to categorise the origin of audit events (Parquet file, manual add, dummy GP removal)

• Added ParticipantAuditMessage model as the queue message contract for audit events

• Added IAuditQueueSender interface and AuditQueueSender implementation to enqueue audit messages to participant-audit-queue; optionally writes request snapshots to blob storage (audit-request-snapshots container) and stores the blob URI as RawDataRef

• Added new AuditWriter Azure Function (AuditWriterFunction) triggered by participant-audit-queue; persists audit records to the database

• Added unit tests

Context

https://nhsd-jira.digital.nhs.uk/browse/DTOSS-12573

Type of changes

• Refactoring (non-breaking change)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would change existing functionality)
• Bug fix (non-breaking change which fixes an issue)

Checklist

• I am familiar with the contributing guidelines
• I have followed the code style of the project
• I have added tests to cover my changes
• I have updated the documentation accordingly
• This PR is a result of pair or mob programming

---

Sensitive Information Declaration

To ensure the utmost confidentiality and protect your and others privacy, we kindly ask you to NOT including PII (Personal Identifiable Information) / PID (Personal Identifiable Data) or any other sensitive data in this PR (Pull Request) and the codebase changes. We will remove any PR that do contain any sensitive information. We really appreciate your cooperation in this matter.

• I confirm that neither PII/PID nor sensitive data are included in this PR and the codebase changes.

[github-actions]

Your Pull Request title must meet the conventional commit standards, please see the following documentation - https://www.conventionalcommits.org/en/v1.0.0/#specification

[github-actions]

Unit Test Results

✔️ Tests 1027 / 1027 - passed in 66.7s
📝 Coverage 47.05%
📏 4911 / 10757 lines covered 🌿 1190 / 2210 branches covered
🔍 click here for more details

✏️ updated for commit 1ef4bfd

[Copilot]

Pull request overview

Introduces participant audit logging across ingestion/ServiceNow flows by emitting audit events to an Azure Storage Queue, persisting them to a new PARTICIPANT_AUDIT_LOG table, and optionally snapshotting requests to blob storage via a new Audit Writer Function.

Changes:

• Add ParticipantAuditLog EF model + EF Core migration (incl. indexes) for PARTICIPANT_AUDIT_LOG.
• Add audit message contract (ParticipantAuditMessage), source enum (AuditSource), and queue sender (IAuditQueueSender/AuditQueueSender) wired into existing functions.
• Add AuditWriter Azure Function app (queue-triggered) to persist audits and write snapshots to blob storage, plus unit tests.

Reviewed changes

Copilot reviewed 26 out of 27 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
tests/UnitTests/ServiceNowMessageHandlerTests/ReceiveServiceNowMessageFunctionTests.cs	Update constructor wiring for new audit sender dependency
tests/UnitTests/ParticipantManagementServicesTests/ManageServiceNowParticipantTests/ManageServiceNowParticipantFunctionTests.cs	Update constructor wiring; minor formatting changes in setups/verifications
tests/UnitTests/CaasIntegrationTests/receiveCaasFileTest/ReceiveCaasFileTests.cs	Update constructor wiring for new audit sender dependency
tests/UnitTests/AuditServicesTests/AuditWriterTests/AuditWriterFunctionTests.cs	New tests for audit writer queue->DB/blob behavior
Directory.Packages.props	Add Functions Storage Queues extension package version
application/CohortManager/src/Functions/Shared/Model/ParticipantAuditMessage.cs	New queue message contract for audit events
application/CohortManager/src/Functions/Shared/Model/Enums/AuditSource.cs	New enum to categorize audit event origin
application/CohortManager/src/Functions/Shared/Model/EFModels/ParticipantAuditLog.cs	New EF model for persisted audit records
application/CohortManager/src/Functions/Shared/DataServices.Migrations/Migrations/DataServicesContextModelSnapshot.cs	EF snapshot updated for audit log entity/indexes
application/CohortManager/src/Functions/Shared/DataServices.Migrations/Migrations/20260331161048_AddParticipantAuditLog.Designer.cs	Generated migration designer for audit log table
application/CohortManager/src/Functions/Shared/DataServices.Migrations/Migrations/20260331161048_AddParticipantAuditLog.cs	Migration creating PARTICIPANT_AUDIT_LOG + indexes
application/CohortManager/src/Functions/Shared/DataServices.Database/DataServicesContext.cs	Register audit log entity and indexes in model configuration
application/CohortManager/src/Functions/Shared/Common/Interfaces/IAuditQueueSender.cs	New abstraction for enqueuing audit messages
application/CohortManager/src/Functions/Shared/Common/Extensions/AzureQueueExtension.cs	DI registration extension for audit queue sender
application/CohortManager/src/Functions/Shared/Common/AuditQueueSender.cs	New sender implementation targeting participant-audit-queue
application/CohortManager/src/Functions/ServiceNowIntegration/ServiceNowMessageHandler/ReceiveServiceNowMessageFunction.cs	Emit audit message when receiving/forwarding ServiceNow request
application/CohortManager/src/Functions/ServiceNowIntegration/ServiceNowMessageHandler/Program.cs	Register audit queue sender in host
application/CohortManager/src/Functions/ParticipantManagementServices/ManageServiceNowParticipant/Program.cs	Register audit queue sender in host
application/CohortManager/src/Functions/ParticipantManagementServices/ManageServiceNowParticipant/ManageServiceNowParticipantFunction.cs	Emit audit message after PDS validation and distribution publish
application/CohortManager/src/Functions/Functions.sln	Add AuditWriter project and solution folder
application/CohortManager/src/Functions/CaasIntegration/receiveCaasFile/receiveCaasFile.cs	Emit audit messages per participant from parquet ingest (with batchId)
application/CohortManager/src/Functions/CaasIntegration/receiveCaasFile/Program.cs	Register audit queue sender in host
application/CohortManager/src/Functions/AuditServices/AuditWriter/Program.cs	New function app host wiring for DB + BlobServiceClient
application/CohortManager/src/Functions/AuditServices/AuditWriter/Dockerfile	New container build/publish for AuditWriter function
application/CohortManager/src/Functions/AuditServices/AuditWriter/AuditWriterFunction.cs	New queue-triggered writer to DB + optional blob snapshot
application/CohortManager/src/Functions/AuditServices/AuditWriter/AuditWriter.csproj	New function app project definition
application/CohortManager/compose.core.yaml	Add audit-writer service to local compose stack

Files not reviewed (1)

• application/CohortManager/src/Functions/Shared/DataServices.Migrations/Migrations/20260331161048_AddParticipantAuditLog.Designer.cs: Language not supported

Comments suppressed due to low confidence (1)

application/CohortManager/src/Functions/CaasIntegration/receiveCaasFile/receiveCaasFile.cs:99

• This method is async, but it blocks with Task.WaitAll, which can cause threadpool starvation and prevents proper async exception/cancellation flow. Replace with await Task.WhenAll(allTasks) (and consider propagating/handling exceptions) so the function remains fully asynchronous.

                    await EnqueueAuditMessagesAsync(listOfAllValues, name, batchId, (int)screeningService.ScreeningId);
                    //split list of all into N amount of chunks to be processed as batches.
                    var chunks = listOfAllValues.Chunk(BatchSize).ToList();

                    foreach (var chunk in chunks)
                    {
                        var batch = chunk.ToList();
                        allTasks.Add(
                            _processCaasFile.ProcessRecords(batch, options, screeningService, name)
                        );
                    }

                    // process each of the batches
                    Task.WaitAll(allTasks.ToArray());

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 27 out of 28 changed files in this pull request and generated 8 comments.

Files not reviewed (1)

• application/CohortManager/src/Functions/Shared/DataServices.Migrations/Migrations/20260331161048_AddParticipantAuditLog.Designer.cs: Language not supported

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Warren-Pitterson]

Security Hotspot is a a false negative as all Dockerfiles are set up this way

[sonarqubecloud]

Quality Gate failed

Failed conditions
1 Security Hotspot

See analysis details on SonarQube Cloud