feat: use exception queues

[MWClayson-NHS]

Description

Migrating the HTTP Calls to the Create exception function to use ServiceBus Queues.
DTOSS-11330

Context

No HTTP Overhead these are now async calls so that there is less blocking on functions that will call createException.

Type of changes

• Refactoring (non-breaking change)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would change existing functionality)
• Bug fix (non-breaking change which fixes an issue)

Checklist

• I am familiar with the contributing guidelines
• I have followed the code style of the project
• I have added tests to cover my changes
• I have updated the documentation accordingly
• This PR is a result of pair or mob programming

---

Sensitive Information Declaration

To ensure the utmost confidentiality and protect your and others privacy, we kindly ask you to NOT including PII (Personal Identifiable Information) / PID (Personal Identifiable Data) or any other sensitive data in this PR (Pull Request) and the codebase changes. We will remove any PR that do contain any sensitive information. We really appreciate your cooperation in this matter.

• I confirm that neither PII/PID nor sensitive data are included in this PR and the codebase changes.

[github-actions]

Unit Test Results

✔️ Tests 1017 / 1017 - passed in 67.8s
📝 Coverage 47.12%
📏 4798 / 10483 lines covered 🌿 1175 / 2194 branches covered
🔍 click here for more details

✏️ updated for commit 07d204e

[sonarqubecloud]

Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
20.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[Copilot]

Pull request overview

This PR migrates “create exception” calls away from synchronous HTTP to asynchronous Service Bus messaging, updating application configuration, infrastructure env vars, and local compose setup to support queue/topic-based exception handling.

Changes:

• Replace ExceptionFunctionURL-based exception creation with Service Bus configuration (ExceptionUseServiceBus, CreateExceptionTopic, subscription/connection updates).
• Update the CreateException function to consume from a Service Bus subscription and complete/dead-letter messages explicitly.
• Align config models/tests and environment definitions (Terraform tfvars + docker-compose) with the new exception handling approach.

Reviewed changes

Copilot reviewed 23 out of 23 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tests/UnitTests/ParticipantManagementServicesTests/UpdateBlockedFlagTests/UpdateBlockedFlagTests.cs	Updates unit test config to remove ExceptionFunctionURL dependency.
tests/UnitTests/DurableDemographicTests/DurableDemographicTests.cs	Updates durable demographic tests/config setup to satisfy new required config fields.
infrastructure/tf-core/environments/sandbox.tfvars	Removes ExceptionFunctionURL app_url wiring; adds ExceptionUseServiceBus + CreateExceptionTopic and Service Bus connections across apps.
infrastructure/tf-core/environments/production.tfvars	Same as sandbox for production environment settings.
infrastructure/tf-core/environments/preprod.tfvars	Same as sandbox for preprod environment settings.
infrastructure/tf-core/environments/nft.tfvars	Same as sandbox for NFT environment settings.
infrastructure/tf-core/environments/integration.tfvars	Same as sandbox for integration environment settings.
infrastructure/tf-core/environments/development.tfvars	Same as sandbox for development environment settings (plus local/dev-specific formatting/entries).
application/CohortManager/src/Functions/screeningDataServices/GetValidationExceptions/Program.cs	Removes .AddExceptionHandler() from GetValidationExceptions startup.
application/CohortManager/src/Functions/Shared/Common/Extensions/ExceptionHandlerServiceExtension.cs	Switches the feature flag/env var name to ExceptionUseServiceBus and wires Service Bus sender config.
application/CohortManager/src/Functions/Shared/Common/ExceptionServiceBusConfig.cs	Renames/reshapes config for Service Bus exception sending (ServiceBusConnectionString_client_internal, ExceptionUseServiceBus).
application/CohortManager/src/Functions/ScreeningValidationService/RemoveValidationException/RemoveValidationExceptionConfig.cs	Makes config properties required and adjusts formatting.
application/CohortManager/src/Functions/ScreeningValidationService/LookupValidation/LookupValidationConfig.cs	Removes legacy exception URL config and adds a TODO note in the type declaration.
application/CohortManager/src/Functions/ParticipantManagementServices/UpdateBlockedFlag/UpdateBlockedFlagConfig.cs	Removes ExceptionFunctionURL from config model.
application/CohortManager/src/Functions/ExceptionHandling/CreateException/CreateException.cs	Updates Service Bus trigger binding (subscription/connection name) and adds extra logging around completion.
application/CohortManager/src/Functions/DemographicServices/DemographicDurableFunction/DemographicDurableFunctionConfig.cs	Makes config fields required.
application/CohortManager/src/Functions/CohortDistributionServices/TransformDataService/TransformDataServiceConfig.cs	Removes legacy exception URL config property.
application/CohortManager/src/Functions/CohortDistributionServices/RetrieveCohortRequestAudit/RetrieveCohortRequestAuditConfig.cs	Removes legacy exception URL config property.
application/CohortManager/src/Functions/CohortDistributionServices/RetrieveCohortRequestAudit/Program.cs	Adds an inline TODO comment to reconsider DB registration.
application/CohortManager/src/Functions/CohortDistributionServices/RetrieveCohortDistribution/RetrieveCohortDistributionConfig.cs	Removes legacy exception URL config property.
application/CohortManager/src/Functions/CaasIntegration/RetrieveMeshFile/Program.cs	Removes .AddExceptionHandler() from RetrieveMeshFile startup.
application/CohortManager/compose.core.yaml	Removes ExceptionFunctionURL from multiple services and adds Service Bus exception settings for local compose.
application/CohortManager/compose.cohort-distribution.yaml	Adds Service Bus exception settings to cohort distribution compose services.

Comments suppressed due to low confidence (1)

application/CohortManager/src/Functions/ExceptionHandling/CreateException/CreateException.cs:77

• The error log message in the Service Bus handler catch block says the exception could not be added to a Service Bus topic, but this function is consuming from Service Bus and writing to the database. This is misleading for operational debugging (and it is asserted in unit tests), so consider updating the log text to reflect the actual failure (e.g., processing/persisting the exception failed) and adjust the corresponding unit test expectations.

            _logger.LogInformation("added exception to database and completed message successfully ");
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "exception could not be added to service bus topic. See dead letter storage for more: {ExceptionMessage}", ex.Message);

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The create-exception function app is Service Bus-triggered (CreateException.cs uses Connection = "ServiceBusConnectionString_internal"), but this compose service only sets ServiceBusConnectionString_client_internal. Without ServiceBusConnectionString_internal the trigger binding will fail at runtime. Add ServiceBusConnectionString_internal=... (or change the trigger to use the client connection env var consistently).

Suggested change

	- ServiceBusConnectionString_internal=Endpoint=sb://service-bus;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=SAS_KEY_VALUE;UseDevelopmentEmulator=true;