Operations, Testing, And Best Practices

This page is the operational and engineering checklist for running and evolving the Notification system.

Build And Run

From C:\Users\Ahmed Ismaeel\source\repos\InfrastructureServices:

dotnet build
dotnet run --project Notifications/Notification.Apis

The service exposes:

GET /health

Swagger is enabled when EnableSwagger is true.

Required Infrastructure

Dependency	Purpose
SQL Server	Notification persistence and Hangfire storage.
Redis	Channel config cache and WhatsApp provisioning state cache.
Message bus	Queue delivery through MassTransit.
Hangfire or background job provider	WhatsApp provisioning, template/status checks, background callbacks.
Seq/OpenTelemetry	Logs, metrics, and tracing when enabled.

Important Configuration

Use placeholders in source-controlled files and inject real values at deploy time.

{
  "ConnectionStrings": {
    "NotificationDbConnection": "<sql-server-connection-string>"
  },
  "RedisConfiguration": {
    "KeyPrefix": "notification-service:",
    "Hosts": [
      {
        "Host": "<redis-host>",
        "Port": 6379
      }
    ]
  },
  "Messaging": {
    "TransportType": "AzureServiceBus",
    "AzureServiceBus": {
      "ConnectionString": "<service-bus-connection-string>"
    }
  },
  "WhatsApp": {
    "Provisioning": {
      "InitialApprovalCheckIntervalMinutes": 30,
      "InitialApprovalCheckCount": 5,
      "ApprovalCheckIntervalMinutes": 60,
      "ApprovalTimeoutHours": 48,
      "LockTimeoutSeconds": 5
    }
  }
}

Migrations

AddNotificationInfrastructure(...) registers NotificationDbContext. The API startup calls EnsureCreated(), which checks pending migrations and applies them.

Operationally:

• Review migrations before deployment.
• Back up production databases before schema changes.
• Keep migrations small and reversible where possible.
• Do not mix unrelated schema changes in one migration.

Security

Minimum expectations:

• Do not commit provider credentials, service bus keys, database passwords, Firebase service accounts, or API keys.
• Store provider credentials encrypted or in a managed secret store.
• Keep InternalApiKey and ClientId consistent between callers and the Notification service.
• Validate gateway and endpoint authorization before exposing admin, credentials, provisioning, or seed endpoints.
• Mask sensitive headers and payload fields in logs.
• Avoid storing secrets in NotificationPayload.Data.

The current service code explicitly disables some shared auth/API middleware in startup options. Treat the service as requiring deployment-level protection unless endpoint auth is deliberately enabled and verified.

Observability

Log and trace these identifiers:

• CorrelationId
• NotificationId
• Subdomain
• UserId
• ChannelType
• ProviderType
• provider message id

Useful dashboards:

• sends by channel
• failures by channel and provider
• retry count
• provider latency
• invalid push token count
• WhatsApp provisioning status by subdomain
• queue consumer failures
• background job failures

Testing Strategy

Unit Tests

Add focused tests for:

• payload builders and validation
• NotificationOrchestrator channel selection
• channel behavior when config is missing
• NotificationExecutionPipeline status transitions
• NotificationMetadataExtractor
• retry deserializers
• config resolution for tenant vs source system

Contract Tests

Add serialization tests for:

• every concrete NotificationPayload
• every EmailTemplateData derived type
• SendNotificationEvent
• persisted payload deserialization for retry
• enum string and numeric compatibility where clients depend on it

Integration Tests

Use fake or local providers where possible:

• SMTP test server or MailKit stub
• fake SMS HTTP endpoint
• fake FCM provider abstraction
• fake T2 client for WhatsApp
• test SQL Server database
• test message bus or MassTransit harness

Verify:

• HTTP send persists request and delivery rows
• queue send is consumed and processed
• retry creates a new request
• Notification Center filters match metadata
• device token invalidation works
• WhatsApp provisioning state transitions are persisted

Provider Development Checklist

When adding a provider:

1. Add a provider enum value in contracts.
2. Add credential request/response DTOs.
3. Add a domain credential value object.
4. Add add/edit/delete/get handlers and validators.
5. Extend the provider factory.
6. Capture provider request and response payloads.
7. Map provider errors to useful failure messages.
8. Add integration tests with a fake provider.
9. Document configuration and failure modes.

Payload Best Practices

• Keep payloads small.
• Use stable business identifiers in Data, not large object graphs.
• Populate metadata fields consistently for Notification Center.
• Set Subdomain or SourceSystem deliberately.
• Avoid changing required constructor parameters on persisted payload types.
• Use typed template data instead of ad hoc replacement dictionaries when a template is reused.

Operational Runbook

When delivery is failing:

1. Check /health.
2. Check service logs using correlation id.
3. Verify database connectivity and pending migrations.
4. Verify Redis connectivity.
5. Verify message bus connectivity for queue delivery.
6. Verify channel config lookup for the effective subdomain or source system.
7. Verify provider credentials and provider-side health.
8. Inspect NotificationRequest and related NotificationDelivery rows.
9. For push, validate device token records.
10. For WhatsApp, check provisioning state and template approval status.