as4-411/docs/architecture/testing-strategy.md

Testing Strategy

Testing approach for the as4-411 directory and resolver. Ensures determinism, protocol correctness, integration, and resilience.

Property-based tests for determinism

• Invariant: Same normalized request + same store state ⇒ same ordered list of directives (see resolution-algorithm and ADR-002).
• Use property-based testing (e.g. fast-check, Hypothesis) to generate many (request, store snapshot) pairs and assert that repeated resolution runs produce identical outputs. Vary identifiers, tenant, constraints, and store contents within valid ranges.
• Tie-break and scoring must be deterministic; tests should catch any dependence on iteration order or non-deterministic randomness.

Golden test vectors per rail

• Each rail (or protocol adapter) should have golden test vectors derived from the _rail-template “Sample payloads and test vectors” section.
• Tests: given a fixed request and a small, fixed store (or artifact set), the resolver output must match the golden directive list (primary + alternates, protocol, address, evidence). Update goldens only when the spec or algorithm intentionally changes; review changes.

Integration harness

• Stack: Postgres (migrations applied) + resolver service + sample gateway client (or mock that calls resolve and validates response shape).
• Scenarios: Create participants, identifiers, endpoints, and routing artifacts via Admin API or store; call resolve with various identifiers and constraints; assert directives and resolution_trace. Include multi-tenant isolation: data for tenant A must not appear in resolve for tenant B.
• Run in CI; use container or test DB so that migrations and seed data are reproducible.

Chaos tests for connectors

• Timeouts and retries: Simulate connector backends (SMP, file, GTT) that delay or fail. Assert timeout and retry behavior per ADR-005 and connectors.
• Circuit-breaker: After N failures, connector should open circuit and (per policy) fall back to cache-only or fail closed. Tests should verify circuit state and that no unbounded retries occur.
• Fallback to cache: When external source is unavailable, resolver should use cached data only within max stale window; tests assert no stale data beyond that and correct resolution_trace (e.g. “SMP cache” when SMP is down).