Latest follow-up: Configuration and data qualification raises that category to 9.5/10 and the overall engineering assessment to 9.3/10. This reliability report remains the historical baseline.
TRUSTWEAVE SDK · 10 SEPTEMBER 2026
Reliability improved, with measured limits
Durable occurrence accounting, bounded contention, process-crash recovery and archived-WAL qualification.
9.5/10 for SDK engineering; deployment qualification remains open
The SDK changes and component evidence support 9.5/10, meeting the requested threshold at the same SDK engineering scope as the preceding review. The half-point increase reflects removal of an admission bottleneck, verified transactional counter and WAL behavior, and repeatable load/crash/PITR gates. It does not qualify an actual production deployment.
A further increase would overstate the evidence while the deployed traffic targets, replica behavior, external payment journal and RPO/RTO remain unspecified and unqualified. The separate Deployment and release score stays at 8.5. Those remaining requirements are listed below.
The score is an engineering assessment, not production certification. Candidate bc86c662bddc6c53f70ff9c853603bfeb9ea6385 on codex/reliability-qualification-20260910 passed hosted release-evidence validation. Changes are also present in the original working copy; its other pre-existing edits are outside this candidate assessment. No main-branch merge, publication or deployment was performed.
Implementation changes
- Remove the repeated history scan. Admission reads occurrence consumption from the locked account row. An AFTER INSERT trigger increments the counter transactionally. Duplicate inserts, rejected updates and rollback do not consume occurrences; releases keep replay and occurrence consumption.
- Preserve migration integrity. An atomic backfill counts every retained state. Account-before-reservation locking matches writers, repeated migration never reduces consumption, and the trigger checks that exactly one account was updated, including under row-level security.
- Require WAL acknowledgement. Migration, reserve and reconciliation use transaction-local synchronous commit while preserving remote_apply. This cannot repair disabled fsync, unsafe storage or missing replicas.
- Make reliability evidence a gate. Seven named database tests and six checker tests protect execution and evidence. CI rejects missing scenarios, low throughput, excess latency, inconsistent state or incomplete recovery evidence and retains raw JSON and logs.
Ledger implementation · Migration, qualification and deployment runbook · Acceptance gate
Measured load and recovery
| Measurement | Local Windows | Hosted Linux |
|---|---|---|
| Initial reserve p99 (ms) | 210 | 205 |
| Completed sustained workflows | 7,793 | 15,016 |
| Sustained duration (ms) | 60,148 | 60,212 |
| Sustained workflow p99 (ms) | 485 | 691 |
| Peak pooled connections | 8 | 8 |
| 100,000-row migration (ms) | 125 | 88 |
| Crash restart + state checks (ms) | 1,951 | 764 |
| Restore + state/replay checks (ms) | 14,663 | 9,416 |
Both environments used 32 callers, four ledger objects and an eight-connection pool. The finite contention scenario accepted exactly 2,048 of 4,096 initial requests and 512 later requests after settlement/release. The sustained scenario directs 80% of work to one mandate. Both scenarios finish with zero balance/count violations and no borrowed connections.
Acceptance: reserve p99 ≤ 2 seconds; a ≥ 60-second skewed workload completing ≥ 1,000 workflows with workflow p99 ≤ 5 seconds and ≤ 8 connections. These are explicit component regression thresholds, not an agreed production SLO. The one-minute run is not a long-duration soak; the four objects do not represent four independently failing hosts.
Recovery uses a verified physical base backup and archived WAL. Three post-backup records, including settlement and release evidence, survive recovery to the named target. A later record is deliberately excluded. Full row digests, replay rejection, idempotent reconciliation and new admission are checked. A corrupted backup is rejected by checksum; missing required WAL prevents writable promotion. Process-crash recovery preserves the latest acknowledged snapshot.
Recovery timers include fixture startup and assertions, not incident detection, fencing, external-journal reconciliation or production cutover. The archive is local; this does not establish remote archive durability or production RPO/RTO.
Validation and score
The exact candidate passed the full SDK build, lint/API compatibility, coverage floors, dependency inventories, 24 documentation entry points, independent VI interoperability, required-test/discovery checks, host alerts, notification delivery and build attestation. Hosted JUnit: 3,901 passed, 15 documented optional skips, zero failures/errors. Merged coverage: 57.31% line and 39.84% branch; the existing floors were retained.
| Category | Before | After |
|---|---|---|
| Security and access control | 9.0 | 9.0 |
| Observability and diagnosability | 9.6 | 9.6 |
| Reliability and scale | 9.0 | 9.5 |
| Configuration and data | 9.0 | 9.0 |
| Deployment and release | 8.5 | 8.5 |
| Testing and documentation | 9.5 | 9.5 |
Only Reliability and scale is reassessed. Category total 55.1 / 6 = 9.1833, rounded to 9.2. Scoring rationale · Measurements and evidence hashes · Previous qualification.
Work to go beyond 9.5 and qualify production
- Identify the deployment and approve targets. Record topology, database/storage/pool limits, tenant/mandate distribution, peak and sustained request rates, p95/p99, availability, RPO and RTO. Run the real authorization path including signature verification and downstream dependencies.
- Qualify sustained capacity and overload. Run the agreed peak and headroom workloads plus a long-duration soak against production-like storage and data volume. Require zero invariant violations, stable memory/pool/queue bounds, measured recovery after overload and all agreed latency/error targets.
- Qualify failure and fencing. Exercise host loss, connection loss during commit, replica promotion and network partitions. Demonstrate a single writer, no double execution, preserved replay state and recovery within agreed bounds.
- Qualify complete recovery. Restore from encrypted remote backups and WAL, measure archive lag, authenticate the real payment-journal watermark and reconcile uncertain outcomes while execution/admission stay fenced. Verify retained history and evidence before reopening traffic.
- Retain an exact-version deployment result. Bind the run to the SDK/host artifacts, configuration, database version and resource sizes. Capture monitoring, incident timing and operator recovery evidence. Reassess from those results instead of assigning the requested number in advance.
These require a designated non-production target and agreed objectives. No deployment credentials, test target or RPO/RTO have been supplied for this review.
Evidence archive
Raw measurements, execution records and integrity hashes
- artifact-attestation.json
- contention.json
- documentation.json
- history.json
- hosted-run.json
- junit-contract.json
- local-contention.json
- local-history.json
- local-lock-timeout.json
- local-qualification.json
- local-sustained.json
- local-validation.json
- local-wal-recovery.json
- lock-timeout.json
- qualification.json
- required-tests.json
- source-files.json
- sustained.json
- TEST-org.trustweave.credential.vi.IntentReliabilityTest.xml
- TEST-org.trustweave.credential.vi.IntentWalRecoveryTest.xml
- validation-manifest.json
- wal-missing.log
- wal-recovery.json
- wal-recovery.log
PostgreSQL references: continuous archiving and PITR, WAL durability settings, client transaction settings.