TRUSTWEAVE SDK · 10 SEPTEMBER 2026
Testing and documentation qualified
Full SDK validation, independent conformance, executable examples and retained evidence for an exact committed candidate.
9.0 → 9.5testing and documentation / 10
9.1 / 10overall engineering score
3,894hosted JUnit tests passed
What the score covers
9.5/10 is the engineering assessment of testing and documentation for the credential-free SDK candidate. It is not a code-coverage percentage. External services and HSMs retain explicit qualification gaps; the draft VI profile is a supported subset. No release publication or production deployment is claimed.
Candidate 32a73dfa6d8d6903555e5902d8a50f39485a6b2c on codex/testing-qualification-20260910. Successful hosted release-evidence run · Evidence summary and hashes · Scoring rationale
Implemented and verified
| Workstream | Delivered evidence |
|---|---|
| Full regression and coverage | Full local SDK build, lint/ABI checks and fresh merged XML/HTML coverage; Docker-backed components run, including cloud-wallet, PostgreSQL and local Indy integration. Existing coverage floors remain enforced. |
| Independent conformance | 26 Python-issued cases, three Kotlin-issued reverse profiles and 65 reference evaluations; two explicit stricter merchant-policy differences. A separate 214-transition Python model agrees with real PostgreSQL reservation, replay, occurrence and settlement behavior. |
| Executable documentation | Nine source-copy contracts across seven distinct files, plus 24 executed entry points, including credential verification, merchant authentication, wallet configuration and an idempotent legacy-ledger migration. Documentation checks find no maintained-link, fence or source-copy errors. |
| Candidate identity | Hosted full build, merged coverage, reference suite, alert/notification exercises and artifact attestation pass for the displayed commit. The manifest rejects dirty or mismatched source, malformed test counters and unapproved skips. The downloaded VI JAR attestation was independently verified against the repository, signer workflow, branch and exact commit; verification record. |
Verified examples · Conformance scope and reproduction · Acceptance gates
Validation results
| Evidence | Result |
|---|---|
| Local SDK JUnit | 3,910 total; 15 explicit skips; zero failures/errors. The local tree is dirty and includes one existing scratch test excluded from the candidate. Thirty-four obsolete BBS cases were quarantined before this count; this is not labelled a release candidate. |
| Hosted SDK JUnit | 3,909 total; 15 permitted skips; zero failures/errors. Commit-bound manifest |
| Validation tools and reference | 55 validation-tool tests, 24 documentation entry points and 326 pinned Python-reference tests pass; reference tests qualify that peer, not the SDK. |
| Local discovery and required regressions | 3,831 compiled direct JUnit methods checked; zero invalid signatures. 47 required named tests verified, including every dynamic autonomous vector and the real overload test. |
| Conformance execution | 65 reference evaluations pass the declared expectations. The 214 ledger transitions are assertions within one model test, not 214 additional JUnit methods. |
Local JUnit XML, merged coverage and build log · Named regressions · Compiled discovery · Documentation inventory · Conformance results
15 explicit optional-test skips and their limits
Each row is permitted by an exact reviewed policy; a new skipped test fails qualification. None of these rows counts as executed provider coverage.
| Test | Reason |
|---|---|
| org.trustweave.anchor.cardano.CardanoBlockfrostIntegrationTest anchor and read back round-trip on Preview() | Requires an authorized Blockfrost Preview project and funded test wallet. |
| org.trustweave.anchor.polygon.PolygonBlockchainAnchorClientTest should throw NotFoundException for non-existent transaction() | Live Amoy RPC requires TRUSTWEAVE_TEST_USE_REAL_SERVICES; local transport tests are separate evidence. |
| org.trustweave.kms.cloudhsm.CloudHsmKeyManagementServiceIntegrationTest requireClusterActive does not throw() | Requires an authorized AWS CloudHSM cluster, HSM identity and native library; no hardware custody qualification claimed. |
| org.trustweave.kms.cloudhsm.CloudHsmKeyManagementServiceIntegrationTest cluster is ACTIVE() | Requires an authorized AWS CloudHSM cluster, HSM identity and native library; no hardware custody qualification claimed. |
| org.trustweave.kms.cloudhsm.CloudHsmKeyManagementServiceIntegrationTest generateKey sign and deleteKey round-trip on the HSM() | Requires an authorized AWS CloudHSM cluster, HSM identity and native library; no hardware custody qualification claimed. |
| org.trustweave.googlekms.GoogleKmsProviderTest test create with valid options() | Requires Google Cloud credentials; constructor checks are not live KMS qualification. |
| org.trustweave.kms.pkcs11.Pkcs11KeyManagementServiceIntegrationTest generateKey(P256) returns a handle and getPublicKey round-trips it() | Requires a configured PKCS11 or SoftHSM2 token and native library; no native token qualification claimed. |
| org.trustweave.kms.pkcs11.Pkcs11KeyManagementServiceIntegrationTest sign(P256) produces a signature that verifies with JCA SHA256withECDSA() | Requires a configured PKCS11 or SoftHSM2 token and native library; no native token qualification claimed. |
| org.trustweave.kms.pkcs11.Pkcs11KeyManagementServiceIntegrationTest generateKey(Secp256k1) is rejected with UnsupportedAlgorithm (MVP contract)() | Requires a configured PKCS11 or SoftHSM2 token and native library; no native token qualification claimed. |
| org.trustweave.kms.pkcs11.Pkcs11KeyManagementServiceIntegrationTest generateKey(Ed25519) succeeds on SoftHSM2 (requires v2_6+)() | Requires a configured PKCS11 or SoftHSM2 token and native library; no native token qualification claimed. |
| org.trustweave.kms.pkcs11.Pkcs11KeyManagementServiceIntegrationTest deleteKey removes the key so subsequent getPublicKey returns KeyNotFound() | Requires a configured PKCS11 or SoftHSM2 token and native library; no native token qualification claimed. |
| org.trustweave.testkit.annotations.RequiresPluginExampleTest example test requiring multiple plugins() | Conditional external-plugin example; intentionally unavailable without example provider credentials. |
| org.trustweave.testkit.annotations.RequiresPluginExampleTest example test requiring Google Cloud KMS() | Conditional external-plugin example; intentionally unavailable without example provider credentials. |
| org.trustweave.testkit.annotations.RequiresPluginExampleTest example test requiring AWS KMS() | Conditional external-plugin example; intentionally unavailable without example provider credentials. |
| org.trustweave.integration.InMemoryTrustWeaveIntegrationTest test with external services template() | External-services template requires AWS KMS and Ethereum DID configuration; not executed integration coverage. |
Measured merged coverage
Values show measured coverage / enforced floor. No exclusions or floor reductions were introduced to pass this qualification.
| Scope | Line | Branch |
|---|---|---|
| * | 57.30% / 56% | 39.83% / 38% |
| org/trustweave/revocation/database | 74.15% / 59% | 53.10% / 44% |
| org/trustweave/credential/anchor | 77.87% / 75% | 37.07% / 33% |
| org/trustweave/credential/vi/verification | 91.57% / 83% | 68.68% / 58% |
| org/trustweave/anchor/indy | 82.87% / 82% | 61.13% / 60% |
Exact counters. Shared-host coverage has its own stricter CI policy.
Defects found while qualifying
- The new Python positive checkout fixture omitted the SDK-required product title. The generator and signed fixtures were corrected; merchant negatives now assert the intended error reason.
- Hosted load exposed a test deadline race: a blocked request expired while three separate probes ran. It now has its own deadline, bounded waits and a 90-second whole-test limit; overload, metrics authentication, trace correlation and redaction assertions remain enforced.
- Artifact comparison found 34 stale BBS results from a removed project. They were reversibly quarantined and excluded from active local evidence; hosted evidence comes from a clean candidate checkout. Audit record.
- Candidate evidence previously allowed new skips to go unexplained. The exact skip policy now fails closed and retains each permitted reason.
Remaining work for 10/10
- Qualify external custody, PKCS11 and live provider combinations in authorized environments, including failure and recovery behavior.
- Complete the requirement-to-vector matrix for every supported VI constraint and malformed encoding; add an independent merchant peer with the same authenticated trust profile and external/concurrent ledger traces.
- Review or convert the wider Kotlin snippet inventory beyond the nine protected complete examples. Validate deployment-specific configuration, migrations and external documentation links in their actual environments.
- Qualify published artifacts and deployed operating procedures, including production load, recovery objectives and on-call delivery. The hosted candidate run is not that evidence.
Existing multi-pair and mixed-recurrence combinations remain rejected. Sequential model agreement does not certify concurrent external settlement or uncertain-commit recovery.
Scorecard
| Category | Previous | Now |
|---|---|---|
| Security and access control | 9.0 | 9.0 |
| Observability and diagnosability | 9.6 | 9.6 |
| Reliability and scale | 9.0 | 9.0 |
| Configuration and data | 9.0 | 9.0 |
| Deployment and release | 8.5 | 8.5 |
| Testing and documentation | 9.0 | 9.5 |
54.6 / 6 = 9.1, rounded half-up to 9.1. Only testing/documentation is reassessed; other category scores carry forward. Previous assessment.