A trustworthy pipeline
that still ships nothing.
The evidence machinery became genuinely dependable this week. What holds the score at 8.8 / 10 is unchanged: nothing is published, coverage is static, and one shipped authorization server keeps its security state in unbounded process memory.
This machine had about 1.6 GB free; the Gradle daemon died with a native out-of-memory and a retry lost its test executors. A local build here is not a valid signal, so build, test and coverage evidence comes from CI run 34737199525 at this exact commit, which is green across all 28 steps.
What got better
- Evidence is a declared Gradle output in a dedicated
qualification/directory rather than sharingreports/with Kover — a better fix than the one made on 11 September, because it removes the output-overlap risk entirely. test_workflow_evidence_paths.pyasserts workflow evidence paths match what the build writes: the right response to a defect class that caused three separate CI failures.- Live AWS KMS custody qualification exists — read-only, least-privilege, verifying signatures independently, rejecting an altered challenge, proving replacement keys use distinct material and that a restarted client still resolves the historical public key. The design is careful.
- PKCS#11 custody recovery and the host overload boundary are now tested. Skips fell from 15 to 11, each still justified.
Findings
T1 · high — the AVP authorization server keeps security state in unbounded process memory
AuthorizationEngine enforces replay prevention, single-use and daily spend caps through three plain ConcurrentHashMaps — 33 lines, no eviction, no persistence. Two failures follow. Availability: the keys are caller-supplied, so ordinary traffic grows all three without bound and an attacker does it faster. Correctness: every guarantee is process-local and vanishes on restart, and a two-replica deployment enforces none of the three. This is the module that authorizes payments — and the repository already contains the right pattern, because PostgresIntentLedger does exactly this durably. The server's KDoc documents only the bind address.
T2 · medium — half the shipped servers still have no authentication hook
withAuthentication reached the DID registrar, VC API and status-list servers. The AVP authorization, OIDC4VCI and trust-registry servers did not. The AVP route carries a comment — “unauthenticated by design (it expects a proxy in front)” — which is honest, but a comment is not a control.
T3 · medium — the custody qualification records a control it cannot establish
The test asserts only denied is SignResult.Failure. That type covers KeyNotFound, UnsupportedAlgorithm and Error, so a mistyped ARN, a wrong region or a network fault all satisfy it — and it then writes "unauthorizedKeyRejected": true as a literal. check-custody-evidence.py requires that field, so the gate inherits the blind spot rather than catching it.
T4 · medium — the custody qualification has never run
workflow_dispatch only, zero runs. Live custody is unqualified in fact; no provider has a qualified custody profile.
T5 · medium — the BOM exports modules whose every method throws
Salesforce and ServiceNow are exported via api(project(...)); every public method throws. They fail closed, which is right, but they are advertised through the BOM, both return Any, and neither carries a maturity — while starknet, threebox, tezos and btcr are correctly marked stub.
T6 · medium — coverage is static and the core is the least-covered large module
58.05% → 58.23% line, 40.72% → 40.82% branch in two days. Floors sit at 57.0 / 40.0, just under the measurement, so the gate ratchets nothing. Observability is at 97.5% and verifiable-intent near 90%, while credential-api — about 4,100 lines, the module that issues and verifies credentials — is under half covered, did/registrar near 11%, didcomm near 32%.
T7 · low — the instrumentation promises more than it emits
Ten call sites use Telemetry. Ten of the fifteen declared Operation values never fire, so a host charting them sees a permanent zero series and cannot tell “not instrumented” from “never happened”. 33 of 792 main-source files reference a logger, unchanged.
T8–T10 · low and informational
Nothing published — v0.7.0 tagged, no release, publish job never run. Recent CI on main reads success / failure / success / cancelled / failure: green at HEAD, red on nearly half of recent pushes. Plus the local-build memory problem above, which is environmental.
Scores
| Category | 09-11 | Now | Why |
|---|---|---|---|
| Security and access control | 8.8 | 8.6 | T1 found this round (pre-existing, not a regression), plus T2, T3, T4 |
| Observability and diagnosability | 9.0 | 9.2 | Telemetry SPI, bridge and correlation landed; ten call sites, logging unchanged |
| Reliability and scale | 9.2 | 9.2 | PKCS#11 recovery tested, evidence sound — offset by T1 |
| Configuration and data | 8.8 | 8.9 | Catalog and image gates hold; T5 open |
| Deployment and release | 7.0 | 8.0 | CI green with twelve gates, evidence durable; still nothing published |
| Testing and documentation | 8.5 | 8.7 | 4,005 tests, skips 15→11, 135 script tests; coverage static |
Mean 52.6 ÷ 6 = 8.77, rounded to 8.8.
Why this differs from the other review dated today
docs/reviews/2026-09-13-code-review/ scores this same revision 9.4. That is not a contradiction — it uses a different six-category rubric, and almost the whole gap is two structural effects. Its supply chain and CI category (9.6) measures pipeline health, which is genuinely excellent; this rubric's deployment and release measures whether anything ships, which nothing does. And its rubric has no category where static coverage is the dominant term, where here it drives testing and documentation.
Both readings are defensible. This one continues the series tracked since 05 September, which is why it was used. Both rounds independently found the same three open items: the custody qualification has no completed run, several published surfaces are deliberate partial implementations, and green main evidence is not a promoted release.
Getting above 9.5
Eleven tasks in tasks.json; projected mean on completion 9.52.
Start day one, because they do not compress. A5 (Sonatype namespace verification) and A3 (an AWS account with three qualification keys and an IAM deny) are worth 1.5 and 0.9 category points between them, and both are blocked on something outside the repository.
- Week 1 — A2 make the custody denial check assert an
AccessDeniedcause and derive the evidence from the assertion; A4 extendwithAuthenticationto the remaining three servers; A7 mark Salesforce and ServiceNowstuband stop the BOM advertising them; A11 rehearse evidence-path changes on a branch. - Weeks 1–2 — A1, the one high finding: durable, shared, bounded AVP state with the intent ledger's fail-closed semantics. Two instances against one store must reject the second presentation of a nonce; a restart must not forget.
- Weeks 2–3 — A5 publish something; A9 finish the instrumentation the
Operationenum promises, or trim the enum. - Weeks 2–5 — A6 coverage, 58.2%/40.8% to roughly 75%/60%, top-down from
credential-api, raising each module's floor as it lands. The largest single item and the one that most limits the score. - Weeks 3–4 — A3 run the custody qualification; A8 classify the remaining 102 modules and qualify the GA core.
- Ongoing — A10 deployment-scale ledger qualification.
Evidence and machine-readable assessment
- Full review (Markdown)
- scores.json · findings.json · tasks.json
- ci-evidence.json — 4,005 tests, 0 failures, coverage counters from CI run 34737199525
Remediation pass — 13 September 2026
8.8 → 9.06. Six of the eleven tasks are done, including the one high finding. The five that remain are the five no code change could make: two are blocked outside the repository, two are measured in weeks, and one is soak time.
A1 (T1, high) — durable, shared and bounded authorization state
The three ConcurrentHashMaps are replaced by one interface with one method,
admit, because replay, single-use and daily spend cannot be checked apart from being
recorded without letting two concurrent presentations of the same authorization both pass.
The in-memory store bounds retention by the authorization's own expiry, taken from the authorization rather than from a store-side default; one whose lifetime nothing bounds is refused rather than half-remembered. At capacity, after a sweep, an admission is refused — evicting a live nonce to make room would turn memory pressure into a replay window. The unbounded per-credential mutex map became a fixed 64-stripe array.
The PostgreSQL store follows PostgresIntentLedger: one READ COMMITTED transaction
with synchronous commit required, unique-key conflicts as the replay and double-spend signals,
FOR UPDATE on the payer's row for the day, and a refusal that rolls the whole
transaction back so nothing is recorded. A failed transaction is a denial and is never retried as a
fresh authorization.
Proven against a real PostgreSQL through Testcontainers: two instances against one database reject the second presentation of a nonce and the second consumption of an authorization id; a fresh instance, which is what a restart is, still rejects both; four admissions alternating replicas share one daily cap rather than one each; eight concurrent presentations yield exactly one admission; an unreachable database is a denial, never an admission.
A4 (T2) — all six servers carry the hook
The AVP authorization server and the OID4VCI issuer refuse their mutating routes with 503 until
the host declares what protects them. The comment reading “unauthenticated by design” is
now HostAuthentication.frontedByProxy(…): the same claim, made where the code
can see it. The trust registry composes the shared gate with its existing apiToken
rather than stacking them, so a deployment authenticating with mTLS or a gateway no longer has to
invent a bearer token as well.
OID4VCI needed one addition. /token, /credential,
/deferred_credential and /notification are the protocol surface,
authenticated by the pre-authorized code and the access token the spec defines, and their callers
are wallets, which hold no host credential. HostAuthentication gained a
protocolAuthenticatedPaths declaration — made by the server about its own
routes, not by the host — so the gate covers /api/offer, which mints credential
offers, and leaves the protocol alone.
A2 (T3) — the evidence reports what the run observed
The denial check asserts an AWS AccessDeniedException and explicitly refuses
KeyNotFound, which is what a mistyped ARN or a wrong region produces. Every boolean is
derived from an assertion that ran, through a recorder that refuses to emit a file missing any
required check. check-custody-evidence.py now also requires
denialResultType: "Error" and denialErrorCode: "AccessDeniedException",
so the gate no longer inherits the blind spot it existed to catch.
A9 (T7) — the telemetry enum is a contract again
KMS_VERIFY is gone: KeyManagementService has no verify operation, so
the value could only ever have been a zero series. The other nine are now emitted —
TelemetryDidMethod and TelemetryWallet, plus presentation verification
and revocation checks in credential-api. Both decorators are wired where a host cannot
miss them: DidMethodRegistry.register instruments on the way in and the wallet DSL on
the way out, so a host-supplied plugin is covered without opting in.
scripts/check-telemetry-operations.py fails the build if a declared operation has no
main-source emitter, or an emitter has no declaration; a test-source emitter does not count,
because a host does not run the test suite.
A7 (T5) — the BOM stops advertising capability that throws
Salesforce and ServiceNow are marked stub, removed from
distribution:bom, and their KDoc says so. The unassessed ratchet fell from 102 to
100.
A11 (T9) — the evidence-path defect is gated by a property
scripts/check-workflow-evidence-paths.py asserts the rule rather than a list of
paths: every module-scoped build/… path a workflow reads must be backed by a
declared Gradle task output. Writing it surfaced something worse than the original finding.
test_workflow_evidence_paths.py — the test written to stop this exact defect
— had never run in CI: the discovery pattern was test_check_*.py and the file is
not named that way. All three workflows now discover test_*.py, and a test asserts
that they do.
Scores after remediation
| Category | Reviewed | After | Why |
|---|---|---|---|
| Security and access control | 8.6 | 9.25 | T1, T2 and T3 closed. Held under 9.5 by T4 — the custody qualification still has no completed run. |
| Observability and diagnosability | 9.2 | 9.5 | Every declared operation has an emitter, and a gate fails the build if that stops being true. |
| Reliability and scale | 9.2 | 9.4 | The AVP durability gap is closed. Held by the absence of published RPO, RTO and replica-promotion figures. |
| Configuration and data | 8.9 | 9.3 | The BOM no longer advertises modules that only throw. Held by the 100 modules still unassessed. |
| Deployment and release | 8.0 | 8.1 | The evidence-path class is gated by a property and CI runs every script test. Cannot go higher: nothing is published. |
| Testing and documentation | 8.7 | 8.8 | About 35 tests added, five against a real PostgreSQL. Merged coverage is unchanged, and that is what this measures. |
What is left
- A5 publish — Sonatype namespace verification. Outside the repository.
- A3 run the custody qualification — an AWS account with three keys and an IAM deny. Outside the repository. A2 landed first, so a run will now prove what it records.
- A6 coverage — two to three weeks against
credential-api. The tests added here cover what this pass changed, which is not what T6 measures. - A8 classify the remaining 100 modules — a week of assessment, then qualification evidence that does not compress.
- A10 the ledger at deployment scale — soak time.
Deployment cannot pass 8.1 while nothing ships and testing cannot pass 8.8 while
credential-api is under half covered. That is where the remaining 0.44 lives.
Verification
The box that could not build at review time now can — JDK 21, -Xmx900m,
--max-workers=1, no daemon. ./gradlew build is green in
18m54s: 2,188 tasks, including checkKotlinAbi and koverVerify
for every module and the per-source-set ktlint checks.
- 500 suites, 3,980 tests, 0 failures, 0 errors, 17 environment-gated skips (live custody and PKCS#11 do not run here, which is why the count is under CI's 4,005).
- 150 script tests pass, up from 135.
- Merged coverage: 58.14% line, 40.83% branch, 56.71% instruction.
Coverage did not move, and it moved slightly the wrong way. Against the
review's baseline of 58.23 / 40.82, line coverage is 0.09 points lower and branch 0.01
higher. The new code is well covered, but it is also about 300 new lines of main source, so the
merged line figure is marginally diluted rather than improved. T6 is untouched:
credential-api is exactly where it was, and the testing score rose for the three new
gates, not for coverage.