ADR-00021: Documents as a decoupled service behind an ObjectStorageProvider seam, Postgres-first

Approval Status

Status flow: Proposed → Accepted → Reviewed → Approved (or Rejected), or Not Required. A superseded ADR keeps its file; only its status changes to Superseded by NNNNN.

Governance body	Status	Last update
CTN Project Team	Approved	2026-06-25
CTN Technical Advisory Board	Approved	2026-07-08
CTN Steering Committee	Not Required	2026-07-08
BDI Conformance Team	Pending	—
BDI Framework Team	Pending	—

Context and Problem Statement

The platform needs to store MIME-typed documents — mostly PDFs, also scanned images — with upload metadata, addressable by a stable UUID so other entities can reference them (see REQ-20260625: document store). The first consumer is onboarding: a requestor uploads proof supporting an Onboarding Request (e.g. a registry extract for a business-register identifier, a signed acceptance of the consortium contract conditions). But the same store must serve later needs that have nothing to do with onboarding — periodic re-validation uploads on a Participant, and documents that validate no entity at all (pure storage).

Two architectural questions must be answered together:

1. Where do the bytes physically live? The preferred stack (ADR-00002) names only PostgreSQL — there is no object store (MinIO/S3) provisioned, and deployment is Kubernetes-native. “Blob store” colloquially implies S3.
2. How does a Document relate to the things that reference it? The headline ask was “1:N from Onboarding Request,” but the real requirement is broader: many future entity types will reference documents, and the onboarding reference must survive the Onboarding Request → Participant transition at approval. The house precedent (Verification) carries two nullable typed FK columns (onboarding_request_id, participant_id).

These are hard to reverse (a storage seam and a reference model that other code depends on), surprising (no S3; no owner column despite a stated 1:N), and the result of real trade-offs — hence this ADR.

Considered Options

Option 1: Bytes directly in Postgres, no seam

• Description: A bytea column (or split payload table) on the document entity; commit to Postgres as the byte store.
• Pros: Zero new infra; transactional with metadata; one backup/restore story; fits ADR-00002 exactly.
• Cons: Hard-codes Postgres as the byte store; migrating to an object store later touches the entity and every reader; Postgres soft-degrades past tens of MB/row.

Option 2: External object store now (MinIO/S3), metadata in Postgres

• Description: Provision an S3-compatible store immediately; Postgres holds only metadata + an object key.
• Pros: Scales to large files; keeps the DB lean; matches the colloquial “blob store” expectation.
• Cons: New infrastructure not in the preferred stack (amends ADR-00002 with real ops cost); a second consistency boundary (orphaned objects / dangling rows); new credentials and failure modes — all for files realistically in the hundreds-of-KB-to-a-few-MB range.

Option 3: ObjectStorageProvider SPI, Postgres-first implementation (chosen)

• Description: Core defines a streaming byte-storage SPI keyed by the Document’s UUID; the first implementation stores bytes in a document_blob Postgres table; a later implementation can target MinIO/S3 without touching the Document model or its references. Mirrors the existing onboarding / claim-verification provider seams.
• Pros: No new infra now, yet no lock-in; swapping backends is a new impl, not a data-model migration; consistent with the platform’s SPI-seam idiom.
• Cons: One layer of indirection before any object store exists; the Postgres impl materializes bytes in heap (acceptable under the size cap; true streaming arrives with an object-store impl).

Reference-model sub-options (orthogonal to the substrate)

• R1 — Two nullable typed FKs on the Document (onboarding_request_id, participant_id), matching Verification. Rejected: ties the general store to onboarding/participant concepts and needs a new column per future referrer type.
• R2 — Polymorphic owner_type + owner_id on the Document. Rejected: still models “owned by one,” breaks FK integrity, and diverges from house style.
• R3 — Document carries no owner; the referrer holds the document_id (chosen). The Document knows nothing about who references it; supersession/migration is the referrer swapping which id it points at. A document↔identifier relationship, when a consumer needs one, is the consumer’s, keyed by the canonical IdentifierUri string (stable across the Request→Participant transition), not by a FK on the Document.

Decision Outcome

Chosen option: Option 3 (storage seam, Postgres-first) + R3 (decoupled, referrer-holds-the-link).

A general, standalone DocumentService over two tables:

document                          -- metadata, core-owned, always Postgres
  document_id    UUID PK
  document_type  text  (enum: RegistryExtract | ContractAcceptance | Other)
  content_type   text  (MIME)
  file_name      text
  size_bytes     bigint
  expiry_date    date  NULL       -- per-type policy: forbidden/optional/required
  + MetaEntity (created_at/by, updated_at/by, is_deleted)

document_blob                     -- the Postgres ObjectStorageProvider impl only
  document_id  UUID PK/FK
  bytes        bytea              -- a future S3 impl drops this table entirely

Bytes are owned by the ObjectStorageProvider SPI (streaming InputStream in/out, keyed by document_id, no presigned/direct-URL operation — access is always API-mediated). The Postgres implementation backs the document_blob table.

Rationale

• Business alignment: one trustworthy, auditable home for proof documents, shared across onboarding, re-validation and future consumers, with provenance and a business-validity (expiry) marker.
• Technical considerations: realistic file sizes sit comfortably in Postgres, so Option 2’s infrastructure is unjustified today; the seam removes the lock-in that makes Option 1 risky. R3 keeps the store general — the explicit requirement — and makes the onboarding “migration” a non-event: the link names the URI / lives on the referrer, never the Document.
• Risk assessment: the chief risk is shipping a service with no HTTP exercise (the upload endpoint rides with the first consumer); mitigated by tests driving DocumentService + the Postgres impl directly (store → read → byte round-trip, expiry-policy and allow-list/size enforcement).
• Cost/benefit: a small indirection cost buys backend portability and keeps the Postgres-only stack intact until an object store is genuinely warranted.

Key properties

• Document Type is a closed enum carrying a per-constant expiry policy (FORBIDDEN / OPTIONAL / REQUIRED). Seed: RegistryExtract (OPTIONAL), ContractAcceptance (FORBIDDEN), Other (FORBIDDEN). Grown per consumer.
• Expiry = business-validity, queryable, never a storage trigger — no auto-delete. Explicit deletion is a deliberately deferred, separate concern (is_deleted reserved for it).
• Write-once: bytes and intrinsic metadata are set at creation; corrections / re-validations are new Documents. The only mutations are an admin correction of expiry_date and document_type and the soft-delete flag.
• Ingest guards: configurable MIME allow-list (default application/pdf, image/png, image/jpeg) and a per-MIME-type size cap, enforced in DocumentService. Magic-byte content sniffing is a deferred hardening follow-up.
• Access: authenticated upload; read restricted to the uploader (created_by) and association-admin; everything API-mediated, with an extension point for a future organization-scoped OrgAdminRole.

Consequences

• Positive: backend-portable byte storage; a general store the whole platform can reuse; provenance, expiry and write-once integrity out of the box; no new infra.
• Negative: orphaned Documents are possible once a standalone upload path exists (mitigated for now: upload is one-step through a consumer endpoint, so the reference is created in the same transaction — no orphan); the Postgres impl does not truly stream; REQUIRED expiry has no seed type exercising it yet.
• Neutral: extends ADR-00002 with a seam rather than amending the stack; a future object-store impl and an isolated-origin download path are anticipated, not built.

Implementation Plan

1. Phase 1 (this slice): document + document_blob Liquibase changelog; Document entity + DocumentType enum with expiry policy; ObjectStorageProvider SPI + Postgres impl; DocumentService (store / read / metadata / admin correction); allow-list + per-MIME size config. Tests drive service + impl directly. No HTTP endpoint.
2. Phase 2 (with first consumer): a one-step consumer-owned upload endpoint (e.g. onboarding) that delegates to DocumentService and creates the referencing entity in the same transaction; read/download endpoint (API-mediated).
3. Phase 3 (later): object-store ObjectStorageProvider impl (MinIO/S3, true streaming); explicit deletion + orphan GC; magic-byte sniffing; isolated-origin download offload; OrgAdminRole read access.

Compliance and Validation

• Security review completed (PII at rest, access control, content sniffing)
• Performance impact assessed (metadata reads do not load bytes)
• Integration impact evaluated (first consumer wiring)
• Documentation updated (CONTEXT.md glossary entries)
• Stakeholder approval obtained

Related Documents

• REQ-20260625: document store — driving requirement
• ADR-00002 — preferred stack (extended here)
• ADR-00011 — IdentifierUri (the stable key for a future document↔identifier link)
• ADR-00014 — the SPI-seam idiom this mirrors
• TDR-0001 — Liquibase owns schema
• docs/ai/CONTEXT.md — Documents glossary section

Status History

Date	Status	Notes
2026-06-25	Proposed	Initial proposal from a grill-with-docs design session

Status flow: Proposed → Accepted → Reviewed → Approved (or Rejected).