Files
Claude Opus 5 b5ab885a67
CI (SIAX Cloud) / security (push) Successful in 27s
Security Scan (SIAX Cloud) / security (push) Successful in 30s
CI (SIAX Cloud) / quality (push) Successful in 39s
CI (SIAX Cloud) / security (pull_request) Successful in 43s
CI (SIAX Cloud) / integration (push) Successful in 51s
CI (SIAX Cloud) / quality (pull_request) Successful in 54s
CI (SIAX Cloud) / integration (pull_request) Successful in 1m15s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 13s
feat(analytics): BI-P129 — ConnectorProvider, and four gates on where data lands
BI-P129 names four provider kinds (native, dlt, Meltano/Singer, Airbyte-if-
licensed). This commit ships the seam and the `native` provider; dlt and
Singer follow on this branch.

What is deliberately NOT here: a second ingestion stack. 0pt already has the
P061 connector contract, the P062 runner with durable checkpoints and
crash/resume tests, the schema registry, the quality gate and the idempotency
ledger. A provider's job is to hand back a P061 `Connector`; the highway is
the one that exists.

What is new is the question the P061 stack has no opinion about: an ingestion
run has to LAND somewhere, and where it lands is an analytics contract
question. Four fail-closed gates, each attacked with the guard (must refuse)
AND without it (must succeed — otherwise the guard is decoration):

  1. tier — source data lands in RAW, or NORMALIZED with a declared
     normalization. Never directly in CURATED/SERVING. BI-P003's "no business
     dashboard reads arbitrary raw ingestion tables" is defeated in one line
     by relabelling the landing table SERVING; the tier is therefore checked
     at WRITE, not only at read.
  2. contract coverage — every field discovery actually reported must be
     declared in the BI-P004 contract. An undeclared column is an
     unclassified column. The counterfactual test lands `personnummer` into a
     RAW table whose contract says pii_classification "none", and the runner
     reports success: that is the harm, shown rather than asserted.
  3. physical home — RAW/NORMALIZED land as Parquet/Iceberg in ST0RE
     (st0re#45 shapes), never straight into ClickHouse.
  4. provider readiness — a provider whose own providerReadiness() says
     verified:false cannot run in a production runtime without a recorded,
     named exception. This is the structural form of the honesty rule 0pt#35
     set with MetricFlow.

Airbyte is a declared kind with NO implementation: its server is ELv2, which
restricts offering the software as a managed service, and that is exactly
what 0pt does. Registering or planning one is refused at the registry, in one
place, so a future licence review has a single constant to change.

BI-P129 DoD — "two real source types ingest into canonical analytics tiers" —
is proven with the two real, heterogeneous P061 connectors: a JSONL file on
actual disk (cursor = line offset, real CDC deletes) and an HTTP JSON API
driven through a real fetch/Response cycle (cursor = opaque server token).
Both run through the real runner into the declared RAW tier, paged, with
9 rows landed each and distinct envelope ids.

Verification: 24 tests. Mutation-checked — neutering gates 1 and 2 fails 5 of
them, so they are load-bearing rather than incidental.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 06:22:07 +02:00
..