116304849bd863293f8033ae0750a0b6857c6708
23
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
92eb78ba0a |
fix(upload): an idempotent retry returns the same verified receipt
CI (SIAX Cloud) / sonar (pull_request) Skipped
masterplan-lock / check (push) Successful in 4s
masterplan-lock / check (pull_request) Successful in 5s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 16s
CI (SIAX Cloud) / security (pull_request) Successful in 35s
CI (SIAX Cloud) / integration (pull_request) Successful in 37s
CI (SIAX Cloud) / quality (pull_request) Successful in 43s
A client retry with the same idempotency key replayed the journal and got 'outcome_unknown' even when the first upload had succeeded, because the verified DOC0 receipt was never persisted. Store document/version/checksum/ stage on the attempt (sql/021) and return it on replay; only a missing receipt falls back to observation. No second POST, no duplicate document. |
||
|
|
dade37ee05 |
feat(credits): one-time trial grant so the ledger runs in production
CI (SIAX Cloud) / sonar (pull_request) Skipped
masterplan-lock / check (pull_request) Successful in 4s
masterplan-lock / check (push) Successful in 5s
CI (SIAX Cloud) / security (pull_request) Successful in 16s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 16s
CI (SIAX Cloud) / integration (pull_request) Successful in 9m58s
CI (SIAX Cloud) / quality (pull_request) Successful in 10m25s
- Ledger accepts kind='grant' (a server-owned balance grant, not a payment). - OPT_TRIAL_CREDITS=N grants N credits once per tenant (idempotent key trial:<tenant>) on balance/structure, so reserve -> settle is exercised before any customer purchase. - sql/020 extends the kind constraint (re-runnable). - Tests for the grant helper; full suite 1345 passed. |
||
|
|
0f734297ce |
fix: enforce tenant isolation on credit storage
CI (SIAX Cloud) / integration (push) Skipped
CI (SIAX Cloud) / sonar (push) Skipped
masterplan-lock / check (push) Successful in 5s
CI (SIAX Cloud) / sonar (pull_request) Skipped
masterplan-lock / check (pull_request) Successful in 6s
CI (SIAX Cloud) / security (push) Successful in 20s
Security Scan (SIAX Cloud) / security (push) Successful in 19s
CI (SIAX Cloud) / security (pull_request) Successful in 16s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 16s
CI (SIAX Cloud) / quality (push) Successful in 52s
CI (SIAX Cloud) / integration (pull_request) Successful in 51s
CI (SIAX Cloud) / quality (pull_request) Successful in 1m9s
|
||
|
|
cf37a66d35 |
feat: persist scoped document upload attempts
CI (SIAX Cloud) / integration (push) Skipped
CI (SIAX Cloud) / sonar (push) Skipped
masterplan-lock / check (push) Successful in 5s
CI (SIAX Cloud) / sonar (pull_request) Skipped
masterplan-lock / check (pull_request) Successful in 4s
Security Scan (SIAX Cloud) / security (push) Successful in 18s
CI (SIAX Cloud) / security (pull_request) Successful in 20s
CI (SIAX Cloud) / security (push) Successful in 33s
CI (SIAX Cloud) / integration (pull_request) Successful in 44s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 50s
CI (SIAX Cloud) / quality (push) Successful in 1m2s
CI (SIAX Cloud) / quality (pull_request) Successful in 1m2s
|
||
|
|
bd1c20edb2 |
feat: add guarded document workbench foundations
CI (SIAX Cloud) / integration (push) Skipped
CI (SIAX Cloud) / sonar (push) Skipped
masterplan-lock / check (push) Successful in 5s
Security Scan (SIAX Cloud) / security (push) Successful in 15s
CI (SIAX Cloud) / security (push) Successful in 16s
CI (SIAX Cloud) / sonar (pull_request) Skipped
masterplan-lock / check (pull_request) Successful in 7s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 21s
CI (SIAX Cloud) / security (pull_request) Successful in 25s
CI (SIAX Cloud) / quality (push) Successful in 1m17s
CI (SIAX Cloud) / integration (pull_request) Successful in 54s
CI (SIAX Cloud) / quality (pull_request) Successful in 1m7s
|
||
|
|
04813c34b0 |
feat(collab): order-anteckningar — @siax/collaboration-kernelns första produktkonsument (spår C)
CI (SIAX Cloud) / integration (push) Skipped
CI (SIAX Cloud) / sonar (push) Skipped
masterplan-lock / check (push) Successful in 7s
Security Scan (SIAX Cloud) / security (push) Successful in 16s
CI (SIAX Cloud) / security (push) Successful in 18s
masterplan-lock / check (pull_request) Successful in 8s
CI (SIAX Cloud) / quality (push) Failing after 33s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 22s
CI (SIAX Cloud) / integration (pull_request) Successful in 31s
CI (SIAX Cloud) / security (pull_request) Successful in 23s
CI (SIAX Cloud) / sonar (pull_request) Skipped
CI (SIAX Cloud) / quality (pull_request) Failing after 21s
Kernel 0.3.0 används som capability, inte som paket-dekor: - lib/order-notes-store.ts: varje note ÄR en CollaborationOperation; nextLamportClock för nya klockor, mergeOperationLog/compareOperations för deterministisk listning; optimistisk version -> 409 vid stale skrivning. - lib/order-notes-client.ts: kernel-CollaborationTransport över host-API; offline = medvetet no-op + varning (ingen tyst förlust), 409 -> ConflictError. - app/api/admin/order-notes: requireRouteAuth (fail-closed 401), server-authoritative tenant-guard (403), AUD0-audit via hostens klient. - app/admin/notes: admin-yta med offline-växel, konflikt-banner + reload. - sql/016_order_notes.sql: persistens (DB-tabell, version + Lamport-klocka). Tester: API (happy/conflict 409/empty/400/401/403/404), klient-adapter (offline no-op utan fetch, 409-mappning, presence), SSR-render. 18 nya test. |
||
|
|
c3582838aa |
feat(orders): kanonisk statusmaskin + guardad DB-constraint (ADR-0002, C3)
CI (SIAX Cloud) / integration (push) Skipped
CI (SIAX Cloud) / sonar (push) Skipped
masterplan-lock / check (push) Successful in 1m3s
CI (SIAX Cloud) / security (push) Successful in 1m4s
Security Scan (SIAX Cloud) / security (push) Successful in 1m7s
CI (SIAX Cloud) / quality (push) Successful in 1m52s
CI (SIAX Cloud) / sonar (pull_request) Skipped
masterplan-lock / check (pull_request) Successful in 6s
CI (SIAX Cloud) / security (pull_request) Successful in 18s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 19s
CI (SIAX Cloud) / quality (pull_request) Successful in 1m20s
CI (SIAX Cloud) / integration (pull_request) Successful in 1m58s
|
||
|
|
d272e52ba0 |
fix(db): subscriptions.created_at — migration 014 reparerar GET /api/subscriptions (42703 mätt live)
CI (SIAX Cloud) / integration (push) Has been skipped
masterplan-lock / check (pull_request) Successful in 4s
masterplan-lock / check (push) Successful in 13s
Security Scan (SIAX Cloud) / security (push) Successful in 16s
CI (SIAX Cloud) / security (push) Successful in 21s
CI (SIAX Cloud) / security (pull_request) Successful in 18s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 19s
CI (SIAX Cloud) / integration (pull_request) Successful in 27s
CI (SIAX Cloud) / quality (push) Successful in 37s
CI (SIAX Cloud) / quality (pull_request) Successful in 37s
lib/subscriptions-store.ts listSubscriptions() SELECT:ar created_at men sql/001_init.sql definierade aldrig kolumnen — GET /api/subscriptions har alltså ALWAYS brutit mot Postgres i prod (42703 errorMissingColumn, mätt live mot 0pt.app 2026-09-22: 500). Kolumnen är nu ADD:ad live på 0pt-edge-postgres (idempotent IF NOT EXISTS); denna migration gör ändringen reproducerbar. GET svarar nu 200 []. |
||
|
|
a1557b2a7d |
feat(payments): bind orders to verified tenant and buyer
CI (SIAX Cloud) / integration (push) Has been skipped
masterplan-lock / check (push) Successful in 4s
CI (SIAX Cloud) / security (push) Successful in 14s
masterplan-lock / check (pull_request) Successful in 4s
CI (SIAX Cloud) / security (pull_request) Successful in 14s
Security Scan (SIAX Cloud) / security (push) Successful in 30s
CI (SIAX Cloud) / quality (push) Successful in 35s
CI (SIAX Cloud) / integration (pull_request) Successful in 25s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 54s
CI (SIAX Cloud) / quality (pull_request) Successful in 1m31s
|
||
|
|
195c4611dc | chore: remove redundant search definition index | ||
|
|
1c1d14a3eb | feat: persist versioned search index definitions | ||
|
|
8432d19d8c | fix: scope dataset version idempotency uniqueness to real keys | ||
|
|
298c965448 | feat: add durable dataset version store | ||
|
|
ca0b060c6d | feat(evidence): add durable AUD0 outbox schema | ||
|
|
1a12e8a882 |
feat(analytics): BI-P061 — dashboard releases, with the publish race measured both ways
CI (SIAX Cloud) / quality (push) Successful in 50s
Security Scan (SIAX Cloud) / security (push) Successful in 45s
CI (SIAX Cloud) / security (push) Successful in 1m0s
CI (SIAX Cloud) / integration (push) Successful in 1m12s
CI (SIAX Cloud) / quality (pull_request) Successful in 1m6s
CI (SIAX Cloud) / security (pull_request) Successful in 33s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 32s
CI (SIAX Cloud) / integration (pull_request) Successful in 47s
DRAFT -> PUBLISHED -> SUPERSEDED as a ledger (sql/010) rather than a column on
the frozen version: sql/008 already makes DashboardVersion immutable at the
database, and a lifecycle column there would have meant weakening that.
Rollback mints a NEW version carrying the old document and publishes it, so
"we went back to v3" is a timestamped fact (v5 = rollback_of_version 3) rather
than a hole where v4 used to be. It is verified by BI-P060 canonical hash, and
documentOf() refuses to serve a stored document whose hash no longer matches
the release — a dashboard that renders an unapproved definition is worse than
one that will not open.
The claim worth attacking is "only one release can be live". The application
check is read-then-write, so it is NOT the guarantee, and the evidence says so
in both directions against Postgres 16.15:
guard_absent partial unique index dropped -> BOTH concurrent publishes
commit, dashboard ends with TWO live versions
guard_present index in place -> loser fails 23505 on
opt_analytics_dashboard_release_one_published, one live
evidence/bi-p061/dashboard-release-race.json, reproducible with
scripts/bi-p061-release-race.mjs.
Two findings while writing the tests, both fixed here rather than papered over:
the one-draft-per-dashboard index makes the two-draft race impossible to even
construct (so the race is run as two create-and-publish transactions, which is
what a "save and publish" button does), and an UPDATE matching zero rows
raises nothing — which reads as "refused" to a careless test, so the
immutability test asserts the row exists first.
23 tests: 8 unit + 15 against live Postgres.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
|
2bfbd3f159 |
feat(analytics): sql/009 — execution metadata, metric governance, semantic registry, quarantine, usage ledger
CI (SIAX Cloud) / integration (push) Failing after 38s
CI (SIAX Cloud) / quality (push) Successful in 1m0s
CI (SIAX Cloud) / security (push) Successful in 25s
Security Scan (SIAX Cloud) / security (push) Successful in 28s
CI (SIAX Cloud) / integration (pull_request) Failing after 48s
CI (SIAX Cloud) / quality (pull_request) Successful in 1m16s
CI (SIAX Cloud) / security (pull_request) Successful in 27s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 23s
The migration the previous commits' comments promised. Additive and idempotent
throughout, so the restore drill can replay it onto a live database.
BI-P018 adds principal_id, workload, compiled_query_hash, datasets, metrics,
cache_status and cost_units to the execution record — and deliberately NO
`query_text` column, with a CHECK pinning the hash to 64 hex characters so a
statement smuggled into that field is refused by the database. A test asserts
the absence of `query_text` and `sql` as columns.
BI-P031's DoD gets a Postgres TRIGGER, not just the TypeScript guard: a
certified metric whose meaning_hash changes without a version bump raises. That
is the layer that holds when something writes without going through the
application — a migration script, a psql session, an admin tool. A compiler
opinion does not survive `psql`; a trigger does.
BI-P030 gets a version-history table whose CONTENT is immutable (status and
valid_from may still move) plus a PARTIAL UNIQUE INDEX enforcing at most one
ACTIVE version per model per tenant. Two active versions is a silent
correctness failure in which half the dashboards read a different model, so it
is an index rather than application discipline.
BI-P021's usage ledger is append-only by trigger: update and delete both raise.
BI-P026's freshness table DEFAULTS TO 'UNKNOWN' — a dataset nobody has computed
freshness for must not read as FRESH — and requires a non-empty disclosure,
because a state with no words attached is a state a dashboard renders as a
silent green dot. BI-P025's quarantine record requires a principal and a reason
on release.
A BUG IN THIS MIGRATION, FOUND BY RUNNING IT:
`SELECT 1 FROM pg_constraint WHERE conname = '...'` is NOT schema-scoped.
conname is unique per schema, not per database, so on a database that already
had these tables in `public` every guarded CHECK was silently skipped when
the file was applied into a second schema. The constraints simply were not
there, and the tests that should have proven them PASSED — because the
INSERTs they expected to fail succeeded, and `expect(message).toContain(...)`
on an undefined message reads as a type error, not as "the constraint is
missing". Every guard now scopes via `conrelid = '<table>'::regclass`, and
the fix is proven by applying the file to `public` FIRST and then running the
suite into a fresh schema: 20/20.
Two flaws in my own test harness, also found by running it:
* `sql.unsafe("SET search_path ...; SELECT ...")` is a multi-statement simple
query, and postgres.js returns the LAST statement's result — so a SELECT
silently returned the connection object and two assertions compared against
nonsense. search_path is now pinned on the CONNECTION.
* the fixture INSERT used column names that do not exist in sql/008's dataset
table, which surfaced as a suite-level failure rather than as 20 skips.
20 tests against a real Postgres 16, gated on ANALYTICS_TEST_DATABASE_URL, and
each of them attempts the write the constraint exists to refuse.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
|
84bb9faa4f |
feat(analytics): BI-P002..BI-P012 analytics core — domain model, tiers, contracts, Arrow, ClickHouse engine, isolation, governance
CI (SIAX Cloud) / security (push) Successful in 41s
Security Scan (SIAX Cloud) / security (push) Successful in 40s
CI (SIAX Cloud) / integration (push) Successful in 52s
CI (SIAX Cloud) / quality (push) Successful in 1m5s
CI (SIAX Cloud) / security (pull_request) Successful in 31s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 29s
CI (SIAX Cloud) / integration (pull_request) Successful in 45s
CI (SIAX Cloud) / quality (pull_request) Successful in 53s
Implements the eight BI program prompts that make 0pt the canonical SIAX
analytics capability. Every file is NEW: not one existing file is modified,
and src/analytics/clickhouse-analytics.ts is byte-identical to main.
BI-P002 — the nineteen canonical entities, verbatim, with tenant_id +
project_id on every tenant-owned record. sql/008 gives each one a table.
The four version entities BI-P002 names as immutable are enforced by BEFORE
UPDATE/DELETE triggers, not by a TypeScript `readonly` — proven by direct SQL
against a live Postgres.
BI-P003 — RAW/NORMALIZED/CURATED/SERVING with the read rule enforced:
assertReadableByBiSurface refuses a RAW dataset to a dashboard unless a
recorded exception exists. All eight required declarations are NOT NULL.
BI-P004 — DatasetContract and a fail-closed compatibility gate. Fifteen
breaking-change classes are each tested twice: refused under a minor bump,
accepted under a major one.
BI-P005 — Arrow IPC alongside JSON, with a hand-written FlatBuffers writer so
no npm dependency (or CVE surface) is added. Conformance comes from three
independent implementations, not from self-agreement: pyarrow 25.0.1 and
DuckDB 1.5.5 read identical values from both ClickHouse's Arrow and ours, and
ClickHouse's own Arrow C++ ingests our 20,000-row batch with exact aggregates.
BI-P009 — SiaxAnalyticsEngine with all eight capabilities. The existing
ClickHouseAnalytics is COMPOSED, never replaced. executeQuery (analyst SQL) is
declared and deliberately refused with CAPABILITY_UNAVAILABLE until BI-P017 and
BI-P033 land, rather than opening the raw-SQL hole BI-P011 enumerates.
BI-P010 — four purpose-built tables, not one giant one, and the difference is
measured against the counterfactual: 48.8ms/40,960 rows vs 174.4ms/5,400,000
rows for the same question; 5/660 granules vs 660/1149.
BI-P011 — layered isolation, each layer proven with the layer above removed.
Without the forced predicate the same statement returns 50,050 (both tenants);
with it, t_alpha gets 50. Without a row policy an unrestricted user sees 2
tenants; the restricted user running the IDENTICAL statement sees 1.
BI-P012 — every named limit, fired individually and identified by ClickHouse
error code, plus the half usually skipped: an unrelated tenant is timed while
a real CPU bomb is in flight and must still return the CORRECT answer.
Three defects found by writing tests that can fail:
* `CHECK (array_length(purposes,1) >= 1)` can NEVER fail — array_length is
NULL for '{}' and a NULL CHECK passes. Fixed with coalesce(); the test
that caught it is pinned.
* EXPLAIN ESTIMATE run under the caller's own max_rows_to_read fails with
code 158 exactly when the estimate would be over budget. Planner calls now
use a narrower envelope.
* result_overflow_mode=break returns HTTP 200 with 65,409 of 100,000 rows
and no indication it is short. Every SELECT ships `throw`.
Verified rather than assumed: server7's ClickHouse holds `opt_scraped` (51
tables, 407,793,471 rows, `persons`/`cz_owners` included) — the b0ts plane this
module is prohibited from touching — has no `opt_analytics` database, zero
tenant_id columns and zero row policies. Nothing there was modified; every
proof ran against disposable containers.
Evidence: evidence/bi-p002-p012/{test-transcript.txt,bi-p010-benchmark.json,
bi-p005-arrow-roundtrip.json,server7-clickhouse-probe.md}
Docs: docs/analytics/BI_ANALYTICS_CORE.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
|
9267576b7c |
feat(data-os): P060-P068 durable ingestion plane — source classification gate, connector contract, common envelope, schema evolution, quality enforcement, queryable lineage, idempotent backfill
CI (SIAX Cloud) / integration (push) Successful in 42s
CI (SIAX Cloud) / quality (push) Successful in 1m17s
CI (SIAX Cloud) / security (push) Successful in 46s
Security Scan (SIAX Cloud) / security (push) Successful in 31s
CI (SIAX Cloud) / integration (pull_request) Successful in 36s
CI (SIAX Cloud) / quality (pull_request) Successful in 1m4s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 27s
CI (SIAX Cloud) / security (pull_request) Successful in 24s
BEFORE_STATE (verified, with evidence):
* P060 PARTIAL/BROKEN. The registry persisted rights/provenance/health/
freshness to Postgres, but carried no authorization, no schedule, no
checkpoint, no schema and no owner distinct from created_by — and the
activation DoD was inverted: postgres-source-repository.ts wrote
`${input.status ?? "active"}`, so a source with no rights, no provenance
and no authorization was ACTIVE the instant it was registered. No
activation operation existed at all.
* P061 MISSING. services/api/sources/connectors.py had per-source fetch_*()
functions only — no lifecycle, no cursor, no resume, no health, no
conformance suite.
* P062 MISSING, already documented as such in
services/api/tests/test_known_gaps.py:10-19 (no CDC pipeline anywhere;
workers/pipeline.py has no persistence at all).
* P063 MISSING. schema_version existed as a free string; no comparison,
no policy, no versioned source schema.
* P064 PARTIAL. Rules/runs/issues/scores existed, but a failing rule only
opened an issue and the data flowed on. No quarantine anywhere.
* P065 DUPLICATE (confirmed). Two ER engines: src/er/entity-resolution-
service.ts (canonical) and services/api/workers/resolve.py (simplified,
still called by pipeline.py:110).
* P067 PARTIAL. Lineage was emitted to AUD0 but nothing here could ANSWER a
lineage question — no store, no query, no route.
* P068 MISSING, documented in test_known_gaps.py:44-52 ("no delivery-store
persistence layer exists to dedupe a backfill against").
BUILT (only what was PARTIAL/BROKEN/MISSING):
* P060 sql/007_ingestion_plane.sql adds authorization/schedule/schemas/
checkpoints/ledger/quarantine/lineage tables, owner_id, and flips the
opt_sources status DEFAULT to 'pending' so even a raw psql INSERT cannot
produce an unclassified-but-active source. SourceRegistryService gains
classifySource/activateSource/deactivateSource + authorization/schedule
accessors; registration NEVER activates (asking for status:"active" goes
through the gate, not around it); a service identity cannot authorize a
source. Routes: activate (GET classification / POST activate),
authorization, schedule.
* P061 src/connectors/connector.ts — all nine operations on one interface.
src/connectors/conformance.ts — a runnable suite (15 checks, no "skip"
verdict). HttpJsonConnector (real HTTP, opaque cursor) and
JsonlFileConnector (real file, line offsets, CDC deletes) both pass it.
The suite caught a real defect while being written: an API that drops
next_cursor at the end left the connector unable to resume, which is why
HttpJsonConnectorConfig has cursor_field.
* P062 src/types/ingestion.ts + src/ingestion/{envelope,ingestion-runner}.ts.
One envelope for batch/stream/cdc; durable per-(source,partition)
checkpoints; resume proven by killing a run inside the sink.
* P063 src/schema/{record-schema,compatibility,schema-registry}.ts. Two
enforcement points, both upstream of any consumer write: a discovery gate
before the first read, and per-record conformance before the page reaches
the sink. A producer cannot widen its own policy retroactively.
* P064 src/quality/quality-policy.ts composes the EXISTING ExpectationEngine
(not a second engine) and adds the missing consequence: block /
quarantine / warn, with quarantine durable and recoverable. No policy is
reported as "none_declared", never as a pass; an unexecutable expectation
counts as a violation, never a silent pass.
* P067 src/lineage/{lineage-edge,lineage-store}.ts + GET /api/0pt/lineage.
AUD0 stays the system of record; this is 0pt's own queryable projection.
* P068 opt_ingestion_ledger + IngestionRunner.backfill: bounded, idempotent
on repeat, leaves the live checkpoint alone, still re-emits real changes.
A BUG THIS PASS WROTE AND CAUGHT: the first IngestionRepository draft
combined the idempotency check and its recording into one atomic call
BEFORE the sink. The crash/resume test then delivered 7 of 10 records — the
page in flight when the sink died was already marked delivered, so the
resumed run skipped it as a duplicate. Permanent, silent data loss. Fixed by
splitting peekDeliveries/commitDeliveries and committing both the ledger and
the checkpoint only after the sink resolves.
NOT DONE, stated rather than deferred quietly:
* P065 is NOT met. resolve.py is still a second simplified production
engine. Closing it means rewriting it as an adapter over the canonical
service, which cannot be verified from this repo today: services/api is
not built by this repo's Dockerfile, api.0pt.app answers 404, and NO CI
workflow runs its pytest suite. The duplication is now stated at both
sites in code and the removal plan is in STATUS.md. The new ingestion
plane performs no ER, so it adds no third engine.
* P066 (Iceberg) untouched — ST0RE is owned by another workflow. Contract
read only.
* Delivery is AT-LEAST-ONCE, not exactly-once (no per-partition run lock).
* No CDC *source* exists; the contract is met, a CDC connector is not built.
* 0pt does not schedule itself — the schedule row is a declaration ACT0
would read.
VERIFIED: pnpm typecheck clean; pnpm test 365 passed / 25 skipped (was
311/18); pnpm build clean with all four new routes in the route table; lint
unchanged at 12 pre-existing errors. Real Postgres 16
(tests/p0/ingestion-plane-postgres.realinfra.test.ts, 7 passed) plus a real
`docker restart` of that container with an identical read-back through the
production code path — that is P060's "a restart preserves all sources",
measured.
DEPLOYMENT REALITY (curl, 2026-09-04): 0pt.app/api/health returns 200 from
Vercel with db:not_configured; 0pt.app/api/0pt/sources returns 404;
api.0pt.app/health returns 404; cd.yml still guards on a placeholder Coolify
app id. None of the capability layer runs in production. Nothing here is
LIVE_VERIFIED and STATUS.md says so.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
|
dc6aaa0e10 |
fix(data-os): P014 — close tenant-isolation + idempotency gaps, execute real restore drill
CI (SIAX Cloud) / quality (push) Successful in 51s
Security Scan (SIAX Cloud) / security (push) Successful in 26s
CI (SIAX Cloud) / security (push) Successful in 28s
CI (SIAX Cloud) / integration (push) Successful in 1m1s
CI (SIAX Cloud) / integration (pull_request) Successful in 30s
CI (SIAX Cloud) / security (pull_request) Successful in 24s
CI (SIAX Cloud) / quality (pull_request) Successful in 48s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 22s
Tenant isolation (security-critical): every app/api/0pt/** route reading
tenant_id from client-controlled query/body never cross-checked it against
the token middleware.ts already verified (x-0pt-verified-tenant). Adds a
shared assertTenantMatchesVerified guard and wires it into all 20+ affected
routes. Also closes two full-leak-by-omission bugs found in this pass
(documents/capture and lakehouse/mappings GETs returned every tenant's rows
when tenant_id was simply omitted) and two "no tenant check at all" reads
(rate-policies/[id], lakehouse/mappings/[id]) reachable by ID alone.
Duplicate-delivery idempotency (correctness-critical): POST /api/orders now
honors Idempotency-Key (sql/006, replay-or-409-on-conflict, Postgres +
in-memory); the Python orchestrator's POST /orders/{id}/run — the actual
delivery trigger — is now idempotent under an asyncio lock, closing a real
concurrent-request race, not just the sequential-retry case.
Engine-duplication resolution: adaptive_router.py's docstring wrongly
invited a bypass ("see b0ts for scraping") contradicting the product's own
compliance stance; corrected, and given real (additive, not escalating)
block-signal tracking so a rate-limited/blocked source is excluded from the
plan instead of hammered. Also fixes a previously-undiscovered production
bug: the Python engine's only call into the TS quality bridge
(X-Internal-Token) was unconditionally 401'd by middleware.ts, which never
checked that credential — the advisory quality bridge was unreachable in any
environment enforcing auth.
Restore drill: executed for real against throwaway Docker Postgres
containers (not simulated). Found and fixed two real blockers along the way:
sql/004 referenced opt_datasets, which no migration ever created (chain was
never actually runnable end to end); lib/db.ts hardcoded ssl:"require",
making any non-TLS Postgres (incl. compose.dev.yaml's own local-dev
container) unreachable. After both fixes: full migration chain applies
cleanly, pg_dump/pg_restore round-trip is byte-identical (row counts +
content checksums), and real application code (lib/orders-store.ts,
SourceRegistryService's Postgres repository) reads the restored data back
correctly, including cross-tenant isolation still holding.
82 new tests (55 TS + 27 Python), all passing; full existing suite (291 TS +
55 Python) still green; typecheck clean on every changed/added file.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
|
07b13bd833 |
feat(data-os): Source Registry Postgres persistence + full HTTP route surface
Replaces the in-memory (per-instance, not even per-process-shared) Maps in SourceService/SourceRegistryService with real Postgres persistence, and adds the register/query/describe/health/rights/provenance/freshness HTTP routes that never existed (grep for SourceRegistryService in app/ previously returned nothing). - sql/002_source_registry.sql: opt_sources, opt_source_rights, opt_source_provenance, opt_source_health_checks, opt_source_freshness. Numbered 002 (not 006) because 004's opt_source_rate_policies already forward-references opt_sources, which no sql/ file defined until now — fixes 004's pre-existing broken migration ordering (opt_datasets remains a separate, unrelated, still-open gap in 004, flagged not fixed). - src/repositories/*: SourceRepository / SourceRegistryRepository seams, each with an in-memory implementation (business logic extracted verbatim) and a Postgres implementation, chosen via resolveStoreMode — production still fails hard with no DATABASE_URL (Master-DoD §3), but a configured production now actually works instead of always being blocked. - SourceService / SourceRegistryService: storage moved behind the repository seam; getHealth/getFreshness are now async and tenant-scoped at the accessor itself (needed once they're reachable directly over HTTP, not only after an earlier caller's tenant check). - src/services/source-registry.ts: shared singleton instances for the sources/* route group, fixing the "each route module owned its own private registry" defect (source-rate-policies and .../query previously couldn't see a source registered via POST /api/0pt/sources). - New routes: POST/GET /api/0pt/sources, GET /api/0pt/sources/[id], GET+PUT .../rights, GET+PUT .../provenance, GET+POST .../health, GET .../freshness. - Business logic unchanged: fail-closed purpose gate, cross-tenant NOT_FOUND (no existence oracle, §50), bounded 100-row health history. Tested against a real Postgres container (not mocked) — tests/p0/source-registry-postgres.realinfra.test.ts, env-gated on SOURCE_REGISTRY_TEST_DATABASE_URL, skips honestly when unset. All 249 pre-existing P0 tests + 7 new realinfra tests pass; also smoke-tested the full HTTP route surface end-to-end against next dev + live Postgres. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
|
bda21e87eb |
Live-verify Iceberg write path against ST0RE + close field/contract gaps
Re-runs this session's task spec against 0pt's actual repo state, which
turned out to already have a substantial, tested implementation from a
prior pass (
|
||
|
|
35b6ec63bf |
Add 0pt Iceberg write-path ownership: dataset<->table mapping + source rate policy
Real, tested implementation of docs/11-lakehouse-write-plan.md's items 2-8 (the "future implementation session" that doc named): 0pt is now the canonical writer of Iceberg tables through ST0RE's real, shipped /v1/st0re/lakehouse/* table registry (sax3l/st0re commit 9ea0abb), never persisting lakehouse table/catalog/snapshot state itself. - src/types + src/services for 0pt.dataset_iceberg_mapping.schema.v1 (dataset -> ST0RE tableId reference) and 0pt.source_rate_policy.schema.v1 (the sourceRatePolicy ownership repair, moved out of N0D per ADR-0030). - src/clients/st0re-lakehouse-client.ts: new client matching ST0RE's actual camelCase route contract, with the same mock/HTTP boundary convention as this repo's existing cl0ud/n0d/aud0/st0re clients. Honest caveat in its module doc comment: not verified against a live ST0RE instance in this pass (no reachable ST0RE_URL); every test exercises the deterministic mock. - src/services/iceberg-write-service.ts: registers tables/snapshots through ST0RE, requests N0D jobs only for heavy writes (never executes them in-process), emits all four AUD0 event names (dataset_iceberg_table_mapped / iceberg_write_requested / iceberg_snapshot_created / source_rate_policy_created). - sql/004, packages/schemas/*.json, app/api/0pt/lakehouse + sources/rate-policies routes, tests/p0/*.test.ts (154 passed + 1 honest it.todo for the live-ST0RE case, up from 142 baseline, 0 failures). - Reconciled a real collision: an earlier untracked draft schema/test pair (tests/p0/enterprise-data-os-expansion.test.ts) asserted a different, now-superseded source_rate_policy shape; updated that one test block only. - docs/ICEBERG_WRITE_BOUNDARY.md states the ownership boundary plainly and is explicit throughout about what's real-and-tested vs. contract-only. docs/11-lakehouse-write-plan.md gets a status-update section, not a rewrite. - Data quality rules (quality-check-service.ts, quality-report-service.ts) were not touched -- confirmed still 0pt-owned. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
|
15c65a584c |
feat(api): Vercel-native platform API + Postgres schema
Route Handlers (Vercel Functions): /api/health, /api/estimates (single-source estimate math, ore), /api/catalog, /api/sources (legal sourcing map). sql/001_init.sql = full docs/03 data model (entities + bridge_provenance + platform tables, pgvector). Ready to wire Vercel Postgres. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |