Kernel 0.3.0 används som capability, inte som paket-dekor:
- lib/order-notes-store.ts: varje note ÄR en CollaborationOperation;
nextLamportClock för nya klockor, mergeOperationLog/compareOperations
för deterministisk listning; optimistisk version -> 409 vid stale skrivning.
- lib/order-notes-client.ts: kernel-CollaborationTransport över host-API;
offline = medvetet no-op + varning (ingen tyst förlust), 409 -> ConflictError.
- app/api/admin/order-notes: requireRouteAuth (fail-closed 401),
server-authoritative tenant-guard (403), AUD0-audit via hostens klient.
- app/admin/notes: admin-yta med offline-växel, konflikt-banner + reload.
- sql/016_order_notes.sql: persistens (DB-tabell, version + Lamport-klocka).
Tester: API (happy/conflict 409/empty/400/401/403/404), klient-adapter
(offline no-op utan fetch, 409-mappning, presence), SSR-render. 18 nya test.
Symmetrisk mot våg 3.4 CSV-export: POST /api/orders/import (validate |
dry_run | execute) + /admin/import som renderar @siax/ui ImportWizard
(0.14.0) över @siax/import-export runtime (0.2.0).
- Servern är enda auktoritet: RFC4180-parse (ingen eval), omvalidering
via frysta @siax/contracts-kerneln (validateRows) + hostregler,
exekvering genom BEFINTLIG orderlogik (createOrder, tenant-guardad).
- Fail-closed: 401 utan principal, 400 okänd header, 413 vid radtak
(500), per-rad-atomsäkra inserts — felrad kan aldrig skada OK-rader.
- Partiell import: OK-rader skapas, felrader rapporteras per rad
(row/field/code). AUD0-audit via 0pts enda producer-boundary
(durable outbox under DATABASE_URL).
- Reload-persistens: efter execute läses /api/orders om och de nya
raderna listas på sidan.
- Tester: 12 route-test (happy, partial, auth fail-closed, tak,
no-write validate, header, quoting, rad-fel mitt i batch) + 3
SSR-render-test av ImportWizard över runtime-storen.
- Deps: @siax/ui ^0.14.0, @siax/charts ^0.5.0, @siax/import-export
^0.2.0; pnpm-overrides till contracts 0.10.0 / import-export 0.2.0
(ui 0.14.0 deklarerar 0.11.0/0.2.1 som ännu ej publicerats).
Servern projicerar öppna ordrar (draft/quoted/in_production/review) på
universal-inbox-konvolverna via createInboxItem (buildOrderInboxItem i
lib/order-flow, ADR-0001-mönstret: policy/resolution i serverlagret).
Admin renderar InboxList (@siax/ui 0.11.1) ovanpå orderkön — resolve-
aktionen gateras genom advance_action-state (available krävs). Terminala
statusar ger ingen post. tests/order-flow 6/6; full svit 1144/1144; build ✓.
GET /api/orders returnerar per order frozen advance_action (resolveAction ur
lib/order-flow); admin renderar via ActionGate — policy i serverlagret,
presentation i klienten (DoD §21). Terminala/okända statusar → read_only med
varför-text i stället för missvisande 'klar'. tests/order-flow.test.ts 3/3;
full svit 1138 godkända; typecheck grön.
Mätt live 2026-09-25 mot produktionsdeployen (0pt.app / api.0pt.app,
container shxd6lepxuxfsjsp0ath3z1g-172514162615 på server6, uppe 17,9 h,
Docker-status "healthy" hela tiden):
GET /api/health -> HTTP/1.1 200 OK
{"ok":false,"service":"0pt-web","db":"ok","queue":"not_configured",
"engine":"error","stripe":"not_configured",
"detail":{"engine":"This operation was aborted"}}
GET /api/ready -> HTTP/1.1 200 OK
{"ready":true,"service":"0pt-web","db":"ok","engine":"error",...}
ENGINE_URL pekade på http://0pt-engine:8000. Mätt inifrån containern:
`getent hosts 0pt-engine` ger ingen post, och det finns ingen sådan
container på någon av server5-11. Ett direktanrop mot motorn misslyckas
efter ~5 s (DNS). Datamotorn — 0pt:s hela produkt — var alltså BORTA, och
samtliga signaler ovanför den var gröna: Docker healthy, /api/health 200,
/api/ready 200.
`ok:false` i en 200-kropp är ingen signal. Ingenting i kedjan läser
kroppar; allt läser statuskoder och exitkoder.
isReady skilde tidigare bara på "db ok". Den skillnad som saknades är inte
konfigurerad-vs-ej-konfigurerad utan FRÅNVARANDE-vs-TRASIG:
* not_configured / not_integrated -> fortsatt ready. 0pt är byggt för att
kunna köra utan motor, Stripe och kö (se .env.example).
* error -> INTE ready. Statusen "error" betyder att någon medvetet
konfigurerade beroendet och att en riktig round trip mot det just
misslyckades. Det är inte längre en valfri integration.
Ändrar ingenting i driften i dag: container-HEALTHCHECK pekar på
/api/health (liveness) och Dockerfilen säger uttryckligen att en
readiness-probe inte ska peka dit. Den här ändringen gör bara att den
ärliga ytan faktiskt blir ärlig, så en yttre övervakning kan använda den.
Regressionstest tillagt; verifierat att det FALLER mot den gamla isReady
(1 failed | 18 passed) och passerar mot den nya (24 passed över
health-checks/ready/health).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
getSqlForDurableStore throws SiaxError(DEPENDENCY_UNAVAILABLE, httpStatus 503,
retryable) when a money-bearing store would fall back to an in-memory Map in a
production runtime (the durable-store guard). The /api/0pt/* surface already
maps it via src/http/siax-error-response.ts; the seven legacy routes outside
the middleware matcher did not — the documented 503 leaked as Next's opaque
500 with no body.
- app/api/siax-error-mapping.ts: shared isSiaxError + siaxErrorResponseForLegacy
- orders (POST/GET), orders/[id] (GET/PATCH), subscriptions (GET/POST),
payments/deposit, payments/release, admin/metrics: try/catch -> mapper,
each route's own typed error contract preserved (idempotency conflict 409 etc.)
- webhooks/stripe: store outage -> 503 received:false reason:store_unavailable
so Stripe's delivery retry path stays open (never a silent received:true)
- tests/api/legacy-routes-siax-error-mapping.test.ts: pins the 503 + body
BEFORE_STATE (verified, with evidence):
* P060 PARTIAL/BROKEN. The registry persisted rights/provenance/health/
freshness to Postgres, but carried no authorization, no schedule, no
checkpoint, no schema and no owner distinct from created_by — and the
activation DoD was inverted: postgres-source-repository.ts wrote
`${input.status ?? "active"}`, so a source with no rights, no provenance
and no authorization was ACTIVE the instant it was registered. No
activation operation existed at all.
* P061 MISSING. services/api/sources/connectors.py had per-source fetch_*()
functions only — no lifecycle, no cursor, no resume, no health, no
conformance suite.
* P062 MISSING, already documented as such in
services/api/tests/test_known_gaps.py:10-19 (no CDC pipeline anywhere;
workers/pipeline.py has no persistence at all).
* P063 MISSING. schema_version existed as a free string; no comparison,
no policy, no versioned source schema.
* P064 PARTIAL. Rules/runs/issues/scores existed, but a failing rule only
opened an issue and the data flowed on. No quarantine anywhere.
* P065 DUPLICATE (confirmed). Two ER engines: src/er/entity-resolution-
service.ts (canonical) and services/api/workers/resolve.py (simplified,
still called by pipeline.py:110).
* P067 PARTIAL. Lineage was emitted to AUD0 but nothing here could ANSWER a
lineage question — no store, no query, no route.
* P068 MISSING, documented in test_known_gaps.py:44-52 ("no delivery-store
persistence layer exists to dedupe a backfill against").
BUILT (only what was PARTIAL/BROKEN/MISSING):
* P060 sql/007_ingestion_plane.sql adds authorization/schedule/schemas/
checkpoints/ledger/quarantine/lineage tables, owner_id, and flips the
opt_sources status DEFAULT to 'pending' so even a raw psql INSERT cannot
produce an unclassified-but-active source. SourceRegistryService gains
classifySource/activateSource/deactivateSource + authorization/schedule
accessors; registration NEVER activates (asking for status:"active" goes
through the gate, not around it); a service identity cannot authorize a
source. Routes: activate (GET classification / POST activate),
authorization, schedule.
* P061 src/connectors/connector.ts — all nine operations on one interface.
src/connectors/conformance.ts — a runnable suite (15 checks, no "skip"
verdict). HttpJsonConnector (real HTTP, opaque cursor) and
JsonlFileConnector (real file, line offsets, CDC deletes) both pass it.
The suite caught a real defect while being written: an API that drops
next_cursor at the end left the connector unable to resume, which is why
HttpJsonConnectorConfig has cursor_field.
* P062 src/types/ingestion.ts + src/ingestion/{envelope,ingestion-runner}.ts.
One envelope for batch/stream/cdc; durable per-(source,partition)
checkpoints; resume proven by killing a run inside the sink.
* P063 src/schema/{record-schema,compatibility,schema-registry}.ts. Two
enforcement points, both upstream of any consumer write: a discovery gate
before the first read, and per-record conformance before the page reaches
the sink. A producer cannot widen its own policy retroactively.
* P064 src/quality/quality-policy.ts composes the EXISTING ExpectationEngine
(not a second engine) and adds the missing consequence: block /
quarantine / warn, with quarantine durable and recoverable. No policy is
reported as "none_declared", never as a pass; an unexecutable expectation
counts as a violation, never a silent pass.
* P067 src/lineage/{lineage-edge,lineage-store}.ts + GET /api/0pt/lineage.
AUD0 stays the system of record; this is 0pt's own queryable projection.
* P068 opt_ingestion_ledger + IngestionRunner.backfill: bounded, idempotent
on repeat, leaves the live checkpoint alone, still re-emits real changes.
A BUG THIS PASS WROTE AND CAUGHT: the first IngestionRepository draft
combined the idempotency check and its recording into one atomic call
BEFORE the sink. The crash/resume test then delivered 7 of 10 records — the
page in flight when the sink died was already marked delivered, so the
resumed run skipped it as a duplicate. Permanent, silent data loss. Fixed by
splitting peekDeliveries/commitDeliveries and committing both the ledger and
the checkpoint only after the sink resolves.
NOT DONE, stated rather than deferred quietly:
* P065 is NOT met. resolve.py is still a second simplified production
engine. Closing it means rewriting it as an adapter over the canonical
service, which cannot be verified from this repo today: services/api is
not built by this repo's Dockerfile, api.0pt.app answers 404, and NO CI
workflow runs its pytest suite. The duplication is now stated at both
sites in code and the removal plan is in STATUS.md. The new ingestion
plane performs no ER, so it adds no third engine.
* P066 (Iceberg) untouched — ST0RE is owned by another workflow. Contract
read only.
* Delivery is AT-LEAST-ONCE, not exactly-once (no per-partition run lock).
* No CDC *source* exists; the contract is met, a CDC connector is not built.
* 0pt does not schedule itself — the schedule row is a declaration ACT0
would read.
VERIFIED: pnpm typecheck clean; pnpm test 365 passed / 25 skipped (was
311/18); pnpm build clean with all four new routes in the route table; lint
unchanged at 12 pre-existing errors. Real Postgres 16
(tests/p0/ingestion-plane-postgres.realinfra.test.ts, 7 passed) plus a real
`docker restart` of that container with an identical read-back through the
production code path — that is P060's "a restart preserves all sources",
measured.
DEPLOYMENT REALITY (curl, 2026-09-04): 0pt.app/api/health returns 200 from
Vercel with db:not_configured; 0pt.app/api/0pt/sources returns 404;
api.0pt.app/health returns 404; cd.yml still guards on a placeholder Coolify
app id. None of the capability layer runs in production. Nothing here is
LIVE_VERIFIED and STATUS.md says so.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Tenant isolation (security-critical): every app/api/0pt/** route reading
tenant_id from client-controlled query/body never cross-checked it against
the token middleware.ts already verified (x-0pt-verified-tenant). Adds a
shared assertTenantMatchesVerified guard and wires it into all 20+ affected
routes. Also closes two full-leak-by-omission bugs found in this pass
(documents/capture and lakehouse/mappings GETs returned every tenant's rows
when tenant_id was simply omitted) and two "no tenant check at all" reads
(rate-policies/[id], lakehouse/mappings/[id]) reachable by ID alone.
Duplicate-delivery idempotency (correctness-critical): POST /api/orders now
honors Idempotency-Key (sql/006, replay-or-409-on-conflict, Postgres +
in-memory); the Python orchestrator's POST /orders/{id}/run — the actual
delivery trigger — is now idempotent under an asyncio lock, closing a real
concurrent-request race, not just the sequential-retry case.
Engine-duplication resolution: adaptive_router.py's docstring wrongly
invited a bypass ("see b0ts for scraping") contradicting the product's own
compliance stance; corrected, and given real (additive, not escalating)
block-signal tracking so a rate-limited/blocked source is excluded from the
plan instead of hammered. Also fixes a previously-undiscovered production
bug: the Python engine's only call into the TS quality bridge
(X-Internal-Token) was unconditionally 401'd by middleware.ts, which never
checked that credential — the advisory quality bridge was unreachable in any
environment enforcing auth.
Restore drill: executed for real against throwaway Docker Postgres
containers (not simulated). Found and fixed two real blockers along the way:
sql/004 referenced opt_datasets, which no migration ever created (chain was
never actually runnable end to end); lib/db.ts hardcoded ssl:"require",
making any non-TLS Postgres (incl. compose.dev.yaml's own local-dev
container) unreachable. After both fixes: full migration chain applies
cleanly, pg_dump/pg_restore round-trip is byte-identical (row counts +
content checksums), and real application code (lib/orders-store.ts,
SourceRegistryService's Postgres repository) reads the restored data back
correctly, including cross-tenant isolation still holding.
82 new tests (55 TS + 27 Python), all passing; full existing suite (291 TS +
55 Python) still green; typecheck clean on every changed/added file.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Continues P010 (branch renamed from feat/p010-source-registry-persistence):
closes the three remaining BEFORE_STATE.md gaps (§2 TS/Python split-brain,
§3 missing provenance fields, §4 untested failure modes), all additive to
the production pipeline — the Python QA filter's delivery decision is
unchanged by any of this.
TS/Python bridge (§2):
- New route surface app/api/0pt/quality/* (rules/runs/issues CRUD +
POST /api/0pt/quality/bridge-run) over the shared QualityService
singleton (src/services/quality.ts), fixing the same "no shared
instance" defect the Source Registry had.
- src/quality/quality-bridge.ts auto-provisions a deterministic,
tenant-scoped baseline rule set on first use so the Python engine never
pre-registers rules through the CRUD surface.
- services/api/workers/quality_bridge.py: the Python-side HTTP client.
Feature-flagged off by default (OPT_QUALITY_BRIDGE_ENABLED), fails open
on any error (never raises into run_job, never changes `delivered`).
Advisory only — real per-row gating against the TS engine isn't
implementable yet (ExpectationEngine returns dataset-level pass/fail
counts + samples, not a per-row boolean array); documented, not faked.
- Live-verified end-to-end: `next dev` (AUTH_ENFORCE=false) + the real
Python pipeline hitting the real brreg.no API, round-tripping through
the real bridge route, receiving a real QualityService score.
Provenance (§3): sources/connectors.py's Provenance dataclass gains
retrieved_at, transform, connector_version, producer, quality,
entity_resolution, artifact_ref (all previously absent from the real
dataflow entirely) via a make_provenance() factory used by every connector.
resolve.py stamps entity_resolution on merged records' provenance (never on
unmerged ones). pipeline.py stamps artifact_ref at delivery and quality
once the bridge succeeds.
Failure tests (§4), real fault injection against real code (services/api,
new pytest + respx suite, not vitest mocks):
- dedup-determinism + duplicate-input + out-of-order: resolve.py under
shuffled/interleaved/exhaustive-permutation input, plus an explicit,
verified test of the one case that IS legitimately order-sensitive
(non-transitive similarity in a greedy clusterer) instead of hiding it.
- schema-evolution: respx-mocked upstream responses with added/removed/
alien-shaped fields through the real brreg connector + pipeline.
- restart: connector exceptions mid-job (including one from respx itself)
never crash run_job.
- backfill: the real property that holds given no persistence layer
exists (replay is compute-idempotent) — NOT a fabricated dedup-store
test.
- CDC-interruption and true backfill/delivery-store idempotency:
explicitly documented as not implementable against this codebase
(no CDC pipeline, no delivery store exist anywhere) via skip markers
with reasons, in tests/test_known_gaps.py, rather than silently
dropped or faked.
Testing: 256/256 vitest (249 pre-existing + 7 new bridge route/service
tests) and 28/28 pytest (2 documented skips) green locally. tsc -p
tsconfig.check.json has zero new errors (14 pre-existing errors, all in
the untracked/unaudited src/fleet, src/graph, src/metrics, src/projections
paths this pass does not touch, per BEFORE_STATE.md).
No PR opened, per instructions.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Replaces the in-memory (per-instance, not even per-process-shared) Maps in
SourceService/SourceRegistryService with real Postgres persistence, and adds
the register/query/describe/health/rights/provenance/freshness HTTP routes
that never existed (grep for SourceRegistryService in app/ previously
returned nothing).
- sql/002_source_registry.sql: opt_sources, opt_source_rights,
opt_source_provenance, opt_source_health_checks, opt_source_freshness.
Numbered 002 (not 006) because 004's opt_source_rate_policies already
forward-references opt_sources, which no sql/ file defined until now —
fixes 004's pre-existing broken migration ordering (opt_datasets remains
a separate, unrelated, still-open gap in 004, flagged not fixed).
- src/repositories/*: SourceRepository / SourceRegistryRepository seams,
each with an in-memory implementation (business logic extracted verbatim)
and a Postgres implementation, chosen via resolveStoreMode — production
still fails hard with no DATABASE_URL (Master-DoD §3), but a configured
production now actually works instead of always being blocked.
- SourceService / SourceRegistryService: storage moved behind the
repository seam; getHealth/getFreshness are now async and tenant-scoped
at the accessor itself (needed once they're reachable directly over HTTP,
not only after an earlier caller's tenant check).
- src/services/source-registry.ts: shared singleton instances for the
sources/* route group, fixing the "each route module owned its own
private registry" defect (source-rate-policies and .../query previously
couldn't see a source registered via POST /api/0pt/sources).
- New routes: POST/GET /api/0pt/sources, GET /api/0pt/sources/[id],
GET+PUT .../rights, GET+PUT .../provenance, GET+POST .../health,
GET .../freshness.
- Business logic unchanged: fail-closed purpose gate, cross-tenant
NOT_FOUND (no existence oracle, §50), bounded 100-row health history.
Tested against a real Postgres container (not mocked) —
tests/p0/source-registry-postgres.realinfra.test.ts, env-gated on
SOURCE_REGISTRY_TEST_DATABASE_URL, skips honestly when unset. All 249
pre-existing P0 tests + 7 new realinfra tests pass; also smoke-tested the
full HTTP route surface end-to-end against next dev + live Postgres.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
BACKLOG P0-3: /api/health only ever checked *presence* of DATABASE_URL/
REDIS_URL and always returned ok:true regardless of whether anything was
reachable -- ARCHITECTURE.md's own words, "an uptime monitor watching HTTP
200 would show green for a dying instance".
lib/health-checks.ts adds real dependency probes:
- db: a live `select 1` round trip (not just DATABASE_URL presence)
- engine: a real HTTP call to the FastAPI engine's own GET /health
- stripe: Stripe's documented lightweight balance.retrieve() liveness call
- queue: honestly reports "not_integrated" when REDIS_URL is set but
unused -- no Redis client is wired anywhere in this repo, so reporting
"ok" would fabricate a check that doesn't exist
/api/health stays a liveness check (always 200 while the process can
respond; the body's `ok` field is the honest signal). New GET /api/ready
fails closed with 503 when the load-bearing dependency (db) isn't both
configured and reachable -- the "no /ready route" gap.
contracts/openapi.yaml, docs/architecture/ARCHITECTURE.md, and docs/slo.md
updated to match. 17 new unit tests (dependency-injected fakes, no real
network/DB) + 5 route-level tests.
A live compliance audit against 0pt.app confirmed both endpoints returned
HTTP 200 with zero authentication -- the full order queue and financial
pipeline aggregates (deposit/pipeline totals) were readable by anyone.
Root cause: middleware.ts's matcher only covers /api/0pt/:path*, by design
("does NOT touch ... the unrelated payment/order APIs"), so nothing upstream
of these two handlers ever verified the caller.
Fix: both GET handlers now call the same fail-closed Zitadel verifier
/api/0pt/* relies on (resolveAuthContext, src/auth/context.ts) directly, via
a new thin wrapper (src/auth/require-route-auth.ts) that maps a rejected
decision to the same 401/403 JSON shape middleware.ts already returns. The
wrapper re-verifies the bearer token itself rather than trusting any
x-0pt-verified-* header on the request, since middleware never runs in front
of these routes to overwrite a client-supplied one.
POST /api/orders (anonymous draft/quote creation before the deposit-payment
step) is intentionally left as-is -- only the read paths that leaked data are
gated. Tracked as part of docs/compliance/BACKLOG.md P0-3.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Merges github/main (6 commits: 17-bug audit fix, CI quality gate,
.env.example + .gitignore fix, missing platform clients + registry
services, docs accuracy overhaul) into gitea/main (8 commits: Iceberg
write boundary + live ST0RE verification, lakehouse mapping routes,
Waves 1-4 foundational hardening).
16 conflicts. Both sides had independently written the four
src/clients/*.ts and the two registry services, so those are combined
rather than picked:
- st0re-client.ts: GitHub's endpoint wins on evidence. POST goes to
/v1/st0re/artifacts/register, not /v1/st0re/artifacts -- verified in
sax3l/st0re src/api/routes/artifacts.ts, where the bare path is a
GET-only list route. Gitea's URL normalisation, error-body text and
mock kept.
- n0d-client.ts: field is `status`, not `state`, with N0D's real 13-state
union -- verified in sax3l/n0d packages/schemas/n0d.job.schema.json,
which says explicitly not to add "succeeded". Gitea's mock echo,
URL normalisation and error-body text kept.
- cl0ud-client.ts: Gitea's fail-closed HTTP normalisation kept (a 200
with no `decision` reads as "deny", not as a silent allow). GitHub's
PolicyDecision/PolicyDecisionValue naming kept; PolicyResult retained
as an alias.
- aud0-client.ts: permissive union of both input types (optional
payload/metadata/correlation_id), GitHub's echo return, Gitea's URL
normalisation and error-body text, plus a JSON-parse fallback so a
successful POST with an empty body does not throw.
- dataset-service.ts / source-service.ts: GitHub's record/DTO split and
event naming (0pt.source_created / 0pt.create_source, which the merged
docs already reference) plus Gitea's name validation, description /
created_by fields and actor_id derivation. List filters are the union.
- lakehouse mapping routes: GitHub's shared lakehouse-mapping-registry
singletons (without which every write 400'd "mapping not found") AND
Gitea's Zod parseBody validation. Both, not either.
- src/types/source.ts, src/types/dataset.ts: unions of both vocabularies;
GitHub's `access` legal-sourcing field kept.
- .env.example: GitHub's audited inventory plus Gitea's Zitadel /
AUTH_ENFORCE block.
- CLAUDE.md, docs/ICEBERG_WRITE_BOUNDARY.md: GitHub's corrections kept,
Gitea's live-verification kept and marked as superseding the
"would 400 outright" finding (Gitea had already added the required
X-Tenant-Id/X-Actor-Id headers).
- pnpm-lock.yaml: regenerated with pnpm 9.15.9, not merged.
Also corrects CLAUDE.md's claim that the banned services/scrapers stack
was "since removed" -- the removal PR is open, not merged, and this
branch inherits that code from Gitea main. Nothing here resurrects,
extends or re-wires any of it; services/scrapers/ is byte-identical to
Gitea main.
Verified on the merged tree: typecheck (src/ + tests/) clean;
17 test files, 121 passed, 1 skipped (the realinfra test skipping
honestly on ST0RE HTTP 401), 0 failures. Gitea main baseline was the
same 121/1; GitHub main baseline could not even `pnpm install`
("packages field missing or empty") -- Gitea's pnpm-workspace.yaml fix
resolves that. Whole-repo `typecheck:app` reports 9 errors, an identical
set to Gitea main's baseline, so it is advisory in both CI workflows
rather than deleted. Lint: 16 problems, identical to Gitea baseline.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs: correct stale quality-check-service.ts comment references
Two source comments still cited src/services/quality-check-service.ts as an
existing precedent. It doesn't exist anywhere in this repo's history (git
log --all: zero commits, any branch, ever) -- confirmed during the earlier
full-doc-corpus accuracy pass. Points each comment at the real precedent
instead (document-extraction-service.ts / iceberg-write-service.ts).
Applied from task_05ec36aa, spun off during that pass.
* fix: 17 confirmed bugs from full-repo audit (edge + Python engine)
Found by an 8-finder bug hunt across app/api, src/services, src/clients,
frontend, the Python engine, the data layer, config/infra, and tests, then
independently adversarially verified per finding (32 raw findings -> 27
confirmed real bugs; 5 refuted). This commit applies the 17 confirmed as
safe, mechanical fixes -- no architecture/design decision required. The
other 10 need a decision and are reported separately, not touched here.
Most consequential, app/api/payments/[order_id]/deposit/route.ts:
the client-supplied Origin request header was trusted unsanitized both as
a server-side fetch target (SSRF) and as Stripe Checkout success/cancel
redirect URLs (open redirect tied to a real payment flow), and a failed
order lookup silently fell through to a hardcoded deposit amount instead
of 404. Fixed by reading the order directly via lib/orders-store.ts
(mirroring the sibling release/route.ts, which already did this correctly)
instead of both trusting the Origin header and self-issuing an HTTP fetch.
Also fixed:
- 5 routes (orders, orders/[id], estimates, subscriptions, ai/parse) that
500'd instead of 400ing on a literal JSON `null` body, since req.json()
resolves (not rejects) for that input and bypasses the try/catch guard.
- app/api/0pt/lakehouse/mappings/{,[mappingId]/{,writes/}}route.ts each
instantiated their own private service instances, so a mapping created
via POST was invisible to the sibling GET and writes routes even before
the separately-tracked dataset-registration gap. Extracted a shared
singleton module, src/services/lakehouse-mapping-registry.ts.
- app/api/0pt/documents/[documentId]/extraction/route.ts collapsed every
error (not-configured, policy-denied, policy-approval-required,
not-found) into a blanket 400; now status-mapped like the sibling
sources/[sourceId]/query route.
- app/api/vehicles/route.ts: plate query param wasn't URL-encoded before
being concatenated into the outbound RDW request, allowing extra query
parameters to be injected; added a strict allowlist + encodeURIComponent.
- app/api/uploads/route.ts: POST returned a fake 200 success with
put_url: null even when "configured", instead of continuing to report
501 until presign generation is actually implemented.
- src/adapters/document-intelligence-adapter.ts: the result-polling loop
never checked response.ok, so a non-2xx poll response (auth expiry,
429, 5xx) was silently retried until a generic timeout masked the real
error.
- app/order/[id]/page.tsx: finish() never cleared the production-simulation
setInterval, so it kept firing (and re-rendering) forever after delivery.
- app/admin/page.tsx: the metrics fetch had no response/shape validation
(unlike the adjacent orders fetch), rendering literal "undefined" stat
tiles on a malformed response instead of falling back to <ErrorState/>.
- services/api/workers/pipeline.py: `config.get("seeds") or DEMO_SEEDS`
used truthy-or, so an explicit empty seeds dict was silently replaced
with hardcoded demo org numbers instead of being respected as empty.
- services/api/workers/resolve.py: corroboration (confidence/agreed_sources
bump) no longer fires for two hits from the same source under fuzzy
match -- only a genuinely independent source now counts, per docs/06's
own ">=2 independent sources agree" definition -- and a merged record's
provenance array now actually includes the corroborating source's
fields instead of only a bare source-name string.
- services/api/orchestrator/main.py: create_order's int(...) on
unvalidated client volume/confidence fields raised on any
decimal-formatted input (int("92.5") -> ValueError); now coerced via
float() first with a safe default. /vehicles/{plate} and /screen also
now catch connector exceptions and degrade gracefully, matching
pipeline.py's existing pattern for the same connectors.
- lib/orders-store.ts: postgres.js (pinned v3.4.9) returns bigint/numeric
columns as strings by default; getOrder/listOrders/setStatus were
silently handing callers stringified money fields. Added a narrow
per-row coercion instead of touching the shared client's global type
map (which would affect unrelated tables' bigint/numeric columns too).
- lib/subscriptions-store.ts: the in-memory fallback returned
oldest-first while the Postgres branch explicitly orders newest-first;
now consistent, mirroring the identical fix already in orders-store.ts.
- app/api/auth/login/route.ts: DEV_LOGIN_KEY was compared with a plain
!==, a non-constant-time comparison; replaced with crypto.timingSafeEqual.
Verified before commit: tsc --noEmit clean; vitest run 88 passed, 1 todo,
0 failures (13 files, unchanged from before this commit -- no existing
test's assertions needed to change). services/api/workers/resolve.py's
source-novelty and provenance-merge logic additionally verified by direct
execution against synthetic same-source/cross-source inputs (no Python
test suite exists in this repo to add to). services/api/workers/pipeline.py
and services/api/orchestrator/main.py verified with python -m py_compile
(no httpx/fastapi installed in this environment to run them directly).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(engine): treat explicit seeds:null as missing to avoid 500 in run_job
Four-eyes follow-up to this PR's own seeds change. Switching from
`config.get('seeds') or DEMO_SEEDS` to `config['seeds'] if 'seeds' in config`
correctly stopped an explicit empty {} from being backfilled with DEMO_SEEDS,
but regressed on client-controlled `{"seeds": null}`: seeds became None and
the downstream `seeds.get(market, [])` raised AttributeError -> 500. That path
is reachable (POST /orders stores raw body.config; POST /orders/{id}/run passes
it straight to run_job). Using `config.get('seeds') is not None` keeps the
intended behaviour (explicit {} honoured, never silently seeded with demo data)
while restoring the None-tolerance the old idiom had.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Real, tested implementation of docs/11-lakehouse-write-plan.md's items 2-8
(the "future implementation session" that doc named): 0pt is now the
canonical writer of Iceberg tables through ST0RE's real, shipped
/v1/st0re/lakehouse/* table registry (sax3l/st0re commit 9ea0abb), never
persisting lakehouse table/catalog/snapshot state itself.
- src/types + src/services for 0pt.dataset_iceberg_mapping.schema.v1
(dataset -> ST0RE tableId reference) and 0pt.source_rate_policy.schema.v1
(the sourceRatePolicy ownership repair, moved out of N0D per ADR-0030).
- src/clients/st0re-lakehouse-client.ts: new client matching ST0RE's actual
camelCase route contract, with the same mock/HTTP boundary convention as
this repo's existing cl0ud/n0d/aud0/st0re clients. Honest caveat in its
module doc comment: not verified against a live ST0RE instance in this
pass (no reachable ST0RE_URL); every test exercises the deterministic mock.
- src/services/iceberg-write-service.ts: registers tables/snapshots through
ST0RE, requests N0D jobs only for heavy writes (never executes them
in-process), emits all four AUD0 event names
(dataset_iceberg_table_mapped / iceberg_write_requested /
iceberg_snapshot_created / source_rate_policy_created).
- sql/004, packages/schemas/*.json, app/api/0pt/lakehouse + sources/rate-policies
routes, tests/p0/*.test.ts (154 passed + 1 honest it.todo for the live-ST0RE
case, up from 142 baseline, 0 failures).
- Reconciled a real collision: an earlier untracked draft schema/test pair
(tests/p0/enterprise-data-os-expansion.test.ts) asserted a different,
now-superseded source_rate_policy shape; updated that one test block only.
- docs/ICEBERG_WRITE_BOUNDARY.md states the ownership boundary plainly and
is explicit throughout about what's real-and-tested vs. contract-only.
docs/11-lakehouse-write-plan.md gets a status-update section, not a rewrite.
- Data quality rules (quality-check-service.ts, quality-report-service.ts)
were not touched -- confirmed still 0pt-owned.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Three new, tenant-agnostic P0 capabilities:
- 0pt.query_source: bounded, timeout/row-capped ad-hoc SQL query execution
against registered database sources, via a Fabric/OneLake adapter
(Azure AD credential-based mssql connection), with AUD0 audit events
and an app/api/0pt/sources/[sourceId]/query endpoint.
- 0pt.document_ocr (document capture + extraction): mailbox-based document
intake via Microsoft Graph, OCR/field extraction via Azure Document
Intelligence, with capture/extraction API endpoints and services that
never fabricate extracted values.
- 0pt.write_external_record (external writeback): a generic writeback-adapter
contract plus a concrete Odoo bills adapter that matches partners via
Fabric, backing an external-writeback-service for pushing records to
downstream systems of record.
Known interim gaps:
- src/adapters/local-blob-store.ts is an interim ST0RE substitute pending
the real ST0RE artifact-store integration.
- mssql is used by the Fabric/OneLake adapter but deliberately not yet
added to package.json (pending `pnpm add mssql` once the concurrent
session's package.json/dependency changes settle).
A separate, concurrently-active session has unrelated in-flight changes
in this same working tree (data-platform catalog/lineage/feature-store,
dataset quality, lakehouse/Iceberg docs, deploy scaffolding, package.json/
pnpm-lock.yaml updates, etc.) — none of that is touched or included here.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Missing GET caused a 405 → r.json() threw → error state shown instead of
the calm 'storage not connected' info note. Now returns 501+[] so the page
properly shows storageOff and 'No files yet.' empty state.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Catalog IDs: synced /api/catalog and /katalog/[id] to consistent English slugs
(nordic-companies, tech-decisionmakers, vehicles-fleet, financial-series,
buying-signals, sanctions-pep); View links from catalog page were all broken
- Added 2 missing catalog detail entries: buying-signals, sanctions-pep
- featured-products-section: link cards to /katalog/{id} instead of hardcoded demo URL
- hero-section: 'See a dataset' now links to /katalog/nordic-companies
- layout.tsx: mount <Toaster> (sonner was installed but never mounted)
- bestall/forskott: pay() tries Stripe checkout first, falls back to demo mode
- api/payments/[order_id]/deposit: Stripe Checkout Session creation (501 if unconfigured)
- app/not-found.tsx: custom 404 page matching mono design system
- philosophy-section: fix TS type error on PRINCIPLES array
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
lib/services (11 categories: data, scraping, mapping, normalization, consolidation->CRM/ERP, documents/OCR, enrichment, ICP, ML, BI, feeds across the 3 flows); /tjanster catalog; /bestall/beskriv NL intake; /api/ai/parse (engine LLM or heuristic NL->config); CRM/ERP destinations; Tjänster in nav. Sells every custom data service through the platform. tsc 0, build green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Edge /api/vehicles (RDW NL, live now) + /api/screen (OpenSanctions, live now), both defer to engine when ENGINE_URL set. Engine: /vehicles/{plate} + /screen endpoints; workers/resolve (blocking+fuzzy entity resolution, now used by pipeline); workers/queue (in-process job queue); sources/financials (Companies House accounts + XBRL path). py_compile + tsc + next build green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
All nav targets now exist on PortalShell; env-guarded actions degrade gracefully. tsc 0, build green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Env-guarded so build stays green: /api/payments/[order_id]/deposit+release + /api/webhooks/stripe (lazy Stripe, 50/50 escrow, 501 until keys); /api/auth login|me|logout (HMAC signed-session scaffold, guarded dev login); lib/engine bridges edge->FastAPI (estimates defers to ENGINE_URL when set); components/embed for NocoDB/Metabase signed URLs; .env.example full inventory. tsc 0, build green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lib/db (env-guarded postgres-js), lib/orders-store (createOrder/getOrder/listOrders/setStatus, DB-or-memory), /api/orders (POST create+estimate, GET list), /api/orders/[id] (GET, PATCH status with escrow-safe state machine). Activates fully on DATABASE_URL/POSTGRES_URL.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
admin
Claude