11 Commits

Include renames

Author	SHA1	Message	Date
adminandClaude Sonnet 5	15a0dbe3b1	fix(security): require auth on GET /api/orders and GET /api/admin/metrics ... A live compliance audit against 0pt.app confirmed both endpoints returned HTTP 200 with zero authentication -- the full order queue and financial pipeline aggregates (deposit/pipeline totals) were readable by anyone. Root cause: middleware.ts's matcher only covers /api/0pt/:path*, by design ("does NOT touch ... the unrelated payment/order APIs"), so nothing upstream of these two handlers ever verified the caller. Fix: both GET handlers now call the same fail-closed Zitadel verifier /api/0pt/* relies on (resolveAuthContext, src/auth/context.ts) directly, via a new thin wrapper (src/auth/require-route-auth.ts) that maps a rejected decision to the same 401/403 JSON shape middleware.ts already returns. The wrapper re-verifies the bearer token itself rather than trusting any x-0pt-verified-* header on the request, since middleware never runs in front of these routes to overwrite a client-supplied one. POST /api/orders (anonymous draft/quote creation before the deposit-payment step) is intentionally left as-is -- only the read paths that leaked data are gated. Tracked as part of docs/compliance/BACKLOG.md P0-3. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-08-11 04:06:13 +02:00
SIAX Ops	44e83ad1de	resolve: README adopts main (newer, fail-closed auth docs)	2026-08-09 11:30:18 +02:00
Claude (SIAX platform session)andClaude Opus 5	bcf285f057	merge: unify GitHub main into Gitea main (0pt) ... Merges github/main (6 commits: 17-bug audit fix, CI quality gate, .env.example + .gitignore fix, missing platform clients + registry services, docs accuracy overhaul) into gitea/main (8 commits: Iceberg write boundary + live ST0RE verification, lakehouse mapping routes, Waves 1-4 foundational hardening). 16 conflicts. Both sides had independently written the four src/clients/*.ts and the two registry services, so those are combined rather than picked: - st0re-client.ts: GitHub's endpoint wins on evidence. POST goes to /v1/st0re/artifacts/register, not /v1/st0re/artifacts -- verified in sax3l/st0re src/api/routes/artifacts.ts, where the bare path is a GET-only list route. Gitea's URL normalisation, error-body text and mock kept. - n0d-client.ts: field is `status`, not `state`, with N0D's real 13-state union -- verified in sax3l/n0d packages/schemas/n0d.job.schema.json, which says explicitly not to add "succeeded". Gitea's mock echo, URL normalisation and error-body text kept. - cl0ud-client.ts: Gitea's fail-closed HTTP normalisation kept (a 200 with no `decision` reads as "deny", not as a silent allow). GitHub's PolicyDecision/PolicyDecisionValue naming kept; PolicyResult retained as an alias. - aud0-client.ts: permissive union of both input types (optional payload/metadata/correlation_id), GitHub's echo return, Gitea's URL normalisation and error-body text, plus a JSON-parse fallback so a successful POST with an empty body does not throw. - dataset-service.ts / source-service.ts: GitHub's record/DTO split and event naming (0pt.source_created / 0pt.create_source, which the merged docs already reference) plus Gitea's name validation, description / created_by fields and actor_id derivation. List filters are the union. - lakehouse mapping routes: GitHub's shared lakehouse-mapping-registry singletons (without which every write 400'd "mapping not found") AND Gitea's Zod parseBody validation. Both, not either. - src/types/source.ts, src/types/dataset.ts: unions of both vocabularies; GitHub's `access` legal-sourcing field kept. - .env.example: GitHub's audited inventory plus Gitea's Zitadel / AUTH_ENFORCE block. - CLAUDE.md, docs/ICEBERG_WRITE_BOUNDARY.md: GitHub's corrections kept, Gitea's live-verification kept and marked as superseding the "would 400 outright" finding (Gitea had already added the required X-Tenant-Id/X-Actor-Id headers). - pnpm-lock.yaml: regenerated with pnpm 9.15.9, not merged. Also corrects CLAUDE.md's claim that the banned services/scrapers stack was "since removed" -- the removal PR is open, not merged, and this branch inherits that code from Gitea main. Nothing here resurrects, extends or re-wires any of it; services/scrapers/ is byte-identical to Gitea main. Verified on the merged tree: typecheck (src/ + tests/) clean; 17 test files, 121 passed, 1 skipped (the realinfra test skipping honestly on ST0RE HTTP 401), 0 failures. Gitea main baseline was the same 121/1; GitHub main baseline could not even `pnpm install` ("packages field missing or empty") -- Gitea's pnpm-workspace.yaml fix resolves that. Whole-repo `typecheck:app` reports 9 errors, an identical set to Gitea main's baseline, so it is advisory in both CI workflows rather than deleted. Lint: 16 problems, identical to Gitea baseline. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-07-28 12:38:01 +02:00
Claude (SIAX platform session)	1c1e2cb649	fix(auth): make the AUTH_ENFORCE default fail-closed on /api/0pt/* ... The auth seam read `process.env.AUTH_ENFORCE === 'true'`, so an absent env var meant NO authentication: any fresh container, local run, DR restore or redeploy that dropped the variable served /api/0pt/* anonymously, taking the tenant from the caller-supplied X-Tenant-Id header. 0pt.app is deployed and was answering anonymously (verified: an invalid Bearer token plus an arbitrary X-Tenant-Id returned 200), so this was a live hole, not latent. * authEnforced() now returns true unless AUTH_ENFORCE is explicitly "false" (trimmed, case-insensitive). Unset/empty/misspelled => ENFORCED. * assertAuthConfigSafe() throws AuthMisconfigurationError when NODE_ENV=production and enforcement resolved off, naming AUTH_ENFORCE. It runs at middleware module load (startup) and on the env-driven request path, so a production instance can neither boot nor serve a single unauthenticated request with auth switched off. * .env.example no longer teaches AUTH_ENFORCE=false; README's Deploy section states the fail-closed posture (no DEPLOY.md in this repo). * 7 new tests, incl. the regression case: with AUTH_ENFORCE unset and no `enforce` option, an unauthenticated request is rejected with 401. No test was inverted or deleted -- none asserted the open default; the three enforce:false cases now describe the explicit local-dev escape hatch. Tests: 121 passed / 1 skipped before, 128 passed / 1 skipped after. Typecheck and build clean; advisory lint unchanged (16 pre-existing problems). Scope: auth default only. The matcher still covers /api/0pt/* alone, and the handlers still take tenant_id from the query/body rather than the verified principal -- both described in the PR body, neither fixed here.	2026-07-27 13:28:24 +02:00
SIAX Foundation BotandClaude Opus 4.8	1d76d50f3d	feat(auth,foundation): Wave 2 flag-gated Zitadel JWT verify + validation + eslint + tests ... Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-25 14:32:34 +02:00
SIAX Foundation Bot	844220d9f5	feat(foundation): Wave 3 integrity/persistence/build fixes	2026-07-25 13:54:36 +02:00
adminandClaude Opus 4.8	e6d3101501	fix: 17 confirmed bugs from full-repo audit (edge + Python engine) (#5) ... * docs: correct stale quality-check-service.ts comment references Two source comments still cited src/services/quality-check-service.ts as an existing precedent. It doesn't exist anywhere in this repo's history (git log --all: zero commits, any branch, ever) -- confirmed during the earlier full-doc-corpus accuracy pass. Points each comment at the real precedent instead (document-extraction-service.ts / iceberg-write-service.ts). Applied from task_05ec36aa, spun off during that pass. * fix: 17 confirmed bugs from full-repo audit (edge + Python engine) Found by an 8-finder bug hunt across app/api, src/services, src/clients, frontend, the Python engine, the data layer, config/infra, and tests, then independently adversarially verified per finding (32 raw findings -> 27 confirmed real bugs; 5 refuted). This commit applies the 17 confirmed as safe, mechanical fixes -- no architecture/design decision required. The other 10 need a decision and are reported separately, not touched here. Most consequential, app/api/payments/[order_id]/deposit/route.ts: the client-supplied Origin request header was trusted unsanitized both as a server-side fetch target (SSRF) and as Stripe Checkout success/cancel redirect URLs (open redirect tied to a real payment flow), and a failed order lookup silently fell through to a hardcoded deposit amount instead of 404. Fixed by reading the order directly via lib/orders-store.ts (mirroring the sibling release/route.ts, which already did this correctly) instead of both trusting the Origin header and self-issuing an HTTP fetch. Also fixed: - 5 routes (orders, orders/[id], estimates, subscriptions, ai/parse) that 500'd instead of 400ing on a literal JSON `null` body, since req.json() resolves (not rejects) for that input and bypasses the try/catch guard. - app/api/0pt/lakehouse/mappings/{,[mappingId]/{,writes/}}route.ts each instantiated their own private service instances, so a mapping created via POST was invisible to the sibling GET and writes routes even before the separately-tracked dataset-registration gap. Extracted a shared singleton module, src/services/lakehouse-mapping-registry.ts. - app/api/0pt/documents/[documentId]/extraction/route.ts collapsed every error (not-configured, policy-denied, policy-approval-required, not-found) into a blanket 400; now status-mapped like the sibling sources/[sourceId]/query route. - app/api/vehicles/route.ts: plate query param wasn't URL-encoded before being concatenated into the outbound RDW request, allowing extra query parameters to be injected; added a strict allowlist + encodeURIComponent. - app/api/uploads/route.ts: POST returned a fake 200 success with put_url: null even when "configured", instead of continuing to report 501 until presign generation is actually implemented. - src/adapters/document-intelligence-adapter.ts: the result-polling loop never checked response.ok, so a non-2xx poll response (auth expiry, 429, 5xx) was silently retried until a generic timeout masked the real error. - app/order/[id]/page.tsx: finish() never cleared the production-simulation setInterval, so it kept firing (and re-rendering) forever after delivery. - app/admin/page.tsx: the metrics fetch had no response/shape validation (unlike the adjacent orders fetch), rendering literal "undefined" stat tiles on a malformed response instead of falling back to <ErrorState/>. - services/api/workers/pipeline.py: `config.get("seeds") or DEMO_SEEDS` used truthy-or, so an explicit empty seeds dict was silently replaced with hardcoded demo org numbers instead of being respected as empty. - services/api/workers/resolve.py: corroboration (confidence/agreed_sources bump) no longer fires for two hits from the same source under fuzzy match -- only a genuinely independent source now counts, per docs/06's own ">=2 independent sources agree" definition -- and a merged record's provenance array now actually includes the corroborating source's fields instead of only a bare source-name string. - services/api/orchestrator/main.py: create_order's int(...) on unvalidated client volume/confidence fields raised on any decimal-formatted input (int("92.5") -> ValueError); now coerced via float() first with a safe default. /vehicles/{plate} and /screen also now catch connector exceptions and degrade gracefully, matching pipeline.py's existing pattern for the same connectors. - lib/orders-store.ts: postgres.js (pinned v3.4.9) returns bigint/numeric columns as strings by default; getOrder/listOrders/setStatus were silently handing callers stringified money fields. Added a narrow per-row coercion instead of touching the shared client's global type map (which would affect unrelated tables' bigint/numeric columns too). - lib/subscriptions-store.ts: the in-memory fallback returned oldest-first while the Postgres branch explicitly orders newest-first; now consistent, mirroring the identical fix already in orders-store.ts. - app/api/auth/login/route.ts: DEV_LOGIN_KEY was compared with a plain !==, a non-constant-time comparison; replaced with crypto.timingSafeEqual. Verified before commit: tsc --noEmit clean; vitest run 88 passed, 1 todo, 0 failures (13 files, unchanged from before this commit -- no existing test's assertions needed to change). services/api/workers/resolve.py's source-novelty and provenance-merge logic additionally verified by direct execution against synthetic same-source/cross-source inputs (no Python test suite exists in this repo to add to). services/api/workers/pipeline.py and services/api/orchestrator/main.py verified with python -m py_compile (no httpx/fastapi installed in this environment to run them directly). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(engine): treat explicit seeds:null as missing to avoid 500 in run_job Four-eyes follow-up to this PR's own seeds change. Switching from `config.get('seeds') or DEMO_SEEDS` to `config['seeds'] if 'seeds' in config` correctly stopped an explicit empty {} from being backfilled with DEMO_SEEDS, but regressed on client-controlled `{"seeds": null}`: seeds became None and the downstream `seeds.get(market, [])` raised AttributeError -> 500. That path is reachable (POST /orders stores raw body.config; POST /orders/{id}/run passes it straight to run_job). Using `config.get('seeds') is not None` keeps the intended behaviour (explicit {} honoured, never silently seeded with demo data) while restoring the None-tolerance the old idiom had. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-22 18:52:19 +02:00
Simon AxelssonandClaude Opus 5	01cc225ffc	fix: add missing 0pt platform clients + registry services, wire test/typecheck scripts ... The 0pt data-engine build has been broken on main since 2026-07-08 (commit c897311). document-capture-service.ts, document-extraction-service.ts, source-query-service.ts, external-writeback-service.ts (and, more recently, dataset-iceberg-mapping-service.ts, iceberg-write-service.ts, source-rate-policy-service.ts) all import six platform client modules and two registry services that were never committed: src/clients/{aud0,cl0ud,n0d, st0re}-client.ts, src/services/{dataset,source}-service.ts, plus src/types/{dataset,source}.ts. `next build` failed with 17 "Module not found" errors, and every route under app/api/0pt/* returned HTTP 500 in dev with the Turbopack compile error leaked straight into the JSON response body. This adds the missing files, following the same fetch-based client shape already established by src/clients/st0re-lakehouse-client.ts (thin interface, HTTP implementation, deterministic in-memory mock used when no base URL is configured) and the same "pure core, thin in-memory registry, AUD0-evented mutation" shape the existing SourceRatePolicyService/DatasetIcebergMappingService already use for DatasetService/SourceService. Each new client mock is documented as a mock/degrade path, never a fabricated success, per this repo's CLAUDE.md honesty doctrine. Also wires up "typecheck" and "test" npm scripts (tsc --noEmit / vitest run) and adds vitest as a devDependency -- previously `pnpm test` failed immediately with ERR_PNPM_NO_SCRIPT even though 13 test files (~89 cases) already existed under tests/p0/. Verified: `pnpm run typecheck` clean, `pnpm run test` 88 passed / 1 todo (pre-existing live-ST0RE-only skip), `pnpm run build` exits 0 (39/39 routes compiled, only the pre-existing intentional optional `mssql` import warning remains). Re-verified live on port 4097: GET /api/0pt/sources/rate-policies, GET /api/0pt/lakehouse/mappings, POST /api/0pt/documents/capture, GET /api/0pt/documents/[id]/extraction, and both GET/POST /api/0pt/sources/[id]/query now return real application responses (200/400/405) instead of HTTP 500 with a leaked compile error. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-07-14 01:16:50 +02:00
adminandClaude Sonnet 5	bda21e87eb	Live-verify Iceberg write path against ST0RE + close field/contract gaps ... Re-runs this session's task spec against 0pt's actual repo state, which turned out to already have a substantial, tested implementation from a prior pass (35b6ec6): dataset<->Iceberg-table mapping, source rate policy, and an ST0RE lakehouse client that was honestly marked "not live-verified". This pass found api.st0re.siax.io IS reachable from this environment and used it -- first via curl to safely discover the real contract, then through 0pt's own updated code -- to drive a real namespace -> table -> snapshot write against production ST0RE (task item 3's "write ONE first test dataset" proof). That surfaced and fixed three real bugs the mock-only pass could not have caught: - ST0RE requires X-Tenant-Id/X-Actor-Id headers on every lakehouse route; the previous HTTP client sent none. Fixed via a threaded St0reRequestContext. - IcebergTableInput.datasetId is a foreign key into ST0RE's OWN dataset table, not a free-form string -- sending 0pt's dataset_id 400s live with a Postgres FK violation. Fixed by no longer sending it; 0pt's dataset<->table linkage lives entirely in DatasetIcebergMapping. - ST0RE enforces a real unique constraint on lakehouse namespaces (tenant_id, project_id, name) with no upsert route. Fixed with a client-side lookup-or-create (listNamespaces + findOrRegisterNamespace). Also closed a field-level gap against the task's literal schema spec: 0pt.dataset_iceberg_mapping.schema.v1 now carries namespace_id (a real reference to ST0RE's LakehouseNamespace.namespaceId, not guessed) and mapped_at, both additive alongside the existing namespace/created_at fields (sql/005, JSON schema, types, service). tests/p0/iceberg-write-service.realinfra.test.ts replaces the prior it.todo with a real SEAM-pattern live test: attempts the real HTTP call, skips honestly if ST0RE is unreachable, never fabricates success. It ran successfully against production ST0RE this session (real namespace/table/ snapshot UUIDs logged in docs/ICEBERG_WRITE_BOUNDARY.md). Two ST0RE-side cross-tenant read bugs were found and flagged (not fixed here -- out of scope for this repo): GET .../namespaces ignores its own tenant header and query filters; GET .../tables/:id doesn't check the caller's tenant against the table's owner. npm test: 155 passed, 0 failed, 0 todo (up from 154 passed + 1 todo). npx tsc --noEmit: 1 pre-existing, unrelated error in quality-report-service.ts (present before this pass, part of the wider uncommitted P0 data-platform build already in this repo) -- not touched. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-13 00:18:10 +02:00
adminandClaude Sonnet 5	35b6ec63bf	Add 0pt Iceberg write-path ownership: dataset<->table mapping + source rate policy ... Real, tested implementation of docs/11-lakehouse-write-plan.md's items 2-8 (the "future implementation session" that doc named): 0pt is now the canonical writer of Iceberg tables through ST0RE's real, shipped /v1/st0re/lakehouse/* table registry (sax3l/st0re commit 9ea0abb), never persisting lakehouse table/catalog/snapshot state itself. - src/types + src/services for 0pt.dataset_iceberg_mapping.schema.v1 (dataset -> ST0RE tableId reference) and 0pt.source_rate_policy.schema.v1 (the sourceRatePolicy ownership repair, moved out of N0D per ADR-0030). - src/clients/st0re-lakehouse-client.ts: new client matching ST0RE's actual camelCase route contract, with the same mock/HTTP boundary convention as this repo's existing cl0ud/n0d/aud0/st0re clients. Honest caveat in its module doc comment: not verified against a live ST0RE instance in this pass (no reachable ST0RE_URL); every test exercises the deterministic mock. - src/services/iceberg-write-service.ts: registers tables/snapshots through ST0RE, requests N0D jobs only for heavy writes (never executes them in-process), emits all four AUD0 event names (dataset_iceberg_table_mapped / iceberg_write_requested / iceberg_snapshot_created / source_rate_policy_created). - sql/004, packages/schemas/*.json, app/api/0pt/lakehouse + sources/rate-policies routes, tests/p0/*.test.ts (154 passed + 1 honest it.todo for the live-ST0RE case, up from 142 baseline, 0 failures). - Reconciled a real collision: an earlier untracked draft schema/test pair (tests/p0/enterprise-data-os-expansion.test.ts) asserted a different, now-superseded source_rate_policy shape; updated that one test block only. - docs/ICEBERG_WRITE_BOUNDARY.md states the ownership boundary plainly and is explicit throughout about what's real-and-tested vs. contract-only. docs/11-lakehouse-write-plan.md gets a status-update section, not a rewrite. - Data quality rules (quality-check-service.ts, quality-report-service.ts) were not touched -- confirmed still 0pt-owned. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-12 19:46:47 +02:00
adminandClaude Sonnet 5	c89731115c	feat: add 0pt.query_source, 0pt.document_ocr, 0pt.write_external_record capabilities ... Three new, tenant-agnostic P0 capabilities: - 0pt.query_source: bounded, timeout/row-capped ad-hoc SQL query execution against registered database sources, via a Fabric/OneLake adapter (Azure AD credential-based mssql connection), with AUD0 audit events and an app/api/0pt/sources/[sourceId]/query endpoint. - 0pt.document_ocr (document capture + extraction): mailbox-based document intake via Microsoft Graph, OCR/field extraction via Azure Document Intelligence, with capture/extraction API endpoints and services that never fabricate extracted values. - 0pt.write_external_record (external writeback): a generic writeback-adapter contract plus a concrete Odoo bills adapter that matches partners via Fabric, backing an external-writeback-service for pushing records to downstream systems of record. Known interim gaps: - src/adapters/local-blob-store.ts is an interim ST0RE substitute pending the real ST0RE artifact-store integration. - mssql is used by the Fabric/OneLake adapter but deliberately not yet added to package.json (pending `pnpm add mssql` once the concurrent session's package.json/dependency changes settle). A separate, concurrently-active session has unrelated in-flight changes in this same working tree (data-platform catalog/lineage/feature-store, dataset quality, lakehouse/Iceberg docs, deploy scaffolding, package.json/ pnpm-lock.yaml updates, etc.) — none of that is touched or included here. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-08 19:32:43 +02:00