* docs: full accuracy overhaul, grounded claim-by-claim in real code Every doc in this repo (CLAUDE.md, README.md, SECURITY.md, docs/*.md) was audited and rewritten by a multi-pass process: independent rewrite per cluster, adversarial re-verification of that rewrite against the actual code, then a final cross-doc consistency pass — each stage re-checking claims against the repo directly rather than trusting the previous stage. Two corrections matter most: 1. docs/ICEBERG_WRITE_BOUNDARY.md, docs/11-lakehouse-write-plan.md, and docs/P0_QUERY_SOURCE_FLOW.md asserted src/services/quality-check-service.ts, quality-report-service.ts, src/data-platform/services/catalog-service.ts, and sql/002_p0_dataset_quality.sql/sql/003_*.sql as real, tested code. git log --all across every branch shows zero commits for any of these paths, ever. 0pt's data-quality/QA enforcement does not exist in code. Corrected everywhere, with a new docs/P0_DATASET_QUALITY_FLOW.md added as an explicit target-spec (not-yet-built) replacement. 2. github.com/sax3l/0pt (this repo) and git.siax.io/sax3l/0pt (Gitea) are two diverged remotes sharing part of the same history (services/api/ workers/adaptive_router.py is byte-identical between them). Gitea's main independently accumulated a WAF-bypass/proxy-rotation scraper stack in three 2026-07-13 commits, since removed via a Gitea PR; GitHub's main never had it. Which remote is canonical/deployed is unresolved — flagged consistently across the corpus, not decided here. Also newly and consistently documented: the tenant_id/project_id-read-from- client-body gap on the P0 lakehouse/rate-policy/source-query routes (no auth/session system exists anywhere in this repo yet); that every CL0UD/ N0D/AUD0/ST0RE platform client is real but mock-only (zero process.env reads for any of their *_URL vars); that two P0 mutation routes (lakehouse/mappings, sources/rate-policies) always fail over HTTP today because their prerequisite entity can never be created via any route; a schema bug in sql/004 (FK references to opt_datasets/opt_sources, which don't exist); and that 0pt.app's live deployment was independently confirmed (curled this pass) to be running this exact codebase with DATABASE_URL/REDIS_URL unset in production. New file: docs/12-edge-capability-api.md — the edge-native P0 TypeScript capability contract (app/api/0pt/*, /api/screen, /api/ai/parse), previously undocumented and distinct from docs/04-api.md's FastAPI-engine-only scope. Verified before commit: tsc --noEmit clean; vitest run 88 passed, 1 todo, 0 failures (13 files) — docs-only change, no source touched. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * docs: reflect PR #5's lakehouse-mapping-registry + status-code fixes Follow-up to keep docs/12-edge-capability-api.md accurate once both PRs land: fix/0pt-full-bug-hunt (#5) fixed the shared-instance bug in the lakehouse mapping routes and added proper status-code mapping to the document extraction route. This doc was written against the pre-fix state in the same pass that added it, so it needs the same correction. Does not claim the lakehouse write path fully works now -- the deeper dataset-registration gap (no route ever calls DatasetService.registerDataset) is unchanged and still blocks it end-to-end; only the shared-instance half of the bug is fixed. --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
23 KiB
12 · Edge-native P0 capability API (Next.js / TypeScript)
This doc did not exist before this pass. It covers the edge-native TypeScript API surface — Next.js Route Handlers under app/api/0pt/*, plus app/api/screen/route.ts and app/api/ai/parse/route.ts — as a contract distinct from docs/04-api.md's FastAPI engine (services/api/orchestrator/main.py, runs on a VPS). Everything here runs on Vercel. The two layers are not the same service, and as of this pass are only partially wired together (docs/04-api.md documents one confirmed case where the edge layer calls an engine route that doesn't exist).
Every route below was read directly from source this pass (app/api/0pt/**/route.ts, app/api/screen/route.ts, app/api/ai/parse/route.ts, and their backing src/services/*.ts / src/types/*.ts). Where a claim below is about runtime behavior rather than just the shape of the code (e.g. "this call always fails"), it's derived by tracing the actual code path, not by running it — flagged as such.
Capability-registry status (verified against packages/capability-registry/src/registry.ts in sax3l/cl0ud)
14 distinct 0pt.* capabilities are registered today. Every single one has surfaces containing only 'api' (plus 'job' for the two heaviest writes), and mcpEnabled/cliEnabled are false on all 14, with no exception. Any doc, comment, or assumption that a siax 0pt ... CLI or an MCP tool for 0pt already exists is describing a target, not current reality.
| Capability | Surfaces | Route(s) below | Notes |
|---|---|---|---|
0pt.query_source |
api |
POST /api/0pt/sources/{sourceId}/query |
riskLevel L2, piiLevel high |
0pt.document_ocr |
api |
POST/GET /api/0pt/documents/capture, POST/GET /api/0pt/documents/{documentId}/extraction |
riskLevel L2, piiLevel high |
0pt.write_external_record |
job (not api) |
none — no HTTP route exists | see "Capabilities with no route" below |
0pt.lakehouse_mapping_manage |
api |
POST/GET /api/0pt/lakehouse/mappings, GET .../{mappingId} |
riskLevel L2 |
0pt.lakehouse_write |
api, job |
POST /api/0pt/lakehouse/mappings/{mappingId}/writes |
riskLevel L3 |
0pt.source_rate_policy_manage |
api |
POST/GET /api/0pt/sources/rate-policies, GET .../{id} |
riskLevel L1 |
0pt.screen_sanctions |
api |
GET /api/screen |
riskLevel L2 |
0pt.write_iceberg_table |
api, job |
(invoked internally by 0pt.lakehouse_mapping_manage/0pt.lakehouse_write routes via IcebergWriteService) |
riskLevel L3, piiLevel high |
0pt.estimate_compute, 0pt.vehicle_lookup, 0pt.catalog_list, 0pt.source_registry_list, 0pt.order_create, 0pt.order_payment |
api |
app/api/estimates, app/api/vehicles, app/api/catalog, app/api/sources, app/api/orders, app/api/payments/* |
not under app/api/0pt/ — out of this doc's assigned scope, not read in detail this pass |
POST /api/ai/parse has no entry at all in the capability registry (grep-verified: no ai/parse or ai_parse or ai.parse string anywhere in registry.ts). It is real, working code, but sits entirely outside the capability model — no declared risk level, no audit requirement, no policy gate.
Routes
0pt.document_ocr — document capture + extraction
POST /api/0pt/documents/capture — app/api/0pt/documents/capture/route.ts → DocumentCaptureService.captureFromMailbox
body: {
mailbox: { tenant_id, client_id, client_secret, mailbox_address, folder }, -- Microsoft Graph / Entra creds for the mailbox itself
since_iso?, top?, max_pages?,
tenant_id, project_id, actor_id, correlation_id -- SIAX platform scoping (unrelated to mailbox.tenant_id above)
}
-> 201 { scanned_messages, captured_documents: DocumentRecord[], errors: string[] }
-> 400 { error } on missing/invalid mailbox creds or missing tenant_id/project_id/actor_id
Note the naming collision: body.mailbox.tenant_id is the Microsoft 365 / Entra tenant the mailbox lives in; body.tenant_id is the SIAX platform tenant. Same field name, two unrelated identifiers, both required, both plain strings in the request body.
Polls the mailbox via Microsoft Graph (app-only), downloads PDF attachments, writes bytes to LocalBlobStore (src/adapters/local-blob-store.ts — an explicitly-marked interim local-disk store, because st0re-client.ts only registers artifact metadata today and has no real bytes-upload endpoint yet), registers the artifact with ST0RE (mock client — see "Mock-only platform clients" below), and enqueues an N0D job placeholder for extraction.
GET /api/0pt/documents/capture — same file → DocumentCaptureService.listDocuments
query: ?tenant_id=&project_id= (both optional filters)
-> 200 DocumentRecord[]
POST /api/0pt/documents/{documentId}/extraction — app/api/0pt/documents/[documentId]/extraction/route.ts → DocumentExtractionService.extractDocument
body: { tenant_id, project_id, actor_id, correlation_id, model_id }
-> 201 DocumentExtractionRecord
-> 501 { error } if Document Intelligence is not configured (missing DOC_INTELLIGENCE_ENDPOINT/DOC_INTELLIGENCE_KEY) — "refusing to fabricate a result"
-> 403 { error } if CL0UD policy denies
-> 409 { error } if CL0UD policy requires approval
-> 404 { error } if the document doesn't exist
-> 400 { error } for any other failure (e.g. missing model_id)
Corrected in fix/0pt-full-bug-hunt (PR #5, merged separately): previously every one of the above collapsed into a blanket 400, which made "not configured" indistinguishable from "bad request" from the caller's side. Now status-mapped, matching the convention already used by app/api/0pt/sources/[sourceId]/query/route.ts.
Every field in the response is evidence-based: a field is only populated when Azure Document Intelligence actually returned it, always carrying DI's own confidence; not-found fields are {value: null, confidence: ..., found: false}, never guessed. This is enforced twice (adapter + service layer) per the file's own doc comment.
GET /api/0pt/documents/{documentId}/extraction — same file → DocumentExtractionService.listExtractionsForDocument
-> 200 DocumentExtractionRecord[]
-> 404 { error } on any thrown error (including errors unrelated to "not found" — the route maps every exception here to 404)
This pair actually works end-to-end, unlike the routes below — documentCaptureService and documentExtractionService are both exported as module-level singletons (src/services/document-capture-service.ts / document-extraction-service.ts) specifically so the extraction route can see documents the capture route created in the same process. This is the one place in app/api/0pt/* that got the shared-instance pattern right.
0pt.lakehouse_mapping_manage / 0pt.lakehouse_write — Iceberg write path
POST /api/0pt/lakehouse/mappings — app/api/0pt/lakehouse/mappings/route.ts → IcebergWriteService.mapDatasetToIcebergTable
body: { tenant_id, project_id, dataset_id, dataset_iceberg_mapping_id, namespace, table_name, storage_uri, catalog_ref, table_format?, actor_id, correlation_id }
-> 201 { mapping: DatasetIcebergMapping, iceberg_table_id, policy_decision_id }
-> 400 { error }
GET /api/0pt/lakehouse/mappings — same file → DatasetIcebergMappingService.listMappings
query: ?tenant_id=&project_id=&dataset_id=
-> 200 DatasetIcebergMapping[]
GET /api/0pt/lakehouse/mappings/{mappingId} — separate file, .../[mappingId]/route.ts → DatasetIcebergMappingService.getMapping
-> 200 DatasetIcebergMapping
-> 404 { error }
POST /api/0pt/lakehouse/mappings/{mappingId}/writes — separate file, .../[mappingId]/writes/route.ts → IcebergWriteService.requestIcebergWrite
body: { actor_id, correlation_id, snapshot_type?, schema_version, manifest_uri, record_count?, size_bytes?, heavy? }
-> 202 { snapshot_id, iceberg_table_id, policy_decision_id, n0d_job_id? }
-> 400 { error }
Confirmed this pass — the POST above still always fails, though one of its two root causes is now fixed:
mapDatasetToIcebergTable's first line isawait this.datasetService.getDataset(input.dataset_id), which throwsDataset {id} not foundunless the dataset was already registered.DatasetService.registerDatasetis never called from any route in this repo (grep-verified acrossapp/andsrc/) — there is noPOST /api/0pt/datasetsroute, despiteapp/api/0pt/route.ts's own index claiming/api/0pt/datasetsexists (it doesn't — see "The/api/0ptindex is wrong" below). So every call toPOST /api/0pt/lakehouse/mappingsthrowsDataset {id} not found, unconditionally, regardless of input, because no HTTP path can ever get a dataset into the registry first. This half is still open — whether to addPOST /api/0pt/datasets/POST /api/0pt/sourcesroutes is a scope decision, not a mechanical fix.Fixed in
fix/0pt-full-bug-hunt(PR #5, merged separately): the three route files above previously each constructed their own separate instance of the relevant service —.../[mappingId]/route.tshad a second, independentDatasetIcebergMappingService, and.../[mappingId]/writes/route.tshad a thirdIcebergWriteServicewith its own fresh internalDatasetIcebergMappingService, so a mapping created viaPOST /api/0pt/lakehouse/mappingswas invisible to both sibling routes even setting the dataset-registration issue aside. All three route files now import shareddatasetService/mappingService/writeServicesingletons fromsrc/services/lakehouse-mapping-registry.tsinstead. This alone does not make the write path succeed end-to-end — the dataset-registration gap above still blocks it — but it removes a second, independent failure mode.
0pt.source_rate_policy_manage — source rate policies
POST /api/0pt/sources/rate-policies — app/api/0pt/sources/rate-policies/route.ts → SourceRatePolicyService.createSourceRatePolicy
body: CreateSourceRatePolicyInput { source_rate_policy_id, tenant_id, project_id, source_id, rate_limit_per_minute, crawl_budget, compliance_notes?, domain?, robots_ok?, access?, status?, correlation_id?, created_by? }
-> 201 SourceRatePolicy
-> 400 { error }
GET /api/0pt/sources/rate-policies — same file → SourceRatePolicyService.listSourceRatePolicies
query: ?tenant_id=&project_id=&source_id=
-> 200 SourceRatePolicy[]
GET /api/0pt/sources/rate-policies/{sourceRatePolicyId} — separate file → SourceRatePolicyService.getSourceRatePolicy
-> 200 SourceRatePolicy
-> 404 { error }
Confirmed this pass — the POST above always fails too, same root cause as lakehouse mappings:
createSourceRatePolicycheckssourceService.getSource(input.source_id)before creating the policy, and throwsSource {id} not foundotherwise.SourceService.createSourceis never called from any route in this repo — there is noPOST /api/0pt/sourcesroute (see next section). Every call toPOST /api/0pt/sources/rate-policiesthrowsSource {id} not found, unconditionally. The.../{sourceRatePolicyId}GET route also constructs its own separateSourceRatePolicyServiceinstance, independent of.../rate-policies/route.ts's — so even a hypothetically-successful create wouldn't be visible there either.
0pt.query_source — bounded synchronous query
POST /api/0pt/sources/{sourceId}/query — app/api/0pt/sources/[sourceId]/query/route.ts → SourceQueryService.querySource
body: { query (required, non-empty string), params?, tenant_id (required), project_id (required), actor_id?, correlation_id?, max_rows?, timeout_ms?, connection_config (required — FabricOneLakeConnectionConfig: {credentials:{tenantId,clientId,clientSecret}, sqlEndpoint, database, scope?, requestTimeoutMs?}) }
-> 200 { source_id, rows, row_count, truncated, duration_ms, bounds:{max_rows,timeout_ms} }
-> 400 { error } (missing query/tenant_id/project_id/connection_config)
-> 404 { error } (source not found)
-> 422 { error } (SourceQueryUnsupportedError — wrong source_type or adapter_id)
-> 504 { error } (SourceQueryTimeoutError)
The route's own inline comment already documents the in-memory-isolation gap for this route: "a source registered via POST /api/0pt/sources will not be resolvable here until SourceService is backed by shared persistence."
Two things worth adding to what that comment says:
- There is no
POST /api/0pt/sourcesroute in this repo at all. The comment describes a route that does not exist as a file anywhere underapp/api/(verified — onlysources/rate-policies*andsources/[sourceId]/queryexist underapp/api/0pt/sources/; the non-0pt-prefixedapp/api/sources/route.tsis a completely different,GET-only, hardcoded 9-entry list — thedim_sourcetransparency-moat catalog described indocs/03-data-model.mdanddocs/04-api.md, not a per-tenantSourceregistry). So this isn't "not yet resolvable across processes" — it's "there is currently no way, via any HTTP endpoint, to create aSourceat all."SourceService.createSourceis only ever called from tests.- Even with a
Sourceregistered directly in code, the only supported adapter (adapter_id: "fabric_onelake_sql") depends on themssqlnpm package, which is not inpackage.jsonas of this pass (src/adapters/fabric-onelake-adapter.ts's own header comment: deliberately not added yet to avoid colliding with a separate in-flight session'spackage.jsonchanges).query()throws a clear, explicit error untilmssqlis installed — by design, not silently.Net effect:
POST /api/0pt/sources/{sourceId}/querycannot succeed against any HTTP-only workflow today, for two independent, stacked reasons — no way to register aSource, and no query driver installed even if one existed.
0pt.write_external_record — no route exists
src/services/external-writeback-service.ts is real, tested TypeScript (tests/p0/external-writeback-service.test.ts) implementing "POST a validated, already-approved structured record to a registered external-system adapter" (worked example: Odoo Vendor Bill via src/adapters/odoo-bills-adapter.ts). The capability registry itself documents this plainly: surfaces: ['job'], apiEnabled: false, with a comment noting it's "invoked today only via direct import, not yet wired to a live surface." There is no app/api/0pt/* route for it. Not part of this doc's route list because there is no route to document.
The /api/0pt index is materially wrong
GET /api/0pt (app/api/0pt/route.ts) returns a static endpoints array as self-documentation:
["/api/0pt/sources", "/api/0pt/datasets", "/api/0pt/data-contracts",
"/api/0pt/quality/checks", "/api/0pt/quality/reports", "/api/0pt/features/views",
"/api/0pt/documents/capture", "/api/0pt/documents/{documentId}/extraction"]
Of these 8 listed paths, only 2 exist (documents/capture, documents/{documentId}/extraction). The other 6 — sources, datasets, data-contracts, quality/checks, quality/reports, features/views — have no corresponding route file anywhere in this repo. quality/checks and quality/reports in particular line up exactly with the confirmed-fabricated quality-check-service work described elsewhere in this doc corpus (CLAUDE.md → README "Known issues") — this index was apparently written against a planned surface, not the real one. Conversely, 6 routes that do exist are missing from this list entirely: lakehouse/mappings (×3) and sources/rate-policies (×2), plus sources/{sourceId}/query. This file is outside this doc's assigned scope to edit (not one of docs/03-data-model.md/docs/04-api.md) — flagging it here since it's the most visible, single-request way a caller would discover this API's shape, and it's currently misleading.
0pt.screen_sanctions — GET /api/screen
GET /api/screen?q=<name>
-> 200 { matches: [] } if q is empty
-> 200 (engine response, verbatim) if ENGINE_URL is set and reachable and engine returns ok
-> 200 { source: "opensanctions", matches: [{name, score, schema}] } direct OpenSanctions fallback
-> 200 { matches: [], source: "opensanctions" } on fetch failure or non-2xx
Calls engineFetch('/screen?q=...') first (services/api/orchestrator/main.py's real GET /screen route — see docs/04-api.md), falls back to api.opensanctions.org directly otherwise. Unlike /api/ai/parse below, this fallback path is a working reconciliation between the edge and engine layers, not a silent bug — both paths return the same shape. Comment in the route itself: "VERIFY commercial terms before production volume" — not verified by this pass either.
POST /api/ai/parse — no capability mapping, engine call silently no-ops
body: { text }
-> 200 (engine response, verbatim) if ENGINE_URL set, reachable, and engine returns ok
-> 200 { config: {...}, rationale, heuristic: true } local keyword-heuristic fallback otherwise
-> 400 { error } if text is empty
As documented in docs/04-api.md: services/api/orchestrator/main.py has no /ai/parse route at all, so engineFetch("/ai/parse", ...) always 404s and this route always falls through to its local heuristic (MARKETS/SERVICE/FIELDS/DEST regex tables in the route file itself), even when the engine is up and configured. Not a crash — just a permanently-dark code path (if (eng && eng.ok) is never true for this one route). Has no entry in the capability registry (see top of this doc) — no declared risk level, audit requirement, or policy gate, unlike every other route documented above.
Cross-cutting gap 1: tenant_id read from the request body, not a session
Hard platform rule (per this doc corpus's platform-wide conventions): tenant_id must be server-bound to an authenticated session, never read from client request body/query/header. This is violated, today, in:
app/api/0pt/lakehouse/mappings/route.ts(POST bodytenant_id/project_id)app/api/0pt/lakehouse/mappings/[mappingId]/writes/route.ts(indirectly —actor_id/correlation_idfrom body feed into a call keyed off a mapping whose owntenant_idwas set by the equally-unauthenticated POST above)app/api/0pt/sources/rate-policies/route.ts(POST bodytenant_id/project_id)app/api/0pt/sources/[sourceId]/query/route.ts(POST bodytenant_id/project_id, explicitly required — 400 if missing)src/services/document-capture-service.ts(captureFromMailbox'sctx.tenant_idcomes straight fromapp/api/0pt/documents/capture/route.ts's request body; the siblingGETroute also takestenant_id/project_idas query-string filters, which is a softer version of the same problem — it doesn't enforce isolation, it just optionally filters an already-unscoped in-memory list)
This is not a deliberate two-tier design choice — it exists because no authentication or session system exists anywhere in this repo yet (see app/api/orders/route.ts's own GET handler, which has the literal comment "org scoping added with auth" — i.e., not yet; and docs/09-build-plan.md Phase 3, which is where auth is planned, not built). Building that auth design is explicitly Simon's call — this doc doesn't invent one. Until it lands, any caller can claim to be any tenant_id on every route above.
Cross-cutting gap 2: in-memory-only persistence, and worse
Every service backing app/api/0pt/* (SourceService, DatasetService, DatasetIcebergMappingService, SourceRatePolicyService, DocumentCaptureService, DocumentExtractionService) stores state in a plain in-memory Map — nothing survives a process restart, and (per the findings above) in most cases nothing is even shared across route files within the same running process, because each route file constructs its own fresh service instance rather than importing a shared singleton. The one exception is documentCaptureService/documentExtractionService, which are correctly exported as module-level singletons for exactly this reason.
Two SQL tables exist for two of these seven types (opt_dataset_iceberg_mappings, opt_source_rate_policies — sql/004_iceberg_write_boundary.sql), but nothing in app/api/0pt/* reads or writes through Postgres — the routes only ever touch the in-memory Maps. See docs/03-data-model.md for the full SQL-side picture, including a confirmed broken foreign-key reference in that same migration file.
Mock-only platform clients
SourceService, DocumentCaptureService, DocumentExtractionService, IcebergWriteService, and DatasetIcebergMappingService all construct CL0UD/N0D/AUD0/ST0RE clients (src/clients/{cl0ud,n0d,aud0,st0re,st0re-lakehouse}-client.ts) via createXClient(baseUrl?) with no baseUrl ever passed by any route in this repo. createXClient returns a deterministic in-memory mock whenever baseUrl is falsy (verified in each client file), and no route or service anywhere reads CL0UD_URL/N0D_URL/AUD0_URL/ST0RE_URL from process.env (grep-verified this pass, zero hits outside comments and the it.todo in tests/p0/iceberg-write-service.test.ts — this corrects an earlier version of this section, which claimed ST0RE_URL is read by the two ST0RE clients; it is not, matching README's "Known issues" #2, docs/02-architecture.md's "Platform clients" section, docs/08-infrastructure.md's "Platform-client URLs" section, and docs/ICEBERG_WRITE_BOUNDARY.md's "Verification status" section, all independently verified against the same grep). None of the five clients has an env fallback wired at all — every one requires an explicit baseUrl/st0reBaseUrl constructor argument that nothing in this repo ever supplies. Practically: every policy check (cl0ud.evaluatePolicy) always returns whatever the mock's canned decision is, and every audit event (aud0.ingestEvent) is a no-op that never leaves the process. This is real, typechecked, tested code — not a stub file — but it has never been exercised against a live CL0UD/N0D/AUD0/ST0RE instance from this repo.
Open questions for Simon
- Should
app/api/0pt/route.ts's index be fixed to list real routes, or is a stale/aspirational index acceptable short-term? (Not this doc's file to edit.) - Should a
POST /api/0pt/sources(andPOST /api/0pt/datasets) route be added soSource/Datasetregistration is reachable at all over HTTP — this is the blocking prerequisite for0pt.query_source,0pt.source_rate_policy_manage, and0pt.lakehouse_mapping_manageto ever succeed via the API as currently wired? - Should the in-memory registry services (
SourceService,DatasetService, etc.) move to shared singletons (likedocumentCaptureService) as an interim fix, or wait for real Postgres-backed persistence? - Whether/when
mssqlgets added topackage.jsonso thefabric_onelake_sqladapter can actually run.