adminandClaude Sonnet 5
2902a26bac
* docs: full accuracy overhaul, grounded claim-by-claim in real code Every doc in this repo (CLAUDE.md, README.md, SECURITY.md, docs/*.md) was audited and rewritten by a multi-pass process: independent rewrite per cluster, adversarial re-verification of that rewrite against the actual code, then a final cross-doc consistency pass — each stage re-checking claims against the repo directly rather than trusting the previous stage. Two corrections matter most: 1. docs/ICEBERG_WRITE_BOUNDARY.md, docs/11-lakehouse-write-plan.md, and docs/P0_QUERY_SOURCE_FLOW.md asserted src/services/quality-check-service.ts, quality-report-service.ts, src/data-platform/services/catalog-service.ts, and sql/002_p0_dataset_quality.sql/sql/003_*.sql as real, tested code. git log --all across every branch shows zero commits for any of these paths, ever. 0pt's data-quality/QA enforcement does not exist in code. Corrected everywhere, with a new docs/P0_DATASET_QUALITY_FLOW.md added as an explicit target-spec (not-yet-built) replacement. 2. github.com/sax3l/0pt (this repo) and git.siax.io/sax3l/0pt (Gitea) are two diverged remotes sharing part of the same history (services/api/ workers/adaptive_router.py is byte-identical between them). Gitea's main independently accumulated a WAF-bypass/proxy-rotation scraper stack in three 2026-07-13 commits, since removed via a Gitea PR; GitHub's main never had it. Which remote is canonical/deployed is unresolved — flagged consistently across the corpus, not decided here. Also newly and consistently documented: the tenant_id/project_id-read-from- client-body gap on the P0 lakehouse/rate-policy/source-query routes (no auth/session system exists anywhere in this repo yet); that every CL0UD/ N0D/AUD0/ST0RE platform client is real but mock-only (zero process.env reads for any of their *_URL vars); that two P0 mutation routes (lakehouse/mappings, sources/rate-policies) always fail over HTTP today because their prerequisite entity can never be created via any route; a schema bug in sql/004 (FK references to opt_datasets/opt_sources, which don't exist); and that 0pt.app's live deployment was independently confirmed (curled this pass) to be running this exact codebase with DATABASE_URL/REDIS_URL unset in production. New file: docs/12-edge-capability-api.md — the edge-native P0 TypeScript capability contract (app/api/0pt/*, /api/screen, /api/ai/parse), previously undocumented and distinct from docs/04-api.md's FastAPI-engine-only scope. Verified before commit: tsc --noEmit clean; vitest run 88 passed, 1 todo, 0 failures (13 files) — docs-only change, no source touched. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * docs: reflect PR #5's lakehouse-mapping-registry + status-code fixes Follow-up to keep docs/12-edge-capability-api.md accurate once both PRs land: fix/0pt-full-bug-hunt (#5) fixed the shared-instance bug in the lakehouse mapping routes and added proper status-code mapping to the document extraction route. This doc was written against the pre-fix state in the same pass that added it, so it needs the same correction. Does not claim the lakehouse write path fully works now -- the deeper dataset-registration gap (no route ever calls DatasetService.registerDataset) is unchanged and still blocks it end-to-end; only the shared-instance half of the bug is fixed. --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
54 lines
5.5 KiB
Markdown
54 lines
5.5 KiB
Markdown
# 01 · Product
|
||
|
||
## Vision
|
||
|
||
0pt is the most flexible, honest data-services platform for the Nordic + European B2B market. Customers order any kind of data work — from ready-made datasets to bespoke pipelines — and get full visibility into where every value came from and how confident it is. The differentiator is **honest accuracy**: provenance per field, freshness tracking, source transparency. Incumbents (Apollo, Cognism, ZoomInfo) overclaim; 0pt shows its work.
|
||
|
||
## The three flows (everything reduces to these)
|
||
|
||
Every service maps to one of three intake/delivery flows (`Flow` in `lib/services.ts`: `catalog` | `custom` | `subscription`).
|
||
|
||
*Correction: this section previously said "nine categories (scraping, structuring, datasets, ICP/leads, BI, ML, documents, subscriptions, custom)" — that list doesn't match `lib/services.ts`'s actual catalog and never did (no `structuring`, `datasets`, `subscriptions`, or `custom` service id exists there). The real catalog today (`lib/services.ts`, `SERVICES`) has 11 entries: data delivery, web scraping, mapping & matching, normalization, consolidation → CRM/ERP, documents & OCR, enrichment, ICP & leads, ML jobs, BI & market analysis, feeds & monitoring.* These are **tags/filters within the three flows**, not separate machinery — each entry carries a `flow` field pointing back to `catalog` / `custom` / `subscription`.
|
||
|
||
1. **Catalog product** — ready-made, browse → sample → buy → deliver. Exists in Postgres or produced fast from the pipeline. Margin + scale.
|
||
2. **Custom job** — does not exist yet; must be scoped. `configure (AI-assist) → estimate → manual scope/quote confirm → produce → QA → validate → deliver`. The hero flow. Consulting in a portal.
|
||
3. **Subscription / feed** — recurring product or job: lead feeds, freshness monitoring, ongoing enrichment. `configure → recurring payment → recurring delivery/webhook`.
|
||
|
||
## Order lifecycle (state machine)
|
||
|
||
```
|
||
DRAFT → QUOTED → DEPOSIT_PENDING → IN_PRODUCTION → REVIEW
|
||
→ (REVISION → IN_PRODUCTION)* → ACCEPTED → DELIVERED → [SUBSCRIPTION_ACTIVE]
|
||
```
|
||
|
||
- **DRAFT**: configurator in progress.
|
||
- **QUOTED**: estimate produced (catalog = instant; custom = after scope confirm).
|
||
- **DEPOSIT_PENDING**: awaiting 50 % deposit.
|
||
- **IN_PRODUCTION**: deposit captured → workers run. Customer watches live.
|
||
- **REVIEW**: QA passed threshold → customer validates (table/map/schema/quality).
|
||
- **REVISION**: customer requested changes → escrow untouched → back to production.
|
||
- **ACCEPTED**: customer approves → remaining 50 % released from escrow.
|
||
- **DELIVERED**: exports + destinations unlocked; provenance log + quality report attached.
|
||
- **SUBSCRIPTION_ACTIVE**: if converted to a feed.
|
||
|
||
This is real, working code today, not just a spec: `app/api/orders/[id]/route.ts`'s `PATCH` handler enforces exactly this sequence (lowercase: `draft`, `quoted`, `deposit_pending`, `in_production`, `review`, `revision`, `accepted`, `delivered`) as its allowed-status list. One gap: `SUBSCRIPTION_ACTIVE` is **not** in that allowed list yet — feed conversion (Phase 7, `docs/09-build-plan.md`) isn't wired into the order status machine.
|
||
|
||
## Business / escrow model
|
||
|
||
- **50 / 50.** Deposit (50 %) captured to start production. Balance (50 %) **held** and only captured/released on **ACCEPTED**. Revision never moves the held amount.
|
||
- Pricing models: pay-per-job, volume, subscription (monthly), custom quote.
|
||
- Estimate is a function of: selected field weights × markets × volume × entity multiplier × confidence-target multiplier + enrichment. Custom jobs get a manual scope/quote gate before DEPOSIT_PENDING.
|
||
- Stripe: deposit = capture now (`app/api/payments/[order_id]/deposit/route.ts`); balance = **a second Stripe Checkout session created on accept**, keyed to the order (`app/api/payments/[order_id]/release/route.ts`). *Correction: earlier drafts of this doc left the balance mechanism open ("either a held authorization … or a second invoice"). The second-Checkout-session approach is what's actually implemented — there is no held-PaymentIntent-capture path in the code.* Both routes are env-guarded on `STRIPE_SECRET_KEY` and return `501` until it's configured (`lib/stripe.ts`).
|
||
|
||
## Customer DB access (designed, not a checkbox)
|
||
|
||
Default: the customer **pushes** data (upload/export) — we rarely need to read their DB. If we must: a **read-only, table-scoped, time-boxed** role the customer creates, IP-allowlisted, ideally against a replica. Creds encrypted, deleted after the job. Everything logged. This is a selling point to compliance-heavy buyers, and it requires a DPA.
|
||
|
||
## Personas (brief — for copy + prioritization)
|
||
|
||
SaaS-Martin (RevOps, wants clean ICP/leads), Bilhandlar-Bengt (vehicle + company data via VehIQ), Rekryterar-Rebecka (people/firmographics), Investor-Ivar (diligence, sanctions/UBO, scrutinizes compliance), Ekonomi-Erik (financials, scrutinizes compliance), Data-David (raw datasets + API). Ivar and Erik are *why* honest sourcing + DPA must be airtight.
|
||
|
||
## First-launch scope (de-risked)
|
||
|
||
Do **not** ship "20 markets × 11 categories." Ship the thinnest sellable slice: **Nordic company data** (the open stuff — GLEIF + Brreg + verified sources) + **vehicle data** (VehIQ synergy) as a couple of catalog datasets, **plus the custom-job intake**, Stripe, and delivery to download + into FiveCRM/Data Loader. First customers are Protosell internally + existing dealer relationships — services you already deliver, productized. Then expand markets as the sourcing map fills in.
|