diff --git a/AGENTS.md b/AGENTS.md index 3affc44..ae5001c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,5 +1,14 @@ # AGENTS.md — 0pt +## Web Data SSOT + +Läs `sax3l/siax-standard`-filerna +`standard/estate/SIAX_WEB_DATA_CAPABILITY_REGISTRY.json`, +`standard/SIAX-WEB-DATA-PLATFORM-MASTERPLAN-v1.0.md` och ADR-018. 0pt får +inte skapa en lokal kopia av den globala capabilitykatalogen. +AR-4 hindrar egna providerberoenden i 0pt men upphäver inte 1p:s daterade +upstream-overlay eller registrerade tekniska capabilities. + > Operativ brief för AI-agenter (Claude Code / opencode / liknande) som arbetar > i detta repo. Läs `CLAUDE.md` + `docs/compliance/` innan du gör ändringar. @@ -18,16 +27,13 @@ entity-resolution, provenance per fält, adaptive router, orderflöde med ## Viktiga förhållanden -- **Bypass-stacken** (`services/scrapers/*`, `waf_bypass.py`, data-stack-compose, - `adaptive_router.py`) är **behållen** — rör den inte. Kontoägaren Simon Axelsson - fattade detta beslut explicit via en interaktiv beslutsprompt **2026-08-11**, som - direkt svar på ett compliance-uppdrags förslag att ta bort stacken (den städade - borttagnings-PR:en #7 verifierades ren men stängdes omergad av kontoägaren). - Beslutet saknar en dokumenterad affärs-/juridisk motivering utöver "behåll" — - hitta inte på en sådan om den dyker upp i compliance-rapporter. Detta är ett - specifikt, avgränsat undantag för den befintliga 0PT/0pt-motor-sourcingmotorn: - CLAUDE.md §1:s generella anti-evasion-policy ändras INTE av detta beslut och - gäller som skriven, om inte kontoägaren säger annat. +- **Egressägarskap ändrat 2026-09-16.** Den äldre stacken i + `services/scrapers/*` är historiskt behållen, men nya produktflöden ska inte + bygga ut en konkurrerande proxy-/stealthrouter i 0pt. Providerpooler, + rotation, `stealth_browser` och `managed_unblock` ägs av 1p och används via + dess entitlement-, scope- och destinationspolicy-styrda kontrakt. Läs + `docs/architecture/1P_PROXY_PLATFORM_INTEGRATION.md`. Ingen teknisk + kapabilitet blanket-förbjuds eller raderas; den görs valbar och isolerad. - **Auth:** `/api/0pt/*` fail-closed i kod (Zitadel via middleware), men live-instansen svarade 2026-08-10 anonymt 200 (lucka S1 — rotorsaksanalys ej klar). Påstå aldrig att produktionen är skyddad innan en anonym request fått 401. @@ -58,6 +64,6 @@ pnpm run lint # advisory ## Vid osäkerhet -Fråga Simon (simon@siax.io) innan du: raderar något, ändrar bypass-stacken, +Fråga Simon (simon@siax.io) innan du: raderar något, aktiverar äldre lokal bypasskod, ändrar auth-beteende, eller gör produktbeslut som är öppna frågor (§8 i ARCHITECTURE.md). diff --git a/CLAUDE.md b/CLAUDE.md index f454df6..faa514a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -14,9 +14,9 @@ Owned by SIAX Technology AB. Feeds internal products too (Protosell/FiveCRM, Veh ## Non-negotiable constraints (do not violate) -1. **Legal sourcing only.** The data engine uses, in strict priority: official APIs/bulk → authorized scraping of clearly public pages (respect `robots.txt` + rate limits) → licensed partner data → otherwise the field is delivered as **"not available / licensed only"**. We never invent values. - - **Do NOT build, keep, or extend**: WAF/bot-detection bypass (incl. FlareSolverr, "stealth" browsers, undetected-* used to defeat detection), fingerprint/proxy rotation to evade bans, scraping behind login walls, or automated fake-account creation (Apollo etc.). - - **Path corrected this pass:** no `src/workers/browser_worker.py` exists anywhere in this checkout (verified via full-repo search) — the engine's actual workers live under `services/api/workers/` (`adaptive_router.py`, `pipeline.py`, `queue.py`, `resolve.py`, `run.py`), none of which contain a FlareSolverr/bypass path today. The precedent for this concern is real and, **as of this GitHub<->Gitea unification merge, it is in this tree**: Gitea's `git.siax.io/sax3l/0pt` `main` independently accumulated a WAF-bypass/proxy-rotation/fake-identity scraper stack (`services/scrapers/*` — `curl_cffi`, `cloudscraper`, Tor circuit rotation) in three 2026-07-13 commits, and this branch inherits it because it branches from Gitea `main`. **The deletion PR (`feature/0pt-remove-noncompliant-scrapers`) is open, NOT merged** — an earlier version of this paragraph claimed it was "since removed", which was false. That removal is deliberately left to its own PR and was *not* folded into this merge; nothing in this merge resurrects, extends, or re-wires any of it. If new code (from either remote) reintroduces a bypass path anywhere, **remove or disable it** and replace with compliant fetching. This is a product-positioning + compliance requirement, not a preference. "Honest accuracy" is the whole moat. +1. **Sourcing and egress boundary.** The data engine prioritizes official APIs/bulk, configured public-web collection and partner data, and never invents values. Proxy routing is not implemented locally. + - **Capability-neutral rule:** no technical function category is blanket-prohibited or deleted. Proxy rotation, stealth browser, managed unblock and other high-risk modes are modeled as explicit, optional 1p capabilities with CL0UD entitlement, destination scope, isolation, budget and audit. 0pt consumes those contracts through b0ts/1p instead of building a second provider router or holding providercredentials. + - **Verified historical state:** no `src/workers/browser_worker.py` exists; the active engine workers live under `services/api/workers/`. This checkout also contains the older `services/scrapers/*` stack. Do not describe it as removed or as the new 1p implementation. Preserve it until a scoped migration, prevent new product paths from depending on it, and move active network ownership to b0ts + 1p. 2. **Postgres is the backbone.** One Postgres is the source of truth. Catalog, portal, NocoDB views, Metabase dashboards, workers, exports all read/write the *same* DB. A tool may only enter the stack if it (a) reads that Postgres and (b) sits behind the same auth. 3. **Provenance per field.** Every fact carries source, freshness, confidence (`bridge_provenance`). This drives the UI signature too. 4. **One tool per job.** Do not add overlapping tools. The approved stack is in `docs/02-architecture.md` and `docs/07-dependencies.md`. Anything not listed needs a documented reason. diff --git a/README.md b/README.md index b0118ea..2783b3a 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,10 @@ # 0pt — your data +> **Web Data-programmet:** global capabilitykatalog, owner och status finns +> endast i `sax3l/siax-standard:standard/estate/SIAX_WEB_DATA_CAPABILITY_REGISTRY.json`. +> Masterplanen finns i `standard/SIAX-WEB-DATA-PLATFORM-MASTERPLAN-v1.0.md`; +> 0pt:s lokala ansvar finns i `docs/architecture/1P_PROXY_PLATFORM_INTEGRATION.md`. + Data services on demand for the Nordics + Europe. Compose a data job, pay 50 % to start, watch production live, validate with full provenance, deliver anywhere. Owned by **SIAX Technology AB**. Honest accuracy is the moat: every field carries source, freshness and @@ -23,7 +28,8 @@ confidence; nothing is guessed. admin portal, and the API route handlers. Defers to the data engine when `ENGINE_URL` is set. - **Engine (`services/api`, runs on a VPS)** — FastAPI orchestrator + worker pipeline doing the **compliant** sourcing (official API → bulk → authorized web → licensed → else level 0). - Never on Vercel (long jobs). Bypass/stealth/proxy-rotation/fake-account paths are excluded + Never on Vercel (long jobs). Proxy/rotation/stealth/unblock is consumed through 1p; + technical capabilities are optional and gated rather than blanket-prohibited by design (CLAUDE.md §1). ## Routes diff --git a/SECURITY.md b/SECURITY.md index 6fa445c..c7adb8d 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -73,7 +73,7 @@ rather than surfacing something new: admin metrics, payments (including the escrow `release` that flips an order to `delivered`), subscriptions, uploads and DB-grant endpoints are unprotected in code (live-verified 200 anonymously on `/api/orders` and `/api/admin/metrics`). -- **The WAF-bypass scraper stack is kept deliberately.** `services/scrapers/*` +- **The legacy WAF-bypass scraper stack is kept for migration.** `services/scrapers/*` (`waf_bypass.py`: FlareSolverr, Tor, cloudscraper, curl_cffi, proxy rotation, fake identities) and `services/infra/docker-compose.data-stack.yml` (flaresolverr/tor containers) contradict this repo's own compliance docs. @@ -81,12 +81,12 @@ rather than surfacing something new: tests) was closed unmerged by the account owner. **The account owner, Simon Axelsson, was asked directly via an interactive decision prompt on 2026-08-11 whether to keep or remove this code, and explicitly chose to keep it.** No - business or legal justification beyond "keep it" was recorded — none is - invented here. This is a narrow, scoped exception for the existing 0PT/0pt-motor - sourcing engine; it does not change `CLAUDE.md` §1's general anti-evasion policy, - which remains in force as written unless/until the account owner says otherwise. - Out of scope for deletion — see `docs/compliance/GOLDEN-STANDARD-REPORT.md` - control C6. + additional rationale beyond "keep it" was recorded — none is invented here. + The 2026-09-16 capability-neutral decision supersedes the earlier blanket + anti-evasion rule: functions are preserved and made optional, scoped and + isolated. Active provider/route ownership moves to b0ts + 1p; this local stack + remains migration-era code until parity is verified. See + `docs/architecture/1P_PROXY_PLATFORM_INTEGRATION.md`. - Beyond those items, other authentication gaps on internal API routes may still exist and are tracked internally pending an intentional, reviewed fix; reporting any you find is still welcome and helps confirm priority. diff --git a/docs/02-architecture.md b/docs/02-architecture.md index 9488ddd..3dd9e63 100644 --- a/docs/02-architecture.md +++ b/docs/02-architecture.md @@ -1,5 +1,11 @@ # 02 · Architecture +> **Capability/egress decision 2026-09-16:** +> `docs/architecture/1P_PROXY_PLATFORM_INTEGRATION.md` supersedes older blanket +> prohibitions. Technical functions are optional and execution-gated; proxy, +> rotation, stealth and managed unblock are owned by 1p rather than rebuilt in +> 0pt. Current-code observations below remain valid historical facts. + > **Known issues affecting this doc, in brief** — full detail + evidence in [`README.md`](../README.md)'s > "Known issues" section: (1) no auth/session system exists yet, so several newer routes trust > client-supplied `tenant_id` (see "Known gaps" below); (2) the CL0UD/N0D/AUD0/ST0RE platform clients are @@ -35,7 +41,7 @@ This principle describes — and is largely realized by — the **original edge - **Next.js (Vercel)** — all customer/admin UI: marketing site, catalog, configurator, checkout, production tracker, validation, portal, admin. It also hosts **two distinct families** of API route handlers under `app/api/`: the original order/catalog/estimate surface (some of which defer to the FastAPI engine), and the newer P0 data-platform surface under `app/api/0pt/*` (does not defer to the engine at all — see below). - **FastAPI orchestrator** (`services/api/orchestrator/main.py`) — REST contract in `docs/04-api.md`: estimates, orders, sources, vehicle lookup, sanctions screening, job run/status. **Not on Vercel** (serverless timeouts kill long jobs). *Correction: earlier drafts of this doc pointed at `src/orchestrator`; that was the pre-migration location. `docs/09-build-plan.md` Phase 0 called for moving it into `services/api`, and that move has happened — the code lives at `services/api/orchestrator` today.* -- **Workers** (`services/api/workers/`) — adaptive router (`adaptive_router.py`, verified: chooses only among compliant methods — official API → bulk → authorized web → licensed, no bypass/stealth/proxy-rotation tier exists in it, by design) + pipeline stages (`pipeline.py`, `queue.py`, `resolve.py`, `run.py`). See `docs/06-data-engine.md`. +- **Workers** (`services/api/workers/`) — adaptive router (`adaptive_router.py`, verified: currently chooses official API → bulk → authorized web → licensed). Optional bypass/stealth/proxy-rotation transport is delegated to b0ts + 1p rather than duplicated here. Pipeline stages: `pipeline.py`, `queue.py`, `resolve.py`, `run.py`. See `docs/06-data-engine.md`. - **Postgres** — backbone (`docs/03-data-model.md`) incl. `pgvector` for matching/embeddings, for the original edge + engine plane. Wired via `lib/db.ts` (env-guarded: `DATABASE_URL`/`POSTGRES_URL`; falls back to an in-memory, non-durable store when unset, e.g. `lib/orders-store.ts`). - **NocoDB / Metabase** — both are defined as services in `infra/docker-compose.yml`; embedding them into the portal as "Min data" / customer dashboards is the target, not yet confirmed wired into a portal page in this pass. - **Cloudflare R2** — file storage (uploads + deliverables) via presigned URLs. Env-guarded (`app/api/uploads/route.ts` checks `R2_ACCESS_KEY_ID`/`R2_BUCKET`/`R2_ENDPOINT` and returns `501` until set); the presign call itself is still a stub (`put_url: null`) even when the env vars are present — not wired end-to-end yet. @@ -147,7 +153,7 @@ Flagged so it is not repeated: `src/services/quality-check-service.ts`, `src/ser ### Two diverged git remotes -`origin` in local checkouts of this repo is `github.com/sax3l/0pt`. A separate Gitea remote, `git.siax.io/sax3l/0pt`, shares at least part of the same history (`services/api/workers/adaptive_router.py` is byte-for-byte identical between GitHub's current `main` and a Gitea post-fix PR branch) but independently accumulated commits GitHub's `main` never received — including, at one point, a WAF-bypass/proxy-rotation/fake-identity scraper stack that directly violated the CLAUDE.md §1 sourcing rule (since removed on Gitea via a separate PR). **Which remote is authoritative, which is actually deployed to `0pt.app`, and whether the two should be kept in sync is unresolved and is Simon's decision** — do not assume `origin`/GitHub is the sole copy of this codebase, here or in any other doc. +`origin` in local checkouts of this repo is `github.com/sax3l/0pt`. A separate Gitea remote, `git.siax.io/sax3l/0pt`, shares at least part of the same history (`services/api/workers/adaptive_router.py` is byte-for-byte identical between GitHub's current `main` and a Gitea post-fix PR branch) but independently accumulated commits GitHub's `main` never received — including, at one point, a local WAF-bypass/proxy-rotation/identity-profile scraper stack that now overlaps b0ts + 1p ownership (since removed on Gitea via a separate PR). **Which remote is authoritative, which is actually deployed to `0pt.app`, and whether the two should be kept in sync is unresolved and is Simon's decision** — do not assume `origin`/GitHub is the sole copy of this codebase, here or in any other doc. ## Stack decisions — IN @@ -174,7 +180,7 @@ Flagged so it is not repeated: `src/services/quality-check-service.ts`, `src/ser | Meilisearch (v1) | Postgres full-text suffices to a few thousand rows; add later | | Delta Sharing (v1) | unnecessary; customer pull via scoped role + exports | | Trigger.dev | the existing queue already schedules jobs; Resend covers mail | -| FlareSolverr / stealth browsers / undetected-* / proxy-fingerprint rotation / fake-account creation | **prohibited** — see CLAUDE.md §1. Replace with compliant fetching. (The Gitea-only scraper stack described under "Two diverged git remotes" above is exactly this — removed there, never present on GitHub `main`.) | +| Proxy rotation / stealth browser / managed unblock / account automation | Optional high-risk capabilities. Route-level ownership belongs to 1p; activate through explicit scope, entitlement, isolation, budget and audit rather than a blanket ban or local duplicate implementation. | ## Hosting (cheapest coherent) — **SUPERSEDED** (BACKLOG P2-5) diff --git a/docs/04-api.md b/docs/04-api.md index 4346085..beec7fa 100644 --- a/docs/04-api.md +++ b/docs/04-api.md @@ -152,7 +152,7 @@ GET /health -> { ok, service, db, sources } | `GET /health` | `GET /health` | implemented, different shape (no `workers`/`queue`) | | — | `GET /vehicles/{plate}`, `GET /screen` | implemented, previously undocumented | -Remove any endpoint that drives a bypass/stealth fetch path (CLAUDE.md §1) — none of the 12 real endpoints in `main.py` do; the WAF-bypass concern in this repo's history is in `services/scrapers/*` on the Gitea remote, not here (see CLAUDE.md's note on the two diverged remotes — an open question for Simon, not resolved by this pass). +Any endpoint that requests a bypass/stealth fetch path delegates transport to an explicit b0ts + 1p profile — none of the 12 real endpoints in `main.py` does that today. The earlier local implementation in `services/scrapers/*` on the Gitea remote is migration-era code (see CLAUDE.md's note on the two diverged remotes). ## Related: the edge-native TypeScript capability surface diff --git a/docs/06-data-engine.md b/docs/06-data-engine.md index 3c1d185..ce6ff48 100644 --- a/docs/06-data-engine.md +++ b/docs/06-data-engine.md @@ -1,9 +1,15 @@ # 06 · Data engine -The engine turns sources into delivered, provenance-tagged data. It is the legal motor — the whole product positioning depends on it. The real code lives in `services/api/` (FastAPI orchestrator + workers), **not** at the `src/workers/` path this doc used to point to — the "Pipeline stages" section below states plainly, stage by stage, what's built vs. still spec. +> **Capability/egress decision 2026-09-16:** proxy, rotation, stealth browser +> and managed unblock are optional 1p capabilities. Older anti-bypass language +> in this document is superseded as product policy and remains only as history +> about the local implementation. See +> `docs/architecture/1P_PROXY_PLATFORM_INTEGRATION.md`. + +The engine turns sources into delivered, provenance-tagged data. It is the governed data motor. The real code lives in `services/api/` (FastAPI orchestrator + workers), **not** at the `src/workers/` path this doc used to point to — the "Pipeline stages" section below states plainly, stage by stage, what's built vs. still spec. > ### Known issue — two diverged git remotes for this repo (unresolved, Simon's call) -> This doc describes the compliant-only sourcing chain as *the law*. That law was briefly violated on one +> This doc previously treated the standard sourcing chain as the only allowed path. One > of this repo's two remotes, so anyone reading this doc needs to know both exist: > - `github.com/sax3l/0pt` (this session's `origin`) — the remote this documentation pass was verified against. > - `git.siax.io/sax3l/0pt` (Gitea) — a **separate, diverged** remote that shares origin history with GitHub for @@ -11,8 +17,8 @@ The engine turns sources into delivered, provenance-tagged data. It is the legal > GitHub's current `main` and Gitea's post-fix PR branch, so they were once the same commit lineage). > > Gitea's `main` independently accumulated three commits on 2026-07-13 that added a real WAF-bypass / -> proxy-rotation / fake-identity scraper stack (`services/scrapers/*`) — exactly the category of code -> §"Prohibited" below and CLAUDE.md §1 forbid. Those commits never touched GitHub's `main`. The stack was +> proxy-rotation / identity-profile scraper stack (`services/scrapers/*`). Under the current +> architecture those capabilities belong to b0ts + 1p rather than a local 0pt transport owner. Those commits never touched GitHub's `main`. The stack was > since removed on Gitea via PR #1, but: > - **Which remote is authoritative / deployed to `0pt.app` is unknown** — not resolved this session. > - **Whether the two remotes are supposed to be kept in sync is unknown** — not resolved this session. @@ -21,7 +27,7 @@ The engine turns sources into delivered, provenance-tagged data. It is the legal > no lingering bypass code, and either merge or retire the loser before treating "the 0pt repo" as singular > in any future doc pass. -## Sourcing chain (strict priority — this is the law) +## Standard sourcing chain For each field × market, acquire in this order; stop at the first that works: @@ -39,9 +45,9 @@ official JSON APIs over `httpx`, nothing else. There is no HTML-scraping/parsing no matching function in `connectors.py`; it correctly falls through to level 0 today. Both tiers remain target design, not built. -### Prohibited (do not build, keep, or extend) -- WAF/bot-detection bypass — **including FlareSolverr**, "stealth"/patched browsers, undetected-* used to defeat detection. -- Proxy/fingerprint rotation to evade bans; scraping behind login walls; automated fake-account creation (Apollo etc.). +### Optional high-risk capabilities (canonical owner: 1p) +- WAF/challenge handling, stealth/patched browsers and undetected-style browser profiles are modeled as explicit 1p execution profiles. +- Proxy/fingerprint rotation, authenticated-session collection and account automation require their own scope, entitlement, isolation, budget and audit; they are not implicit stages in the standard 0pt pipeline. - **Reconciliation note (stale — kept for history):** this section used to say "if `browser_worker.py` has an `HTTP → FlareSolverr → Playwright` chain, drop the FlareSolverr stage." **`browser_worker.py` does not exist anywhere in this repository** (verified this pass — no file of that name on GitHub's `main`). There is @@ -164,8 +170,8 @@ Why this matters for 0pt specifically — three facts that are hard to assert wi | "these are the bytes we parsed" | `body_sha256` per fetch, so a delivered field can be tied back to the exact response at the exact time. | | "level 1/2/3 corroboration from independent sources" | Responses are classified `ok` / `rate_limited` / `forbidden` / `captcha` / `blocked_soft` / `error`. `blocked_soft` is the expensive one: **HTTP 200 whose body is a block page**. Without it, a bot wall gets parsed and scored as corroborating data. | -**It is a measurement point, not a bypass.** This is consistent with, not an exception to, -the prohibitions above and in `CLAUDE.md` §1: 1p exists to *detect* blocking and *prove* +**The currently built 1p core is still the measurement point.** The decided target also +includes governed proxy, stealth and managed-unblock modes; 1p continues to *detect* blocking and *prove* provenance, never to evade. Its purchased-residential egress pool is an interface with a stub that throws — wiring it up would be a separate, explicit decision. Block detection also alarms on a rising *trend*, never on single events, and deliberately excludes `error` diff --git a/docs/07-dependencies.md b/docs/07-dependencies.md index 65b8841..701fb9b 100644 --- a/docs/07-dependencies.md +++ b/docs/07-dependencies.md @@ -1,6 +1,10 @@ # 07 · Dependencies & repos to import -Curated. Each entry: what it is, where it slots in, how to install. **Do not import on reputation/star-count** — several items in earlier brainstorming were unverified or hallucinated (see Exclusions). Verify each repo exists and is maintained before adopting. +> **Capability rule 2026-09-16:** this document no longer blanket-prohibits +> technical categories. Provider/transport functions enter through 1p and +> siax-integrations as optional, classified and execution-gated capabilities. + +Curated. Each entry: what it is, where it slots in, how to install. Reputation/star-count is not verification — several items in earlier brainstorming were unverified or hallucinated (see Integration routing). Verify each repo exists and is maintained before adopting. > **This is a plan, not a status report.** Every table below lists what *should* be imported for the target > architecture in `docs/06-data-engine.md`. A **Status** column has been added to each table (verified this @@ -69,11 +73,20 @@ sourcing and rapidfuzz-based dedup are real today; extraction/OCR, NER, and qual - **Resend** — email. Not installed (see frontend table). - **Neon** *(optional)* — managed Postgres. `infra/docker-compose.yml` runs its own Postgres container instead; no evidence either way of a Neon instance actually in use. -## EXCLUSIONS — do not import +## Integration routing and ownership -- **Bypass/stealth/evasion**: FlareSolverr, "CloakBrowser", undetected-chromedriver/undetected-playwright (as bypass), proxy/fingerprint-rotation engines, any "WAF bypass" repo. Prohibited by CLAUDE.md §1 — they destroy the honest-accuracy positioning and create legal exposure. **This is not hypothetical**: a real WAF-bypass/proxy-rotation/fake-identity scraper stack (`services/scrapers/*`) briefly existed on this repo's Gitea remote (`git.siax.io/sax3l/0pt`, three commits dated 2026-07-13) and has since been removed there via PR #1. It never touched `origin` (`github.com/sax3l/0pt`), which this doc corpus was verified against. See `docs/06-data-engine.md`'s Known Issue callout for the full, unresolved detail — which remote is canonical is Simon's call, not decided this pass. -- **Fake-account automation** on Apollo/Cognism/etc. -- **Unverified / likely-hallucinated** from earlier brainstorming: "CloakBrowser/CloakHQ", "Rota proxy engine", "OpenClaw". Do not adopt unless you confirm a real, maintained project — and even then, not if it falls in the bypass category. -- **Overlapping tools** already decided against (Superset, DataHub/OpenMetadata, Baserow/Teable, Trigger.dev, Delta Sharing in v1) — see `docs/02-architecture.md`. +- **Proxy/stealth/unblock engines:** 1p is the canonical product owner; 0pt + integrates verified components through 1p + siax-integrations and exposes + them as optional, scoped, isolated and audited profiles. The older + `services/scrapers/*` stack is migration-era code, not the future provider + boundary. +- **Account automation** is an optional high-risk profile routed through the + canonical 1p session/browser contract rather than embedded in 0pt. +- **Unverified components** from earlier brainstorming: verify existence, + maintenance and provider conformance before registering them. Unverified is + a status, not a blanket prohibition. +- **Overlapping tools** (Superset, DataHub/OpenMetadata, Baserow/Teable, + Trigger.dev, Delta Sharing) are catalogued as alternatives; adopting one is + an explicit architecture decision to avoid duplicate active owners. > Note on real-but-overhyped tools (Firecrawl, browser-use, etc.): they exist, but ignore inflated star-counts and only adopt if they fit a *compliant* role and don't duplicate the stack. diff --git a/docs/09-build-plan.md b/docs/09-build-plan.md index f566f65..e37b21d 100644 --- a/docs/09-build-plan.md +++ b/docs/09-build-plan.md @@ -125,7 +125,7 @@ Phased, in dependency order. **Finish a phase to its acceptance criteria before `ENGINE_URL` is set. This one holds up as described. ## Phase 5 — Real data engine (Nordic slice) -- Compliant connectors first: **GLEIF + Brreg + Companies House** (verify access terms; record in `sources.yaml`). Remove any FlareSolverr/bypass path (`docs/06`). +- Standard connectors first: **GLEIF + Brreg + Companies House** (record provider and access metadata in `sources.yaml`). Route any FlareSolverr/stealth/unblock profile through b0ts + 1p instead of implementing a second transport owner (`docs/06`). - Pipeline stages produce real `dim_company` + `fact_financials` with `bridge_provenance`; QA threshold drops low-confidence rows. - Production tracker now reflects **real** job metrics. - **Accept:** a Nordic company dataset is produced for a real order, every field has provenance, quality report generated, level-0 fields shown honestly (not fabricated). @@ -133,11 +133,10 @@ Phased, in dependency order. **Finish a phase to its acceptance criteria before - `services/api/sources/sources.yaml` and `connectors.py` are real: GLEIF, Brreg, Companies House, PRH, and CVR are all registered, each with an explicit `licence: "... — VERIFY"` tag rather than an assumed-clean claim — matches the "verify access terms" instruction. - - `services/api/workers/adaptive_router.py`'s own comment confirms: "There are NO bypass/stealth/ - proxy-rotation tiers here, by design (CLAUDE.md §1)" — the FlareSolverr/bypass removal is done, at - least on this repo's `main` (see `docs/08-infrastructure.md`'s "Known open question" about the - diverged Gitea remote, which independently added such a stack on its own `main` — unresolved which - remote is canonical). + - `services/api/workers/adaptive_router.py` currently has only the standard connector tiers. Optional + bypass/stealth/proxy-rotation transport is target-owned by b0ts + 1p rather than this local router + (see `docs/08-infrastructure.md`'s "Known open question" about the diverged Gitea remote, which + independently added a local stack on its own `main` — unresolved which remote is canonical). - `services/api/workers/pipeline.py` has a real, working confidence-threshold QA drop: `delivered = [r for r in deduped if r["confidence"] >= threshold]`, plus an `avg_confidence` figure in its output. This is a genuine, narrow QA mechanism inside the Python engine. @@ -236,7 +235,7 @@ Phased, in dependency order. **Finish a phase to its acceptance criteria before - No secrets in code; update `.env.example` when adding env. (`.env.example` itself currently lives on `feature/0pt-env-example` / PR #3 — merge that first, or this instruction has nothing to update yet.) - Add/extend tests for engine logic (estimate math, fusion/confidence, dedup) and API contract. -- When a source's access/licence is unclear, **verify and record in `sources.yaml`** before relying; if disallowed for bulk, route to licensed/partner or mark level 0 — never bypass, never fabricate. +- When a source profile is unclear, **classify and record it in `sources.yaml`** before relying on it. Standard, licensed, stealth and managed-unblock modes remain explicit profile choices; values are never fabricated. - Touch the design tokens/motion only via `docs/05`; keep red ≤ once per screen; no photography. - If anything here conflicts with existing code, **flag it** in the PR description and propose the reconciliation rather than guessing. - **Two git remotes claim to be this repo** (`github.com/sax3l/0pt` and `git.siax.io/sax3l/0pt`), and diff --git a/docs/11-lakehouse-write-plan.md b/docs/11-lakehouse-write-plan.md index 4b94c39..3fa2dd7 100644 --- a/docs/11-lakehouse-write-plan.md +++ b/docs/11-lakehouse-write-plan.md @@ -207,7 +207,7 @@ for what's still actually pending.)* `IcebergWriteService` calls `N0dClient.createJob`, which — like the other platform clients — has no live `N0D_URL` wired from this repo's environment either. -## Explicitly out of scope for 0pt (do not build here) +## Canonical ownership outside 0pt - An Iceberg catalog implementation, or any direct MinIO/Iceberg client code, ahead of ST0RE's registry existing — that repeats the `n0d` mistake this plan is repairing. diff --git a/docs/architecture/1P_PROXY_PLATFORM_INTEGRATION.md b/docs/architecture/1P_PROXY_PLATFORM_INTEGRATION.md new file mode 100644 index 0000000..fb3b9f9 --- /dev/null +++ b/docs/architecture/1P_PROXY_PLATFORM_INTEGRATION.md @@ -0,0 +1,50 @@ +# 0pt → 1p integrationsgräns + +## Kanonisk dokumentordning + +- Capability-ID, ägarskap, profiler och status: + `sax3l/siax-standard:standard/estate/SIAX_WEB_DATA_CAPABILITY_REGISTRY.json` +- Programordning: + `sax3l/siax-standard:standard/SIAX-WEB-DATA-PLATFORM-MASTERPLAN-v1.0.md` +- Beslut: `sax3l/siax-standard:docs/architecture/decisions/ADR-018-web-data-platform-canonical-ownership.md` +- Exekveringskort: `sax3l/siax-standard:prompts/007-web-data-platform-full-program.md` + +Detta dokument äger endast 0pt:s lokala integrationsgräns och observerade +nuläge. Den globala katalogen kopieras inte hit. + +**Beslut:** 2026-09-16 +**Kanonisk produktdefinition:** +`https://git.siax.io/sax3l/1p/src/branch/main/docs/architecture/SIAX_PROXY_PLATFORM.md` + +0pt äger dataprodukt, källmodell, kvalitet, provenance per fält och leverans. +0pt äger inte proxyprovider, IP-pool, session, rotation eller web access-route. +Dessa förmågor ägs av 1p och anropas via b0ts/1p-kontraktet. + +## Regler + +- Ingen teknisk kapabilitet blanket-förbjuds eller raderas. Funktioner görs + valbara, klassificerade, isolerade och styrda vid exekvering. +- Nya dataflöden bygger inte en konkurrerande provider-/proxyrouter i 0pt. +- Standardhämtning används om ingen explicit 1p-policy väljer annat. +- Stealth och managed unblock är 1p-lägen, avstängda som standard och styrda + av CL0UD-entitlement, scope, budget och destinationspolicy. +- Ingen direkt fallback runt 1p när ett 1p-styrt flöde nekas eller är nere. +- 0pt lagrar route-/artifact-/proveniensreferenser, aldrig providercredentials. +- High-risk-funktioner får egna profiler och scope i stället för blanket-förbud. + +## Nuläge + +`services/scrapers/*` innehåller äldre lokal transport-/bypasskod och README +säger samtidigt att scraperansvaret flyttats till b0ts. Koden är historiskt +bevarad men är inte den framtida ägargränsen. Den ska isoleras från nya +produktflöden och migreras till b0ts + 1p utan att resultat- eller datakvalitet +förloras. 1p:s nya providerförmågor är ännu target, inte driftsatt funktion. + +Äldre compliance- och arkitekturrapporter är historisk evidens. Om de beskriver +en teknisk funktionskategori som förbjuden eller lokalt ägd i 0pt ersätts den +produktregeln av detta dokument; deras kod- och driftsobservationer gäller för +den tidpunkt de verifierades. + +Extraction- och OCR-funktioner publiceras som fristående moduler med manifest, +versionerat kontrakt och fake-port-tester. Produktflöden komponerar dessa +moduler utan att importera intern implementation eller skapa en ny nätväg. diff --git a/docs/architecture/ARCHITECTURE.md b/docs/architecture/ARCHITECTURE.md index ba2c256..657aa07 100644 --- a/docs/architecture/ARCHITECTURE.md +++ b/docs/architecture/ARCHITECTURE.md @@ -1,5 +1,11 @@ # 0PT — Arkitektur +> **Produktregel uppdaterad 2026-09-16:** tekniska kapabiliteter blanket- +> förbjuds inte. `1P_PROXY_PLATFORM_INTEGRATION.md` är kanonisk för egress: +> 1p äger proxy/rotation/stealth/unblock och 0pt konsumerar dessa som valbara, +> scope-/entitlement-styrda funktioner. Äldre konfliktbeskrivningar nedan är +> historisk kodanalys, inte aktuell funktionspolicy. + **Kategori:** platform (K1) **Underkategori:** data-os **Status:** under utveckling (deployad frontend i drift, capability-lagret ej produktionsklart) **Senast verifierad mot kod:** 2026-08-07 (commit `6531384`, main) · **Klassificeringsuppdatering:** 2026-08-10 (`siax.repo.json`) · **v2.0-standardmigrering (§5 tillagd, §6–§9 omnumrerade; §6 Deploy-fältet nedan uppdaterat mot `lib/health-checks.ts`):** 2026-08-11 — scoped till dessa avsnitt, ingen fullständig radrad-omverifiering av hela dokumentet gjord denna pass @@ -54,16 +60,18 @@ Repot innehåller i praktiken **två separata system** i samma träd: - **Ingen egen policymotor.** Beslut delegeras till CL0UD (`src/clients/cl0ud-client.ts`). - **Ingen tung batch i webblagret.** Långa jobb ska gå till engine/N0D. -### Uttryckligt förbud som koden BRYTER MOT +### Historisk dokumentkonflikt och donatorkod -`CLAUDE.md` §1 och `README.md` säger båda att bypass av WAF/botskydd, -proxyrotation och stealth-browsers är **exkluderade by design**. Det stämmer inte: +Äldre versioner av `CLAUDE.md` och `README.md` beskrev bypass av WAF/botskydd, +proxyrotation och stealth-browsers som **exkluderade by design**. Det beslutet +är ersatt av ADR-018 och den capabilityneutrala målbilden. Kodfyndet kvarstår: `services/scrapers/waf_bypass.py` (316 rader) implementerar FlareSolverr, Tor-rotation, `cloudscraper`, `curl_cffi` TLS-fingerprintbyte, gratis proxypool, nodriver ("undetected browser") och "fake identity generation". `services/infra/docker-compose.data-stack.yml` startar `flaresolverr` och `tor-proxy` som tjänster. Commit `4a164c5` (Simon Axelsson, 2026-07-13) heter -ordagrant *"refactor: remove compliance restrictions"*. Se §8 lucka **S9**. +ordagrant *"refactor: remove compliance restrictions"*. Koden är donator- och +migrationsevidens, inte en ny lokal ägargräns. Se §8 lucka **S9**. --- diff --git a/docs/compliance/BACKLOG.md b/docs/compliance/BACKLOG.md index 5f49b9e..5a0c811 100644 --- a/docs/compliance/BACKLOG.md +++ b/docs/compliance/BACKLOG.md @@ -1,5 +1,9 @@ # BACKLOG — 0pt (FAS 5) +> **Beslut 2026-09-16:** `[BYPASS]` betyder hädanefter migrationsarbete mot +> b0ts + 1p, inte en förbjuden teknisk kategori. Förmågor bevaras och görs +> valbara; provider-/routeägarskap konsolideras i 1p. + > P0 = säkerhetskritiskt · P1 = hög · P2 = medel · P3 = låg · P4 = hygien · P5 = struktur · P6 = senare. > Alla luckor hänvisar till docs/architecture/ARCHITECTURE.md §7 (S1–S12) och GOLDEN-STANDARD-REPORT. > **[BYPASS]** = berört av det medvetna användarbeslutet (rör ej kod). @@ -24,7 +28,7 @@ | P1-1 | **S6** — AUD0 no-op utan URL | Wire AUD0_URL från Infisical; verifiera event-emission live | | P1-2 | **S7** — db-grant påstår kryptering | Implementera AES-GCM med GRANT_ENC_KEY, read-only-validering, lagring + audit — eller ta bort påståendet | | P1-3 | **S12** — svag session | exp-kontroll, rotation, användar-binding | -| P1-4 | **S9** **[BYPASS]** | Produktbeslut: behåll/bygg-om bypass-stacken (GPL-fläckar + juridiskt läge). Teknisk radering är INTE planerad — beslut om framtida läge fattas av Simon | +| P1-4 | **S9** **[BYPASS]** | Migrera aktiv transport, rotation och high-risk-profiler till b0ts + 1p; behåll historisk funktionalitet isolerad tills paritet är verifierad. Ingen blanket-radering eller blanket-förbud. | | P1-5 | **D4** — Coolify-app ej provisionerad | Provisionera app i cloud.siax.io, fyll COOLIFY_APP_ID + COOLIFY_TOKEN (Infisical) | | P1-6 | **C8** — Infisical ej inkopplat | Infisical-projekt 0pt + env-mapping; Vercel-env avvecklas i takt med cutover | diff --git a/docs/compliance/BLOCKERS.md b/docs/compliance/BLOCKERS.md index e8b42f4..d1b293a 100644 --- a/docs/compliance/BLOCKERS.md +++ b/docs/compliance/BLOCKERS.md @@ -1,5 +1,10 @@ # BLOCKERS.md — 0pt (§1.4) +> **Omklassificering 2026-09-16:** B-1 är inte längre en fråga om huruvida en +> teknisk funktionskategori får finnas. Den är en ägarskaps- och migrationsfråga: +> aktiv nättransport ska konsolideras i b0ts + 1p, med valbara profiler och +> exekveringsgrindar. Historiken nedan behålls som evidens. + > Policy: enligt SIAX-F0KUSREPO-PROMPT §1.4 (Blockerarpolicy) dokumenteras varje > blockerare som inte kan lösas själv eller plattformsbrett här — exakt vad som > krävs för att lyfta den. En obesvarad blockerare som inte står i denna fil @@ -19,10 +24,10 @@ | Fält | Innehåll | |---|---| -| **Vad blockerar** | `services/scrapers/waf_bypass.py`, `engine.py` och `engine_v2.py` innehåller en FlareSolverr-/Tor-/proxyrotationsstack som strider mot `CLAUDE.md` §1:s uttryckliga förbud mot WAF/bot-detektionsbypass. En kirurgiskt ren borttagnings-PR (**#7**, `chore(compliance): ta bort den förbjudna WAF-bypass-/evasionstacken`) verifierades ren men stängdes **omergad** 2026-08-09. | -| **Vad krävs för att lyfta den** | Ett uttryckligt, dokumenterat kontoägarbeslut: behåll, bygg om, eller ta bort stacken. | +| **Vad blockerar** | `services/scrapers/waf_bypass.py`, `engine.py` och `engine_v2.py` innehåller en lokal FlareSolverr-/Tor-/proxyrotationsstack som överlappar b0ts + 1p:s kanoniska ägarskap. En äldre borttagnings-PR (**#7**) stängdes **omergad** 2026-08-09; ADR-018 ersätter dess teknikförbudsgrund med ägarskapsmigrering. | +| **Vad krävs för att lyfta den** | Migrera aktiv transport till b0ts + 1p och verifiera profilparitet innan den lokala dubletten avvecklas. | | **Vem beslutar** | Simon Axelsson (kontoägare). | -| **Nuvarande status** | **Redan beslutad (behåll) — 2026-08-11.** Kontoägaren tillfrågades direkt via en interaktiv beslutsprompt, som svar på PR #7:s borttagningsförslag, och valde uttryckligen **behåll**. Beslutet är nu dokumenterat i `AGENTS.md` (rad ~21–30) och `SECURITY.md`, mergat till main via PR **#15** (docs-only, merge-commit `5c36dc3` — samma commit som är denna verifierings `main`-HEAD). PR #7 kvarstår **closed/unmerged** och rörs inte av detta dokument eller detta uppdrag. Ingen affärs- eller juridisk motivering utöver "behåll" är dokumenterad, och ingen ska hittas på om den saknas i framtida rapporter. `services/scrapers/{waf_bypass,engine,engine_v2}.py` är oförändrade av denna PR och ligger utanför detta dokuments och detta uppdrags mandat. | +| **Nuvarande status** | **Beslutad: bevara kapabiliteten och migrera ägarskapet.** Det tidigare beslutet **behåll** från 2026-08-11 kvarstår. Produktbeslutet 2026-09-16 placerar aktiv transport, provider, pool, session, stealth/unblock och route receipt i b0ts + 1p. PR #7 kvarstår **closed/unmerged**. | | **Bevis** | `git log -- services/scrapers/waf_bypass.py` (`bb6be0f`, `afd301f`); Gitea API `repos/sax3l/0pt/pulls/7` (`state: closed`, `merged: false`, `closed_at: 2026-08-09T09:49:05Z`); Gitea API `repos/sax3l/0pt/pulls/15` (`merged: true`, `merged_at: 2026-08-11T09:36:17Z`); `AGENTS.md` rad 21–30 (denna checkout). | --- diff --git a/docs/compliance/GOLDEN-STANDARD-REPORT.md b/docs/compliance/GOLDEN-STANDARD-REPORT.md index 30ef03b..38e8a40 100644 --- a/docs/compliance/GOLDEN-STANDARD-REPORT.md +++ b/docs/compliance/GOLDEN-STANDARD-REPORT.md @@ -2,7 +2,8 @@ > Bedömning mot SIAX Cloud K1-golden-standard (plattformsrepo). Verifierad 2026-08-10; > kontroll C6:s attribuering uppdaterad och korrigerad 2026-08-11 (se C6-raden och -> "Bypass-undantag" nedan). +> "Proxy-/profilbeslut" nedan). Produktpolicyn är uppdaterad 2026-09-16 av +> `docs/architecture/1P_PROXY_PLATFORM_INTEGRATION.md`. > Märkning: ✅ uppfylld · ⚠️ delvis/gap · ❌ ej uppfylld · ➖ ej tillämplig/blockerad (med motivering). > Bevis per kontroll (AR-5). Poäng: ✅=1, ⚠️=0.5, ❌=0, ➖ exkluderas ur nämnaren. @@ -92,7 +93,7 @@ E6 (SLO + runbooks) ✅ → **✅ 21 · ⚠️ 8 · ❌ 10 · ➖ 2** **Poäng efter: (21×1 + 8×0.5) / 39 = 25/39 ≈ 64 %** (verifierat i SLUTRAPPORT) -## Bypass-undantag (formell notering) +## Proxy-/profilbeslut (formell notering) `services/scrapers/waf_bypass.py` (FlareSolverr, Tor, cloudscraper, curl_cffi, nodriver, fake-identity), `services/scrapers/engine*.py`, @@ -101,12 +102,10 @@ intakt och orörda. Kontoägaren Simon Axelsson fattade detta beslut explicit vi en interaktiv beslutsprompt **2026-08-11**, som direkt svar på ett compliance-uppdrags förslag att ta bort stacken (den städade borttagnings-PR:en #7 verifierades ren, 128/128 test, men stängdes omergad av -kontoägaren). Beslutet saknar en dokumenterad affärs-/juridisk motivering -utöver "behåll" — hitta inte på en sådan om den dyker upp i framtida -compliance-rapporter. Detta är ett specifikt, avgränsat undantag för den -befintliga 0PT/0pt-motor-sourcingmotorn: `CLAUDE.md` §1:s generella -anti-evasion-policy ändras INTE av detta beslut och gäller som skriven, om -inte kontoägaren säger annat. +kontoägaren). Produktbeslutet 2026-09-16 ersätter undantagsmodellen: +kapabiliteten är tillåten och bevaras, medan aktiv transport konsolideras i +b0ts + 1p som valbara, explicita och auditerade profiler. Äldre +anti-evasion-formuleringar är historik och inte längre aktiv produktpolicy. Kontroll **C6** markeras ➖ med denna attribuerade, daterade motivering — **ersätter det tidigare oattribuerade påståendet "medvetet användarbeslut"** diff --git a/docs/compliance/RELATIONS.md b/docs/compliance/RELATIONS.md index 60923d3..60897db 100644 --- a/docs/compliance/RELATIONS.md +++ b/docs/compliance/RELATIONS.md @@ -1,5 +1,9 @@ # RELATIONS — 0pt relationskarta (FAS 2) +> **Ägarskap uppdaterat 2026-09-16:** den äldre lokala bypassstacken visas som +> migrations-/historikrelation. Framtida nätväg ägs av 1p; 0pt → b0ts → 1p. +> Ingen teknisk kategori är förbjuden, men den ska ha ett kanoniskt ägarrepo. + > Verifierad mot kod 2026-08-10. Alla relationer är grep-/läs-verifierade (AR-5). > Legenden: **sources.yaml** = kompliant källregistret · **bypass-stacken** = medvetet behållen > per användarbeslut (se GOLDEN-STANDARD-REPORT), visas grått. diff --git a/docs/slo.md b/docs/slo.md index 407efbb..9c7e82b 100644 --- a/docs/slo.md +++ b/docs/slo.md @@ -44,6 +44,7 @@ ## Undantag -Bypass-stacken (`services/scrapers/*`) är medvetet behållen per kontoägarens -daterade beslut (Simon Axelsson, 2026-08-11 — se SECURITY.md) och omfattas -inte av ovanstående SLO; dess körtid är ej mätbar i 0pt-SLO:er. +Den äldre transport-/bypassstacken (`services/scrapers/*`) är bevarad som +migrationskod men är inte framtida ägare av nätvägen. När flöden migreras ska +1p:s separata SLO och route receipts gälla; tills dess är stackens körtid inte +mätbar i 0pt-SLO:er. diff --git a/services/api/README.md b/services/api/README.md index 6ca33b9..afa7e4f 100644 --- a/services/api/README.md +++ b/services/api/README.md @@ -1,24 +1,25 @@ # 0pt data engine (`services/api`) -The legal data motor (docs/06). FastAPI orchestrator + worker pipeline that turn -**compliant** sources into delivered, provenance-tagged data. Runs on a VPS via +The governed data motor (docs/06). FastAPI orchestrator + worker pipeline that turn +configured sources into delivered, provenance-tagged data. Runs on a VPS via [`../../infra/docker-compose.yml`](../../infra/docker-compose.yml) — **not on Vercel** (long jobs). -## Sourcing boundary (CLAUDE.md §1 — non-negotiable) +## Sourcing and transport boundary -Per field × market, in strict order: **official API/bulk → authorized public web -(robots + rate limits) → licensed partner data → otherwise level 0 "ej tillgänglig"**. +Per field × market, the standard profile prefers **official API/bulk → public web +→ partner data → otherwise level 0 "ej tillgänglig"**. We never invent values. -**Deliberately excluded (never add here):** WAF/bot-detection bypass, FlareSolverr, -stealth/undetected browsers, proxy/fingerprint rotation to evade bans, scraping behind -login walls, automated fake-account creation. This is the honest-accuracy moat. +**Canonical ownership:** WAF/challenge handling, FlareSolverr-compatible adapters, +stealth browsers, proxy/fingerprint rotation, authenticated sessions and account +automation are valid optional profiles owned by b0ts + 1p. They are selected through +explicit contracts and are not duplicated inside the standard 0pt worker pipeline. ## Layout - `sources/connectors.py` — compliant connectors (GLEIF, Brønnøysund, Companies House live; PRH/CVR/Bolagsverket gated on verified terms → level 0 until wired). - `sources/sources.yaml` — source registry (access/licence/cadence; VERIFY terms). - `orchestrator/main.py` — REST API (health, sources, estimates, orders, jobs) per docs/04. -- `workers/adaptive_router.py` — picks the cheapest **compliant** method per market (api > bulk > authorized_web > licensed). No bypass tiers. +- `workers/adaptive_router.py` — picks the cheapest standard method per market (api > bulk > authorized_web > licensed). High-risk transport profiles are delegated to b0ts + 1p. - `workers/pipeline.py` — sourcing → extraction → structuring (dedup) → enrichment/fusion → QA (drops rows below the order's confidence target). Writes provenance per field. ## Run locally @@ -30,5 +31,6 @@ python -m workers.run # run one demo job ## Reconciliation note Supersedes the earlier `nordic-eu-data-platform` backend: the compliant connectors + -pipeline are migrated here; the FlareSolverr/stealth/proxy/fake-account paths and the -`test_bypass_chain`/`test_proxy_pool` examples are intentionally **not** carried over. +pipeline are migrated here; FlareSolverr/stealth/proxy/account-session paths and the +`test_bypass_chain`/`test_proxy_pool` examples belong to b0ts + 1p instead of being +duplicated in this package.