unify: merge the GitHub line into Gitea, settled against real service contracts #6
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: admin/0pt#6
Reference in New Issue
Block a user
+8 Gitea, +6 GitHub — both sides had work the other lacked. Merged so both survive.
An independent reviewer was pointed at this branch with one instruction: find what was
dropped. Verdict: 0 dropped behaviours, 17 confirmed survived.
The conflicts were settled against real contracts, not by preference
The four
src/clients/*.tsfiles were the crux — both sides rewrote them independently.Rather than pick a side, the resolver read the target services' actual contracts:
st0re-client.ts— Gitea POSTed to/v1/st0re/artifacts. Readingsax3l/st0resrc/api/routes/artifacts.tsshows that path is registered GET-only; the registerendpoint is
/v1/st0re/artifacts/register, which is GitHub's. Gitea's client wouldnever have registered anything. GitHub's endpoint wins on evidence. Gitea's URL
normalisation, response-body-in-error and required
retention_policy/data_residencyall kept.
n0d-client.ts— the two sides disagreed onstatevsstatusand on the enum.sax3l/n0dpackages/schemas/n0d.job.schema.jsonrequiresstatuswith a 13-stateenum and carries the note "Do not add 'succeeded' — that is 'completed' under a
different name." GitHub's field name was right, Gitea's vocabulary was wrong. Both
historical type names kept as aliases so no import breaks.
cl0ud-client.ts— kept Gitea's fail-closed normalisation(
body.decision ?? "deny"). GitHub returnedres.json()raw, so a 200 with a missingdecisionfield would have read as permitted. This is the security-relevant half.What the reviewer confirmed survived
GitHub's 17 audit bug-fixes (byte-identical), the CI quality gate, the
.gitignoreblanket-
.envfix and its!.env.examplenegation, the added platform clients andregistry services — plus Gitea's Iceberg write-boundary work, its Zod
parseBodygates on all four lakehouse routes, its Zitadel/
AUTH_ENFORCEenv block, and itsX-Tenant-Id/X-Actor-Idthreading.Measured
pnpm installpackages field missing)pnpm testtypecheck(scoped)GitHub's 88 tests are a strict subset of Gitea's file set, so no GitHub-only test was
lost. The 1 skip is honest:
iceberg-write-service.realinfra.test.tsskips becauseST0RE returned 401, and says so rather than fabricating a pass.
Two things left unsettled ON PURPOSE
packages/schemas/0pt.{lineage_event,signal}.schema.json— the two lines disagree onfield vocabulary (
producerobject vs flat string;entity_id/observed_at/confidence_levelvsentity_key/detected_at/numericconfidence). Nothing in therepo reads, writes or validates against either file, so the code cannot decide it. Both
vocabularies are retained as optional properties with the disagreement recorded in each
schema's own
description. Whoever builds signal detection picks one pair and deletesthe other.
Also:
typecheckis scoped tosrc/+tests/as blocking and whole-repo as advisory.The 9 whole-repo errors are all Wave-2 Zod schemas typing a field as
stringwhere theservice wants a narrower union — tightening them changes which request bodies the routes
accept, which is a behaviour change that does not belong in a merge.
One review finding, fixed
The tree asserted the banned scraper stack was "excluded by design" while all 64 files of
services/scrapers/are present. The README now states the policy and the state, andpoints at PR #1, which exists to delete them.
Deleting them here would fold a policy change into a history merge.
Backup:
backup/pre-github-unify-2026-07-28.* docs: correct stale quality-check-service.ts comment references Two source comments still cited src/services/quality-check-service.ts as an existing precedent. It doesn't exist anywhere in this repo's history (git log --all: zero commits, any branch, ever) -- confirmed during the earlier full-doc-corpus accuracy pass. Points each comment at the real precedent instead (document-extraction-service.ts / iceberg-write-service.ts). Applied from task_05ec36aa, spun off during that pass. * fix: 17 confirmed bugs from full-repo audit (edge + Python engine) Found by an 8-finder bug hunt across app/api, src/services, src/clients, frontend, the Python engine, the data layer, config/infra, and tests, then independently adversarially verified per finding (32 raw findings -> 27 confirmed real bugs; 5 refuted). This commit applies the 17 confirmed as safe, mechanical fixes -- no architecture/design decision required. The other 10 need a decision and are reported separately, not touched here. Most consequential, app/api/payments/[order_id]/deposit/route.ts: the client-supplied Origin request header was trusted unsanitized both as a server-side fetch target (SSRF) and as Stripe Checkout success/cancel redirect URLs (open redirect tied to a real payment flow), and a failed order lookup silently fell through to a hardcoded deposit amount instead of 404. Fixed by reading the order directly via lib/orders-store.ts (mirroring the sibling release/route.ts, which already did this correctly) instead of both trusting the Origin header and self-issuing an HTTP fetch. Also fixed: - 5 routes (orders, orders/[id], estimates, subscriptions, ai/parse) that 500'd instead of 400ing on a literal JSON `null` body, since req.json() resolves (not rejects) for that input and bypasses the try/catch guard. - app/api/0pt/lakehouse/mappings/{,[mappingId]/{,writes/}}route.ts each instantiated their own private service instances, so a mapping created via POST was invisible to the sibling GET and writes routes even before the separately-tracked dataset-registration gap. Extracted a shared singleton module, src/services/lakehouse-mapping-registry.ts. - app/api/0pt/documents/[documentId]/extraction/route.ts collapsed every error (not-configured, policy-denied, policy-approval-required, not-found) into a blanket 400; now status-mapped like the sibling sources/[sourceId]/query route. - app/api/vehicles/route.ts: plate query param wasn't URL-encoded before being concatenated into the outbound RDW request, allowing extra query parameters to be injected; added a strict allowlist + encodeURIComponent. - app/api/uploads/route.ts: POST returned a fake 200 success with put_url: null even when "configured", instead of continuing to report 501 until presign generation is actually implemented. - src/adapters/document-intelligence-adapter.ts: the result-polling loop never checked response.ok, so a non-2xx poll response (auth expiry, 429, 5xx) was silently retried until a generic timeout masked the real error. - app/order/[id]/page.tsx: finish() never cleared the production-simulation setInterval, so it kept firing (and re-rendering) forever after delivery. - app/admin/page.tsx: the metrics fetch had no response/shape validation (unlike the adjacent orders fetch), rendering literal "undefined" stat tiles on a malformed response instead of falling back to <ErrorState/>. - services/api/workers/pipeline.py: `config.get("seeds") or DEMO_SEEDS` used truthy-or, so an explicit empty seeds dict was silently replaced with hardcoded demo org numbers instead of being respected as empty. - services/api/workers/resolve.py: corroboration (confidence/agreed_sources bump) no longer fires for two hits from the same source under fuzzy match -- only a genuinely independent source now counts, per docs/06's own ">=2 independent sources agree" definition -- and a merged record's provenance array now actually includes the corroborating source's fields instead of only a bare source-name string. - services/api/orchestrator/main.py: create_order's int(...) on unvalidated client volume/confidence fields raised on any decimal-formatted input (int("92.5") -> ValueError); now coerced via float() first with a safe default. /vehicles/{plate} and /screen also now catch connector exceptions and degrade gracefully, matching pipeline.py's existing pattern for the same connectors. - lib/orders-store.ts: postgres.js (pinned v3.4.9) returns bigint/numeric columns as strings by default; getOrder/listOrders/setStatus were silently handing callers stringified money fields. Added a narrow per-row coercion instead of touching the shared client's global type map (which would affect unrelated tables' bigint/numeric columns too). - lib/subscriptions-store.ts: the in-memory fallback returned oldest-first while the Postgres branch explicitly orders newest-first; now consistent, mirroring the identical fix already in orders-store.ts. - app/api/auth/login/route.ts: DEV_LOGIN_KEY was compared with a plain !==, a non-constant-time comparison; replaced with crypto.timingSafeEqual. Verified before commit: tsc --noEmit clean; vitest run 88 passed, 1 todo, 0 failures (13 files, unchanged from before this commit -- no existing test's assertions needed to change). services/api/workers/resolve.py's source-novelty and provenance-merge logic additionally verified by direct execution against synthetic same-source/cross-source inputs (no Python test suite exists in this repo to add to). services/api/workers/pipeline.py and services/api/orchestrator/main.py verified with python -m py_compile (no httpx/fastapi installed in this environment to run them directly). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(engine): treat explicit seeds:null as missing to avoid 500 in run_job Four-eyes follow-up to this PR's own seeds change. Switching from `config.get('seeds') or DEMO_SEEDS` to `config['seeds'] if 'seeds' in config` correctly stopped an explicit empty {} from being backfilled with DEMO_SEEDS, but regressed on client-controlled `{"seeds": null}`: seeds became None and the downstream `seeds.get(market, [])` raised AttributeError -> 500. That path is reachable (POST /orders stores raw body.config; POST /orders/{id}/run passes it straight to run_job). Using `config.get('seeds') is not None` keeps the intended behaviour (explicit {} honoured, never silently seeded with demo data) while restoring the None-tolerance the old idiom had. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>Merges github/main (6 commits: 17-bug audit fix, CI quality gate, .env.example + .gitignore fix, missing platform clients + registry services, docs accuracy overhaul) into gitea/main (8 commits: Iceberg write boundary + live ST0RE verification, lakehouse mapping routes, Waves 1-4 foundational hardening). 16 conflicts. Both sides had independently written the four src/clients/*.ts and the two registry services, so those are combined rather than picked: - st0re-client.ts: GitHub's endpoint wins on evidence. POST goes to /v1/st0re/artifacts/register, not /v1/st0re/artifacts -- verified in sax3l/st0re src/api/routes/artifacts.ts, where the bare path is a GET-only list route. Gitea's URL normalisation, error-body text and mock kept. - n0d-client.ts: field is `status`, not `state`, with N0D's real 13-state union -- verified in sax3l/n0d packages/schemas/n0d.job.schema.json, which says explicitly not to add "succeeded". Gitea's mock echo, URL normalisation and error-body text kept. - cl0ud-client.ts: Gitea's fail-closed HTTP normalisation kept (a 200 with no `decision` reads as "deny", not as a silent allow). GitHub's PolicyDecision/PolicyDecisionValue naming kept; PolicyResult retained as an alias. - aud0-client.ts: permissive union of both input types (optional payload/metadata/correlation_id), GitHub's echo return, Gitea's URL normalisation and error-body text, plus a JSON-parse fallback so a successful POST with an empty body does not throw. - dataset-service.ts / source-service.ts: GitHub's record/DTO split and event naming (0pt.source_created / 0pt.create_source, which the merged docs already reference) plus Gitea's name validation, description / created_by fields and actor_id derivation. List filters are the union. - lakehouse mapping routes: GitHub's shared lakehouse-mapping-registry singletons (without which every write 400'd "mapping not found") AND Gitea's Zod parseBody validation. Both, not either. - src/types/source.ts, src/types/dataset.ts: unions of both vocabularies; GitHub's `access` legal-sourcing field kept. - .env.example: GitHub's audited inventory plus Gitea's Zitadel / AUTH_ENFORCE block. - CLAUDE.md, docs/ICEBERG_WRITE_BOUNDARY.md: GitHub's corrections kept, Gitea's live-verification kept and marked as superseding the "would 400 outright" finding (Gitea had already added the required X-Tenant-Id/X-Actor-Id headers). - pnpm-lock.yaml: regenerated with pnpm 9.15.9, not merged. Also corrects CLAUDE.md's claim that the banned services/scrapers stack was "since removed" -- the removal PR is open, not merged, and this branch inherits that code from Gitea main. Nothing here resurrects, extends or re-wires any of it; services/scrapers/ is byte-identical to Gitea main. Verified on the merged tree: typecheck (src/ + tests/) clean; 17 test files, 121 passed, 1 skipped (the realinfra test skipping honestly on ST0RE HTTP 401), 0 failures. Gitea main baseline was the same 121/1; GitHub main baseline could not even `pnpm install` ("packages field missing or empty") -- Gitea's pnpm-workspace.yaml fix resolves that. Whole-repo `typecheck:app` reports 9 errors, an identical set to Gitea main's baseline, so it is advisory in both CI workflows rather than deleted. Lint: 16 problems, identical to Gitea baseline. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>