Simon AxelssonandClaude Sonnet 5
dc6aaa0e10
CI (SIAX Cloud) / quality (push) Successful in 51s
Security Scan (SIAX Cloud) / security (push) Successful in 26s
CI (SIAX Cloud) / security (push) Successful in 28s
CI (SIAX Cloud) / integration (push) Successful in 1m1s
CI (SIAX Cloud) / integration (pull_request) Successful in 30s
CI (SIAX Cloud) / security (pull_request) Successful in 24s
CI (SIAX Cloud) / quality (pull_request) Successful in 48s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 22s
Tenant isolation (security-critical): every app/api/0pt/** route reading
tenant_id from client-controlled query/body never cross-checked it against
the token middleware.ts already verified (x-0pt-verified-tenant). Adds a
shared assertTenantMatchesVerified guard and wires it into all 20+ affected
routes. Also closes two full-leak-by-omission bugs found in this pass
(documents/capture and lakehouse/mappings GETs returned every tenant's rows
when tenant_id was simply omitted) and two "no tenant check at all" reads
(rate-policies/[id], lakehouse/mappings/[id]) reachable by ID alone.
Duplicate-delivery idempotency (correctness-critical): POST /api/orders now
honors Idempotency-Key (sql/006, replay-or-409-on-conflict, Postgres +
in-memory); the Python orchestrator's POST /orders/{id}/run — the actual
delivery trigger — is now idempotent under an asyncio lock, closing a real
concurrent-request race, not just the sequential-retry case.
Engine-duplication resolution: adaptive_router.py's docstring wrongly
invited a bypass ("see b0ts for scraping") contradicting the product's own
compliance stance; corrected, and given real (additive, not escalating)
block-signal tracking so a rate-limited/blocked source is excluded from the
plan instead of hammered. Also fixes a previously-undiscovered production
bug: the Python engine's only call into the TS quality bridge
(X-Internal-Token) was unconditionally 401'd by middleware.ts, which never
checked that credential — the advisory quality bridge was unreachable in any
environment enforcing auth.
Restore drill: executed for real against throwaway Docker Postgres
containers (not simulated). Found and fixed two real blockers along the way:
sql/004 referenced opt_datasets, which no migration ever created (chain was
never actually runnable end to end); lib/db.ts hardcoded ssl:"require",
making any non-TLS Postgres (incl. compose.dev.yaml's own local-dev
container) unreachable. After both fixes: full migration chain applies
cleanly, pg_dump/pg_restore round-trip is byte-identical (row counts +
content checksums), and real application code (lib/orders-store.ts,
SourceRegistryService's Postgres repository) reads the restored data back
correctly, including cross-tenant isolation still holding.
82 new tests (55 TS + 27 Python), all passing; full existing suite (291 TS +
55 Python) still green; typecheck clean on every changed/added file.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
42 lines
2.2 KiB
SQL
42 lines
2.2 KiB
SQL
-- P014: fills a real, previously-undiscovered gap in the migration chain.
|
|
-- sql/004_iceberg_write_boundary.sql's own comment already documented (but
|
|
-- did not fix) that `opt_dataset_iceberg_mappings.dataset_id REFERENCES
|
|
-- opt_datasets (dataset_id)` points at a table NO sql/ file ever defined --
|
|
-- meaning `psql -d opt < sql/001... < sql/002... < sql/004...` (exactly the
|
|
-- flow docs/runbook/backup-restore.md's own "Målbild" describes, and exactly
|
|
-- what a real restore drill executes) has never actually been runnable
|
|
-- end to end. This was found by running that exact sequence for real against
|
|
-- a throwaway Postgres container during P014's restore drill, not by
|
|
-- inspecting the code.
|
|
--
|
|
-- Numbered 003 (the previously-unused gap between 002_source_registry.sql
|
|
-- and 004_iceberg_write_boundary.sql) so the existing ordering -- 001 -> 002
|
|
-- -> 004 -> 005 -- becomes 001 -> 002 -> 003 -> 004 -> 005 without renumbering
|
|
-- anything already shipped.
|
|
--
|
|
-- Columns mirror src/services/dataset-service.ts's `DatasetRecord` exactly
|
|
-- (that file's own doc comment: "Storage shape: nullable columns, exactly as
|
|
-- the eventual `opt_datasets` table would hold them") -- this is the real
|
|
-- application contract, not a guess. `dataset_type`/`status` CHECK
|
|
-- constraints mirror src/types/dataset.ts's `DatasetType`/`DatasetStatus`
|
|
-- unions verbatim. DatasetService itself remains in-memory-only in this
|
|
-- pass (wiring it to this table is a separate, larger change than closing
|
|
-- the migration-chain gap); this table exists so the SCHEMA a fresh
|
|
-- database gets by replaying sql/*.sql matches what 004/005 already assume.
|
|
|
|
CREATE TABLE IF NOT EXISTS opt_datasets (
|
|
dataset_id TEXT PRIMARY KEY,
|
|
tenant_id TEXT NOT NULL,
|
|
project_id TEXT NOT NULL,
|
|
name TEXT NOT NULL,
|
|
dataset_type TEXT NOT NULL CHECK (dataset_type IN ('raw', 'structured', 'enriched', 'curated', 'feature', 'derived')),
|
|
status TEXT NOT NULL DEFAULT 'active' CHECK (status IN ('active', 'archived', 'deprecated')),
|
|
description TEXT,
|
|
correlation_id TEXT,
|
|
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
|
updated_at TIMESTAMPTZ,
|
|
created_by TEXT
|
|
);
|
|
|
|
CREATE INDEX IF NOT EXISTS opt_datasets_tenant_project_idx ON opt_datasets (tenant_id, project_id);
|