adminandClaude Sonnet 5
bda21e87eb
Re-runs this session's task spec against 0pt's actual repo state, which
turned out to already have a substantial, tested implementation from a
prior pass (35b6ec6): dataset<->Iceberg-table mapping, source rate policy,
and an ST0RE lakehouse client that was honestly marked "not live-verified".
This pass found api.st0re.siax.io IS reachable from this environment and
used it -- first via curl to safely discover the real contract, then
through 0pt's own updated code -- to drive a real namespace -> table ->
snapshot write against production ST0RE (task item 3's "write ONE first
test dataset" proof). That surfaced and fixed three real bugs the
mock-only pass could not have caught:
- ST0RE requires X-Tenant-Id/X-Actor-Id headers on every lakehouse route;
the previous HTTP client sent none. Fixed via a threaded
St0reRequestContext.
- IcebergTableInput.datasetId is a foreign key into ST0RE's OWN dataset
table, not a free-form string -- sending 0pt's dataset_id 400s live with
a Postgres FK violation. Fixed by no longer sending it; 0pt's
dataset<->table linkage lives entirely in DatasetIcebergMapping.
- ST0RE enforces a real unique constraint on lakehouse namespaces
(tenant_id, project_id, name) with no upsert route. Fixed with a
client-side lookup-or-create (listNamespaces + findOrRegisterNamespace).
Also closed a field-level gap against the task's literal schema spec:
0pt.dataset_iceberg_mapping.schema.v1 now carries namespace_id (a real
reference to ST0RE's LakehouseNamespace.namespaceId, not guessed) and
mapped_at, both additive alongside the existing namespace/created_at
fields (sql/005, JSON schema, types, service).
tests/p0/iceberg-write-service.realinfra.test.ts replaces the prior
it.todo with a real SEAM-pattern live test: attempts the real HTTP call,
skips honestly if ST0RE is unreachable, never fabricates success. It ran
successfully against production ST0RE this session (real namespace/table/
snapshot UUIDs logged in docs/ICEBERG_WRITE_BOUNDARY.md).
Two ST0RE-side cross-tenant read bugs were found and flagged (not fixed
here -- out of scope for this repo): GET .../namespaces ignores its own
tenant header and query filters; GET .../tables/:id doesn't check the
caller's tenant against the table's owner.
npm test: 155 passed, 0 failed, 0 todo (up from 154 passed + 1 todo).
npx tsc --noEmit: 1 pre-existing, unrelated error in
quality-report-service.ts (present before this pass, part of the wider
uncommitted P0 data-platform build already in this repo) -- not touched.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
21 lines
1.2 KiB
SQL
21 lines
1.2 KiB
SQL
-- 0pt Iceberg write-path ownership: field-level gap closure pass
|
|
-- (docs/ICEBERG_WRITE_BOUNDARY.md "Live verification" section, 2026-07-13).
|
|
--
|
|
-- 004_iceberg_write_boundary.sql shipped opt_dataset_iceberg_mappings without
|
|
-- namespace_id/mapped_at. This session's task spec names those fields
|
|
-- literally (0pt.dataset_iceberg_mapping.schema.v1: dataset_id,
|
|
-- iceberg_table_id, namespace_id, mapped_at, mapped_by), and this session
|
|
-- also live-verified (real HTTP calls against api.st0re.siax.io) that
|
|
-- ST0RE's namespace registry (/v1/st0re/lakehouse/namespaces) returns a real
|
|
-- namespaceId distinct from the bare namespace name -- so namespace_id is a
|
|
-- genuine reference, not a renamed duplicate of the existing `namespace`
|
|
-- column. Both new columns are nullable additive columns; no data migration
|
|
-- needed since 004 has not been applied to a live database by any session
|
|
-- (same honest status as 002/003 before it).
|
|
|
|
ALTER TABLE opt_dataset_iceberg_mappings
|
|
ADD COLUMN IF NOT EXISTS namespace_id TEXT,
|
|
ADD COLUMN IF NOT EXISTS mapped_at TIMESTAMPTZ NOT NULL DEFAULT now();
|
|
|
|
CREATE INDEX IF NOT EXISTS opt_dataset_iceberg_mappings_namespace_idx ON opt_dataset_iceberg_mappings (namespace_id);
|