Simon AxelssonandClaude Sonnet 5
9740e20818
CI (SIAX Cloud) / security (push) Successful in 23s
Security Scan (SIAX Cloud) / security (push) Successful in 22s
CI (SIAX Cloud) / integration (push) Successful in 29s
CI (SIAX Cloud) / quality (push) Successful in 45s
CI (SIAX Cloud) / security (pull_request) Successful in 27s
CI (SIAX Cloud) / integration (pull_request) Successful in 31s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 29s
CI (SIAX Cloud) / quality (pull_request) Successful in 48s
Continues P010 (branch renamed from feat/p010-source-registry-persistence):
closes the three remaining BEFORE_STATE.md gaps (§2 TS/Python split-brain,
§3 missing provenance fields, §4 untested failure modes), all additive to
the production pipeline — the Python QA filter's delivery decision is
unchanged by any of this.
TS/Python bridge (§2):
- New route surface app/api/0pt/quality/* (rules/runs/issues CRUD +
POST /api/0pt/quality/bridge-run) over the shared QualityService
singleton (src/services/quality.ts), fixing the same "no shared
instance" defect the Source Registry had.
- src/quality/quality-bridge.ts auto-provisions a deterministic,
tenant-scoped baseline rule set on first use so the Python engine never
pre-registers rules through the CRUD surface.
- services/api/workers/quality_bridge.py: the Python-side HTTP client.
Feature-flagged off by default (OPT_QUALITY_BRIDGE_ENABLED), fails open
on any error (never raises into run_job, never changes `delivered`).
Advisory only — real per-row gating against the TS engine isn't
implementable yet (ExpectationEngine returns dataset-level pass/fail
counts + samples, not a per-row boolean array); documented, not faked.
- Live-verified end-to-end: `next dev` (AUTH_ENFORCE=false) + the real
Python pipeline hitting the real brreg.no API, round-tripping through
the real bridge route, receiving a real QualityService score.
Provenance (§3): sources/connectors.py's Provenance dataclass gains
retrieved_at, transform, connector_version, producer, quality,
entity_resolution, artifact_ref (all previously absent from the real
dataflow entirely) via a make_provenance() factory used by every connector.
resolve.py stamps entity_resolution on merged records' provenance (never on
unmerged ones). pipeline.py stamps artifact_ref at delivery and quality
once the bridge succeeds.
Failure tests (§4), real fault injection against real code (services/api,
new pytest + respx suite, not vitest mocks):
- dedup-determinism + duplicate-input + out-of-order: resolve.py under
shuffled/interleaved/exhaustive-permutation input, plus an explicit,
verified test of the one case that IS legitimately order-sensitive
(non-transitive similarity in a greedy clusterer) instead of hiding it.
- schema-evolution: respx-mocked upstream responses with added/removed/
alien-shaped fields through the real brreg connector + pipeline.
- restart: connector exceptions mid-job (including one from respx itself)
never crash run_job.
- backfill: the real property that holds given no persistence layer
exists (replay is compute-idempotent) — NOT a fabricated dedup-store
test.
- CDC-interruption and true backfill/delivery-store idempotency:
explicitly documented as not implementable against this codebase
(no CDC pipeline, no delivery store exist anywhere) via skip markers
with reasons, in tests/test_known_gaps.py, rather than silently
dropped or faked.
Testing: 256/256 vitest (249 pre-existing + 7 new bridge route/service
tests) and 28/28 pytest (2 documented skips) green locally. tsc -p
tsconfig.check.json has zero new errors (14 pre-existing errors, all in
the untracked/unaudited src/fleet, src/graph, src/metrics, src/projections
paths this pass does not touch, per BEFORE_STATE.md).
No PR opened, per instructions.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
54 lines
2.4 KiB
Python
54 lines
2.4 KiB
Python
"""
|
|
Honest documentation of two of the seven requested failure-test categories
|
|
that are NOT implemented as real fault-injection tests in this pass, and
|
|
why — matching this repo's established convention of stating a gap in code
|
|
(BEFORE_STATE.md, docs/P0_DATASET_QUALITY_FLOW.md) rather than silently
|
|
dropping it or faking coverage. These are `pytest.skip` markers, not
|
|
disguised passes, so they show up in the test report as explicitly
|
|
unaddressed rather than vanishing from view.
|
|
|
|
* CDC-interruption: this repo has NO change-data-capture pipeline
|
|
anywhere — no Debezium/WAL-slot consumer, no CDC producer, no
|
|
replication-slot code, confirmed by a full-repo search
|
|
(`grep -rn "debezium\\|wal_level\\|replication.slot\\|cdc" services/ src/`
|
|
finds nothing on this topic). A "CDC-interruption" fault-injection
|
|
test needs a CDC stream to interrupt; fabricating one to have
|
|
something to test would be building fictional infrastructure and
|
|
testing the fiction, not this codebase — the exact failure mode this
|
|
project's CLAUDE.md and BEFORE_STATE.md explicitly warn against
|
|
("never fabricate").
|
|
* True backfill idempotency (dedup against a delivery store): the
|
|
pipeline has no persistence layer at all (see workers/pipeline.py —
|
|
every run_job() call recomputes from scratch, nothing is written to a
|
|
database). "Backfill doesn't create duplicate deliveries" requires a
|
|
delivery store to check duplicates against, which doesn't exist.
|
|
test_pipeline_fault_injection.py::TestBackfillReplay covers the
|
|
narrower, real property that DOES hold today (replaying an identical
|
|
order config is compute-idempotent — byte-identical output, no drift)
|
|
and documents this exact distinction inline.
|
|
"""
|
|
import pytest
|
|
|
|
|
|
@pytest.mark.skip(
|
|
reason=(
|
|
"No CDC pipeline exists anywhere in this repo (verified by full-repo "
|
|
"search) — nothing to interrupt without inventing fictional "
|
|
"infrastructure. See this file's module docstring."
|
|
)
|
|
)
|
|
def test_cdc_interruption_not_implementable_against_real_code():
|
|
pass
|
|
|
|
|
|
@pytest.mark.skip(
|
|
reason=(
|
|
"No delivery-store persistence layer exists to dedupe a backfill "
|
|
"against — the compute-idempotency property that DOES exist is "
|
|
"covered for real in test_pipeline_fault_injection.py::"
|
|
"TestBackfillReplay. See this file's module docstring."
|
|
)
|
|
)
|
|
def test_true_backfill_idempotency_not_implementable_against_real_code():
|
|
pass
|