CI (SIAX Cloud) / security (push) Successful in 23s
Security Scan (SIAX Cloud) / security (push) Successful in 22s
CI (SIAX Cloud) / integration (push) Successful in 29s
CI (SIAX Cloud) / quality (push) Successful in 45s
CI (SIAX Cloud) / security (pull_request) Successful in 27s
CI (SIAX Cloud) / integration (pull_request) Successful in 31s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 29s
CI (SIAX Cloud) / quality (pull_request) Successful in 48s
Continues P010 (branch renamed from feat/p010-source-registry-persistence):
closes the three remaining BEFORE_STATE.md gaps (§2 TS/Python split-brain,
§3 missing provenance fields, §4 untested failure modes), all additive to
the production pipeline — the Python QA filter's delivery decision is
unchanged by any of this.
TS/Python bridge (§2):
- New route surface app/api/0pt/quality/* (rules/runs/issues CRUD +
POST /api/0pt/quality/bridge-run) over the shared QualityService
singleton (src/services/quality.ts), fixing the same "no shared
instance" defect the Source Registry had.
- src/quality/quality-bridge.ts auto-provisions a deterministic,
tenant-scoped baseline rule set on first use so the Python engine never
pre-registers rules through the CRUD surface.
- services/api/workers/quality_bridge.py: the Python-side HTTP client.
Feature-flagged off by default (OPT_QUALITY_BRIDGE_ENABLED), fails open
on any error (never raises into run_job, never changes `delivered`).
Advisory only — real per-row gating against the TS engine isn't
implementable yet (ExpectationEngine returns dataset-level pass/fail
counts + samples, not a per-row boolean array); documented, not faked.
- Live-verified end-to-end: `next dev` (AUTH_ENFORCE=false) + the real
Python pipeline hitting the real brreg.no API, round-tripping through
the real bridge route, receiving a real QualityService score.
Provenance (§3): sources/connectors.py's Provenance dataclass gains
retrieved_at, transform, connector_version, producer, quality,
entity_resolution, artifact_ref (all previously absent from the real
dataflow entirely) via a make_provenance() factory used by every connector.
resolve.py stamps entity_resolution on merged records' provenance (never on
unmerged ones). pipeline.py stamps artifact_ref at delivery and quality
once the bridge succeeds.
Failure tests (§4), real fault injection against real code (services/api,
new pytest + respx suite, not vitest mocks):
- dedup-determinism + duplicate-input + out-of-order: resolve.py under
shuffled/interleaved/exhaustive-permutation input, plus an explicit,
verified test of the one case that IS legitimately order-sensitive
(non-transitive similarity in a greedy clusterer) instead of hiding it.
- schema-evolution: respx-mocked upstream responses with added/removed/
alien-shaped fields through the real brreg connector + pipeline.
- restart: connector exceptions mid-job (including one from respx itself)
never crash run_job.
- backfill: the real property that holds given no persistence layer
exists (replay is compute-idempotent) — NOT a fabricated dedup-store
test.
- CDC-interruption and true backfill/delivery-store idempotency:
explicitly documented as not implementable against this codebase
(no CDC pipeline, no delivery store exist anywhere) via skip markers
with reasons, in tests/test_known_gaps.py, rather than silently
dropped or faked.
Testing: 256/256 vitest (249 pre-existing + 7 new bridge route/service
tests) and 28/28 pytest (2 documented skips) green locally. tsc -p
tsconfig.check.json has zero new errors (14 pre-existing errors, all in
the untracked/unaudited src/fleet, src/graph, src/metrics, src/projections
paths this pass does not touch, per BEFORE_STATE.md).
No PR opened, per instructions.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
165 lines
7.7 KiB
Python
165 lines
7.7 KiB
Python
"""
|
|
Tests for workers/quality_bridge.py (the Python half of the P010 TS/Python
|
|
bridge) and its wiring into workers/pipeline.run_job. Verifies the
|
|
fail-open contract end to end: disabled, not configured, timeout,
|
|
non-2xx, malformed body, and success — and, critically, that `delivered`
|
|
rows from run_job are IDENTICAL regardless of which of those the bridge
|
|
call resolves to (the bridge is advisory, never a second gate).
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import httpx
|
|
import pytest
|
|
import respx
|
|
|
|
from workers.pipeline import run_job
|
|
from workers.quality_bridge import run_quality_bridge
|
|
|
|
BRIDGE_URL = "http://edge.local/api/0pt/quality/bridge-run"
|
|
BRREG_URL = "https://data.brreg.no/enhetsregisteret/api/enheter/923609016"
|
|
|
|
VALID_BRREG_BODY = {
|
|
"navn": "Volvo Personvagnar Norge AS",
|
|
"konkurs": False,
|
|
"forretningsadresse": {"adresse": ["Storgata 1"]},
|
|
}
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def _clean_env(monkeypatch):
|
|
monkeypatch.delenv("OPT_QUALITY_BRIDGE_ENABLED", raising=False)
|
|
monkeypatch.delenv("OPT_QUALITY_BRIDGE_URL", raising=False)
|
|
monkeypatch.delenv("EDGE_URL", raising=False)
|
|
monkeypatch.delenv("INTERNAL_TOKEN", raising=False)
|
|
|
|
|
|
class TestBridgeDirectly:
|
|
@pytest.mark.asyncio
|
|
async def test_disabled_by_default_makes_zero_network_calls(self, monkeypatch):
|
|
# No respx mock registered at all — if this made an HTTP call it would raise.
|
|
result = await run_quality_bridge(tenant_id="t_a", project_id="p_1", dataset_id="ds_1", rows=[])
|
|
assert result.status == "disabled"
|
|
assert result.score is None
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_enabled_but_no_url_configured(self, monkeypatch):
|
|
monkeypatch.setenv("OPT_QUALITY_BRIDGE_ENABLED", "true")
|
|
result = await run_quality_bridge(tenant_id="t_a", project_id="p_1", dataset_id="ds_1", rows=[])
|
|
assert result.status == "not_configured"
|
|
|
|
@pytest.mark.asyncio
|
|
@respx.mock
|
|
async def test_successful_run_is_parsed(self, monkeypatch):
|
|
monkeypatch.setenv("OPT_QUALITY_BRIDGE_ENABLED", "true")
|
|
monkeypatch.setenv("OPT_QUALITY_BRIDGE_URL", "http://edge.local")
|
|
respx.post(BRIDGE_URL).mock(return_value=httpx.Response(201, json={
|
|
"run": {"run_id": "dqrun_1", "status": "succeeded", "score": 100},
|
|
"issues": [],
|
|
}))
|
|
result = await run_quality_bridge(
|
|
tenant_id="t_a", project_id="p_1", dataset_id="ds_1", rows=[{"name": "X", "confidence": 0.9}]
|
|
)
|
|
assert result.status == "ok"
|
|
assert result.score == 100
|
|
assert result.run_id == "dqrun_1"
|
|
assert result.issue_count == 0
|
|
|
|
@pytest.mark.asyncio
|
|
@respx.mock
|
|
async def test_timeout_fails_open(self, monkeypatch):
|
|
monkeypatch.setenv("OPT_QUALITY_BRIDGE_ENABLED", "true")
|
|
monkeypatch.setenv("OPT_QUALITY_BRIDGE_URL", "http://edge.local")
|
|
respx.post(BRIDGE_URL).mock(side_effect=httpx.TimeoutException("timed out"))
|
|
result = await run_quality_bridge(tenant_id="t_a", project_id="p_1", dataset_id="ds_1", rows=[])
|
|
assert result.status == "unavailable"
|
|
assert result.detail == "timeout"
|
|
|
|
@pytest.mark.asyncio
|
|
@respx.mock
|
|
async def test_non_2xx_fails_open(self, monkeypatch):
|
|
monkeypatch.setenv("OPT_QUALITY_BRIDGE_ENABLED", "true")
|
|
monkeypatch.setenv("OPT_QUALITY_BRIDGE_URL", "http://edge.local")
|
|
respx.post(BRIDGE_URL).mock(return_value=httpx.Response(500, json={"error": "boom"}))
|
|
result = await run_quality_bridge(tenant_id="t_a", project_id="p_1", dataset_id="ds_1", rows=[])
|
|
assert result.status == "unavailable"
|
|
assert "500" in result.detail
|
|
|
|
@pytest.mark.asyncio
|
|
@respx.mock
|
|
async def test_malformed_body_fails_open_not_raises(self, monkeypatch):
|
|
monkeypatch.setenv("OPT_QUALITY_BRIDGE_ENABLED", "true")
|
|
monkeypatch.setenv("OPT_QUALITY_BRIDGE_URL", "http://edge.local")
|
|
respx.post(BRIDGE_URL).mock(return_value=httpx.Response(201, content=b"not json"))
|
|
result = await run_quality_bridge(tenant_id="t_a", project_id="p_1", dataset_id="ds_1", rows=[])
|
|
assert result.status == "unavailable"
|
|
|
|
@pytest.mark.asyncio
|
|
@respx.mock
|
|
async def test_connection_refused_fails_open(self, monkeypatch):
|
|
monkeypatch.setenv("OPT_QUALITY_BRIDGE_ENABLED", "true")
|
|
monkeypatch.setenv("OPT_QUALITY_BRIDGE_URL", "http://edge.local")
|
|
respx.post(BRIDGE_URL).mock(side_effect=httpx.ConnectError("refused"))
|
|
result = await run_quality_bridge(tenant_id="t_a", project_id="p_1", dataset_id="ds_1", rows=[])
|
|
assert result.status == "unavailable"
|
|
|
|
|
|
class TestBridgeWiredIntoThePipeline:
|
|
"""The part that matters most: whatever the bridge does, `delivered`
|
|
(the rows the customer actually receives) must not change."""
|
|
|
|
@pytest.mark.asyncio
|
|
@respx.mock
|
|
async def test_missing_tenant_id_skips_the_bridge_honestly(self):
|
|
respx.get(BRREG_URL).mock(return_value=httpx.Response(200, json=VALID_BRREG_BODY))
|
|
result = await run_job("order_notenant", {"markets": ["NO"], "confidence": 50, "seeds": {"NO": ["923609016"]}})
|
|
assert result["metrics"]["quality_bridge"]["status"] == "not_configured"
|
|
assert len(result["records"]) == 1 # delivery unaffected
|
|
|
|
@pytest.mark.asyncio
|
|
@respx.mock
|
|
async def test_bridge_disabled_delivers_identically_to_bridge_enabled_and_failing(self, monkeypatch):
|
|
respx.get(BRREG_URL).mock(return_value=httpx.Response(200, json=VALID_BRREG_BODY))
|
|
config = {
|
|
"markets": ["NO"], "confidence": 50, "seeds": {"NO": ["923609016"]},
|
|
"tenant_id": "t_bridge_pipeline", "project_id": "p_1",
|
|
}
|
|
|
|
monkeypatch.delenv("OPT_QUALITY_BRIDGE_ENABLED", raising=False)
|
|
disabled_result = await run_job("order_bridge_off", config)
|
|
|
|
monkeypatch.setenv("OPT_QUALITY_BRIDGE_ENABLED", "true")
|
|
monkeypatch.setenv("OPT_QUALITY_BRIDGE_URL", "http://edge.local")
|
|
respx.post(BRIDGE_URL).mock(side_effect=httpx.ConnectError("edge unreachable"))
|
|
failing_result = await run_job("order_bridge_fail", config)
|
|
|
|
assert disabled_result["metrics"]["rows"] == failing_result["metrics"]["rows"] == 1
|
|
assert [r["name"] for r in disabled_result["records"]] == [r["name"] for r in failing_result["records"]]
|
|
assert disabled_result["metrics"]["quality_bridge"]["status"] == "disabled"
|
|
assert failing_result["metrics"]["quality_bridge"]["status"] == "unavailable"
|
|
|
|
@pytest.mark.asyncio
|
|
@respx.mock
|
|
async def test_bridge_success_stamps_quality_provenance_without_dropping_or_adding_rows(self, monkeypatch):
|
|
respx.get(BRREG_URL).mock(return_value=httpx.Response(200, json=VALID_BRREG_BODY))
|
|
monkeypatch.setenv("OPT_QUALITY_BRIDGE_ENABLED", "true")
|
|
monkeypatch.setenv("OPT_QUALITY_BRIDGE_URL", "http://edge.local")
|
|
respx.post(BRIDGE_URL).mock(return_value=httpx.Response(201, json={
|
|
"run": {"run_id": "dqrun_9", "status": "succeeded", "score": 100},
|
|
"issues": [],
|
|
}))
|
|
result = await run_job(
|
|
"order_bridge_ok",
|
|
{
|
|
"markets": ["NO"], "confidence": 50, "seeds": {"NO": ["923609016"]},
|
|
"tenant_id": "t_bridge_ok", "project_id": "p_1",
|
|
},
|
|
)
|
|
assert result["metrics"]["rows"] == 1
|
|
assert result["metrics"]["quality_bridge"]["status"] == "ok"
|
|
assert result["metrics"]["quality_bridge"]["score"] == 100
|
|
for p in result["records"][0]["provenance"]:
|
|
assert p["quality"] == {
|
|
"score": 100, "run_id": "dqrun_9", "run_status": "succeeded", "checked_by": "0pt-quality-bridge",
|
|
}
|
|
assert p["artifact_ref"] == "0pt:pipeline_delivery:order_bridge_ok"
|