feat(0pt): PII-Gate GDPR-guardrail for personal-data sources (fail-closed)

PiiGateService makes any future personal-data source (ratsit.se and other
directory domains) technically GDPR-OK before the first byte is stored:
- recorded legal basis (art. 6) with basis_ref evidence, no policy => fail-closed
- data minimisation via field allowlist (art. 5(1)(c))
- storage limitation + enforceRetention purge (art. 5(1)(e))
- sha256-only identity keys + log redaction (art. 25/32)
- accessStatement / erase subject-rights surfaces (art. 15/17)
- robots/rate confirmation required for web+api sources; no stealth/bypass
- audit trail via AUD0 at risk L3 (0pt.pii_*)
- gate enabled by default in SourceQueryService

10 new tests; full suite 166 pass | 1 skip; typecheck (blocking gate) clean.
This commit is contained in:
2026-09-15 02:40:52 +02:00
committed by admin
parent 9cf797b5c8
commit 12080fc517
6 changed files with 678 additions and 1 deletions
+58
View File
@@ -0,0 +1,58 @@
# PII Guardrail — GDPR requirement for personal-data sources (0pt)
Status: **enforced, fail-closed.** No 0pt pipeline is crawling personuppgifter
today; the `PiiGateService` ships so that any future source that touches
personal data (Ratsit, Merinfo, Hitta, …) is **technically GDPR-OK before the
first byte is stored**.
## The rule
1. **Classify.** A source is personal-data if it self-flags
(`touches_personal_data`) or is a known directory domain (ratsit.se,
merinfo.se, hitta.se, mrkoll.se, eniro.se, birthday.se, upplysning.se).
2. **No policy → no processing.** `assertCanStore()` throws unless a
`PiiPolicy` with a valid `legal_basis` + `basis_ref` is on record.
3. **Everything is evidenced.** Each decision is emitted to AUD0 at risk L3
(`0pt.pii_policy_created | pii_data_retained | pii_erased |
pii_retention_purged`).
## GDPR article mapping
| Requirement | Implementation |
|---|---|
| Legal basis — art. 6 | `legal_basis` ∈ legitimate_interest/consent/contract/legal_obligation + `basis_ref` (LIA doc id, consent record id, contract clause, provision). Cannot be empty. |
| Data minimisation — art. 5(1)(c) | `field_allowlist`; storing any other field throws `PiiMinimisationError`. Empty allowlist = nothing stored. |
| Storage limitation — art. 5(1)(e) | `retention_days`; `enforceRetention()` purges expired rows. `0` = fetch-and-drop, nothing persisted. |
| Integrity/confidentiality — art. 32 | Only sha256 hashes of identity keys stored (`recordStored`); plaintext identity never written; log/trace redaction (`redact()`) for personnummer/e-mail/phone. |
| Right of access — art. 15 | `accessStatement(subjectHash)` — what is held on a subject, across sources. |
| Right to erasure — art. 17 | `erase({ subject_hash, reason })` tombstones every stored value for the subject. |
| Data protection by design — art. 25 | Gate defaults ON in `SourceQueryService`; no bypass knobs. |
## When someone requests a personal-data source (e.g. Ratsit)
- [x] Confirm the source is actually needed and no anonymised/official
alternative exists (official register > directory listing).
- [x] Write the **Legitimate Interest Assessment** (or record consent /
contract / legal obligation) and store its id in `basis_ref`.
LIA minimum: purpose, necessity, individuals' rights impact, mitigations.
- [x] **VERIFY current terms + robots.txt + availability** on the live site.
Do not hardcode "catsit.com allows" — re-check each enable.
- [x] Set `robots_and_rate_confirmed: true` + a `SourceRatePolicy` matching
the source's own limits (legacy N0D default for `ratsit.se` was `1/min` —
treat that as a ceiling, not a target).
- [x] Define the minimum `field_allowlist` (e.g. only `street_address`, not
`phone`).
- [x] Invoke `recordStored` for every stored value and run `enforceRetention`
on a schedule (e.g. daily cron).
- [x] Record a DPIA note if processing is "large scale" (art. 35) — bulk
directory scraping of living persons is large-scale by default.
## Never
- Pretend a source has a legal basis it does not have; `basis_ref` must point
to a real artifact.
- Store identity keys in plaintext (`pseudonymize_keys` defaults to true).
- Use stealth/bot-bypass/rate-evasion — excluded by CLAUDE.md §1 regardless of
GDPR.
- Claim compliance on a source we have not re-verified this season (`VERIFY`
markers are intentional).
+324
View File
@@ -0,0 +1,324 @@
/**
* src/services/pii-gate-service.ts
* ==================================
* 0pt.pii_* — the GDPR guardrail (fail-closed). Enforced by default wherever
* a SourceQueryService is constructed; consulted by workers before they
* persist anything for a personal-data source.
*
* Owns 0pt.pii_policy.schema.v1 and the in-memory identity registry that
* backs the subject rights surfaces (access/erasure, GDPR art. 15/17) and
* the retention worker (GDPR art. 5(1)(e)). Every decision is emitted to
* AUD0 as risk L3 events so CL0UD gets an audit trail without polling 0pt.
*
* Directory domains known to publish personuppgifter (ratsit.se, …) are
* pre-classified as personal: until a PiiPolicy with a recorded lawful
* basis exists for them, the gate REJECTS storage/query of those sources.
*/
import { createHash } from "node:crypto";
import type {
PiiPolicy,
CreatePiiPolicyInput,
PiiIdentityKey,
PiiErasureRequest,
PiiAccessStatement,
} from "../types/pii";
import type { Source } from "../types/source";
import { createAud0Client, type Aud0Client, type Aud0Event } from "../clients/aud0-client";
/** Domain publishes personuppgifter (directory/register listings). */
const KNOWN_DIRECTORY_DOMAINS = [
"ratsit.se",
"merinfo.se",
"hitta.se",
"mrkoll.se",
"eniro.se",
"birthday.se",
"upplysning.se",
];
const DECISION_TEMPLATE = "PII source %s: %s";
export class PiiPolicyExistsError extends Error {}
export class PiiLegalBasisError extends Error {}
export class PiiMinimisationError extends Error {}
export class PiiRetentionError extends Error {}
export class PiiRobotsRateError extends Error {}
export class PiiGateService {
private policies: Map<string, PiiPolicy> = new Map();
private registry: PiiIdentityKey[] = [];
private aud0: Aud0Client;
constructor(opts?: { aud0BaseUrl?: string }) {
this.aud0 = createAud0Client(opts?.aud0BaseUrl);
}
/** True if the source either self-flags or is a known directory domain. */
classifySourcePersonal(source: Pick<Source, "source_id" | "source_type" | "name" | "touches_personal_data">): boolean {
if (source.touches_personal_data === true) return true;
if (source.source_type !== "web" && source.source_type !== "api") return false;
const host = (source.name || "").toLowerCase().replace(/^https?:\/\//, "").replace(/\/.*$/, "");
return KNOWN_DIRECTORY_DOMAINS.some((d) => host === d || host.endsWith("." + d));
}
/* ------------------------------------------------------------------ *
* Policies
* ------------------------------------------------------------------ */
async createPiiPolicy(input: CreatePiiPolicyInput): Promise<PiiPolicy> {
if (this.policies.has(input.source_id)) {
throw new Error(`PiiPolicy already exists for source ${input.source_id}`);
}
// Hard validation — a policy is the lawful-basis gate, so it cannot be
// written with a half-legal posture.
if (!input.basis_ref.trim()) throw new Error("basis_ref is required (LIA id, consent id, contract clause or legal provision)");
if (!Number.isFinite(input.retention_days) || input.retention_days < 0) {
throw new Error("retention_days must be a non-negative integer (0 = fetch-and-drop)");
}
if (!Array.isArray(input.field_allowlist) || input.field_allowlist.length === 0) {
throw new Error("field_allowlist must contain at least one field (data minimisation)");
}
const now = new Date().toISOString();
const policy: PiiPolicy = {
source_id: input.source_id,
tenant_id: input.tenant_id,
project_id: input.project_id,
touches_personal_data: true,
legal_basis: input.legal_basis,
basis_ref: input.basis_ref,
retention_days: input.retention_days,
field_allowlist: [...input.field_allowlist],
pseudonymize_keys: input.pseudonymize_keys ?? true,
robots_and_rate_confirmed: input.robots_and_rate_confirmed ?? false,
created_at: now,
updated_at: null,
created_by: input.created_by ?? null,
};
this.policies.set(input.source_id, policy);
await this.safeEmit({
event_id: `evt_0pt_pii_policy_created_${input.source_id}`,
event_type: "0pt.pii_policy_created",
capability_id: "0pt.pii_policy_created",
tenant_id: input.tenant_id,
project_id: input.project_id,
actor_id: input.created_by ?? "system",
resource_type: "source_pii_policy",
resource_id: input.source_id,
risk_level: "L3",
payload: { legal_basis: input.legal_basis, retention_days: input.retention_days, field_count: input.field_allowlist.length },
});
return policy;
}
async getPiiPolicy(sourceId: string): Promise<PiiPolicy | null> {
return this.policies.get(sourceId) ?? null;
}
async listPiiPolicies(tenantId?: string): Promise<PiiPolicy[]> {
const all = Array.from(this.policies.values());
return tenantId ? all.filter((p) => p.tenant_id === tenantId) : all;
}
/* ------------------------------------------------------------------ *
* Enforcement (fail-closed)
* ------------------------------------------------------------------ */
/**
* Gate for a store/query of `fields` against `source`. Throws unless the
* source is non-personal or a lawful, minimised, retention-bounded,
* robots/rate-confirmed policy is on record.
*/
async assertCanStore(
source: Pick<Source, "source_id" | "source_type" | "name" | "touches_personal_data">,
fields: string[],
tenantId: string,
): Promise<void> {
if (!this.classifySourcePersonal(source)) return;
const policy = this.policies.get(source.source_id);
if (!policy) {
throw new PiiLegalBasisError(
DECISION_TEMPLATE.replace("%s", source.source_id).replace(
"%s",
`is a personal-data source with no PiiPolicy on record — a lawful basis (${KNOWN_DIRECTORY_DOMAINS.includes(source.name || "") ? "including robots.txt + rate limit confirmation" : "art. 6 GDPR"}) must be registered before any processing`,
),
);
}
if (policy.retention_days === 0 && fields.length > 0) {
throw new PiiRetentionError(
DECISION_TEMPLATE.replace("%s", source.source_id).replace(
"%s",
"policy forbids retention (retention_days = 0); no fields may be stored",
),
);
}
const disallowed = fields.filter((f) => !policy.field_allowlist.includes(f));
if (disallowed.length > 0) {
throw new PiiMinimisationError(
DECISION_TEMPLATE.replace("%s", source.source_id).replace(
"%s",
`field(s) outside the allowlist: ${disallowed.join(", ")} (GDPR art. 5(1)(c) minimisation)`,
),
);
}
if ((source.source_type === "web" || source.source_type === "api") && !policy.robots_and_rate_confirmed) {
throw new PiiRobotsRateError(
DECISION_TEMPLATE.replace("%s", source.source_id).replace(
"%s",
"robots.txt / rate-limit not confirmed — required for web & api personal-data sources; re-check the source terms (do not hardcode)",
),
);
}
}
/* ------------------------------------------------------------------ *
* Subject rights: access (art. 15) & erasure (art. 17), retention (art. 5)
* ------------------------------------------------------------------ */
/**
* Register that one value was stored. Only hashes are persisted; the
* plaintext identity never reaches the registry.
*/
async recordStored(input: {
source: Pick<Source, "source_id" | "source_type" | "name" | "touches_personal_data">;
tenantId: string;
identityKey: string;
subjectId: string;
field: string;
artifactRef?: string;
}): Promise<PiiIdentityKey> {
await this.assertCanStore(input.source, [input.field], input.tenantId);
const policy = this.policies.get(input.source.source_id)!;
const now = Date.now();
const key: PiiIdentityKey = {
key_hash: hash256(`${sourceIdSalt(input.source.source_id)}:${input.identityKey}`),
subject_hash: hash256(input.subjectId),
source_id: input.source.source_id,
tenant_id: input.tenantId,
field: input.field,
artifact_ref: input.artifactRef ?? null,
stored_at: new Date(now).toISOString(),
expires_at: new Date(now + policy.retention_days * 86_400_000).toISOString(),
status: "active",
};
this.registry.push(key);
await this.safeEmit({
event_id: `evt_0pt_pii_retained_${input.source.source_id}_${key.key_hash.slice(0, 12)}`,
event_type: "0pt.pii_data_retained",
capability_id: "0pt.pii_retain_identity",
tenant_id: input.tenantId,
project_id: "",
actor_id: "system",
resource_type: "pii_identity_key",
resource_id: key.key_hash,
risk_level: "L3",
payload: { source_id: input.source.source_id, field: input.field, expires_at: key.expires_at },
});
return key;
}
/** Right of access — everything currently held on a subject across sources. */
async accessStatement(subjectHash: string): Promise<PiiAccessStatement> {
const rows = this.registry.filter((r) => r.subject_hash === subjectHash && r.status === "active");
return { subject_hash: subjectHash, rows, generated_at: new Date().toISOString() };
}
/** Right to erasure — tombstone every held row for the subject. */
async erase(request: PiiErasureRequest): Promise<number> {
let count = 0;
for (const row of this.registry) {
if (row.subject_hash === request.subject_hash && row.status === "active") {
row.status = "erased";
count += 1;
}
}
await this.safeEmit({
event_id: `evt_0pt_pii_erased_${request.subject_hash.slice(0, 12)}_${Date.now()}`,
event_type: "0pt.pii_erased",
capability_id: "0pt.pii_erase",
tenant_id: "",
project_id: "",
actor_id: request.actor_id ?? "system",
resource_type: "pii_subject",
resource_id: request.subject_hash,
risk_level: "L3",
payload: { reason: request.reason, row_count: count },
});
return count;
}
/** Storage limitation — purge rows past their retention deadline. */
async enforceRetention(now: Date = new Date()): Promise<{ purged: number }> {
let purged = 0;
const keep: PiiIdentityKey[] = [];
for (const row of this.registry) {
if (row.expires_at <= now.toISOString() && row.status === "active") {
row.status = "purged";
purged += 1;
} else {
keep.push(row);
}
}
if (purged > 0) {
await this.safeEmit({
event_id: `evt_0pt_pii_retention_${now.getTime()}`,
event_type: "0pt.pii_retention_purged",
capability_id: "0pt.pii_enforce_retention",
tenant_id: "",
project_id: "",
actor_id: "system",
resource_type: "pii_identity_key",
resource_id: now.toISOString(),
risk_level: "L3",
payload: { row_count: purged },
});
}
return { purged };
}
/* ------------------------------------------------------------------ *
* Log hygiene — no personuppgifter in logs/metrics/traces
* ------------------------------------------------------------------ */
/** Scrub personnummer, e-mail and Swedish phone numbers from a log line. */
redact(line: string): string {
return line
.replace(/\b\d{8}[-–]\d{4}\b/g, "[PERSONNUMMER]")
.replace(/\b\d{6}[-–]\d{4}\b/g, "[PERSONNUMMER]")
.replace(/\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b/g, "[EMAIL]")
.replace(/(?:\+46|0046|0)\s?[1-9][\d\s.-]{7,13}/g, "[TELEFON]");
}
/* ------------------------------------------------------------------ */
private async safeEmit(event: Omit<Aud0Event, "app_id" | "created_at">): Promise<void> {
const full: Aud0Event = {
...event,
app_id: "0pt",
created_at: new Date().toISOString(),
};
try {
await this.aud0.ingestEvent(full);
} catch {
// Audit is important but must never take down a compliance decision or
// a retention pass. The caller's state change has already happened.
}
}
}
function sourceIdSalt(sourceId: string): string {
return `0pt-pii-v1:${sourceId}`;
}
/** sha256 hex digest — never the plaintext. */
export function hash256(value: string): string {
return createHash("sha256").update(value, "utf8").digest("hex");
}
+13 -1
View File
@@ -20,6 +20,7 @@
import type { Source } from "../types/source";
import { SourceService } from "./source-service";
import { PiiGateService } from "./pii-gate-service";
import { createAud0Client, type Aud0Client } from "../clients/aud0-client";
import { FabricOneLakeAdapter, type FabricOneLakeConnectionConfig, type AdapterQueryResult } from "../adapters/fabric-onelake-adapter";
@@ -76,16 +77,27 @@ function clamp(value: number, min: number, max: number): number {
export class SourceQueryService {
private sourceService: SourceService;
private aud0: Aud0Client;
/** GDPR guardrail — on by default, can be shared via opts so callers use one registry. */
private piiGate: PiiGateService;
constructor(opts?: { sourceService?: SourceService; aud0BaseUrl?: string }) {
constructor(opts?: { sourceService?: SourceService; aud0BaseUrl?: string; piiGate?: PiiGateService }) {
// Accepts an existing SourceService so callers (e.g. the API route) share
// one in-memory registry instead of each constructing an empty one.
this.sourceService = opts?.sourceService ?? new SourceService();
this.aud0 = createAud0Client(opts?.aud0BaseUrl);
this.piiGate = opts?.piiGate ?? new PiiGateService({ aud0BaseUrl: opts?.aud0BaseUrl });
}
async querySource(input: QuerySourceInput): Promise<QuerySourceResult> {
const startedAt = Date.now();
// GDPR fail-closed: personal-data sources (self-flagged or known directory
// domains such as ratsit.se) may not be processed without a PiiPolicy.
// Runs BEFORE the capability/type gate so the lawful-basis error is the
// one the caller sees for personal data.
const preflight = await this.sourceService.getSource(input.source_id);
await this.piiGate.assertCanStore(preflight, [], input.tenant_id);
const source = await this.getQueryableSource(input.source_id);
const maxRows = clamp(input.max_rows ?? SOURCE_QUERY_DEFAULT_MAX_ROWS, 1, SOURCE_QUERY_HARD_MAX_ROWS);
+102
View File
@@ -0,0 +1,102 @@
/**
* src/types/pii.ts
* =================
* 0pt.pii_policy.schema.v1 — SIAX Cloud GDPR guardrail for sources that
* CAN touch personuppgifter (personal data). Purpose: make processing of
* personal data "tekniskt OK" per GDPR (art. 5, 6, 25, 32, 17, 15):
* recorded legal basis, storage limitation, data minimisation,
* pseudonymisation, and working access/erasure surfaces.
*
* The guardrail only ever OPENS when a lawful basis is on record; nothing in
* 0pt today crawls personal data, so this ships as a fail-closed boundary:
* no policy → no storage (and, for directory domains like ratsit.se, no
* query at all).
*/
/** GDPR art. 6(1) legal bases 0pt accepts for personal-data sources. */
export type LegalBasis = "legitimate_interest" | "consent" | "contract" | "legal_obligation";
/** Compliance posture for one source that touches personal data. */
export interface PiiPolicy {
source_id: string;
tenant_id: string;
project_id: string;
touches_personal_data: true;
legal_basis: LegalBasis;
/**
* Pointer to the evidence: a recorded Legitimate Interest Assessment
* (LIA) document id, the consent record id, the contract clause, or the
* legal provision. Must exist for audit/DPIA purposes — a bare claim is
* not enough.
*/
basis_ref: string;
/**
* Max storage in days. 0 = fetch-and-drop only (never retained).
* The retention worker deletes rows where stored_at + retention_days
* < now (art. 5(1)(e)).
*/
retention_days: number;
/**
* The ONLY fields that may be stored for this source (art. 5(1)(c)
* minimisation). Empty list = nothing may be stored.
*/
field_allowlist: string[];
/**
* Pseudonymise identity keys (personnummer, email, reg-number) before
* storage; only sha256 digests are ever written (art. 25).
*/
pseudonymize_keys: boolean;
/**
* Must be true for web/api sources: robots.txt checked, rate limit in
* force (SourceRatePolicy), no auth-wall / bot-bypass technique in use.
*/
robots_and_rate_confirmed: boolean;
created_at: string;
updated_at: string | null;
created_by: string | null;
}
export interface CreatePiiPolicyInput {
source_id: string;
tenant_id: string;
project_id: string;
legal_basis: LegalBasis;
basis_ref: string;
retention_days: number;
field_allowlist: string[];
pseudonymize_keys?: boolean;
robots_and_rate_confirmed?: boolean;
created_by?: string;
}
/**
* Registry row for one stored personal-data value. Only hashes are held —
* the key_hash lets 0pt locate the subject (right of access, art. 15) and
* the subject_hash lets it erase every artifact for that subject across
* sources (right to erasure, art. 17) without ever storing the plaintext
* identity.
*/
export interface PiiIdentityKey {
key_hash: string;
subject_hash: string;
source_id: string;
tenant_id: string;
field: string;
artifact_ref: string | null;
stored_at: string;
expires_at: string;
status: "active" | "erased" | "purged";
}
export interface PiiErasureRequest {
subject_hash: string;
reason: "deleted_by_subject" | "retention_expired" | "consent_withdrawn";
actor_id?: string;
correlation_id?: string;
}
export interface PiiAccessStatement {
subject_hash: string;
rows: PiiIdentityKey[];
generated_at: string;
}
+8
View File
@@ -63,6 +63,13 @@ export interface Source {
adapter_id?: string;
/** Mirrors dim_source.access (docs/03-data-model.md). */
access?: SourceAccess;
/**
* Explicit self-flag that this source CAN expose personuppgifter. When
* true (or when the source is a known directory domain like ratsit.se),
* the PiiGateService requires a PiiPolicy with a recorded lawful basis
* before any storage/query (fail-closed, GDPR art. 6).
*/
touches_personal_data?: boolean;
description?: string;
status: SourceStatus;
correlation_id?: string;
@@ -86,6 +93,7 @@ export interface CreateSourceInput {
name: string;
adapter_id?: string;
access?: SourceAccess;
touches_personal_data?: boolean;
description?: string;
status?: SourceStatus;
correlation_id?: string;
+173
View File
@@ -0,0 +1,173 @@
import { describe, it, expect, vi } from "vitest";
import { PiiGateService } from "@/src/services/pii-gate-service";
import { SourceService } from "@/src/services/source-service";
import { SourceQueryService } from "@/src/services/source-query-service";
import type { FabricOneLakeConnectionConfig } from "@/src/adapters/fabric-onelake-adapter";
/** Minimal config — never reached (the pii gate fires first). */
const EMPTY_CFG = {} as FabricOneLakeConnectionConfig;
describe("PiiGateService — GDPR guardrail (fail-closed)", () => {
const ratsit = {
source_id: "src_ratsit_1",
tenant_id: "t1",
project_id: "p1",
source_type: "web" as const,
name: "ratsit.se",
status: "active" as const,
created_at: new Date().toISOString(),
};
const company = {
source_id: "src_company_1",
tenant_id: "t1",
project_id: "p1",
source_type: "api" as const,
name: "allabolag.se",
status: "active" as const,
created_at: new Date().toISOString(),
};
it("classifies known directory domains as personal-data sources", () => {
const gate = new PiiGateService();
expect(gate.classifySourcePersonal(ratsit)).toBe(true);
expect(gate.classifySourcePersonal({ ...ratsit, name: "www.ratsit.se" })).toBe(true);
expect(gate.classifySourcePersonal(company)).toBe(false);
expect(gate.classifySourcePersonal({ ...company, touches_personal_data: true })).toBe(true);
});
it("rejects processing a personal-data source with no recorded lawful basis", async () => {
const gate = new PiiGateService();
await expect(gate.assertCanStore(ratsit, [], "t1")).rejects.toThrowError(/no PiiPolicy/);
});
it("rejects half-legal policies (missing basis_ref, zero retention, empty allowlist)", async () => {
const gate = new PiiGateService();
await expect(
gate.createPiiPolicy({ source_id: "s1", tenant_id: "t1", project_id: "p1", legal_basis: "legitimate_interest", basis_ref: " ", retention_days: 90, field_allowlist: ["address"] }),
).rejects.toThrow(/basis_ref is required/);
await expect(
gate.createPiiPolicy({ source_id: "s1", tenant_id: "t1", project_id: "p1", legal_basis: "consent", basis_ref: "consent-record-1", retention_days: -1, field_allowlist: ["address"] }),
).rejects.toThrow(/retention_days/);
await expect(
gate.createPiiPolicy({ source_id: "s1", tenant_id: "t1", project_id: "p1", legal_basis: "contract", basis_ref: "clause-9", retention_days: 90, field_allowlist: [] }),
).rejects.toThrow(/data minimisation/);
});
it("blocks out-of-allowlist fields (minimisation) and unconfirmed web sources", async () => {
const gate = new PiiGateService();
await gate.createPiiPolicy({
source_id: ratsit.source_id, tenant_id: "t1", project_id: "p1",
legal_basis: "consent", basis_ref: "consent-record-42",
retention_days: 90, field_allowlist: ["name", "address"],
});
await expect(gate.assertCanStore(ratsit, ["name", "phone"], "t1")).rejects.toThrowError(/outside the allowlist/);
await expect(gate.assertCanStore(ratsit, ["name"], "t1")).rejects.toThrowError(/robots.txt \/ rate-limit not confirmed/);
const ratsitConfirmed = { ...ratsit, source_id: "src_ratsit_confirmed" };
await gate.createPiiPolicy({
source_id: ratsitConfirmed.source_id, tenant_id: "t1", project_id: "p1",
legal_basis: "consent", basis_ref: "consent-record-42",
retention_days: 90, field_allowlist: ["name", "address"], robots_and_rate_confirmed: true,
});
await expect(gate.assertCanStore(ratsitConfirmed, ["name"], "t1")).resolves.toBeUndefined();
});
it("forbids any storage when retention_days = 0", async () => {
const gate = new PiiGateService();
await gate.createPiiPolicy({
source_id: "src_drop_1", tenant_id: "t1", project_id: "p1",
legal_basis: "legitimate_interest", basis_ref: "lia-2026-001",
retention_days: 0, field_allowlist: ["name"], robots_and_rate_confirmed: true,
});
await expect(
gate.assertCanStore({ ...company, source_id: "src_drop_1", name: "ratsit.se" }, ["name"], "t1"),
).rejects.toThrowError(/forbids retention/);
});
it("stores ruled bases and enables access/erasure without plaintext identity", async () => {
const gate = new PiiGateService();
await gate.createPiiPolicy({
source_id: ratsit.source_id, tenant_id: "t1", project_id: "p1",
legal_basis: "legal_obligation", basis_ref: "prop-§5",
retention_days: 30, field_allowlist: ["street_address"], robots_and_rate_confirmed: true,
});
const key = await gate.recordStored({
source: ratsit, tenantId: "t1",
identityKey: "20250101-1234", subjectId: "subj-77", field: "street_address", artifactRef: "st0re/dataset-a",
});
expect(key.key_hash).toMatch(/^[0-9a-f]{64}$/);
expect(key.key_hash).not.toContain("20250101");
const statement = await gate.accessStatement(key.subject_hash);
expect(statement.rows).toHaveLength(1);
expect(statement.rows[0].field).toBe("street_address");
const erased = await gate.erase({ subject_hash: key.subject_hash, reason: "deleted_by_subject" });
expect(erased).toBe(1);
expect((await gate.accessStatement(key.subject_hash)).rows).toHaveLength(0);
});
it("retention worker purges rows past their deadline", async () => {
const gate = new PiiGateService();
vi.useFakeTimers();
vi.setSystemTime(new Date("2026-08-01T00:00:00Z"));
await gate.createPiiPolicy({
source_id: ratsit.source_id, tenant_id: "t1", project_id: "p1",
legal_basis: "legitimate_interest", basis_ref: "lia-2026-002",
retention_days: 30, field_allowlist: ["street_address"], robots_and_rate_confirmed: true,
});
await gate.recordStored({ source: ratsit, tenantId: "t1", identityKey: "k-1", subjectId: "s-1", field: "street_address" });
vi.setSystemTime(new Date("2026-09-15T00:00:00Z"));
const { purged } = await gate.enforceRetention(new Date("2026-09-15T00:00:00Z"));
vi.useRealTimers();
expect(purged).toBe(1);
});
it("redacts personnummer, e-mail and phone from log lines", () => {
const gate = new PiiGateService();
const out = gate.redact(
"subject=202501011234 res=" +
'evt=19720101-1234 mail=simon@siax.io tel=070-123 45 67 sök=ok',
);
expect(out).not.toMatch(/\d{8}-\d{4}/);
expect(out).toContain("[PERSONNUMMER]");
expect(out).toContain("[EMAIL]");
expect(out).toContain("[TELEFON]");
});
it("SourceQueryService blocks a personal-data source by default (fail-closed)", async () => {
const sourceService = new SourceService();
await sourceService.createSource({ ...ratsit });
const service = new SourceQueryService({ sourceService });
await expect(
service.querySource({
source_id: ratsit.source_id,
query: "SELECT 1",
tenant_id: "t1", project_id: "p1",
connection_config: EMPTY_CFG,
}),
).rejects.toThrowError(/no PiiPolicy/);
});
it("SourceQueryService passes non-personal sources straight through to capability checks", async () => {
const sourceService = new SourceService();
await sourceService.createSource({
source_id: "src_safe_1", tenant_id: "t1", project_id: "p1",
source_type: "web", name: "example.org", status: "active",
});
const service = new SourceQueryService({ sourceService });
// Gate passes; the sync path then correctly rejects non-database sources.
await expect(
service.querySource({
source_id: "src_safe_1",
query: "SELECT 1",
tenant_id: "t1", project_id: "p1",
connection_config: EMPTY_CFG,
}),
).rejects.toThrowError(/source_type "web"/);
});
});