* feat(entitlements): partition the self-host bypass so connector capabilities fall through to grants; capability_grants.source accepts 'connector' Sovereign plan WS3 PR3: ships dark, nothing changes for hosted. - lib/entitlements/keys.ts: CONNECTOR_CAPABILITIES = bank_sync, skatteverket, org_lookup, migration (services Accounted operates that a self-hosted instance cannot provide itself) + isConnectorCapability(). Separate from PAID_CAPABILITIES and outside the trial-seed trigger on purpose: a hosted company can never hold a connector grant. - lib/entitlements/has-capability.ts: isPaywallBypassed() -> isBypassedFor(key). Hosted: byte-identical (dev / DISABLE_PAYWALL bypass, FORCE_PAYWALL wins, else the grant lookup). Self-host: local capabilities always on (FORCE_PAYWALL included, as the existing test demands); connector capabilities behave like hosted, i.e. dev bypass, FORCE_PAYWALL, else the grant lookup where the connector sync will write source='connector' rows. getCompanyEntitlements on a self-host: local paid keys + active connector keys, state 'paid' with an active connector grant else 'none' (never the hosted trial copy). - Migration 20260820122000: capability_grants.source CHECK gains 'connector', found through pg_constraint (the CHECK was declared inline and auto-named; Postgres stores IN as = ANY, matched accordingly). pg-real test: connector accepted, unknown source rejected, upsert on the (scope, key, source) identity, trial seed writes no connector rows. - Tests: self-hosted connector matrix (local all-on without DB, connector gated by grant/expiry, dev bypass all-on, FORCE_PAYWALL gates connector keys only, bulk resolution, entitlements shape); two pre-existing tests that asserted the old "self-host holds connector keys" contract updated to the new one. Verified: full unit suite green, pg-real suite for lib/entitlements green against a local supabase/postgres with every migration applied, lint ratchet, guards. Deferred to the instance-wiring PR: adding the connector extensions to the self-host Docker preset (dead-end upsells until a key can be issued). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(entitlements): fold the self-host branch into the existing grants query One .or(scopeFilter), not two: the duplicated helper pushed the no-phantom-columns unresolvable-expression count to 380/379. Behaviour is unchanged; the self-host matrix tests still pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(connect): hosted connector-key registry + validate RPC + entitlements endpoint; instance sync writes connector grants hourly Sovereign plan WS3 PR4 ("key infra enabling manual sales"), stacked on the entitlement partition (#1747). Nothing is purchasable yet; this is the plumbing both ends need before the first manually issued key. Hosted side: - Migration 20260820123000: connector_keys (SHA-256 key_hash, prefix, org_number, pinned instance_url, scopes, status, Stripe ids, current_period_end, per-minute rate limit, active_company_count, last_seen/synced) and connector_usage_events (per-request metering, separate from metered_events whose company_id references hosted companies). RLS on, NO policies: service role only. RPC validate_and_increment_connector_key copies the api_keys pattern (FOR UPDATE, minute window, suspended reported not counted, revoked = no row) and is REVOKEd from PUBLIC/anon/authenticated, GRANTed to service_role. pg-real test covers validate/count, unknown+revoked, suspended, rate limit, execute privileges per role, RLS invisibility, usage cascade. - lib/connect/contract.ts (shared wire types), lib/connect/hosted/keys.ts (generate/hash/validate -> 401/403/429 mapping), with-connector-auth.ts (Bearer or X-Connector-Key, one usage row per request, 500 envelope on handler throw), /api/connect/entitlements GET + POST (records active_company_count, pins instance_url on first report, never moves a pinned one), scripts/issue-connector-key.ts (dry run unless --confirm, prints the key once + the .env lines). Instance side: - lib/connect/instance/config.ts (GNUBOK_CONNECTOR_KEY, GNUBOK_CONNECT_URL default https://app.gnubok.se), sync.ts: reports the active company count and writes source='connector' grants for every company x covered scope, expires_at = min(now+72h, period_end+3d); 401/403 or a non-active status deletes them (freeze-and-retain); network/5xx/429 leave them alone. /api/connector/sync/cron (hourly) runs it; not_configured without a key. - Crontab generator gains EXTRA_JOBS (variant-only jobs not in vercel.json, with reasons) + drift tests; docker/crontab.self-hosted regenerated with the hourly sync. Docs (SELF-HOSTING connector section, env templates), DECISIONS. Tests: 52 new unit tests (keys, auth wrapper, route, config, sync outcomes and grant arithmetic, cron route, crontab EXTRA_JOBS) + 7 pg-real tests run locally against supabase/postgres with every migration applied. no-phantom-columns ceiling +1 with a reason (the bulk grant upsert). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(connect): Enable Banking proxy for self-hosted instances, with a secret-free ownership ledger and a global rate budget Sovereign plan WS3 PR5a, stacked on the connector-key infra (#1748). A self-hosted instance with a `bank_sync`-scoped connector key can now connect a bank through Arcim's PSD2 credentials; the bank session id and all transaction data stay in the instance's own database (founder decision: tokens on the instance, proxy stateless). - Migration 20260820124000: `connector_connections` (secret-free ledger: sha256 of the EB session id + account uids, service-role only), `connector_upstream_counters` + RPC `connector_reserve_upstream` (global budget under EB Annex 1 §5's 300/min, shared with hosted), and `connector_keys.limits` jsonb; validate RPC v2 returns limits. All RPCs REVOKEd from PUBLIC/anon/authenticated, GRANTed service_role. pg-real covers all of it. - EB JWT minting moved to lib/connect/upstreams/enable-banking-jwt.ts (core must not import @/extensions/); the extension re-exports it, tests unchanged. - lib/connect/hosted/{state,ledger,upstream-budget}.ts: HMAC-signed connector state (15-min TTL) so the consent redirect can use OUR registered EB callback and bounce back to the instance, no per-instance redirect URI at EB; the callback route gains that connector branch. - app/api/connect/bank/[...path]: path allowlist (aspsps, auth, sessions, accounts/{uid}/{balances,transactions}), never open passthrough. POST /auth enforces the per-company connection quota + rewrites redirect/state; reads/deletes verify ledger ownership; every upstream call takes the global budget (429 + Retry-After when exhausted). - issue-connector-key.ts: scopes default bank_sync,skatteverket (TIC out of v1), --bank/skv-connections-per-company + --sync-min-interval. - Docs (SELF-HOSTING: bank connector live), DECISIONS. Verified: 52 connect unit tests + 13 pg-real (run locally against supabase/postgres with all migrations) + EB extension suite (225, jwt relocation intact); full unit suite 15 979 green; tsc, guards, lint clean. Not in this PR: SKV broker (PR5b) and instance wiring (PR6). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(connect): update ledger pg test to re-versioned migration 20260831200000 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * fix(connect): redact opaque path segments before usage metering; correct stale RPC-source comment GET/DELETE /sessions/{id} and /accounts/{uid}/... carry the raw EB session id / account uid in the pathname; metering persisted it in cleartext next to the ledger that stores only sha256(handle). Opaque segments (UUID, long hex, long base64url) now become ':id' before the connector_usage_events insert. Migration comment now cites the real prior RPC source (20260831190000). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * fix(connect): percent-encoded path segments count as opaque in metering redaction Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * fix(connect): PR #1751 review batch: https-only EB URL, body-covering timeout, quota reservation, delete-after-success, doc fix Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * fix(connect): bind the /sessions code exchange to its verified pending state; ceiling +1 Verified state signature, key/service match, and an existing pending row now precede the EB exchange; a concurrently consumed state closes the just-minted upstream session and 409s. no-phantom-columns ceiling 391 for countHeldConnections' computed .or() timestamp filter. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb --------- Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Emil <emilmattsson14@gmail.com>
202 lines
9.3 KiB
PL/PgSQL
202 lines
9.3 KiB
PL/PgSQL
-- Connector proxy: per-connection ledger, global upstream budget, key limits.
|
|
--
|
|
-- WS3 PR5. The hosted proxy under /api/connect/* brokers Enable Banking (and,
|
|
-- next, Skatteverket) for self-hosted instances. Tokens stay on the instance;
|
|
-- the proxy holds NO usable credential per customer. What it needs hosted-side
|
|
-- is only: (1) a way to recognise a connection an instance obtained through
|
|
-- its own key so it cannot use one it did not create or one belonging to
|
|
-- another instance, (2) a global rate budget so all connector traffic plus
|
|
-- hosted stays under Enable Banking's shared quota (Annex 1 §5: 20 rps /
|
|
-- 300 rpm / 10 000 per hour), and (3) per-key entitlement limits (e.g. one
|
|
-- bank connection per company) that the package sells.
|
|
--
|
|
-- Everything here is service-role only (RLS on, no policies); the validate and
|
|
-- reserve RPCs are SECURITY DEFINER and REVOKEd from PUBLIC/anon/authenticated.
|
|
|
|
-- 1. Per-key limits (the sellable package shape). Defaulted so existing keys
|
|
-- get sane values without a data backfill.
|
|
ALTER TABLE public.connector_keys
|
|
ADD COLUMN limits jsonb NOT NULL DEFAULT
|
|
'{"bank_connections_per_company": 1, "skv_connections_per_company": 1, "sync_min_interval_s": 0}'::jsonb;
|
|
|
|
-- 2. Connection ledger: one row per bank/SKV connection an instance holds,
|
|
-- identified by the SHA-256 of the upstream handle (EB session id, SKV
|
|
-- access token). The handle value itself is NEVER stored: the row proves
|
|
-- "this key created this connection for this company_ref" and nothing more.
|
|
CREATE TABLE public.connector_connections (
|
|
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
|
connector_key_id uuid NOT NULL REFERENCES public.connector_keys(id) ON DELETE CASCADE,
|
|
service text NOT NULL CHECK (service IN ('bank', 'skatteverket')),
|
|
-- The instance's own company id, opaque to us: the "per company" unit for
|
|
-- the entitlement limit. Never resolvable to a hosted company.
|
|
company_ref text NOT NULL,
|
|
provider text,
|
|
-- SHA-256 of the live handle (EB session_id / SKV access_token). Rotated on
|
|
-- refresh. Unique per service so a handle maps to at most one ledger row.
|
|
handle_hash text,
|
|
-- SHA-256 of the SKV refresh token, so a refresh exchange can be tied back
|
|
-- to its connection without storing the token.
|
|
refresh_hash text,
|
|
-- Bank account uids this connection covers (EB), for ownership checks on
|
|
-- /accounts/{uid}/... calls.
|
|
account_uids text[] NOT NULL DEFAULT '{}',
|
|
status text NOT NULL DEFAULT 'pending' CHECK (status IN ('pending', 'active', 'revoked')),
|
|
-- Signed connector-state nonce while a consent round-trip is in flight, so
|
|
-- the callback can find the pending row.
|
|
pending_state text,
|
|
created_at timestamptz NOT NULL DEFAULT now(),
|
|
activated_at timestamptz,
|
|
last_used_at timestamptz,
|
|
revoked_at timestamptz
|
|
);
|
|
|
|
CREATE INDEX idx_connector_connections_key ON public.connector_connections (connector_key_id, service, status);
|
|
CREATE INDEX idx_connector_connections_company ON public.connector_connections (connector_key_id, company_ref, service) WHERE status = 'active';
|
|
CREATE UNIQUE INDEX idx_connector_connections_handle ON public.connector_connections (service, handle_hash) WHERE handle_hash IS NOT NULL;
|
|
CREATE INDEX idx_connector_connections_pending_state ON public.connector_connections (pending_state) WHERE pending_state IS NOT NULL;
|
|
|
|
ALTER TABLE public.connector_connections ENABLE ROW LEVEL SECURITY;
|
|
-- No policies: service role only.
|
|
|
|
-- 3. Global upstream budget. One shared counter per (service, window), so all
|
|
-- connector traffic is bounded well below the ASPSP-provider quota that is
|
|
-- shared with hosted. Same shape and RPC pattern as agent_rate_counters.
|
|
CREATE TABLE public.connector_upstream_counters (
|
|
service text NOT NULL,
|
|
window_kind text NOT NULL CHECK (window_kind IN ('minute', 'hour')),
|
|
window_key text NOT NULL,
|
|
count integer NOT NULL DEFAULT 0,
|
|
updated_at timestamptz NOT NULL DEFAULT now(),
|
|
PRIMARY KEY (service, window_kind, window_key)
|
|
);
|
|
|
|
ALTER TABLE public.connector_upstream_counters ENABLE ROW LEVEL SECURITY;
|
|
CREATE POLICY connector_upstream_counters_no_select ON public.connector_upstream_counters FOR SELECT USING (false);
|
|
|
|
-- Reserve one upstream call against the global budget. Returns { ok, scope,
|
|
-- retry_after_sec }. Atomic increment-then-check with rollback, so two
|
|
-- concurrent proxy requests cannot both slip past the ceiling.
|
|
CREATE OR REPLACE FUNCTION public.connector_reserve_upstream(
|
|
p_service text,
|
|
p_minute_max integer,
|
|
p_hour_max integer
|
|
) RETURNS jsonb
|
|
LANGUAGE plpgsql SECURITY DEFINER
|
|
SET search_path = public
|
|
AS $$
|
|
DECLARE
|
|
v_minute_key text := to_char(now() AT TIME ZONE 'UTC', 'YYYY-MM-DD"T"HH24:MI');
|
|
v_hour_key text := to_char(now() AT TIME ZONE 'UTC', 'YYYY-MM-DD"T"HH24');
|
|
v_minute_count integer;
|
|
v_hour_count integer;
|
|
BEGIN
|
|
INSERT INTO public.connector_upstream_counters (service, window_kind, window_key, count)
|
|
VALUES (p_service, 'minute', v_minute_key, 1)
|
|
ON CONFLICT (service, window_kind, window_key)
|
|
DO UPDATE SET count = connector_upstream_counters.count + 1, updated_at = now()
|
|
RETURNING count INTO v_minute_count;
|
|
|
|
IF v_minute_count > p_minute_max THEN
|
|
UPDATE public.connector_upstream_counters SET count = count - 1
|
|
WHERE service = p_service AND window_kind = 'minute' AND window_key = v_minute_key;
|
|
RETURN jsonb_build_object('ok', false, 'scope', 'minute', 'retry_after_sec', 60);
|
|
END IF;
|
|
|
|
INSERT INTO public.connector_upstream_counters (service, window_kind, window_key, count)
|
|
VALUES (p_service, 'hour', v_hour_key, 1)
|
|
ON CONFLICT (service, window_kind, window_key)
|
|
DO UPDATE SET count = connector_upstream_counters.count + 1, updated_at = now()
|
|
RETURNING count INTO v_hour_count;
|
|
|
|
IF v_hour_count > p_hour_max THEN
|
|
UPDATE public.connector_upstream_counters SET count = count - 1
|
|
WHERE service = p_service AND window_kind = 'hour' AND window_key = v_hour_key;
|
|
UPDATE public.connector_upstream_counters SET count = count - 1
|
|
WHERE service = p_service AND window_kind = 'minute' AND window_key = v_minute_key;
|
|
RETURN jsonb_build_object('ok', false, 'scope', 'hour', 'retry_after_sec', 3600);
|
|
END IF;
|
|
|
|
RETURN jsonb_build_object('ok', true);
|
|
END;
|
|
$$;
|
|
|
|
REVOKE ALL ON FUNCTION public.connector_reserve_upstream(text, integer, integer) FROM PUBLIC, anon, authenticated;
|
|
GRANT EXECUTE ON FUNCTION public.connector_reserve_upstream(text, integer, integer) TO service_role;
|
|
|
|
-- 4. validate RPC v2: also return the key's limits, so the proxy has the
|
|
-- entitlement caps in the same round-trip as auth. Additive: the return
|
|
-- columns 1-7 are unchanged from 20260831190000, `limits` is appended.
|
|
DROP FUNCTION IF EXISTS public.validate_and_increment_connector_key(text);
|
|
CREATE FUNCTION public.validate_and_increment_connector_key(p_key_hash text)
|
|
RETURNS TABLE(
|
|
connector_key_id uuid,
|
|
org_number text,
|
|
instance_url text,
|
|
scopes text[],
|
|
status text,
|
|
current_period_end timestamptz,
|
|
rate_limited boolean,
|
|
limits jsonb
|
|
)
|
|
LANGUAGE plpgsql SECURITY DEFINER
|
|
SET search_path = public
|
|
AS $$
|
|
DECLARE
|
|
v_id uuid;
|
|
v_org_number text;
|
|
v_instance_url text;
|
|
v_scopes text[];
|
|
v_status text;
|
|
v_period_end timestamptz;
|
|
v_limits jsonb;
|
|
v_rate_limit_rpm integer;
|
|
v_request_count integer;
|
|
v_window_start timestamptz;
|
|
BEGIN
|
|
SELECT ck.id, ck.org_number, ck.instance_url, ck.scopes, ck.status, ck.current_period_end, ck.limits,
|
|
ck.rate_limit_rpm, ck.request_count, ck.rate_limit_window_start
|
|
INTO v_id, v_org_number, v_instance_url, v_scopes, v_status, v_period_end, v_limits,
|
|
v_rate_limit_rpm, v_request_count, v_window_start
|
|
FROM public.connector_keys ck
|
|
WHERE ck.key_hash = p_key_hash
|
|
AND ck.revoked_at IS NULL
|
|
AND ck.status <> 'revoked'
|
|
FOR UPDATE;
|
|
|
|
IF v_id IS NULL THEN
|
|
RETURN;
|
|
END IF;
|
|
|
|
IF v_status <> 'active' THEN
|
|
UPDATE public.connector_keys SET last_seen_at = now() WHERE id = v_id;
|
|
RETURN QUERY SELECT v_id, v_org_number, v_instance_url, v_scopes, v_status, v_period_end, false, v_limits;
|
|
RETURN;
|
|
END IF;
|
|
|
|
IF v_window_start IS NULL OR v_window_start < now() - interval '1 minute' THEN
|
|
UPDATE public.connector_keys
|
|
SET request_count = 1, rate_limit_window_start = now(), last_seen_at = now()
|
|
WHERE id = v_id;
|
|
RETURN QUERY SELECT v_id, v_org_number, v_instance_url, v_scopes, v_status, v_period_end, false, v_limits;
|
|
RETURN;
|
|
END IF;
|
|
|
|
IF v_request_count >= v_rate_limit_rpm THEN
|
|
RETURN QUERY SELECT v_id, v_org_number, v_instance_url, v_scopes, v_status, v_period_end, true, v_limits;
|
|
RETURN;
|
|
END IF;
|
|
|
|
UPDATE public.connector_keys
|
|
SET request_count = request_count + 1, last_seen_at = now()
|
|
WHERE id = v_id;
|
|
|
|
RETURN QUERY SELECT v_id, v_org_number, v_instance_url, v_scopes, v_status, v_period_end, false, v_limits;
|
|
END;
|
|
$$;
|
|
|
|
REVOKE ALL ON FUNCTION public.validate_and_increment_connector_key(text) FROM PUBLIC, anon, authenticated;
|
|
GRANT EXECUTE ON FUNCTION public.validate_and_increment_connector_key(text) TO service_role;
|
|
|
|
COMMENT ON TABLE public.connector_connections IS
|
|
'Secret-free ledger of bank/SKV connections a self-hosted instance obtained through its connector key. handle_hash = sha256 of the upstream handle; the handle never rests here. Service-role only.';
|