* feat(entitlements): partition the self-host bypass so connector capabilities fall through to grants; capability_grants.source accepts 'connector' Sovereign plan WS3 PR3: ships dark, nothing changes for hosted. - lib/entitlements/keys.ts: CONNECTOR_CAPABILITIES = bank_sync, skatteverket, org_lookup, migration (services Accounted operates that a self-hosted instance cannot provide itself) + isConnectorCapability(). Separate from PAID_CAPABILITIES and outside the trial-seed trigger on purpose: a hosted company can never hold a connector grant. - lib/entitlements/has-capability.ts: isPaywallBypassed() -> isBypassedFor(key). Hosted: byte-identical (dev / DISABLE_PAYWALL bypass, FORCE_PAYWALL wins, else the grant lookup). Self-host: local capabilities always on (FORCE_PAYWALL included, as the existing test demands); connector capabilities behave like hosted, i.e. dev bypass, FORCE_PAYWALL, else the grant lookup where the connector sync will write source='connector' rows. getCompanyEntitlements on a self-host: local paid keys + active connector keys, state 'paid' with an active connector grant else 'none' (never the hosted trial copy). - Migration 20260820122000: capability_grants.source CHECK gains 'connector', found through pg_constraint (the CHECK was declared inline and auto-named; Postgres stores IN as = ANY, matched accordingly). pg-real test: connector accepted, unknown source rejected, upsert on the (scope, key, source) identity, trial seed writes no connector rows. - Tests: self-hosted connector matrix (local all-on without DB, connector gated by grant/expiry, dev bypass all-on, FORCE_PAYWALL gates connector keys only, bulk resolution, entitlements shape); two pre-existing tests that asserted the old "self-host holds connector keys" contract updated to the new one. Verified: full unit suite green, pg-real suite for lib/entitlements green against a local supabase/postgres with every migration applied, lint ratchet, guards. Deferred to the instance-wiring PR: adding the connector extensions to the self-host Docker preset (dead-end upsells until a key can be issued). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(entitlements): fold the self-host branch into the existing grants query One .or(scopeFilter), not two: the duplicated helper pushed the no-phantom-columns unresolvable-expression count to 380/379. Behaviour is unchanged; the self-host matrix tests still pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(connect): hosted connector-key registry + validate RPC + entitlements endpoint; instance sync writes connector grants hourly Sovereign plan WS3 PR4 ("key infra enabling manual sales"), stacked on the entitlement partition (#1747). Nothing is purchasable yet; this is the plumbing both ends need before the first manually issued key. Hosted side: - Migration 20260820123000: connector_keys (SHA-256 key_hash, prefix, org_number, pinned instance_url, scopes, status, Stripe ids, current_period_end, per-minute rate limit, active_company_count, last_seen/synced) and connector_usage_events (per-request metering, separate from metered_events whose company_id references hosted companies). RLS on, NO policies: service role only. RPC validate_and_increment_connector_key copies the api_keys pattern (FOR UPDATE, minute window, suspended reported not counted, revoked = no row) and is REVOKEd from PUBLIC/anon/authenticated, GRANTed to service_role. pg-real test covers validate/count, unknown+revoked, suspended, rate limit, execute privileges per role, RLS invisibility, usage cascade. - lib/connect/contract.ts (shared wire types), lib/connect/hosted/keys.ts (generate/hash/validate -> 401/403/429 mapping), with-connector-auth.ts (Bearer or X-Connector-Key, one usage row per request, 500 envelope on handler throw), /api/connect/entitlements GET + POST (records active_company_count, pins instance_url on first report, never moves a pinned one), scripts/issue-connector-key.ts (dry run unless --confirm, prints the key once + the .env lines). Instance side: - lib/connect/instance/config.ts (GNUBOK_CONNECTOR_KEY, GNUBOK_CONNECT_URL default https://app.gnubok.se), sync.ts: reports the active company count and writes source='connector' grants for every company x covered scope, expires_at = min(now+72h, period_end+3d); 401/403 or a non-active status deletes them (freeze-and-retain); network/5xx/429 leave them alone. /api/connector/sync/cron (hourly) runs it; not_configured without a key. - Crontab generator gains EXTRA_JOBS (variant-only jobs not in vercel.json, with reasons) + drift tests; docker/crontab.self-hosted regenerated with the hourly sync. Docs (SELF-HOSTING connector section, env templates), DECISIONS. Tests: 52 new unit tests (keys, auth wrapper, route, config, sync outcomes and grant arithmetic, cron route, crontab EXTRA_JOBS) + 7 pg-real tests run locally against supabase/postgres with every migration applied. no-phantom-columns ceiling +1 with a reason (the bulk grant upsert). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(connect): Enable Banking proxy for self-hosted instances, with a secret-free ownership ledger and a global rate budget Sovereign plan WS3 PR5a, stacked on the connector-key infra (#1748). A self-hosted instance with a `bank_sync`-scoped connector key can now connect a bank through Arcim's PSD2 credentials; the bank session id and all transaction data stay in the instance's own database (founder decision: tokens on the instance, proxy stateless). - Migration 20260820124000: `connector_connections` (secret-free ledger: sha256 of the EB session id + account uids, service-role only), `connector_upstream_counters` + RPC `connector_reserve_upstream` (global budget under EB Annex 1 §5's 300/min, shared with hosted), and `connector_keys.limits` jsonb; validate RPC v2 returns limits. All RPCs REVOKEd from PUBLIC/anon/authenticated, GRANTed service_role. pg-real covers all of it. - EB JWT minting moved to lib/connect/upstreams/enable-banking-jwt.ts (core must not import @/extensions/); the extension re-exports it, tests unchanged. - lib/connect/hosted/{state,ledger,upstream-budget}.ts: HMAC-signed connector state (15-min TTL) so the consent redirect can use OUR registered EB callback and bounce back to the instance, no per-instance redirect URI at EB; the callback route gains that connector branch. - app/api/connect/bank/[...path]: path allowlist (aspsps, auth, sessions, accounts/{uid}/{balances,transactions}), never open passthrough. POST /auth enforces the per-company connection quota + rewrites redirect/state; reads/deletes verify ledger ownership; every upstream call takes the global budget (429 + Retry-After when exhausted). - issue-connector-key.ts: scopes default bank_sync,skatteverket (TIC out of v1), --bank/skv-connections-per-company + --sync-min-interval. - Docs (SELF-HOSTING: bank connector live), DECISIONS. Verified: 52 connect unit tests + 13 pg-real (run locally against supabase/postgres with all migrations) + EB extension suite (225, jwt relocation intact); full unit suite 15 979 green; tsc, guards, lint clean. Not in this PR: SKV broker (PR5b) and instance wiring (PR6). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(connect): Skatteverket broker + data proxy for self-hosted instances (tokens stay on the instance) Sovereign plan WS3 PR5b, stacked on the bank proxy (#1751). A self-hosted instance with a `skatteverket`-scoped connector key can now run the BankID consent, file VAT/AGI and sync skattekonto through Arcim's registered Skatteverket client; the SKV tokens are returned to the instance and stored (encrypted) there. - lib/connect/upstreams/skatteverket-oauth.ts: core-side SKV OAuth + data helpers (authorize URL, code/refresh exchange with Arcim's client secret, the four backing-API base URLs, the API-gateway Client_Id/Client_Secret headers). Core can't import @/extensions/, so this duplicates the extension's endpoints/scope set (one integrator = Arcim), mirroring the EB JWT relocation. - app/api/connect/skv/oauth/authorize-url: builds the authorize URL against OUR registered redirect_uri + a signed connector state, per-company SKV connection quota, pending ledger row. - app/api/connect/skv/oauth/token: exchanges/refreshes and RETURNS the tokens to the instance; the ledger keeps only sha256(access_token) + sha256(refresh_token). - app/api/connect/skv/api/[...path]: allowlist over moms / skattekonto / agd-inlamning / agd-period. The instance sends the user's SKV Bearer (as X-Connector-Upstream-Authorization) + X-Connector-Key; the proxy checks the token hash against the ledger, adds Arcim's gateway credentials (never exposed to the instance), forwards. Same per-key + global budget as bank. - The Skatteverket extension /callback gains the connector branch (isConnectorState -> 302 back to the instance; code never exchanged there). - Docs (SELF-HOSTING: SKV connector live) + DECISIONS. Tests: SKV oauth lib, authorize-url, token, data proxy, callback connector branch (all green; 74 connect + 425 connect/SKV). tsc, guards, lint clean; no-phantom-columns held at 380 (literal update branches). Not in this PR: instance-side wiring (PR6). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(connect): SKV broker review+skeptic batch: state-bound exchange, owned-only refresh, identity-number redaction, quota reservation, https-only bases, docs dedupe Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * docs(self-host): restore the instance-wiring qualifier in the connector section Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * fix(connect): PR #1757 review batch 2: redirect 'error' on credential fetches, mandatory ledger writes before token return, MD037 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * fix(connect): reject encoded path separators in SKV data-proxy segments (traversal guard) The WHATWG parser normalizes raw dot segments before the route runs; what survives is an encoded separator inside a segment (a%2Fb, ..%2Fx, a%5Cb), which would escape the allowlisted service base once the upstream fetch re-normalizes. splitPath now decodes each segment and rejects dot segments and separator-bearing values. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb --------- Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Emil <emilmattsson14@gmail.com>
141 lines
5.8 KiB
TypeScript
141 lines
5.8 KiB
TypeScript
import type { SupabaseClient } from '@supabase/supabase-js'
|
|
import { NextResponse, after } from 'next/server'
|
|
import { createServiceClientNoCookies } from '@/lib/auth/api-keys'
|
|
import { createLogger, type Logger } from '@/lib/logger'
|
|
import { CONNECTOR_KEY_HEADER, CONNECTOR_KEY_PREFIX } from '../contract'
|
|
import { validateConnectorKey, type ValidatedConnectorKey } from './keys'
|
|
|
|
/**
|
|
* Wrapper for the hosted connector endpoints (app/api/connect/*), the
|
|
* connector twin of lib/api/v1/with-api-v1.ts.
|
|
*
|
|
* 1. Extracts the key from `Authorization: Bearer gnubok_ck_...` or, for
|
|
* proxied calls where Authorization carries an upstream token, from
|
|
* `X-Connector-Key`.
|
|
* 2. Validates it through the atomic RPC (rate-limited): 401 unknown /
|
|
* revoked, 403 suspended, 429 over the per-minute limit.
|
|
* 3. Runs the handler with a service-role client and the validated key.
|
|
* 4. Records one connector_usage_events row (metering, never blocking).
|
|
*
|
|
* These routes are reached by self-hosted instances with a key, never by a
|
|
* browser session, so there is no cookie/MFA handling here; the proxy
|
|
* middleware already lets /api/* through for bearer callers.
|
|
*/
|
|
|
|
export interface ConnectorContext {
|
|
requestId: string
|
|
log: Logger
|
|
supabase: SupabaseClient
|
|
key: ValidatedConnectorKey
|
|
}
|
|
|
|
type ConnectorHandler = (request: Request, ctx: ConnectorContext) => Promise<NextResponse | Response>
|
|
|
|
export function extractConnectorKey(request: Request): string | null {
|
|
// A Bearer value is the connector credential only when it looks like one
|
|
// (gnubok_ck_ prefix); otherwise it is an UPSTREAM token on a proxied call
|
|
// and the connector key rides in X-Connector-Key. Hashing the upstream
|
|
// token instead would 401 every such request. A non-prefixed Bearer with
|
|
// no X-Connector-Key still falls through to the format check's 401.
|
|
const auth = request.headers.get('authorization')
|
|
const bearer = auth?.startsWith('Bearer ') ? auth.slice(7).trim() || null : null
|
|
if (bearer?.startsWith(CONNECTOR_KEY_PREFIX)) return bearer
|
|
const header = request.headers.get(CONNECTOR_KEY_HEADER)
|
|
if (header?.trim()) return header.trim()
|
|
return bearer
|
|
}
|
|
|
|
/**
|
|
* Opaque path segments (UUIDs, long hex, long base64url tokens) become ':id'
|
|
* before a path is persisted for metering. Literal route words (sessions,
|
|
* accounts, balances, aspsps, ...) survive, so the metric keys stay useful.
|
|
*/
|
|
// The 10+-digit rule covers Swedish identity numbers riding in SKV data-proxy
|
|
// paths (personnummer 10/12 digits, orgnr 10, redovisare12 16+orgnr): they are
|
|
// personal data of the instance's downstream clients and must never rest in
|
|
// metering. Period segments (YYYYMM, 6 digits) survive for metric granularity.
|
|
const OPAQUE_SEGMENT = /^(?:[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}|[0-9a-f]{16,}|[A-Za-z0-9_-]{20,}|\d{10,})$/i
|
|
|
|
export function redactEndpoint(pathname: string): string {
|
|
return pathname
|
|
.split('/')
|
|
// A percent-encoded segment is opaque too: URL.pathname does not decode,
|
|
// so an encoded handle would otherwise slip the pattern while the route
|
|
// decodes and uses it.
|
|
.map((segment) => (OPAQUE_SEGMENT.test(segment) || segment.includes('%') ? ':id' : segment))
|
|
.join('/')
|
|
}
|
|
|
|
export function withConnectorAuth(
|
|
operation: string,
|
|
handler: ConnectorHandler,
|
|
options: { service?: string } = {},
|
|
): (request: Request) => Promise<Response> {
|
|
const service = options.service ?? operation.split('.')[1] ?? operation
|
|
return async function wrapped(request: Request): Promise<Response> {
|
|
const requestId = `conn_${crypto.randomUUID()}`
|
|
const log = createLogger(`connect/${operation}`, { requestId, operation })
|
|
const supabase = createServiceClientNoCookies()
|
|
|
|
const presented = extractConnectorKey(request)
|
|
if (!presented) {
|
|
return NextResponse.json(
|
|
{ error: 'Missing connector key', code: 'CONNECTOR_KEY_MISSING' },
|
|
{ status: 401, headers: { 'X-Request-Id': requestId } },
|
|
)
|
|
}
|
|
const validation = await validateConnectorKey(presented, supabase)
|
|
if (!validation.ok) {
|
|
log.warn('connector auth rejected', { code: validation.code })
|
|
return NextResponse.json(
|
|
{ error: validation.error, code: validation.code },
|
|
{ status: validation.status, headers: { 'X-Request-Id': requestId } },
|
|
)
|
|
}
|
|
|
|
let response: Response
|
|
try {
|
|
response = await handler(request, { requestId, log, supabase, key: validation.key })
|
|
} catch (err) {
|
|
log.error('connector handler failed', err)
|
|
response = NextResponse.json(
|
|
{ error: 'Internal error', code: 'INTERNAL_ERROR' },
|
|
{ status: 500 },
|
|
)
|
|
}
|
|
response.headers.set('X-Request-Id', requestId)
|
|
|
|
// Metering: one row per request, never on the critical path. The path is
|
|
// REDACTED first: proxied paths carry the EB session id / account uid as
|
|
// segments, and the whole ledger design is that those handles never rest
|
|
// hosted-side (connector_connections stores sha256 only). Persisting the
|
|
// raw pathname would put the cleartext handle in connector_usage_events.
|
|
const endpoint = (() => {
|
|
try {
|
|
return redactEndpoint(new URL(request.url).pathname)
|
|
} catch {
|
|
return null
|
|
}
|
|
})()
|
|
const recordUsage = async (): Promise<void> => {
|
|
const { error: usageError } = await supabase.from('connector_usage_events').insert({
|
|
connector_key_id: validation.key.id,
|
|
service,
|
|
endpoint,
|
|
status_code: response.status,
|
|
})
|
|
if (usageError) log.warn('usage event not recorded', { err: usageError.message })
|
|
}
|
|
try {
|
|
// Off the response path: the caller should not wait on metering.
|
|
// Same pattern as lib/webhooks/dispatch-kick.ts.
|
|
after(() => recordUsage())
|
|
} catch {
|
|
// Outside a request scope (unit tests): record inline.
|
|
await recordUsage()
|
|
}
|
|
|
|
return response
|
|
}
|
|
}
|