Files
accounted/lib/connect/instance/sync.ts
T
Jakob Wennberg 0ff1b05553 feat(connect): hosted connector-key registry + validate RPC + entitlements endpoint; instance sync writes connector grants hourly (#1748)
* feat(entitlements): partition the self-host bypass so connector capabilities fall through to grants; capability_grants.source accepts 'connector'

Sovereign plan WS3 PR3: ships dark, nothing changes for hosted.

- lib/entitlements/keys.ts: CONNECTOR_CAPABILITIES = bank_sync,
  skatteverket, org_lookup, migration (services Accounted operates that a
  self-hosted instance cannot provide itself) + isConnectorCapability().
  Separate from PAID_CAPABILITIES and outside the trial-seed trigger on
  purpose: a hosted company can never hold a connector grant.
- lib/entitlements/has-capability.ts: isPaywallBypassed() -> isBypassedFor(key).
  Hosted: byte-identical (dev / DISABLE_PAYWALL bypass, FORCE_PAYWALL wins,
  else the grant lookup). Self-host: local capabilities always on
  (FORCE_PAYWALL included, as the existing test demands); connector
  capabilities behave like hosted, i.e. dev bypass, FORCE_PAYWALL, else the
  grant lookup where the connector sync will write source='connector' rows.
  getCompanyEntitlements on a self-host: local paid keys + active connector
  keys, state 'paid' with an active connector grant else 'none' (never the
  hosted trial copy).
- Migration 20260820122000: capability_grants.source CHECK gains
  'connector', found through pg_constraint (the CHECK was declared inline
  and auto-named; Postgres stores IN as = ANY, matched accordingly).
  pg-real test: connector accepted, unknown source rejected, upsert on the
  (scope, key, source) identity, trial seed writes no connector rows.
- Tests: self-hosted connector matrix (local all-on without DB, connector
  gated by grant/expiry, dev bypass all-on, FORCE_PAYWALL gates connector
  keys only, bulk resolution, entitlements shape); two pre-existing tests
  that asserted the old "self-host holds connector keys" contract updated
  to the new one.

Verified: full unit suite green, pg-real suite for lib/entitlements green
against a local supabase/postgres with every migration applied, lint
ratchet, guards. Deferred to the instance-wiring PR: adding the connector
extensions to the self-host Docker preset (dead-end upsells until a key can
be issued).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(entitlements): fold the self-host branch into the existing grants query

One .or(scopeFilter), not two: the duplicated helper pushed the
no-phantom-columns unresolvable-expression count to 380/379. Behaviour is
unchanged; the self-host matrix tests still pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(connect): hosted connector-key registry + validate RPC + entitlements endpoint; instance sync writes connector grants hourly

Sovereign plan WS3 PR4 ("key infra enabling manual sales"), stacked on the
entitlement partition (#1747). Nothing is purchasable yet; this is the
plumbing both ends need before the first manually issued key.

Hosted side:
- Migration 20260820123000: connector_keys (SHA-256 key_hash, prefix,
  org_number, pinned instance_url, scopes, status, Stripe ids,
  current_period_end, per-minute rate limit, active_company_count,
  last_seen/synced) and connector_usage_events (per-request metering,
  separate from metered_events whose company_id references hosted
  companies). RLS on, NO policies: service role only. RPC
  validate_and_increment_connector_key copies the api_keys pattern (FOR
  UPDATE, minute window, suspended reported not counted, revoked = no row)
  and is REVOKEd from PUBLIC/anon/authenticated, GRANTed to service_role.
  pg-real test covers validate/count, unknown+revoked, suspended, rate
  limit, execute privileges per role, RLS invisibility, usage cascade.
- lib/connect/contract.ts (shared wire types), lib/connect/hosted/keys.ts
  (generate/hash/validate -> 401/403/429 mapping),
  with-connector-auth.ts (Bearer or X-Connector-Key, one usage row per
  request, 500 envelope on handler throw), /api/connect/entitlements GET +
  POST (records active_company_count, pins instance_url on first report,
  never moves a pinned one), scripts/issue-connector-key.ts (dry run unless
  --confirm, prints the key once + the .env lines).

Instance side:
- lib/connect/instance/config.ts (GNUBOK_CONNECTOR_KEY, GNUBOK_CONNECT_URL
  default https://app.gnubok.se), sync.ts: reports the active company count
  and writes source='connector' grants for every company x covered scope,
  expires_at = min(now+72h, period_end+3d); 401/403 or a non-active status
  deletes them (freeze-and-retain); network/5xx/429 leave them alone.
  /api/connector/sync/cron (hourly) runs it; not_configured without a key.
- Crontab generator gains EXTRA_JOBS (variant-only jobs not in vercel.json,
  with reasons) + drift tests; docker/crontab.self-hosted regenerated with
  the hourly sync. Docs (SELF-HOSTING connector section, env templates),
  DECISIONS.

Tests: 52 new unit tests (keys, auth wrapper, route, config, sync outcomes
and grant arithmetic, cron route, crontab EXTRA_JOBS) + 7 pg-real tests
run locally against supabase/postgres with every migration applied.
no-phantom-columns ceiling +1 with a reason (the bulk grant upsert).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore(connect): update pg test to re-versioned migration 20260831190000

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb

* fix(connect): RPC errors answer 503 not 401; X-Connector-Key wins over Authorization

A hosted DB error mapped to 401 made the instance sync treat a pooler
blip as key revocation and delete its entire connector grant cache,
zeroing the 72h offline grace. 503 lands in the sync's keep-grants
branch (already test-pinned). Bearer-first extraction hashed the
upstream token on dual-header proxied calls, 401ing the exact shape
X-Connector-Key exists for.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb

* fix(connect): sync deletes grants only on a body-proven connector rejection, never bare 401/403

A WAF challenge page, edge deployment protection, or an egress proxy
answers 401/403 without the hosted app ever running; trusting status
alone wiped the instance's 72h offline grant cache within the hour.
Deletion now requires the hosted route's own rejection code in the
JSON body; codeless 401/403 keeps grants (server_error branch).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb

* fix(connect): PR #1748 review batch: https-only connect URL, atomic pin, prefix-gated Bearer, deferred metering, entitlements validation, integer months

- GNUBOK_CONNECT_URL must be https (http only for loopback); invalid or
  plaintext URLs disable the connector instead of sending the key.
- instance_url pin update filters on IS NULL; a lost race re-reads and
  reports the winner's pin.
- extractConnectorKey: a Bearer is the connector credential only with
  the gnubok_ck_ prefix; upstream Bearer falls through to X-Connector-Key.
- Usage metering runs via after() off the response path (inline outside
  a request scope).
- Sync validates entitlements shape: unknown status or malformed
  current_period_end keeps grants (server_error), never deletes.
- issue-connector-key rejects fractional --months.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb

---------

Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Emil <emilmattsson14@gmail.com>
2026-08-31 20:51:17 +02:00

237 lines
9.7 KiB
TypeScript

import type { SupabaseClient } from '@supabase/supabase-js'
import { CONNECTOR_CAPABILITIES } from '@/lib/entitlements/keys'
import { createLogger } from '@/lib/logger'
import { fetchAllRows } from '@/lib/supabase/fetch-all'
import {
CONNECTOR_ENTITLEMENTS_PATH,
type ConnectorEntitlements,
type ConnectorSyncReport,
} from '../contract'
import { getConnectorConfig, type ConnectorConfig } from './config'
const log = createLogger('connector-sync')
/**
* Grant rows ARE the offline cache. Each sync re-stamps the connector grants
* to expire at min(now + 72h, period_end + 3d): the hosted service only has
* to be reachable once in three days, and a lapsed subscription freezes the
* connector capabilities within days even if the sync keeps running. Uses
* the existing expiry check in lib/entitlements, zero new cache code.
*/
export const CONNECTOR_GRANT_TTL_MS = 72 * 60 * 60 * 1000
export const CONNECTOR_PERIOD_GRACE_MS = 3 * 24 * 60 * 60 * 1000
const UPSERT_CHUNK = 500
export type ConnectorSyncOutcome =
| 'not_configured'
| 'synced'
| 'revoked'
| 'network_error'
| 'server_error'
export interface ConnectorSyncResult {
outcome: ConnectorSyncOutcome
companies: number
grantsUpserted: number
grantsDeleted: number
status?: string
httpStatus?: number
scopes?: string[]
expiresAt?: string
message?: string
}
export interface ConnectorSyncOptions {
fetchImpl?: typeof fetch
now?: Date
config?: ConnectorConfig | null
instanceUrl?: string | null
appVersion?: string | null
}
export function connectorGrantExpiry(now: Date, currentPeriodEnd: string | null): string {
const ttl = now.getTime() + CONNECTOR_GRANT_TTL_MS
if (!currentPeriodEnd) return new Date(ttl).toISOString()
const periodGrace = new Date(currentPeriodEnd).getTime() + CONNECTOR_PERIOD_GRACE_MS
return new Date(Math.min(ttl, periodGrace)).toISOString()
}
async function deleteConnectorGrants(supabase: SupabaseClient, keepKeys: string[] | null): Promise<number> {
let query = supabase.from('capability_grants').delete({ count: 'exact' }).eq('source', 'connector')
if (keepKeys && keepKeys.length > 0) {
query = query.not('capability_key', 'in', `(${keepKeys.join(',')})`)
}
const { error, count } = await query
if (error) throw new Error(`Failed to delete connector grants: ${error.message}`)
return count ?? 0
}
/**
* One sync run: report the active company count to the hosted service, then
* translate the answer into source='connector' capability grants for every
* company on this instance.
*
* 200 active -> upsert grants for scopes, drop grants for scopes no
* longer covered (synced)
* 200 non-active -> delete all connector grants (freeze-and-retain) (revoked)
* 401 / 403 -> delete all connector grants (revoked)
* network error,
* 429, 5xx, other -> leave grants alone, they expire on their own (network_error / server_error)
*/
export async function syncConnectorEntitlements(
supabase: SupabaseClient,
options: ConnectorSyncOptions = {},
): Promise<ConnectorSyncResult> {
const config = options.config === undefined ? getConnectorConfig() : options.config
if (!config) return { outcome: 'not_configured', companies: 0, grantsUpserted: 0, grantsDeleted: 0 }
const now = options.now ?? new Date()
const fetchImpl = options.fetchImpl ?? fetch
const companies = await fetchAllRows<{ id: string }>(({ from, to }) =>
supabase
.from('companies')
.select('id')
.is('archived_at', null)
.order('id', { ascending: true })
.range(from, to),
)
const companyIds = companies.map((c) => c.id)
const report: ConnectorSyncReport = {
active_company_count: companyIds.length,
...(options.instanceUrl ? { instance_url: options.instanceUrl } : {}),
...(options.appVersion ? { app_version: options.appVersion } : {}),
}
let response: Response
try {
response = await fetchImpl(`${config.baseUrl}${CONNECTOR_ENTITLEMENTS_PATH}`, {
method: 'POST',
headers: {
Authorization: `Bearer ${config.key}`,
'Content-Type': 'application/json',
Accept: 'application/json',
},
body: JSON.stringify(report),
})
} catch (err) {
const message = err instanceof Error ? err.message : String(err)
log.warn('connector sync: hosted service unreachable, keeping existing grants', { message })
return { outcome: 'network_error', companies: companyIds.length, grantsUpserted: 0, grantsDeleted: 0, message }
}
if (response.status === 401 || response.status === 403) {
// Delete the grant cache ONLY on a genuine connector rejection, proven by
// the hosted route's own JSON body code (with-connector-auth always sends
// one). A bare-status 401/403 also comes from layers where the app never
// ran: a Vercel WAF challenge page, edge deployment protection, an egress
// proxy at the self-host. Trusting the status alone let any of those wipe
// a paying instance's 72h offline grace within the hour: the same failure
// class as the RPC-error-to-401 mapping, one layer up. Anything without a
// known rejection code keeps the grants and expires naturally.
let rejectionCode: string | null = null
try {
const body = (await response.clone().json()) as { code?: string }
if (typeof body.code === 'string') rejectionCode = body.code
} catch {
// Non-JSON body (challenge/error page): not a connector rejection.
}
const isConnectorRejection =
rejectionCode === 'CONNECTOR_KEY_MISSING' ||
rejectionCode === 'CONNECTOR_KEY_INVALID' ||
rejectionCode === 'CONNECTOR_KEY_SUSPENDED'
if (isConnectorRejection) {
const grantsDeleted = await deleteConnectorGrants(supabase, null)
log.warn('connector sync: key rejected by hosted service, connector grants removed', {
httpStatus: response.status,
code: rejectionCode,
grantsDeleted,
})
return { outcome: 'revoked', companies: companyIds.length, grantsUpserted: 0, grantsDeleted, httpStatus: response.status }
}
log.warn('connector sync: 401/403 without a connector rejection code (edge/proxy?), keeping existing grants', {
httpStatus: response.status,
code: rejectionCode,
})
return { outcome: 'server_error', companies: companyIds.length, grantsUpserted: 0, grantsDeleted: 0, httpStatus: response.status }
}
if (!response.ok) {
log.warn('connector sync: hosted service error, keeping existing grants', { httpStatus: response.status })
return { outcome: 'server_error', companies: companyIds.length, grantsUpserted: 0, grantsDeleted: 0, httpStatus: response.status }
}
let entitlements: ConnectorEntitlements
try {
const body = (await response.json()) as { data?: ConnectorEntitlements }
// Full shape validation, not just presence: an UNKNOWN status string must
// land in keep-grants (server_error), not fall through to the
// "status !== 'active'" delete below (contract drift or a tampering
// middlebox must never wipe the cache), and a malformed
// current_period_end would make connectorGrantExpiry throw mid-write.
if (
!body.data ||
!['active', 'suspended', 'revoked'].includes(body.data.status as string) ||
!Array.isArray(body.data.scopes) ||
!body.data.scopes.every((s) => typeof s === 'string') ||
!(
body.data.current_period_end === null ||
body.data.current_period_end === undefined ||
(typeof body.data.current_period_end === 'string' &&
Number.isFinite(new Date(body.data.current_period_end).getTime()))
)
) {
throw new Error('unexpected entitlements payload')
}
entitlements = body.data
} catch (err) {
const message = err instanceof Error ? err.message : String(err)
log.warn('connector sync: unreadable entitlements payload, keeping existing grants', { message })
return { outcome: 'server_error', companies: companyIds.length, grantsUpserted: 0, grantsDeleted: 0, httpStatus: response.status, message }
}
if (entitlements.status !== 'active') {
const grantsDeleted = await deleteConnectorGrants(supabase, null)
log.warn('connector sync: key not active, connector grants removed', { status: entitlements.status, grantsDeleted })
return { outcome: 'revoked', companies: companyIds.length, grantsUpserted: 0, grantsDeleted, status: entitlements.status }
}
const scopes = entitlements.scopes.filter((s) => (CONNECTOR_CAPABILITIES as readonly string[]).includes(s))
const expiresAt = connectorGrantExpiry(now, entitlements.current_period_end)
let grantsUpserted = 0
if (scopes.length > 0 && companyIds.length > 0) {
const rows = companyIds.flatMap((companyId) =>
scopes.map((capabilityKey) => ({
company_id: companyId,
team_id: null,
capability_key: capabilityKey,
source: 'connector',
expires_at: expiresAt,
})),
)
for (let i = 0; i < rows.length; i += UPSERT_CHUNK) {
const chunk = rows.slice(i, i + UPSERT_CHUNK)
const { error } = await supabase
.from('capability_grants')
.upsert(chunk, { onConflict: 'company_id,team_id,capability_key,source' })
if (error) throw new Error(`Failed to upsert connector grants: ${error.message}`)
grantsUpserted += chunk.length
}
}
// Scopes the subscription no longer covers (or none at all): drop them.
const grantsDeleted = await deleteConnectorGrants(supabase, scopes.length > 0 ? scopes : null)
log.info('connector sync complete', { companies: companyIds.length, scopes, grantsUpserted, grantsDeleted, expiresAt })
return {
outcome: 'synced',
companies: companyIds.length,
grantsUpserted,
grantsDeleted,
status: entitlements.status,
scopes,
expiresAt,
httpStatus: response.status,
}
}