* feat(settings): per-company opt-in for data analysis of bookkeeping outcomes (#1346)
Adds company_settings.data_analysis_opt_in (default false, no grandfathering)
and gates every path that reads bookkeeping outcomes across companies on it:
POST /api/agent/categorize/outcome stops writing calibration samples for
companies that have not opted in, and the backtest / calibration-fit scripts
filter to opted-in company ids. One helper (lib/company/data-analysis.ts)
is the single gate for future analysis paths. A toggle on Inställningar >
Företag states plainly what is analysed (proposed vs booked account, amount,
confidence; no free text, no personal data) in sv and en. The flag is UI-only
by design: consent is a human action, so it is absent from the v1 REST / MCP
settings pick lists.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nAd8XJ2RPCmG2eKoLBdna
* fix(settings): make data-analysis consent copy true for the backtest path (#1346)
Addresses adversarial review findings on PR #2007:
- Findings 1-3 (consent narrower than the gated processing): the flag also
gates scripts/backtest-categorize.ts, which re-runs transaction
descriptions, merchant names and matched underlag through the model. The
sv/en toggle help and disclosure now state that explicitly as "evaluation
runs" and no longer claim that free text or underlag are excluded. The
migration header and COMMENT, the lib/company/data-analysis.ts docstring,
the backtest script header and the DECISIONS line say the same. Kept the
gate (un-gating would put the script back to reading every company with
no consent at all). A test pins that both locales name those inputs and
contain no "no free text / no underlag" denial.
- Finding 4 (member sees an active switch that RLS rejects): the toggle is
now enabled only for owner/admin, matching the company_settings update
policy; the disclosure says only administrators can change the choice.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nAd8XJ2RPCmG2eKoLBdna
* fix(scripts): address round-2 review findings (#1346)
1. [minor] Opted-in company filter was an unbounded PostgREST `in` list in
the URL (scripts/fit-categorize-calibration.ts, scripts/backtest-categorize.ts).
Both scripts now read the opted-in ids through a shared, paginated helper
(listDataAnalysisOptedInCompanyIds, fetchAllRows so the pre-fetch no longer
caps at 1000) and query per chunk of 100 ids (chunkCompanyIds). The fit
script pages each chunk on the id PK; the backtest merges per-chunk
results and re-cuts to the N most recent overall. Early exit on zero
opt-ins is kept. Pinned with tests in lib/company/__tests__/data-analysis.test.ts.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nAd8XJ2RPCmG2eKoLBdna
* fix(scripts): coerce a null transaction description in the backtest (#1346)
The typed row from the chunked consent query made description nullable,
which TransactionForSelect does not accept; fall back to the original
description or an empty string, as the untyped row did implicitly before.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nAd8XJ2RPCmG2eKoLBdna
---------
Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Two handlers hand-rolled supabase.auth.getUser() and therefore skipped the
MFA (AAL2) gate on hosted: DELETE /api/transactions/[id] and
GET /api/transactions. Both sat next to a sibling handler that was already
wrapped, and the raw-route-auth ratchet exempted a file as soon as any
withRouteContext call appeared in it, so they were never flagged.
GET /api/documents/[id]/integrity and POST /api/agent/categorize called
requireAuth() directly (MFA enforced, but no request id, no completion log,
no canonical error envelope). All four are now withRouteContext handlers
with identical company scoping and responses; the transaction delete keeps
its viewer rejection via requireWrite.
The guard now judges each top-level export segment of a route file on its
own, so a wrapped handler no longer exempts a hand-rolled sibling. Baseline
is unchanged (mcp-oauth/authorize remains the one grandfathered file).
Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The highest-leverage quality lever for real users. A prod read showed the
majority are cold-start (365 companies, 32.7k unbooked transactions, median 0
counterparty templates), so the LLM selector carries them — and it was only
seeing the bank line (merchant + amount), never the receipt.
- lib/agent/categorize/underlag.ts: gathers the matched receipt/invoice text
for a transaction (receipts.matched_transaction_id + invoice_inbox_items
.matched_transaction_id + the transaction's own attached document) and renders
it as bounded Swedish text — supplier, date, total, moms, line items. Same
sources the categorization intent reads, as a string not a tool loop. Core
queries the tables directly (no @/extensions import). Best-effort: '' on any
failure.
- POST /api/agent/categorize gathers it server-side when the caller didn't
supply `underlag`, so the model reasons over the actual supplier + line items.
Server-side only, no client change. 31 categorize tests green; lint + guards +
scoped typecheck clean.
Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Turns the selector's raw confidence into a score that means what it says.
- lib/agent/categorize/calibration.ts: the engine. Isotonic regression
(pool-adjacent-violators, distribution-free + monotonic) over
(confidence, was_correct) samples → a calibrator; plus reliabilityByBucket,
ECE, and bandFor(). bandFor NEVER returns 'auto' without a fitted calibrator
(no silent booking on an unproven score) and never auto-books above an amount
cap. 12 engine tests (overconfidence pulled down, underconfidence lifted,
monotonicity, ECE, band gating).
- Measurement loop: migration categorize_calibration_samples (append-only,
company-scoped RLS, confidence CHECK [0,1]) + POST /api/agent/categorize/
outcome logging one sample (proposed vs actually booked) fire-and-forget from
QuickReviewDialog on a successful book (sandbox skipped). AiCategorizeProposal
surfaces the proposal metadata via onProposal.
- scripts/fit-categorize-calibration.ts (read-only): prints the reliability
diagram + ECE + fitted calibrator once data has accumulated.
Fitting needs a few hundred real outcomes, so nothing calibrates today — the
loop starts collecting, and "säker" stays uncalibrated (no auto-book) until the
data proves it. 131 unit tests green; RLS covered by a pg-real test.
Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
The auto-booking cascade end to end (retrieval → selector), minus the write.
- lib/agent/categorize/candidates.ts (Tier 1): assembles the deterministic
candidate slate for a transaction — the learned counterparty template
(strongest, carries its own VAT) plus mapping rules / patterns / per-merchant
history via the same engine gnubok_suggest_categories uses. No model call.
Deduped by account (highest confidence wins), capped; suggestions get the
category's default VAT treatment derived.
- POST /api/agent/categorize: loads the transaction + company VAT context,
runs Tier 1 → Tier 2 selectAccount, returns the proposed account + VAT +
confidence + reasoning + the candidate slate. Never posts anything — the
caller renders an approval card. Gated on configured (any provider incl.
local), same gates as /api/agent/ask.
12 tests: candidate merge/dedupe/VAT-derivation, and the route (401/429/400/
403/404/503 + happy path threading entity type, VAT, underlag, samples).
Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Jakob Wennberg