feat(categorize): Tier 1 candidate gathering + the proposal route (#1781)

The auto-booking cascade end to end (retrieval → selector), minus the write.

- lib/agent/categorize/candidates.ts (Tier 1): assembles the deterministic
  candidate slate for a transaction — the learned counterparty template
  (strongest, carries its own VAT) plus mapping rules / patterns / per-merchant
  history via the same engine gnubok_suggest_categories uses. No model call.
  Deduped by account (highest confidence wins), capped; suggestions get the
  category's default VAT treatment derived.
- POST /api/agent/categorize: loads the transaction + company VAT context,
  runs Tier 1 → Tier 2 selectAccount, returns the proposed account + VAT +
  confidence + reasoning + the candidate slate. Never posts anything — the
  caller renders an approval card. Gated on configured (any provider incl.
  local), same gates as /api/agent/ask.

12 tests: candidate merge/dedupe/VAT-derivation, and the route (401/429/400/
403/404/503 + happy path threading entity type, VAT, underlag, samples).

Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Jakob Wennberg
2026-08-21 13:32:09 +02:00
committed by GitHub
co-authored by Jakob Wennberg Claude Opus 4.8
parent b17878e58f
commit e7a5e65ecf
5 changed files with 452 additions and 0 deletions
+1
View File
@@ -1143,3 +1143,4 @@ One line per decision: `[YYYY-MM-DD] <decision>: <why>`. Appended by agents and
[2026-08-21] Skeptic pass on #1778: routing endpoint switched from router.project-osrm.org (demo server, non-commercial only) to FOSSGIS routing.openstreetmap.de (fair use, attribution shown in UI); lookup click-gated instead of as-you-type per Nominatim's no-autocomplete policy; OSMF/FOSSGIS disclosed as independent recipients, not underbiträden.
[2026-08-21] SCHABLONINTAKT_RATE_BY_CLOSING_YEAR backfilled 2020-2024 (SLR 30 Nov per Riksgalden: -0.09/-0.10/0.23 floored to 0.5 %, 1.94 %, 2.62 %) and the rate now resolves lazily (resolveSchablonintaktRate: 0 when no 212X account carried an opening balance): the table only covered 2025/2026 and the builder consulted it unconditionally, so every AB closing a pre-2025 year got a generic 500 at bokslut step 3 (126 open FY2024 periods on prod, incl. a byra trial). 2019 and earlier stay unmapped on purpose: the 100 %-of-SLR rule keys on beskattningsar STARTING 2019-01-01+ (prop. 2017/18:245), so a 2019 closing can be a brutet ar under the old 72 % factor. Unmapped-year-with-fonder now raises SCHABLONINTAKT_RATE_NOT_CONFIGURED (typed, 500 so runtime-error clustering still flags the missed December update) instead of INTERNAL_ERROR.
[2026-08-21] RIP-4 "optimal" auto-booking cascade, Tier 2 = the provider-agnostic account SELECTOR (lib/agent/categorize/select-account.ts), built per the 2026 research (artifact dc0c2760): the model does NOT free-form a categorizer; it CHOOSES from a closed set — the deterministic candidate accounts (Tier 1) + the 19 standard business categories (each maps deterministically to a BAS account via getDefaultAccountForCategory) + "needs_review". So the model can't invent an account, account/VAT stays deterministic and validated, and it runs on any provider (Bedrock or a local model) via getAiService().generateStructured. Founder chose the optimal path (model selects on EVERY transaction, LLM calls are fine), so confidence uses SELF-CONSISTENCY (default 3 samples, majority vote, agreement fraction) combined with the model's stated confidence and floored by the winning candidate's deterministic confidence — never the model's verbalized confidence alone (research: systematically overconfident). reasoning field precedes choice in the schema (reason-before-choice). needs_review is never auto-applied. Calibration of the combined score → the auto-book/suggest/review gate is a later tier. Not yet wired: Tier 1 candidate gathering (counterparty templates + getSuggestedCategories) + a route + the ApprovalCard UI (next PRs).
[2026-08-21] RIP-4 cascade Tier 1 (candidate gathering) + the proposal route. lib/agent/categorize/candidates.ts assembles the deterministic candidate slate for a transaction exactly like the gnubok_suggest_categories MCP tool (mapping_rules + per-merchant history via buildMerchantHistory/getSuggestedCategories + the learned counterparty template), NO model call, deduped by account (highest confidence wins) and capped. Suggestions carry no VAT so it derives the category default (getDefaultVatTreatmentForCategory); the counterparty template carries its own. POST /api/agent/categorize runs Tier 1 → Tier 2 selectAccount and returns the proposal + the candidate slate; it NEVER posts (the caller renders an approval card). Gated on getAiStatus().configured (any provider incl. local), same gates as /api/agent/ask (auth via requireAuth, rate, membership, sandbox, capability). Next: wire the transaction row to this route + the ApprovalCard (UI, visual sign-off), then calibration + the auto-book gate.
@@ -0,0 +1,109 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { NextResponse } from 'next/server'
import { createMockRequest, parseJsonResponse } from '@/tests/helpers'
const requireAuthMock = vi.fn()
vi.mock('@/lib/auth/require-auth', () => ({ requireAuth: () => requireAuthMock() }))
vi.mock('@/lib/company/context', () => ({ getActiveCompanyId: vi.fn().mockResolvedValue('company-1') }))
const checkRate = vi.fn()
vi.mock('@/lib/rate-limits/agent', () => ({
checkAgentRateLimit: () => checkRate(),
agentRateLimitResponseBody: () => ({ error: 'rate' }),
}))
vi.mock('@/lib/sandbox/guard', () => ({ guardSandbox: vi.fn().mockResolvedValue(null) }))
const requireCapability = vi.fn()
vi.mock('@/lib/entitlements/has-capability', () => ({ requireCapability: () => requireCapability() }))
vi.mock('@/lib/entitlements/keys', () => ({ CAPABILITY: { ai: 'ai' } }))
const aiStatus = vi.fn()
vi.mock('@/lib/ai', () => ({ getAiStatus: () => aiStatus() }))
const gatherCandidates = vi.fn()
vi.mock('@/lib/agent/categorize/candidates', () => ({ gatherCandidates: (...a: unknown[]) => gatherCandidates(...a) }))
const selectAccount = vi.fn()
vi.mock('@/lib/agent/categorize/select-account', () => ({ selectAccount: (...a: unknown[]) => selectAccount(...a) }))
import { POST } from '../route'
// supabase router: membership + transactions + companies + company_settings.
function makeSupabase(opts: { tx?: unknown } = {}) {
return {
from(table: string) {
const rows: Record<string, unknown> = {
company_members: { user_id: 'user-1' },
transactions: opts.tx === undefined ? { id: 'tx-1' } : opts.tx,
companies: { entity_type: 'aktiebolag' },
company_settings: { vat_registered: true },
}
const chain = {
select: () => chain,
eq: () => chain,
maybeSingle: async () => ({ data: rows[table] ?? null }),
}
return chain
},
}
}
const supabase = makeSupabase()
const VALID_TX = '11111111-1111-4111-8111-111111111111'
const body = (o: Record<string, unknown> = {}) => ({ transaction_id: VALID_TX, ...o })
beforeEach(() => {
vi.clearAllMocks()
requireAuthMock.mockResolvedValue({ user: { id: 'user-1' }, supabase, error: null })
checkRate.mockResolvedValue({ ok: true })
requireCapability.mockResolvedValue(null)
aiStatus.mockReturnValue({ configured: true })
gatherCandidates.mockResolvedValue([{ account: '5410', label: 'Material', vatTreatment: 'standard_25', source: 'counterparty_template', confidence: 0.9 }])
selectAccount.mockResolvedValue({
account: '5410', category: null, vatTreatment: 'standard_25', reverseCharge: false,
confidence: 0.86, modelConfidence: 'high', agreement: 1, reasoning: 'r',
choice: { kind: 'candidate', account: '5410' }, model: 'qwen3.8', fromCandidate: true,
})
})
describe('POST /api/agent/categorize', () => {
it('401 when unauthenticated', async () => {
requireAuthMock.mockResolvedValue({ user: null, supabase, error: NextResponse.json({ error: 'x' }, { status: 401 }) })
expect((await POST(createMockRequest('/x', { method: 'POST', body: body() }))).status).toBe(401)
})
it('429 when rate limited', async () => {
checkRate.mockResolvedValue({ ok: false })
expect((await POST(createMockRequest('/x', { method: 'POST', body: body() }))).status).toBe(429)
})
it('400 on a missing/invalid transaction_id', async () => {
expect((await POST(createMockRequest('/x', { method: 'POST', body: {} }))).status).toBe(400)
expect((await POST(createMockRequest('/x', { method: 'POST', body: { transaction_id: 'nope' } }))).status).toBe(400)
})
it('403 without the ai capability', async () => {
requireCapability.mockResolvedValue(NextResponse.json({ error: 'pay' }, { status: 403 }))
expect((await POST(createMockRequest('/x', { method: 'POST', body: body() }))).status).toBe(403)
})
it('503 when no backend is configured', async () => {
aiStatus.mockReturnValue({ configured: false })
const res = await POST(createMockRequest('/x', { method: 'POST', body: body() }))
const { status, body: b } = await parseJsonResponse<{ code: string }>(res)
expect(status).toBe(503)
expect(b.code).toBe('ai_unconfigured')
expect(selectAccount).not.toHaveBeenCalled()
})
it('404 when the transaction is not found / not this company', async () => {
requireAuthMock.mockResolvedValue({ user: { id: 'user-1' }, supabase: makeSupabase({ tx: null }), error: null })
const res = await POST(createMockRequest('/x', { method: 'POST', body: body() }))
expect(res.status).toBe(404)
expect(selectAccount).not.toHaveBeenCalled()
})
it('returns the selection + candidate slate on the happy path', async () => {
const res = await POST(createMockRequest('/x', { method: 'POST', body: body({ samples: 3, underlag: 'Biltema AB 499 kr' }) }))
const { status, body: b } = await parseJsonResponse<{
data: { account: string; confidence: number; candidates: { account: string }[] }
}>(res)
expect(status).toBe(200)
expect(b.data.account).toBe('5410')
expect(b.data.confidence).toBe(0.86)
expect(b.data.candidates[0].account).toBe('5410')
// entity type + vat_registered threaded from the company rows; underlag + samples passed through
expect(selectAccount).toHaveBeenCalledWith(
expect.objectContaining({ entityType: 'aktiebolag', vatRegistered: true, underlag: 'Biltema AB 499 kr', samples: 3 }),
)
})
})
+114
View File
@@ -0,0 +1,114 @@
import { NextResponse } from 'next/server'
import { z } from 'zod'
import { requireAuth } from '@/lib/auth/require-auth'
import { getActiveCompanyId } from '@/lib/company/context'
import { checkAgentRateLimit, agentRateLimitResponseBody } from '@/lib/rate-limits/agent'
import { guardSandbox } from '@/lib/sandbox/guard'
import { requireCapability } from '@/lib/entitlements/has-capability'
import { CAPABILITY } from '@/lib/entitlements/keys'
import { getAiStatus } from '@/lib/ai'
import { gatherCandidates } from '@/lib/agent/categorize/candidates'
import { selectAccount } from '@/lib/agent/categorize/select-account'
import { getErrorMessage as getUserErrorMessage } from '@/lib/errors/get-error-message'
import type { EntityType, Transaction } from '@/types'
/**
* POST /api/agent/categorize: a provider-agnostic booking proposal for one
* transaction — the auto-booking cascade end to end (Tier 1 retrieval → Tier 2
* selector), minus the write.
*
* It gathers the deterministic candidate accounts (counterparty templates,
* rules, history) with NO model call, then has the model SELECT among them
* (self-consistency sampled), and returns the proposed account + VAT +
* confidence + reasoning + the candidate slate. It never posts anything: the
* caller (the transaction row) renders the proposal as an approval card.
*
* Runs on any configured backend (Bedrock or a local model), so it is gated on
* `configured`, not `assistantAvailable` — same as /api/agent/ask.
*/
const Schema = z.object({
transaction_id: z.string().uuid(),
company_id: z.string().uuid().optional(),
/** Extracted receipt/invoice text, if the caller already has it. */
underlag: z.string().max(24_000).optional(),
/** Self-consistency samples (default 3). */
samples: z.number().int().min(1).max(5).optional(),
})
export async function POST(request: Request): Promise<Response> {
const { user, supabase, error } = await requireAuth()
if (error) return error
const rate = await checkAgentRateLimit(supabase, user.id)
if (!rate.ok) return NextResponse.json(agentRateLimitResponseBody(rate), { status: 429 })
let body: unknown
try {
body = await request.json()
} catch {
return NextResponse.json({ error: 'Invalid JSON' }, { status: 400 })
}
const parsed = Schema.safeParse(body)
if (!parsed.success) {
return NextResponse.json({ error: 'Ogiltig förfrågan.', type: 'validation_error' }, { status: 400 })
}
const companyId = parsed.data.company_id ?? (await getActiveCompanyId(supabase, user.id))
if (!companyId) return NextResponse.json({ error: 'No active company' }, { status: 400 })
const { data: membership } = await supabase
.from('company_members')
.select('user_id')
.eq('company_id', companyId)
.eq('user_id', user.id)
.maybeSingle()
if (!membership) return NextResponse.json({ error: 'Forbidden' }, { status: 403 })
const blocked = await guardSandbox(supabase, companyId)
if (blocked) return blocked
const capBlocked = await requireCapability(supabase, companyId, CAPABILITY.ai)
if (capBlocked) return capBlocked
if (!getAiStatus().configured) {
return NextResponse.json(
{ error: 'Assistenten är inte konfigurerad på den här installationen.', code: 'ai_unconfigured' },
{ status: 503 },
)
}
const { data: tx } = await supabase
.from('transactions')
.select('id, merchant_name, description, original_description, amount, date, currency, category, is_business')
.eq('id', parsed.data.transaction_id)
.eq('company_id', companyId)
.maybeSingle()
if (!tx) return NextResponse.json({ error: 'Transaktionen hittades inte.' }, { status: 404 })
const [{ data: company }, { data: settings }] = await Promise.all([
supabase.from('companies').select('entity_type').eq('id', companyId).maybeSingle(),
supabase.from('company_settings').select('vat_registered').eq('company_id', companyId).maybeSingle(),
])
try {
const candidates = await gatherCandidates(supabase, companyId, tx as Transaction)
const selection = await selectAccount({
transaction: {
merchantName: (tx as Transaction).merchant_name,
description: (tx as Transaction).description,
amount: (tx as Transaction).amount,
date: (tx as Transaction).date,
currency: (tx as Transaction).currency,
},
underlag: parsed.data.underlag,
candidates,
entityType: ((company?.entity_type as EntityType | undefined) ?? 'enskild_firma'),
vatRegistered: settings?.vat_registered ?? false,
samples: parsed.data.samples,
})
return NextResponse.json({ data: { ...selection, candidates } })
} catch (err) {
return NextResponse.json({ error: getUserErrorMessage(err) }, { status: 500 })
}
}
@@ -0,0 +1,107 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import type { SupabaseClient } from '@supabase/supabase-js'
import type { Transaction } from '@/types'
const getSuggestedCategories = vi.fn()
vi.mock('@/lib/transactions/category-suggestions', () => ({
getSuggestedCategories: (...a: unknown[]) => getSuggestedCategories(...a),
buildMerchantHistory: () => new Map(),
merchantHistoryFor: () => ({}),
}))
const findCounterpartyTemplate = vi.fn()
vi.mock('@/lib/bookkeeping/counterparty-templates', () => ({
findCounterpartyTemplate: (...a: unknown[]) => findCounterpartyTemplate(...a),
formatCounterpartyName: (n: string) => n,
}))
import { gatherCandidates } from '../candidates'
// Permissive chainable supabase: every builder method returns the chain, and
// the chain is awaitable (resolves {data: []}) so the two list queries settle.
function mockSupabase(): SupabaseClient {
const chain: Record<string, unknown> = {}
for (const m of ['from', 'select', 'or', 'eq', 'is', 'not', 'neq', 'order', 'limit']) {
chain[m] = () => chain
}
chain.then = (resolve: (v: { data: unknown[] }) => unknown) => resolve({ data: [] })
return chain as unknown as SupabaseClient
}
const TX = { id: 't1', merchant_name: 'Biltema', description: 'Kortköp', original_description: 'BILTEMA' } as unknown as Transaction
function cpMatch(over: Record<string, unknown> = {}) {
return {
template: {
debit_account: '5410',
counterparty_name: 'Biltema',
vat_treatment: 'standard_25',
occurrence_count: 4,
category: 'expense_consumables',
...over,
},
matchMethod: 'exact_normalized',
confidence: 0.9,
}
}
beforeEach(() => {
vi.clearAllMocks()
findCounterpartyTemplate.mockResolvedValue(null)
getSuggestedCategories.mockReturnValue([])
})
describe('gatherCandidates', () => {
it('puts the counterparty template first, carrying its own VAT', async () => {
findCounterpartyTemplate.mockResolvedValue(cpMatch())
const out = await gatherCandidates(mockSupabase(), 'c1', TX)
expect(out[0]).toMatchObject({
account: '5410',
source: 'counterparty_template',
vatTreatment: 'standard_25',
confidence: 0.9,
})
expect(out[0].matchReason).toContain('4 tidigare')
})
it('derives VAT for rule/pattern/history suggestions from their category', async () => {
getSuggestedCategories.mockReturnValue([
{ category: 'expense_bank_fees', label: 'Bankavgift', account: '6570', confidence: 0.8, source: 'mapping_rule' },
{ category: 'expense_software', label: 'Programvara', account: '5420', confidence: 0.6, source: 'pattern' },
])
const out = await gatherCandidates(mockSupabase(), 'c1', TX)
const bank = out.find((c) => c.account === '6570')!
const soft = out.find((c) => c.account === '5420')!
expect(bank.vatTreatment).toBeNull() // bank fees are VAT-exempt
expect(soft.vatTreatment).toBe('standard_25')
})
it('de-duplicates by account, keeping the highest confidence', async () => {
findCounterpartyTemplate.mockResolvedValue(cpMatch({ debit_account: '5410' }))
getSuggestedCategories.mockReturnValue([
{ category: 'expense_consumables', label: 'Material', account: '5410', confidence: 0.56, source: 'history' },
])
const out = await gatherCandidates(mockSupabase(), 'c1', TX)
const fivefour = out.filter((c) => c.account === '5410')
expect(fivefour).toHaveLength(1)
expect(fivefour[0].source).toBe('counterparty_template') // 0.9 > 0.56
})
it('skips suggestions with no account and sorts by confidence, capped', async () => {
getSuggestedCategories.mockReturnValue([
{ category: 'expense_other', label: 'x', account: null, confidence: 0.9, source: 'pattern' },
{ category: 'expense_office', label: 'Kontor', account: '6110', confidence: 0.5, source: 'history' },
{ category: 'expense_travel', label: 'Resor', account: '5800', confidence: 0.7, source: 'mapping_rule' },
])
const out = await gatherCandidates(mockSupabase(), 'c1', TX, 2)
expect(out.map((c) => c.account)).toEqual(['5800', '6110']) // no null, sorted desc, capped at 2
})
it('returns just the suggestions when there is no counterparty match', async () => {
getSuggestedCategories.mockReturnValue([
{ category: 'expense_office', label: 'Kontor', account: '6110', confidence: 0.5, source: 'history' },
])
const out = await gatherCandidates(mockSupabase(), 'c1', TX)
expect(out).toHaveLength(1)
expect(out[0].source).toBe('history')
})
})
+121
View File
@@ -0,0 +1,121 @@
import type { SupabaseClient } from '@supabase/supabase-js'
import {
getSuggestedCategories,
buildMerchantHistory,
merchantHistoryFor,
} from '@/lib/transactions/category-suggestions'
import {
findCounterpartyTemplate,
formatCounterpartyName,
} from '@/lib/bookkeeping/counterparty-templates'
import { getDefaultVatTreatmentForCategory } from '@/lib/bookkeeping/category-mapping'
import type { MappingRule, Transaction, VatTreatment } from '@/types'
import type { AccountCandidate } from './select-account'
/**
* Tier 1 of the auto-booking cascade: deterministic candidate generation.
*
* Assembles the ranked slate of candidate accounts for one transaction from
* the company's own memory: a learned counterparty template (the strongest
* signal) plus mapping rules, keyword patterns and per-merchant history. This
* is the same engine the `gnubok_suggest_categories` MCP tool uses; it runs
* with NO model call. The slate is what the Tier-2 selector reasons over.
*
* Company-scoped throughout. Returns at most `limit` candidates, de-duplicated
* by account (highest confidence wins), highest confidence first.
*/
const MAX_HISTORY_ROWS = 200
export async function gatherCandidates(
supabase: SupabaseClient,
companyId: string,
transaction: Transaction,
limit = 8,
): Promise<AccountCandidate[]> {
// The company's own rules plus the global (null-company) defaults. Two static
// queries rather than one dynamic `.or('company_id.eq.<id>,...')`, which the
// no-phantom-columns scanner can't resolve (and it would trip the ceiling).
const [companyRulesRes, globalRulesRes, historyRes, cpMatch] = await Promise.all([
supabase
.from('mapping_rules')
.select('*')
.eq('company_id', companyId)
.eq('is_active', true)
.order('priority', { ascending: false }),
supabase
.from('mapping_rules')
.select('*')
.is('company_id', null)
.eq('is_active', true)
.order('priority', { ascending: false }),
// Counterparty-keyed history: only the same merchant's past bookings, so
// global frequency padding can't drown the signal in noise.
supabase
.from('transactions')
.select('category, merchant_name, description, original_description')
.eq('company_id', companyId)
.not('is_business', 'is', null)
.neq('category', 'uncategorized')
.neq('category', 'private')
.order('date', { ascending: false })
.limit(MAX_HISTORY_ROWS),
findCounterpartyTemplate(supabase, companyId, transaction),
])
const mappingRules = [
...((companyRulesRes.data ?? []) as MappingRule[]),
...((globalRulesRes.data ?? []) as MappingRule[]),
]
const merchantHistory = buildMerchantHistory(historyRes.data ?? [])
const raw: AccountCandidate[] = []
// 1. Learned counterparty template — the strongest signal (carries its own VAT).
if (cpMatch?.template.debit_account) {
const t = cpMatch.template
raw.push({
account: t.debit_account,
label: formatCounterpartyName(t.counterparty_name),
vatTreatment: (t.vat_treatment as VatTreatment | null) ?? null,
source: 'counterparty_template',
confidence: cpMatch.confidence,
matchReason: `${t.occurrence_count ?? 0} tidigare bokföringar`,
})
}
// 2. Rules / pattern / history suggestions. They don't carry VAT, so derive
// the category's default treatment (the selector can still flag reverse charge).
const suggestions = getSuggestedCategories(
transaction,
mappingRules,
merchantHistoryFor(
merchantHistory,
transaction.merchant_name,
transaction.original_description ?? transaction.description,
),
)
for (const s of suggestions) {
if (!s.account) continue
raw.push({
account: s.account,
label: s.label,
vatTreatment: getDefaultVatTreatmentForCategory(s.category),
source: s.source,
confidence: s.confidence,
matchReason: s.match_reason,
})
}
return dedupeByAccount(raw).slice(0, limit)
}
/** Keep one candidate per account (the highest-confidence one), highest confidence first. */
function dedupeByAccount(candidates: AccountCandidate[]): AccountCandidate[] {
const best = new Map<string, AccountCandidate>()
for (const c of candidates) {
const existing = best.get(c.account)
if (!existing || c.confidence > existing.confidence) best.set(c.account, c)
}
return [...best.values()].sort((a, b) => b.confidence - a.confidence)
}