Files
accounted/lib/import/opening-balance/parser.ts
T
Jakob Wennberg 39204cc0de UX polish bundle: Enable Banking lookback + sync progress, invoice inbox, matching previews (#548)
* fix(import): dedup opening-balance rows when account numbers differ only in whitespace

The parser's merge map keyed on the post-strip account_number, but rows like
"1930", " 1930 " and "1.930" could leak as separate entries when the
upstream string contained non-breaking spaces or zero-width chars that the
old .replace(/[^0-9]/g, '') ran on already-stripped output. Strip those
explicitly in the raw string and use /\D/g for the digit extraction.

Also adds defense-in-depth dedup inside OpeningBalanceEditStep so any
duplicates that survive the parser collapse before the user sees them.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(enable-banking): anchor lookback picker to fiscal year, not days

Replaces the 90/180/365 days dropdown on the account-selection screen
with three explicit modes:

- "Senaste 90 dagar (snabbt)" — fastest path, matches PSD2 ceiling
- "Sedan räkenskapsårets början" (default) — resolves via
  fiscal_year_start_month, surfaces the literal date inline
- "Anpassat datum" — free date picker OR "Föregående räkenskapsårets start"

When the resulting range exceeds 90 days, the picker now surfaces a
quiet helper that points users at the SIE/bankfil import for older
history, so they don't waste an account-selection round-trip discovering
that banks usually cap at ~90 days.

The PATCH /accounts handler accepts initial_lookback_from_date alongside
initial_lookback_days; the new helper getCurrentFiscalYearStart() in
lib/company/fiscal-year.ts is reused.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(enable-banking): dedicated sync progress modal replaces silent spinner

After the user confirms account selection, transactions fetch in the
background for 30–60 seconds. Previously this showed only the Spara-button
spinner with no indication of duration or what was happening — users
described being stuck on the page.

The new BankSyncProgressDialog opens immediately on Save, lists the enabled
accounts being synced, and disables manual close until the PATCH resolves.
On completion it shows the imported count and the actual date range the
bank returned, plus an amber escape hatch to SIE/bankfil import when the
returned range was truncated by >7 days from what was requested.

Failure path surfaces in the same modal rather than as a destructive toast
that disappears.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(invoice-inbox): drop duplicate Skapa leverantör button

The inbox detail panel had its own supplier-creation button that fired
/api/suppliers + match-supplier. The same action is reachable from the
supplier-invoice form's "Skapa & välj" card (showAISupplierHint), which
also prefills more fields. The duplicate button is gone; a quiet inline
hint replaces it so the user still knows why no supplier matched.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ui(invoice-inbox): surface currency and totals above the long metadata tail

Move Valuta / Totalt / Moms in FIELD_DEFS so they sit immediately under
Leverantör / Org.nr / VAT-nr. These are the fields the user reads first
when triaging an inbox item; burying them after nine metadata fields
forces unnecessary scrolling on every single invoice.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(invoice-inbox): accept .eml forwards and log rejected attachments

Gmail's "Forward as attachment" packages the original email as message/rfc822,
which our MIME allowlist silently dropped. Adds mailparser so we can unwrap
the inner attachments and ingest them under the inner email's subject/from.

Also persists every rejected attachment as an invoice_inbox_items row with
status='error', so users can see what was dropped instead of guessing why
nothing showed up in their inbox.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(supplier-invoices): redirect back to inbox after creating from invoice-inbox

When the leverantörsfaktura form is opened from an invoice-inbox item, every
successful create previously kicked the user out to /supplier-invoices or the
just-created invoice's detail page — derailing the "process the next
document" workflow. The Tillbaka button likewise routed to the
supplier-invoice list rather than the inbox they came from.

Adds an afterCreate helper that lands inbox-originated submissions at
/e/general/invoice-inbox and preserves the original target everywhere else.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ui(pending): show transaction/document context for match-and-attach reviews

The granskning page previously rendered attach_document_to_transaction and
match_transaction_invoice operations through the generic key/value preview,
so reviewers saw "document file name: Faktura.pdf / transaction amount:
-216 USD" without any visual indication of which two things were being
paired. The MCP tool already returns enriched preview data; we just
needed dedicated layouts.

Adds:
- AttachDocumentPreview — two-card layout (Transaktion | Dokument) with
  a "Visa dokument" button that fetches a signed download URL on demand
- MatchTransactionInvoicePreview — same layout (Transaktion | Faktura)
- DocumentViewButton — reusable signed-URL opener

Also tightens the matching tools' descriptions so AI clients are nudged
to verify human-readable context before staging.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review): address PR #548 review feedback

- invoice-inbox: hoist mailparser to a static import. The extension system
  generates a static import tree via setup:extensions and disallows dynamic
  imports — await import('mailparser') worked in dev but could fail in
  production standalone builds.
- enable-banking AccountPickerDialog: guard the Save path when "Anpassat
  datum" + "Specifikt datum" is selected with an empty date. Without this,
  lookback.body resolves to null and the PATCH silently falls back to the
  backend's 120-day default, ignoring the user's intent.
- enable-banking BankSyncProgressDialog: drop the empty-body useEffect.
  Close-prevention is already handled inline via the onOpenChange guard +
  onPointerDownOutside + onEscapeKeyDown handlers.
- lib/company/fiscal-year: pin both operands of daysBetween() to UTC when
  parsing ISO date strings. Mixing a UTC-parsed date with new Date() (local
  time) drifts by one day in any timezone east of UTC.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review): address compliance swarm + Swedish review feedback

Three actionable items from the post-fix compliance scan; the rest were
false positives or out of scope.

- enable-banking PATCH /accounts: reject future initial_lookback_from_date
  with 400 instead of silently falling through to the 120-day default.
  Compliance V2.2.
- AttachDocumentPreview: promote the overwrite warning to a destructive
  banner with BFL 7 kap context when the existing document is marked as
  räkenskapsinformation. A muted footnote was too easy to skip past for
  a verifikationsunderlag replacement.
- MatchTransactionInvoicePreview: surface transaction_date + invoice_date
  in the staged preview so reviewers can spot date drift before approving
  (BFL 5 kap 6§ — verifikation date must align with affärshändelse). Also
  shows a quiet hint when the two dates differ by > 31 days. Tool's SELECT
  + stage payload extended accordingly.

Skipped (with rationale):
- V5.3 inner.filename path traversal — lib/core/documents/document-service.ts
  already sanitizes filenames before constructing storage paths.
- V5.2 magic-number MIME — pre-existing pattern for all email attachments;
  scope is codebase-wide.
- V1.2 att.id composite ID — only used as a DB column value, never a path.
- V13.1 / CM-8 SBOM/SCA — repository-wide policy, not this PR.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review): address second-round compliance + Swedish review feedback

Compliance Swarm (defense-in-depth + valid finds):
- invoice-inbox: sanitise .eml inner attachment filenames and content-types
  before they flow into uploadAndExtract or the raw_email_payload JSONB.
  document-service already strips bad chars before constructing storage
  paths, but the swarm flagged the upstream input as unsanitised — easier
  to add a thin sanitiseFilename/sanitiseMime layer than to argue about
  defense-in-depth. Caps lengths too.
- DocumentViewButton: validate documentId as a UUID before interpolating
  into /api/documents/:id — staged preview_data is Record<string, unknown>
  on the wire, so refusing junk early gives a clearer error and keeps the
  internal API from seeing oddly-shaped path segments. (Compliance V1.2.)

Swedish review:
- MatchTransactionInvoicePreview: drop the BFL 5 kap 6§ citation from the
  date-drift hint — that section governs verifikationsinnehåll, not a 31-day
  tolerance. The hint stays (the practical concern is real) but no longer
  pretends to quote a legislated threshold.
- fiscal-year: document the implicit assumption that entity_type reflects
  the company's current tax-year status, not a mid-conversion state.

Skipped (with rationale):
- V5.2 magic-number MIME — pre-existing pattern across all email attachments.
- A.8.12 signed URL via window.open — pre-existing pattern shared with
  JournalEntryAttachments.tsx; refactor to server-side redirect is broader scope.
- A.8.15 logRejection failure path — pre-existing console.error pattern.
- CC9.2 mailparser vendor review / SBOM — out of PR scope.
- CC6.1 IDOR — /api/documents/:id already enforces company_id; false positive.
- Swedish #1 räkenskapsinformation flag origin — server-side already derives
  the flag from document_attachments.journal_entry_id in the staging tool;
  not caller-trusted.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review): fail-safe BFL warning + preserve merge validation errors

Two findings from the third compliance pass; both worth addressing.

- AttachDocumentPreview: treat an absent
  existing_document_is_rakenskapsinformation flag as räkenskapsinformation
  rather than as "safe to overwrite". The MCP staging tool sets the flag
  deterministically from document_attachments.journal_entry_id today, but
  a future code path that forgets it would silently downgrade the BFL 7
  kap warning. Only an explicit `=== false` from the server keeps the
  muted note path.
- Opening-balance merge: union validation_errors when collapsing duplicate
  account_number rows, both in the parser and the EditStep useState
  initializer. Previously a warning that fired on row 5 (e.g. BAS-class
  mismatch) was silently dropped if row 2 of the same account had no error,
  risking misclassified IB data downstream. Added a parser test covering
  the union behaviour for two rows of a class-3 (resultatkonto) account.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 12:10:12 +02:00

247 lines
8.4 KiB
TypeScript
Raw Blame History

This file contains invisible Unicode characters
This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
import * as XLSX from 'xlsx'
import { detectColumns } from './column-detector'
import { getBASReference } from '@/lib/bookkeeping/bas-reference'
import { readWorkbookFromBuffer } from '../shared/workbook-reader'
import type {
DetectedColumns,
ParsedOpeningBalanceRow,
OpeningBalanceParseResult,
} from './types'
/**
* Parse a numeric value from a cell, handling Swedish decimal commas,
* thousand separators, and various string formats.
*/
export function parseAmount(value: unknown): number {
if (value === null || value === undefined || value === '') return 0
if (typeof value === 'number') return Math.round(value * 100) / 100
const str = String(value).trim()
if (str === '' || str === '-') return 0
// Remove thousand separators (spaces and dots before comma)
// Then replace comma with dot for parsing
const cleaned = str
.replace(/\s/g, '') // Remove spaces (thousand separator)
.replace(/\.(?=\d{3})/g, '') // Remove dots used as thousand separators
.replace(',', '.') // Swedish decimal comma → dot
const num = parseFloat(cleaned)
if (isNaN(num)) return 0
return Math.round(num * 100) / 100
}
/**
* Parse an opening balance file (Excel or CSV) and return structured rows
* with validation and BAS account matching.
*
* @param buffer - Raw file buffer
* @param filename - Original filename (used for format detection)
* @param columnOverrides - Optional manual column mapping (from column mapping step)
*/
export function parseOpeningBalanceFile(
buffer: ArrayBuffer,
filename: string,
columnOverrides?: DetectedColumns,
): OpeningBalanceParseResult {
const workbook = readWorkbookFromBuffer(buffer, filename)
// Pick the sheet with the most rows (heuristic for multi-sheet workbooks)
let bestSheet = workbook.SheetNames[0]
let bestRowCount = 0
for (const name of workbook.SheetNames) {
const sheet = workbook.Sheets[name]
const range = XLSX.utils.decode_range(sheet['!ref'] || 'A1')
const rowCount = range.e.r - range.s.r + 1
if (rowCount > bestRowCount) {
bestRowCount = rowCount
bestSheet = name
}
}
const sheet = workbook.Sheets[bestSheet]
const rawData: string[][] = XLSX.utils.sheet_to_json(sheet, {
header: 1,
defval: '',
raw: false,
})
if (rawData.length < 2) {
return {
filename,
sheet_name: bestSheet,
total_rows: 0,
detected_columns: columnOverrides || {
account_number_col: 0,
account_name_col: null,
layout: 'net',
balance_col: null,
debit_col: null,
credit_col: null,
confidence: 0,
},
headers: rawData[0]?.map((h) => String(h)) || [],
preview_rows: [],
rows: [],
total_debit: 0,
total_credit: 0,
is_balanced: true,
warnings: ['Filen innehåller för få rader.'],
}
}
const headers = rawData[0].map((h) => String(h))
const dataRows = rawData.slice(1)
// Detect or use overridden columns
const columns = columnOverrides || detectColumns(headers, dataRows)
const rows: ParsedOpeningBalanceRow[] = []
const warnings: string[] = []
const seenAccounts = new Map<string, number>() // account_number → first row_index
for (let i = 0; i < dataRows.length; i++) {
const row = dataRows[i]
const rawAccountNumber = String(row[columns.account_number_col] || '')
.replace(/[ ]/g, '') // strip NBSP and zero-width chars
.trim()
// Skip empty rows
if (!rawAccountNumber) continue
// Clean account number — strip every non-digit (whitespace, dots, dashes, letters)
const accountNumber = rawAccountNumber.replace(/\D/g, '')
// Skip non-4-digit account numbers (likely header/total rows)
if (!/^\d{4}$/.test(accountNumber)) {
// Could be a summary/total row — skip silently unless it looked intentional
if (rawAccountNumber.length > 0 && !/^(summa|total|sum|samman)/i.test(rawAccountNumber)) {
warnings.push(`Rad ${i + 2}: "${rawAccountNumber}" är inte ett giltigt kontonummer (4 siffror) — hoppades över`)
}
continue
}
// Get account name from file or BAS reference
const accountNameFromFile = columns.account_name_col !== null
? String(row[columns.account_name_col] || '').trim()
: ''
const basRef = getBASReference(accountNumber)
const basMatch = basRef?.account_name ?? null
const accountName = accountNameFromFile || basMatch || `Konto ${accountNumber}`
// Parse amounts
let debitAmount = 0
let creditAmount = 0
if (columns.layout === 'debit_credit') {
debitAmount = parseAmount(columns.debit_col !== null ? row[columns.debit_col] : 0)
creditAmount = parseAmount(columns.credit_col !== null ? row[columns.credit_col] : 0)
} else {
// Net balance: positive → debit, negative → credit
const netAmount = parseAmount(columns.balance_col !== null ? row[columns.balance_col] : 0)
if (netAmount > 0) {
debitAmount = netAmount
} else if (netAmount < 0) {
creditAmount = Math.abs(netAmount)
}
}
// Validate
const validationErrors: string[] = []
const is_valid_account = /^\d{4}$/.test(accountNumber)
if (!is_valid_account) {
validationErrors.push('Ogiltigt kontonummer')
}
if (debitAmount === 0 && creditAmount === 0) {
// Zero-amount rows will be filtered silently
continue
}
if (debitAmount > 0 && creditAmount > 0) {
validationErrors.push('Kontot har både debet- och kreditbelopp')
}
// Warn on P&L accounts (class 3-8)
const accountClass = parseInt(accountNumber.charAt(0), 10)
if (accountClass >= 3 && accountClass <= 8) {
validationErrors.push(`Konto ${accountNumber} är ett resultatkonto (klass ${accountClass}) — ingående balanser ska normalt bara innehålla balanskonton (klass 1-2)`)
}
// Track duplicates
if (seenAccounts.has(accountNumber)) {
warnings.push(`Konto ${accountNumber} förekommer på flera rader — beloppen kommer summeras`)
}
seenAccounts.set(accountNumber, i + 2) // +2 for header row + 1-based
rows.push({
row_index: i + 2,
account_number: accountNumber,
account_name: accountName,
debit_amount: debitAmount,
credit_amount: creditAmount,
is_valid: validationErrors.length === 0,
validation_errors: validationErrors,
bas_match: basMatch,
})
}
// Merge duplicate accounts — keyed on the already-normalized account_number.
// Union validation_errors across rows so a warning that fires on row 5 (e.g.
// BAS-class mismatch) isn't silently dropped because row 2 of the same
// account had no error. Suppressed validation issues on IB-feeding data
// would risk a misclassification propagating into the ledger.
const mergedMap = new Map<string, ParsedOpeningBalanceRow>()
for (const row of rows) {
const existing = mergedMap.get(row.account_number)
if (existing) {
existing.debit_amount = Math.round((existing.debit_amount + row.debit_amount) * 100) / 100
existing.credit_amount = Math.round((existing.credit_amount + row.credit_amount) * 100) / 100
if (!existing.account_name && row.account_name) {
existing.account_name = row.account_name
}
if (row.validation_errors?.length) {
const seen = new Set(existing.validation_errors)
for (const err of row.validation_errors) {
if (!seen.has(err)) existing.validation_errors.push(err)
}
existing.is_valid = existing.is_valid && row.is_valid
}
} else {
mergedMap.set(row.account_number, { ...row, validation_errors: [...row.validation_errors] })
}
}
const mergedRows = Array.from(mergedMap.values())
// Compute totals
let totalDebit = 0
let totalCredit = 0
for (const row of mergedRows) {
totalDebit = Math.round((totalDebit + row.debit_amount) * 100) / 100
totalCredit = Math.round((totalCredit + row.credit_amount) * 100) / 100
}
const diff = Math.round((totalDebit - totalCredit) * 100) / 100
const isBalanced = Math.abs(diff) < 0.01
if (!isBalanced) {
warnings.push(`Debet (${totalDebit.toFixed(2)}) och kredit (${totalCredit.toFixed(2)}) balanserar inte — differens: ${diff.toFixed(2)} SEK`)
}
return {
filename,
sheet_name: bestSheet,
total_rows: mergedRows.length,
detected_columns: columns,
headers,
preview_rows: dataRows.slice(0, 5),
rows: mergedRows,
total_debit: totalDebit,
total_credit: totalCredit,
is_balanced: isBalanced,
warnings,
}
}