fix(api): stabilize report pagination + declare real { data, meta } envelope on v1 single/write endpoints (#811)
* fix(reports): stabilize fetchAllRows paging to stop doubled/dropped balances (#790, #791) PostgREST `.range()` paging is only correct when the underlying query has a stable TOTAL order. Several aggregating report queries (general ledger, trial balance, grundbok, supplier/AR ledgers, etc.) paginated without `.order()`, so on datasets larger than one 1000-row page Postgres could return rows in a different order between requests — silently DUPLICATING or SKIPPING rows on a page boundary and doubling or dropping financial totals. - fetch-all.ts: document the ordering invariant and add an optional `dedupeBy` defense-in-depth that drops cross-page duplicates and warns when it fires (surfaces a missing `.order()` in logs instead of corrupting money). - Add a stable `.order()` (line PK or account_number) to every paginated query in lib/reports/ and the account-balances route; pass `dedupeBy` on the money-aggregating line queries. - Add fetch-all unit tests and update report test fixtures to carry row ids. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(api): declare the real { data, meta } envelope on v1 single/write/204 endpoints (#794) The OpenAPI generator derives each endpoint's documented body purely from its registered `response.success` Zod schema, and that schema is never validated at runtime — so a route could advertise a shape its handler never sends. #802 fixed this for list endpoints; the same drift was latent on single-resource and write endpoints, which declared the bare resource schema instead of the `{ data, meta }` envelope the handlers actually return. - registry.ts: extend `ResponseMetaSchema` with the optional `audit` block and `partial_expansions` list that writes/expansions emit; add the `NoBodyResponse` sentinel so 204 DELETE handlers document a bare 204 instead of a phantom 200. - Wrap every single/write endpoint's `response.success` in `dataEnvelope(...)` (or `NoBodyResponse` for 204s) across the v1 routes. - Add a response-envelope contract test that fails CI if any JSON endpoint forgets to wrap its schema, with binary downloads and 204s as the only exemptions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(reports): extend paging dedupeBy to rc-basis-gaps and opening-balances Address PR review: these two money-aggregating line queries already had the stable `.order('id')` (so paging was correct) but didn't carry `id` in the select, so they couldn't use the `dedupeBy` defense-in-depth that general-ledger and trial-balance got. Select `id` and pass `dedupeBy: r => r.id` so the whole report layer applies the ordering invariant consistently. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jakob Wennberg
co-authored by
Claude Opus 4.8
Claude Opus 4.8
parent
ae17b304d7
commit
fce6faff2c
@@ -0,0 +1,131 @@
|
||||
/**
|
||||
* Response-envelope contract test.
|
||||
*
|
||||
* Every v1 handler returns the canonical `{ data, meta }` envelope — `ok()` and
|
||||
* `created()` wrap a single object, `paginated()` wraps an array, both stamping
|
||||
* the shared `meta` block (see `lib/api/v1/response.ts`). The OpenAPI generator,
|
||||
* however, derives each endpoint's documented body purely from its registered
|
||||
* `response.success` Zod schema, and that schema is NOT validated at runtime —
|
||||
* so nothing stops a route from declaring a shape the handler never sends.
|
||||
*
|
||||
* That is exactly what issue #794 found: every list endpoint declared a bare
|
||||
* `{ <name>: [...] }` object that no handler emits. #802 fixed the list
|
||||
* endpoints (via `listEnvelope`/`dataEnvelope`); the same drift was latent on
|
||||
* the single-resource and write endpoints, which declared the bare resource
|
||||
* schema instead of `{ data, meta }`.
|
||||
*
|
||||
* This test is the regression guard the issue asked for. It asserts EVERY
|
||||
* JSON-returning endpoint declares the `{ data, meta }` envelope with the shared
|
||||
* `ResponseMetaSchema` — so a new endpoint that forgets to wrap its schema
|
||||
* (list OR single) fails CI here instead of shipping a lying spec. Binary
|
||||
* downloads (`response.contentType`) and 204 No Content endpoints
|
||||
* (`NoBodyResponse`) carry no JSON body and are the only exemptions.
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { z } from 'zod'
|
||||
import { listEndpoints, ResponseMetaSchema, NoBodyResponse, listEnvelope, dataEnvelope } from '../registry'
|
||||
// Side-effect import — every route file's registerEndpoint() runs at module
|
||||
// load time and populates the shared ENDPOINTS map.
|
||||
import '../load-routes'
|
||||
|
||||
/** Binary downloads (PDF, SIE text) declare a non-JSON contentType. */
|
||||
function isBinary(success: { contentType?: string }): boolean {
|
||||
return !!success.contentType && success.contentType !== 'application/json'
|
||||
}
|
||||
|
||||
describe('v1 response envelope contract', () => {
|
||||
const endpoints = listEndpoints()
|
||||
|
||||
it('every JSON endpoint declares the { data, meta } envelope with the shared meta schema', () => {
|
||||
// Accumulate every violation so a failing run names ALL offending endpoints
|
||||
// at once (a fresh route that forgets to wrap, plus any that drift later),
|
||||
// instead of failing one-at-a-time across many edit cycles.
|
||||
const violations: string[] = []
|
||||
|
||||
for (const ep of endpoints) {
|
||||
const ctx = `${ep.method} ${ep.path} (${ep.operation})`
|
||||
|
||||
// Exemptions: binary bodies and 204-no-content have no JSON envelope.
|
||||
if (isBinary(ep.response)) continue
|
||||
if (ep.response.success === NoBodyResponse) continue
|
||||
|
||||
const success = ep.response.success
|
||||
if (!(success instanceof z.ZodObject)) {
|
||||
violations.push(`${ctx}: response.success is not a { data, meta } object — wrap it with listEnvelope()/dataEnvelope() (or use NoBodyResponse for 204 / response.contentType for binary).`)
|
||||
continue
|
||||
}
|
||||
|
||||
const shape = (success as z.ZodObject<z.ZodRawShape>).shape
|
||||
const keys = Object.keys(shape).sort()
|
||||
if (keys.length !== 2 || keys[0] !== 'data' || keys[1] !== 'meta') {
|
||||
violations.push(`${ctx}: top-level keys must be [data, meta] — found [${keys.join(', ')}]. The handler returns { data, meta }; declare it with listEnvelope()/dataEnvelope().`)
|
||||
continue
|
||||
}
|
||||
|
||||
// Reference equality: both envelope helpers wire in this exact schema, so
|
||||
// a hand-rolled `{ data, meta: z.object({...}) }` that drifts from the
|
||||
// real meta block is rejected too.
|
||||
if (shape.meta !== ResponseMetaSchema) {
|
||||
violations.push(`${ctx}: meta is not the shared ResponseMetaSchema (use listEnvelope()/dataEnvelope(), don't hand-roll the envelope).`)
|
||||
}
|
||||
}
|
||||
|
||||
expect(
|
||||
violations,
|
||||
`\n${violations.length} v1 endpoint(s) declare a response.success that doesn't match the { data, meta } envelope the handler actually returns:\n\n${violations.map((v) => ` • ${v}`).join('\n')}\n`,
|
||||
).toEqual([])
|
||||
})
|
||||
|
||||
it('list endpoints expose data as an array (the paginated() envelope)', () => {
|
||||
// Detect list endpoints structurally: their `data` is a Zod array. This is
|
||||
// the half of the contract that maps onto paginated() specifically — guards
|
||||
// against a list endpoint drifting from `{ data: [...] }` back to a bare
|
||||
// `{ <name>: [...] }` (which would drop the array out of `data` entirely).
|
||||
const arrayDataEndpoints = endpoints.filter((ep) => {
|
||||
const s = ep.response.success
|
||||
return s instanceof z.ZodObject && (s as z.ZodObject<z.ZodRawShape>).shape.data instanceof z.ZodArray
|
||||
})
|
||||
|
||||
// The 10 cursor-paginated list endpoints (companies, customers, suppliers,
|
||||
// invoices, supplier-invoices, journal-entries, transactions, employees,
|
||||
// salary-runs, webhook deliveries). accounts/fiscal-periods/webhooks nest
|
||||
// their array under a named key inside `data`, so they use dataEnvelope and
|
||||
// are intentionally NOT counted here. A drop below this floor means a
|
||||
// paginated endpoint silently lost its `data: [...]` shape.
|
||||
expect(
|
||||
arrayDataEndpoints.length,
|
||||
`expected the known paginated list endpoints to keep data: z.array(...); found only ${arrayDataEndpoints.length}`,
|
||||
).toBeGreaterThanOrEqual(10)
|
||||
|
||||
for (const ep of arrayDataEndpoints) {
|
||||
const shape = (ep.response.success as z.ZodObject<z.ZodRawShape>).shape
|
||||
expect(
|
||||
shape.meta === ResponseMetaSchema,
|
||||
`${ep.method} ${ep.path}: list envelope meta must be the shared ResponseMetaSchema`,
|
||||
).toBe(true)
|
||||
}
|
||||
})
|
||||
|
||||
it('listEnvelope() and dataEnvelope() produce the canonical { data, meta } shape', () => {
|
||||
const list = listEnvelope(z.object({ id: z.string() }))
|
||||
expect(list instanceof z.ZodObject).toBe(true)
|
||||
expect(Object.keys(list.shape).sort()).toEqual(['data', 'meta'])
|
||||
expect(list.shape.data instanceof z.ZodArray).toBe(true)
|
||||
expect(list.shape.meta === ResponseMetaSchema).toBe(true)
|
||||
|
||||
const data = dataEnvelope(z.object({ id: z.string() }))
|
||||
expect(data instanceof z.ZodObject).toBe(true)
|
||||
expect(Object.keys(data.shape).sort()).toEqual(['data', 'meta'])
|
||||
expect(data.shape.data instanceof z.ZodObject).toBe(true)
|
||||
expect(data.shape.meta === ResponseMetaSchema).toBe(true)
|
||||
})
|
||||
|
||||
it('the shared meta schema carries request_id + api_version', () => {
|
||||
// The envelope helpers are only correct if meta itself is well-formed.
|
||||
expect(ResponseMetaSchema instanceof z.ZodObject).toBe(true)
|
||||
const metaKeys = Object.keys(ResponseMetaSchema.shape)
|
||||
expect(metaKeys).toContain('request_id')
|
||||
expect(metaKeys).toContain('api_version')
|
||||
})
|
||||
})
|
||||
+37
-5
@@ -21,15 +21,31 @@ import type { ZodTypeAny } from 'zod'
|
||||
import type { ApiKeyScope } from '@/lib/auth/api-keys'
|
||||
import { API_V1_VERSION } from './version'
|
||||
|
||||
/**
|
||||
* The audit block surfaced inline on write responses (see `AuditBlock` in
|
||||
* `lib/api/v1/response.ts`) so an agent gets the voucher number / audit-trail
|
||||
* URL without a second round-trip. Every field is optional.
|
||||
*/
|
||||
const ResponseAuditSchema = z.object({
|
||||
voucher_number: z.string().optional(),
|
||||
voucher_url: z.string().optional(),
|
||||
audit_trail_url: z.string().optional(),
|
||||
immutable_at: z.string().optional(),
|
||||
})
|
||||
|
||||
/**
|
||||
* The `meta` block echoed in every v1 response envelope (see
|
||||
* `lib/api/v1/response.ts`). List endpoints additionally populate
|
||||
* `next_cursor`; it is absent on the final page.
|
||||
* `next_cursor`; it is absent on the final page. Writes may surface an
|
||||
* `audit` block, and soft-degraded `?expand=` responses a `partial_expansions`
|
||||
* list — both optional, so reads and lists omit them.
|
||||
*/
|
||||
export const ResponseMetaSchema = z.object({
|
||||
request_id: z.string(),
|
||||
api_version: z.string(),
|
||||
next_cursor: z.string().nullable().optional(),
|
||||
audit: ResponseAuditSchema.optional(),
|
||||
partial_expansions: z.array(z.string()).optional(),
|
||||
})
|
||||
|
||||
/**
|
||||
@@ -66,6 +82,18 @@ export function dataEnvelope<T extends ZodTypeAny>(data: T) {
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Sentinel `response.success` for endpoints that return 204 No Content with an
|
||||
* empty body — e.g. DELETE handlers calling `noContent()`. The OpenAPI
|
||||
* generator emits a bare `204` response (no schema) for these instead of a
|
||||
* `200 { data, meta }`, and the envelope contract test exempts them.
|
||||
*
|
||||
* Identified by REFERENCE equality, so every 204 route MUST import this exact
|
||||
* constant rather than declaring its own `z.object({})` — that is what lets the
|
||||
* generator and the contract test recognise the "no body" intent.
|
||||
*/
|
||||
export const NoBodyResponse = z.object({})
|
||||
|
||||
export type HttpMethod = 'GET' | 'POST' | 'PATCH' | 'PUT' | 'DELETE'
|
||||
|
||||
export type ActionRisk = 'low' | 'medium' | 'high'
|
||||
@@ -327,6 +355,13 @@ export function generateOpenApiSpec(serverUrl: string): OpenApiSpec {
|
||||
? { [def.response.contentType]: { schema: { type: 'string', format: 'binary' } } }
|
||||
: { 'application/json': { schema: zodToJsonSchema(def.response.success) } }
|
||||
|
||||
// 204 No Content endpoints (DELETEs returning noContent()) carry no body —
|
||||
// emit a bare 204 instead of a 200 { data, meta } so the spec stops
|
||||
// advertising a response shape these handlers never send.
|
||||
const successResponse = def.response.success === NoBodyResponse
|
||||
? { '204': { description: 'No Content' } }
|
||||
: { '200': { description: 'Success', content: successContent } }
|
||||
|
||||
const operationDef: Record<string, unknown> = {
|
||||
operationId: def.operation,
|
||||
summary: def.summary,
|
||||
@@ -343,10 +378,7 @@ export function generateOpenApiSpec(serverUrl: string): OpenApiSpec {
|
||||
'x-dry-run-supported': def.dryRunSupported,
|
||||
...(def.scope ? { 'x-required-scope': def.scope } : {}),
|
||||
responses: {
|
||||
'200': {
|
||||
description: 'Success',
|
||||
content: successContent,
|
||||
},
|
||||
...successResponse,
|
||||
'400': { description: 'Validation error', $ref: '#/components/responses/Error' },
|
||||
'401': { description: 'Unauthorized', $ref: '#/components/responses/Error' },
|
||||
'403': { description: 'Insufficient scope', $ref: '#/components/responses/Error' },
|
||||
|
||||
@@ -9,7 +9,7 @@ let results: Array<{ data?: unknown; error?: unknown }>
|
||||
|
||||
function makeBuilder() {
|
||||
const b: Record<string, unknown> = {}
|
||||
for (const m of ['select', 'eq', 'in', 'range']) {
|
||||
for (const m of ['select', 'eq', 'in', 'order', 'range']) {
|
||||
b[m] = vi.fn().mockReturnValue(b)
|
||||
}
|
||||
b.single = vi.fn().mockImplementation(async () => results[resultIdx++] ?? { data: null, error: null })
|
||||
|
||||
@@ -9,7 +9,7 @@ let mockResults: Record<string, MockResult[]>
|
||||
|
||||
function makeBuilder(tableName: string) {
|
||||
const b: Record<string, unknown> = {}
|
||||
for (const m of ['select', 'eq', 'in', 'lt', 'neq', 'range', 'update']) {
|
||||
for (const m of ['select', 'eq', 'in', 'lt', 'neq', 'order', 'range', 'update']) {
|
||||
b[m] = vi.fn().mockReturnValue(b)
|
||||
}
|
||||
const consume = (): MockResult => {
|
||||
|
||||
@@ -124,6 +124,54 @@ describe('generateGeneralLedger', () => {
|
||||
expect(acc1930.closing_balance).toBe(1250)
|
||||
})
|
||||
|
||||
it('does not double a balance when an unstable page boundary re-serves a line (#790/#791)', async () => {
|
||||
// Reproduces the doubling bug's mechanism: a paginated query whose order
|
||||
// was not stable can return the same journal_entry_line on two pages.
|
||||
// Page 1 must be a FULL page (PAGE_SIZE rows) so fetchAllRows fetches a
|
||||
// second page; page 2 re-serves the 5010 line. dedupeBy(line id) must
|
||||
// collapse it so the single 4000 posting totals 4000, not 8000.
|
||||
const PAGE_SIZE = 1000
|
||||
const filler = Array.from({ length: PAGE_SIZE - 1 }, (_, i) => ({
|
||||
id: `f${i}`,
|
||||
account_number: '1930',
|
||||
debit_amount: 0,
|
||||
credit_amount: 0,
|
||||
journal_entries: { entry_date: '2024-01-02', voucher_number: 1, voucher_series: 'A', description: 'filler', source_type: 'manual' },
|
||||
}))
|
||||
const rentLine = {
|
||||
id: 'rent-line-1',
|
||||
account_number: '5010',
|
||||
debit_amount: 4000,
|
||||
credit_amount: 0,
|
||||
journal_entries: { entry_date: '2024-01-15', voucher_number: 2, voucher_series: 'A', description: 'Lokalhyra', source_type: 'manual' },
|
||||
}
|
||||
|
||||
mockResults = {
|
||||
fiscal_periods: [
|
||||
{ data: { period_start: '2024-01-01', period_end: '2024-12-31', opening_balance_entry_id: null }, error: null },
|
||||
],
|
||||
journal_entry_lines: [
|
||||
{ data: [...filler, rentLine], error: null }, // page 1 — full → triggers page 2
|
||||
{ data: [rentLine], error: null }, // page 2 — duplicate of the 5010 line
|
||||
],
|
||||
chart_of_accounts: [
|
||||
{
|
||||
data: [
|
||||
{ account_number: '1930', account_name: 'Företagskonto' },
|
||||
{ account_number: '5010', account_name: 'Lokalhyra' },
|
||||
],
|
||||
error: null,
|
||||
},
|
||||
],
|
||||
}
|
||||
|
||||
const report = await generateGeneralLedger(supabase, 'company-1', 'period-1')
|
||||
|
||||
const acc5010 = report.accounts.find((a) => a.account_number === '5010')!
|
||||
expect(acc5010.total_debit).toBe(4000) // not 8000
|
||||
expect(acc5010.lines).toHaveLength(1) // verifikat listed once, not twice
|
||||
})
|
||||
|
||||
it('computes opening balance from prior period entries', async () => {
|
||||
mockResults = {
|
||||
fiscal_periods: [
|
||||
|
||||
@@ -67,6 +67,54 @@ describe('generateJournalRegister', () => {
|
||||
expect(report.period).toEqual({ start: '2024-01-01', end: '2024-12-31' })
|
||||
})
|
||||
|
||||
it('does not double an entry when an unstable page boundary re-serves a line (#790/#793)', async () => {
|
||||
// Page 1 is a FULL page so fetchAllRows fetches page 2, which re-serves
|
||||
// the 5010 line of voucher 2. dedupeBy(line id) must collapse it so the
|
||||
// grundbok lists the voucher once with a 4000 (not 8000) total.
|
||||
const PAGE_SIZE = 1000
|
||||
const filler = Array.from({ length: PAGE_SIZE - 1 }, (_, i) => ({
|
||||
id: `f${i}`,
|
||||
account_number: '1930',
|
||||
debit_amount: 0,
|
||||
credit_amount: 0,
|
||||
journal_entry_id: 'e0',
|
||||
journal_entries: { id: 'e0', entry_date: '2024-01-02', voucher_number: 1, voucher_series: 'A', description: 'filler', source_type: 'manual', status: 'posted' },
|
||||
}))
|
||||
const rentLine = {
|
||||
id: 'rent-line-1',
|
||||
account_number: '5010',
|
||||
debit_amount: 4000,
|
||||
credit_amount: 0,
|
||||
journal_entry_id: 'e1',
|
||||
journal_entries: { id: 'e1', entry_date: '2024-01-15', voucher_number: 2, voucher_series: 'A', description: 'Lokalhyra', source_type: 'manual', status: 'posted' },
|
||||
}
|
||||
|
||||
mockResults = {
|
||||
fiscal_periods: [
|
||||
{ data: { period_start: '2024-01-01', period_end: '2024-12-31' }, error: null },
|
||||
],
|
||||
journal_entry_lines: [
|
||||
{ data: [...filler, rentLine], error: null }, // page 1 — full → triggers page 2
|
||||
{ data: [rentLine], error: null }, // page 2 — duplicate of the 5010 line
|
||||
],
|
||||
chart_of_accounts: [
|
||||
{
|
||||
data: [
|
||||
{ account_number: '1930', account_name: 'Företagskonto' },
|
||||
{ account_number: '5010', account_name: 'Lokalhyra' },
|
||||
],
|
||||
error: null,
|
||||
},
|
||||
],
|
||||
}
|
||||
|
||||
const report = await generateJournalRegister(supabase, 'company-1', 'period-1')
|
||||
|
||||
const voucher2 = report.entries.find((e) => e.voucher_number === 2)!
|
||||
expect(voucher2.total_debit).toBe(4000) // not 8000
|
||||
expect(voucher2.lines).toHaveLength(1) // listed once, not twice
|
||||
})
|
||||
|
||||
it('produces entries in registration order with correct totals', async () => {
|
||||
mockResults = {
|
||||
fiscal_periods: [
|
||||
|
||||
@@ -5,6 +5,20 @@ const { supabase, mockResult } = createMockSupabase()
|
||||
|
||||
import { generateMonthlyBreakdown } from '../monthly-breakdown'
|
||||
|
||||
// Minimal chainable query mock: every filter/order method returns the same
|
||||
// object; .single()/.range() resolve to the queued result. Tolerant of
|
||||
// query-shape changes such as an added .order() (see fetch-all.ts ordering
|
||||
// invariant) so the tests don't hardcode the exact method chain.
|
||||
function chain(result: unknown) {
|
||||
const c: Record<string, unknown> = {}
|
||||
for (const m of ['select', 'eq', 'in', 'gte', 'lte', 'lt', 'neq', 'order']) {
|
||||
c[m] = () => c
|
||||
}
|
||||
c.single = () => Promise.resolve(result)
|
||||
c.range = () => Promise.resolve(result)
|
||||
return c
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
})
|
||||
@@ -33,38 +47,9 @@ describe('generateMonthlyBreakdown', () => {
|
||||
let callCount = 0
|
||||
supabase.from.mockImplementation(() => {
|
||||
callCount++
|
||||
if (callCount === 1) {
|
||||
// fiscal_periods query
|
||||
return {
|
||||
select: () => ({
|
||||
eq: () => ({
|
||||
eq: () => ({
|
||||
single: () =>
|
||||
Promise.resolve({
|
||||
data: { period_start: '2024-01-01', period_end: '2024-12-31' },
|
||||
error: null,
|
||||
}),
|
||||
}),
|
||||
}),
|
||||
}),
|
||||
}
|
||||
}
|
||||
// journal_entry_lines query
|
||||
return {
|
||||
select: () => ({
|
||||
eq: () => ({
|
||||
eq: () => ({
|
||||
eq: () => ({
|
||||
range: () =>
|
||||
Promise.resolve({
|
||||
data: [],
|
||||
error: null,
|
||||
}),
|
||||
}),
|
||||
}),
|
||||
}),
|
||||
}),
|
||||
}
|
||||
return callCount === 1
|
||||
? chain({ data: { period_start: '2024-01-01', period_end: '2024-12-31' }, error: null })
|
||||
: chain({ data: [], error: null })
|
||||
})
|
||||
|
||||
const result = await generateMonthlyBreakdown(supabase as never, 'company-1', 'period-1')
|
||||
@@ -79,61 +64,37 @@ describe('generateMonthlyBreakdown', () => {
|
||||
let callCount = 0
|
||||
supabase.from.mockImplementation(() => {
|
||||
callCount++
|
||||
if (callCount === 1) {
|
||||
return {
|
||||
select: () => ({
|
||||
eq: () => ({
|
||||
eq: () => ({
|
||||
single: () =>
|
||||
Promise.resolve({
|
||||
data: { period_start: '2024-01-01', period_end: '2024-03-31' },
|
||||
error: null,
|
||||
}),
|
||||
}),
|
||||
}),
|
||||
}),
|
||||
}
|
||||
}
|
||||
return {
|
||||
select: () => ({
|
||||
eq: () => ({
|
||||
eq: () => ({
|
||||
eq: () => ({
|
||||
range: () =>
|
||||
Promise.resolve({
|
||||
data: [
|
||||
{
|
||||
account_number: '3001',
|
||||
debit_amount: 0,
|
||||
credit_amount: 10000,
|
||||
journal_entry: { entry_date: '2024-01-15', status: 'posted', user_id: 'user-1', fiscal_period_id: 'period-1' },
|
||||
},
|
||||
{
|
||||
account_number: '5010',
|
||||
debit_amount: 3000,
|
||||
credit_amount: 0,
|
||||
journal_entry: { entry_date: '2024-01-20', status: 'posted', user_id: 'user-1', fiscal_period_id: 'period-1' },
|
||||
},
|
||||
{
|
||||
account_number: '3001',
|
||||
debit_amount: 0,
|
||||
credit_amount: 5000,
|
||||
journal_entry: { entry_date: '2024-02-10', status: 'posted', user_id: 'user-1', fiscal_period_id: 'period-1' },
|
||||
},
|
||||
{
|
||||
account_number: '6200',
|
||||
debit_amount: 1500,
|
||||
credit_amount: 0,
|
||||
journal_entry: { entry_date: '2024-02-15', status: 'posted', user_id: 'user-1', fiscal_period_id: 'period-1' },
|
||||
},
|
||||
],
|
||||
error: null,
|
||||
}),
|
||||
}),
|
||||
}),
|
||||
}),
|
||||
}),
|
||||
}
|
||||
return callCount === 1
|
||||
? chain({ data: { period_start: '2024-01-01', period_end: '2024-03-31' }, error: null })
|
||||
: chain({
|
||||
data: [
|
||||
{
|
||||
account_number: '3001',
|
||||
debit_amount: 0,
|
||||
credit_amount: 10000,
|
||||
journal_entry: { entry_date: '2024-01-15', status: 'posted', user_id: 'user-1', fiscal_period_id: 'period-1' },
|
||||
},
|
||||
{
|
||||
account_number: '5010',
|
||||
debit_amount: 3000,
|
||||
credit_amount: 0,
|
||||
journal_entry: { entry_date: '2024-01-20', status: 'posted', user_id: 'user-1', fiscal_period_id: 'period-1' },
|
||||
},
|
||||
{
|
||||
account_number: '3001',
|
||||
debit_amount: 0,
|
||||
credit_amount: 5000,
|
||||
journal_entry: { entry_date: '2024-02-10', status: 'posted', user_id: 'user-1', fiscal_period_id: 'period-1' },
|
||||
},
|
||||
{
|
||||
account_number: '6200',
|
||||
debit_amount: 1500,
|
||||
credit_amount: 0,
|
||||
journal_entry: { entry_date: '2024-02-15', status: 'posted', user_id: 'user-1', fiscal_period_id: 'period-1' },
|
||||
},
|
||||
],
|
||||
error: null,
|
||||
})
|
||||
})
|
||||
|
||||
const result = await generateMonthlyBreakdown(supabase as never, 'company-1', 'period-1')
|
||||
@@ -160,61 +121,37 @@ describe('generateMonthlyBreakdown', () => {
|
||||
let callCount = 0
|
||||
supabase.from.mockImplementation(() => {
|
||||
callCount++
|
||||
if (callCount === 1) {
|
||||
return {
|
||||
select: () => ({
|
||||
eq: () => ({
|
||||
eq: () => ({
|
||||
single: () =>
|
||||
Promise.resolve({
|
||||
data: { period_start: '2024-01-01', period_end: '2024-01-31' },
|
||||
error: null,
|
||||
}),
|
||||
}),
|
||||
}),
|
||||
}),
|
||||
}
|
||||
}
|
||||
return {
|
||||
select: () => ({
|
||||
eq: () => ({
|
||||
eq: () => ({
|
||||
eq: () => ({
|
||||
range: () =>
|
||||
Promise.resolve({
|
||||
data: [
|
||||
{
|
||||
account_number: '1930',
|
||||
debit_amount: 10000,
|
||||
credit_amount: 0,
|
||||
journal_entry: { entry_date: '2024-01-15', status: 'posted', user_id: 'user-1', fiscal_period_id: 'period-1' },
|
||||
},
|
||||
{
|
||||
account_number: '2611',
|
||||
debit_amount: 0,
|
||||
credit_amount: 2500,
|
||||
journal_entry: { entry_date: '2024-01-15', status: 'posted', user_id: 'user-1', fiscal_period_id: 'period-1' },
|
||||
},
|
||||
{
|
||||
account_number: '8400',
|
||||
debit_amount: 500,
|
||||
credit_amount: 0,
|
||||
journal_entry: { entry_date: '2024-01-20', status: 'posted', user_id: 'user-1', fiscal_period_id: 'period-1' },
|
||||
},
|
||||
{
|
||||
account_number: '8300',
|
||||
debit_amount: 0,
|
||||
credit_amount: 200,
|
||||
journal_entry: { entry_date: '2024-01-25', status: 'posted', user_id: 'user-1', fiscal_period_id: 'period-1' },
|
||||
},
|
||||
],
|
||||
error: null,
|
||||
}),
|
||||
}),
|
||||
}),
|
||||
}),
|
||||
}),
|
||||
}
|
||||
return callCount === 1
|
||||
? chain({ data: { period_start: '2024-01-01', period_end: '2024-01-31' }, error: null })
|
||||
: chain({
|
||||
data: [
|
||||
{
|
||||
account_number: '1930',
|
||||
debit_amount: 10000,
|
||||
credit_amount: 0,
|
||||
journal_entry: { entry_date: '2024-01-15', status: 'posted', user_id: 'user-1', fiscal_period_id: 'period-1' },
|
||||
},
|
||||
{
|
||||
account_number: '2611',
|
||||
debit_amount: 0,
|
||||
credit_amount: 2500,
|
||||
journal_entry: { entry_date: '2024-01-15', status: 'posted', user_id: 'user-1', fiscal_period_id: 'period-1' },
|
||||
},
|
||||
{
|
||||
account_number: '8400',
|
||||
debit_amount: 500,
|
||||
credit_amount: 0,
|
||||
journal_entry: { entry_date: '2024-01-20', status: 'posted', user_id: 'user-1', fiscal_period_id: 'period-1' },
|
||||
},
|
||||
{
|
||||
account_number: '8300',
|
||||
debit_amount: 0,
|
||||
credit_amount: 200,
|
||||
journal_entry: { entry_date: '2024-01-25', status: 'posted', user_id: 'user-1', fiscal_period_id: 'period-1' },
|
||||
},
|
||||
],
|
||||
error: null,
|
||||
})
|
||||
})
|
||||
|
||||
const result = await generateMonthlyBreakdown(supabase as never, 'company-1', 'period-1')
|
||||
|
||||
@@ -9,7 +9,7 @@ let results: Array<{ data?: unknown; error?: unknown }>
|
||||
|
||||
function makeBuilder() {
|
||||
const b: Record<string, unknown> = {}
|
||||
for (const m of ['select', 'eq', 'in', 'gte', 'lte', 'lt', 'or', 'not', 'range']) {
|
||||
for (const m of ['select', 'eq', 'in', 'gte', 'lte', 'lt', 'or', 'not', 'order', 'range']) {
|
||||
b[m] = vi.fn().mockReturnValue(b)
|
||||
}
|
||||
b.single = vi.fn().mockImplementation(async () => results[resultIdx++] ?? { data: null, error: null })
|
||||
|
||||
@@ -9,7 +9,7 @@ let results: Array<{ data?: unknown; error?: unknown }>
|
||||
|
||||
function makeBuilder() {
|
||||
const b: Record<string, unknown> = {}
|
||||
for (const m of ['select', 'eq', 'in', 'range']) {
|
||||
for (const m of ['select', 'eq', 'in', 'order', 'range']) {
|
||||
b[m] = vi.fn().mockReturnValue(b)
|
||||
}
|
||||
b.single = vi.fn().mockImplementation(async () => results[resultIdx++] ?? { data: null, error: null })
|
||||
|
||||
@@ -11,7 +11,7 @@ let mockResults: Record<string, MockResult[]>
|
||||
|
||||
function makeBuilder(tableName: string) {
|
||||
const b: Record<string, unknown> = {}
|
||||
for (const m of ['select', 'eq', 'in', 'lt', 'lte', 'gte', 'neq', 'range']) {
|
||||
for (const m of ['select', 'eq', 'in', 'lt', 'lte', 'gte', 'neq', 'order', 'range']) {
|
||||
b[m] = vi.fn().mockReturnValue(b)
|
||||
}
|
||||
const consume = (): MockResult => {
|
||||
|
||||
@@ -9,7 +9,7 @@ let results: Array<{ data?: unknown; error?: unknown }>
|
||||
|
||||
function makeBuilder() {
|
||||
const b: Record<string, unknown> = {}
|
||||
for (const m of ['select', 'eq', 'in', 'gte', 'lte', 'lt', 'or', 'not', 'range']) {
|
||||
for (const m of ['select', 'eq', 'in', 'gte', 'lte', 'lt', 'or', 'not', 'order', 'range']) {
|
||||
b[m] = vi.fn().mockReturnValue(b)
|
||||
}
|
||||
b.single = vi.fn().mockImplementation(async () => results[resultIdx++] ?? { data: null, error: null })
|
||||
|
||||
@@ -69,6 +69,8 @@ export async function generateARLedger(
|
||||
.select('*, customer:customers(id, name)')
|
||||
.eq('company_id', companyId)
|
||||
.in('status', ['sent', 'overdue', 'credited'])
|
||||
// Stable total order for correct paging (see fetch-all.ts).
|
||||
.order('id', { ascending: true })
|
||||
.range(from, to)
|
||||
)
|
||||
} catch {
|
||||
|
||||
@@ -209,6 +209,8 @@ export async function estimateArchiveSize(
|
||||
.eq('company_id', companyId)
|
||||
.eq('fiscal_period_id', periodId)
|
||||
.in('status', ['posted', 'reversed'])
|
||||
// Stable total order for correct paging (see fetch-all.ts).
|
||||
.order('id', { ascending: true })
|
||||
.range(from, to)
|
||||
)
|
||||
const ids = periodEntryIds.map((e) => e.id)
|
||||
@@ -345,6 +347,8 @@ async function writeDocuments(
|
||||
)
|
||||
.eq('company_id', companyId)
|
||||
.not('journal_entry_id', 'is', null)
|
||||
// Stable total order for correct paging (see fetch-all.ts).
|
||||
.order('id', { ascending: true })
|
||||
.range(from, to)
|
||||
)
|
||||
|
||||
@@ -730,6 +734,9 @@ async function buildEntryToPeriodMap(
|
||||
query = query.in('fiscal_period_id', periodIds)
|
||||
}
|
||||
|
||||
// Stable total order for correct paging (see fetch-all.ts).
|
||||
query = query.order('id', { ascending: true })
|
||||
|
||||
const entries = await fetchAllRows<{ id: string; fiscal_period_id: string }>(({ from, to }) =>
|
||||
query.range(from, to)
|
||||
)
|
||||
|
||||
@@ -83,6 +83,7 @@ export async function generateGeneralLedger(
|
||||
// Supabase types !inner joins as arrays; for many-to-one (line → entry)
|
||||
// it returns a single object at runtime. Cast via `as any` on the query.
|
||||
const rawLines = await fetchAllRows<{
|
||||
id: string
|
||||
account_number: string
|
||||
debit_amount: number
|
||||
credit_amount: number
|
||||
@@ -97,7 +98,7 @@ export async function generateGeneralLedger(
|
||||
}>(({ from, to }) => {
|
||||
let query = supabase
|
||||
.from('journal_entry_lines')
|
||||
.select('account_number, debit_amount, credit_amount, journal_entry_id, journal_entries!inner(entry_date, voucher_number, voucher_series, description, source_type, company_id, fiscal_period_id, status)')
|
||||
.select('id, account_number, debit_amount, credit_amount, journal_entry_id, journal_entries!inner(entry_date, voucher_number, voucher_series, description, source_type, company_id, fiscal_period_id, status)')
|
||||
.eq('journal_entries.company_id', companyId)
|
||||
.eq('journal_entries.fiscal_period_id', periodId)
|
||||
.in('journal_entries.status', ['posted', 'reversed'])
|
||||
@@ -106,9 +107,13 @@ export async function generateGeneralLedger(
|
||||
query = query.neq('journal_entry_id', obEntryId)
|
||||
}
|
||||
|
||||
// Stable total order on the line PK — paging is only correct with a
|
||||
// deterministic order, else rows duplicate/skip across pages and balances
|
||||
// double or accounts vanish (see fetch-all.ts ordering invariant). The
|
||||
// report re-sorts lines per account below, so this order is invisible.
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
return query.range(from, to) as any
|
||||
})
|
||||
return query.order('id', { ascending: true }).range(from, to) as any
|
||||
}, { dedupeBy: (r) => r.id })
|
||||
|
||||
if (rawLines.length === 0 && openingBalances.size === 0) {
|
||||
return { accounts: [], period: { start: period.period_start, end: period.period_end } }
|
||||
@@ -120,6 +125,7 @@ export async function generateGeneralLedger(
|
||||
.from('chart_of_accounts')
|
||||
.select('account_number, account_name')
|
||||
.eq('company_id', companyId)
|
||||
.order('account_number', { ascending: true })
|
||||
.range(from, to)
|
||||
)
|
||||
|
||||
|
||||
@@ -745,6 +745,7 @@ export async function generateINK2Declaration(
|
||||
.from('chart_of_accounts')
|
||||
.select('account_number, account_name')
|
||||
.eq('company_id', companyId)
|
||||
.order('account_number', { ascending: true })
|
||||
.range(from, to)
|
||||
)
|
||||
|
||||
|
||||
@@ -62,6 +62,7 @@ export async function generateJournalRegister(
|
||||
// Supabase types !inner joins as arrays; for many-to-one (line → entry)
|
||||
// it returns a single object at runtime. Cast via `as any` on the query.
|
||||
const rawLines = await fetchAllRows<{
|
||||
id: string
|
||||
account_number: string
|
||||
debit_amount: number
|
||||
credit_amount: number
|
||||
@@ -75,16 +76,18 @@ export async function generateJournalRegister(
|
||||
source_type: string
|
||||
status: string
|
||||
}
|
||||
}>(({ from, to }) =>
|
||||
supabase
|
||||
}>(({ from, to }) => {
|
||||
return supabase
|
||||
.from('journal_entry_lines')
|
||||
.select('account_number, debit_amount, credit_amount, journal_entry_id, journal_entries!inner(id, entry_date, voucher_number, voucher_series, description, source_type, status, company_id, fiscal_period_id)')
|
||||
.select('id, account_number, debit_amount, credit_amount, journal_entry_id, journal_entries!inner(id, entry_date, voucher_number, voucher_series, description, source_type, status, company_id, fiscal_period_id)')
|
||||
.eq('journal_entries.company_id', companyId)
|
||||
.eq('journal_entries.fiscal_period_id', periodId)
|
||||
.in('journal_entries.status', ['posted', 'reversed'])
|
||||
// Stable total order on the line PK — without it, rows duplicate/skip
|
||||
// across pages and entries appear twice or go missing (see fetch-all.ts).
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
.range(from, to) as any
|
||||
)
|
||||
.order('id', { ascending: true }).range(from, to) as any
|
||||
}, { dedupeBy: (r) => r.id })
|
||||
|
||||
if (rawLines.length === 0) {
|
||||
return { entries: [], total_entries: 0, total_debit: 0, total_credit: 0, period: { start: period.period_start, end: period.period_end } }
|
||||
@@ -96,6 +99,7 @@ export async function generateJournalRegister(
|
||||
.from('chart_of_accounts')
|
||||
.select('account_number, account_name')
|
||||
.eq('company_id', companyId)
|
||||
.order('account_number', { ascending: true })
|
||||
.range(from, to)
|
||||
)
|
||||
|
||||
|
||||
@@ -63,6 +63,8 @@ export async function generateMonthlyBreakdown(
|
||||
.eq('journal_entries.fiscal_period_id', fiscalPeriodId)
|
||||
.eq('journal_entries.company_id', companyId)
|
||||
.eq('journal_entries.status', 'posted')
|
||||
// Stable total order for correct paging (see fetch-all.ts).
|
||||
.order('id', { ascending: true })
|
||||
.range(from, to)
|
||||
)
|
||||
} catch {
|
||||
|
||||
@@ -213,6 +213,7 @@ export async function generateNEDeclaration(
|
||||
.from('chart_of_accounts')
|
||||
.select('account_number, account_name')
|
||||
.eq('company_id', companyId)
|
||||
.order('account_number', { ascending: true })
|
||||
.range(from, to)
|
||||
)
|
||||
|
||||
|
||||
@@ -41,16 +41,20 @@ export async function getOpeningBalances(
|
||||
// for consistency (avoids silent truncation) and joins journal_entries
|
||||
// to enforce company_id ownership (defense in depth alongside RLS).
|
||||
const obLines = await fetchAllRows<{
|
||||
id: string
|
||||
account_number: string
|
||||
debit_amount: number
|
||||
credit_amount: number
|
||||
}>(({ from, to }) =>
|
||||
supabase
|
||||
.from('journal_entry_lines')
|
||||
.select('account_number, debit_amount, credit_amount, journal_entries!inner(company_id)')
|
||||
.select('id, account_number, debit_amount, credit_amount, journal_entries!inner(company_id)')
|
||||
.eq('journal_entry_id', obEntryId)
|
||||
.eq('journal_entries.company_id', companyId)
|
||||
.range(from, to)
|
||||
// Stable total order for correct paging (see fetch-all.ts).
|
||||
.order('id', { ascending: true })
|
||||
.range(from, to),
|
||||
{ dedupeBy: (r) => r.id }
|
||||
)
|
||||
|
||||
for (const line of obLines) {
|
||||
|
||||
@@ -200,6 +200,8 @@ export async function generatePeriodiskSammanstallning(
|
||||
.in('journal_entries.source_type', ['invoice_created', 'credit_note'])
|
||||
.gte('journal_entries.entry_date', start)
|
||||
.lte('journal_entries.entry_date', end)
|
||||
// Stable total order for correct paging (see fetch-all.ts).
|
||||
.order('id', { ascending: true })
|
||||
.range(from, to) as unknown as PromiseLike<{ data: RawLine[] | null; error: { message: string } | null }>,
|
||||
)
|
||||
|
||||
@@ -229,6 +231,8 @@ export async function generatePeriodiskSammanstallning(
|
||||
)
|
||||
`)
|
||||
.in('id', invoiceIds)
|
||||
// Stable total order for correct paging (see fetch-all.ts).
|
||||
.order('id', { ascending: true })
|
||||
.range(from, to) as unknown as PromiseLike<{ data: RawInvoice[] | null; error: { message: string } | null }>,
|
||||
)
|
||||
for (const inv of invoices) invoiceMap.set(inv.id, inv)
|
||||
|
||||
@@ -90,6 +90,7 @@ function pickEntry(row: RcLineRow): EntryFields | null {
|
||||
}
|
||||
|
||||
interface SiblingLineRow {
|
||||
id: string
|
||||
journal_entry_id: string
|
||||
account_number: string
|
||||
debit_amount: number
|
||||
@@ -122,6 +123,8 @@ export async function findRcBasisGaps(
|
||||
.eq('journal_entries.status', 'posted')
|
||||
.gte('journal_entries.entry_date', start)
|
||||
.lte('journal_entries.entry_date', end)
|
||||
// Stable total order for correct paging (see fetch-all.ts).
|
||||
.order('id', { ascending: true })
|
||||
.range(from, to),
|
||||
)) as RcLineRow[]
|
||||
|
||||
@@ -132,9 +135,12 @@ export async function findRcBasisGaps(
|
||||
const siblingLines = await fetchAllRows<SiblingLineRow>(({ from, to }) =>
|
||||
supabase
|
||||
.from('journal_entry_lines')
|
||||
.select('journal_entry_id, account_number, debit_amount, credit_amount')
|
||||
.select('id, journal_entry_id, account_number, debit_amount, credit_amount')
|
||||
.in('journal_entry_id', entryIds)
|
||||
// Stable total order for correct paging (see fetch-all.ts).
|
||||
.order('id', { ascending: true })
|
||||
.range(from, to),
|
||||
{ dedupeBy: (r) => r.id },
|
||||
)
|
||||
|
||||
const basisByEntry = new Map<string, number>()
|
||||
|
||||
@@ -73,6 +73,9 @@ export async function generateSalaryJournal(
|
||||
.eq('salary_runs.period_year', year)
|
||||
.eq('salary_runs.status', 'booked')
|
||||
.order('created_at')
|
||||
// id tiebreaker — created_at is not unique, so it alone is not a stable
|
||||
// total order for paging (see fetch-all.ts).
|
||||
.order('id', { ascending: true })
|
||||
.range(from, to)
|
||||
)
|
||||
|
||||
|
||||
@@ -47,6 +47,8 @@ export async function generateSupplierLedger(
|
||||
.select('*, supplier:suppliers(id, name)')
|
||||
.eq('company_id', companyId)
|
||||
.in('status', ['registered', 'approved', 'partially_paid', 'overdue'])
|
||||
// Stable total order for correct paging (see fetch-all.ts).
|
||||
.order('id', { ascending: true })
|
||||
.range(from, to)
|
||||
)
|
||||
} catch {
|
||||
|
||||
@@ -60,13 +60,14 @@ export async function generateTrialBalance(
|
||||
options.fromDate > period.period_start
|
||||
) {
|
||||
const priorLines = await fetchAllRows<{
|
||||
id: string
|
||||
account_number: string
|
||||
debit_amount: number
|
||||
credit_amount: number
|
||||
}>(({ from, to }) => {
|
||||
let query = supabase
|
||||
.from('journal_entry_lines')
|
||||
.select('account_number, debit_amount, credit_amount, journal_entries!inner(company_id, fiscal_period_id, status, source_type, entry_date)')
|
||||
.select('id, account_number, debit_amount, credit_amount, journal_entries!inner(company_id, fiscal_period_id, status, source_type, entry_date)')
|
||||
.eq('journal_entries.company_id', companyId)
|
||||
.eq('journal_entries.fiscal_period_id', fiscalPeriodId)
|
||||
.in('journal_entries.status', ['posted', 'reversed'])
|
||||
@@ -81,8 +82,9 @@ export async function generateTrialBalance(
|
||||
query = query.neq('journal_entries.source_type', 'year_end')
|
||||
}
|
||||
|
||||
return query.range(from, to)
|
||||
})
|
||||
// Stable total order on the line PK for correct paging (see fetch-all.ts).
|
||||
return query.order('id', { ascending: true }).range(from, to)
|
||||
}, { dedupeBy: (r) => r.id })
|
||||
|
||||
for (const line of priorLines) {
|
||||
const existing = openingBalances.get(line.account_number) || { debit: 0, credit: 0 }
|
||||
@@ -100,13 +102,14 @@ export async function generateTrialBalance(
|
||||
// be missed from both IB and period. The window is sub-second and the
|
||||
// consequence is a single stale report — acceptable.
|
||||
const lines = await fetchAllRows<{
|
||||
id: string
|
||||
account_number: string
|
||||
debit_amount: number
|
||||
credit_amount: number
|
||||
}>(({ from, to }) => {
|
||||
let query = supabase
|
||||
.from('journal_entry_lines')
|
||||
.select('account_number, debit_amount, credit_amount, journal_entries!inner(company_id, fiscal_period_id, status, source_type, entry_date)')
|
||||
.select('id, account_number, debit_amount, credit_amount, journal_entries!inner(company_id, fiscal_period_id, status, source_type, entry_date)')
|
||||
.eq('journal_entries.company_id', companyId)
|
||||
.eq('journal_entries.fiscal_period_id', fiscalPeriodId)
|
||||
.in('journal_entries.status', ['posted', 'reversed'])
|
||||
@@ -132,8 +135,9 @@ export async function generateTrialBalance(
|
||||
query = query.neq('journal_entries.source_type', 'year_end')
|
||||
}
|
||||
|
||||
return query.range(from, to)
|
||||
})
|
||||
// Stable total order on the line PK for correct paging (see fetch-all.ts).
|
||||
return query.order('id', { ascending: true }).range(from, to)
|
||||
}, { dedupeBy: (r) => r.id })
|
||||
|
||||
if (lines.length === 0 && openingBalances.size === 0) {
|
||||
return { rows: [], totalDebit: 0, totalCredit: 0, isBalanced: true }
|
||||
@@ -149,6 +153,7 @@ export async function generateTrialBalance(
|
||||
.from('chart_of_accounts')
|
||||
.select('account_number, account_name, account_class')
|
||||
.eq('company_id', companyId)
|
||||
.order('account_number', { ascending: true })
|
||||
.range(from, to)
|
||||
)
|
||||
|
||||
|
||||
@@ -61,6 +61,9 @@ export async function generateVacationLiability(
|
||||
.eq('is_active', true)
|
||||
.not('vacation_rule', 'in', '(none,semesterersattning)')
|
||||
.order('last_name')
|
||||
// id tiebreaker — last_name is not unique, so it alone is not a stable
|
||||
// total order for paging (see fetch-all.ts).
|
||||
.order('id', { ascending: true })
|
||||
.range(from, to)
|
||||
)
|
||||
|
||||
@@ -80,6 +83,8 @@ export async function generateVacationLiability(
|
||||
.eq('company_id', companyId)
|
||||
.eq('salary_runs.period_year', year)
|
||||
.eq('salary_runs.status', 'booked')
|
||||
// Stable total order for correct paging (see fetch-all.ts).
|
||||
.order('id', { ascending: true })
|
||||
.range(from, to)
|
||||
)
|
||||
|
||||
|
||||
@@ -283,6 +283,8 @@ export async function calculateVatDeclaration(
|
||||
.in('journal_entries.status', ['posted', 'reversed'])
|
||||
.gte('journal_entries.entry_date', start)
|
||||
.lte('journal_entries.entry_date', end)
|
||||
// Stable total order for correct paging (see fetch-all.ts).
|
||||
.order('id', { ascending: true })
|
||||
.range(from, to)
|
||||
)
|
||||
|
||||
|
||||
@@ -0,0 +1,88 @@
|
||||
import { describe, it, expect } from 'vitest'
|
||||
import { fetchAllRows } from '../fetch-all'
|
||||
|
||||
const PAGE_SIZE = 1000
|
||||
|
||||
type Row = { id: string; value?: number }
|
||||
|
||||
/**
|
||||
* Build a queryFn that serves predefined pages keyed by the `from` offset.
|
||||
* Mirrors how `fetchAllRows` drives PostgREST `.range(from, to)`.
|
||||
*/
|
||||
function pagedQuery(pages: Record<number, Row[]>) {
|
||||
return ({ from }: { from: number; to: number }) =>
|
||||
Promise.resolve({ data: pages[from] ?? [], error: null })
|
||||
}
|
||||
|
||||
function makeRows(start: number, count: number): Row[] {
|
||||
return Array.from({ length: count }, (_, i) => ({ id: String(start + i), value: 1 }))
|
||||
}
|
||||
|
||||
describe('fetchAllRows', () => {
|
||||
it('returns a single page as-is and stops (page < PAGE_SIZE)', async () => {
|
||||
const rows = makeRows(0, 3)
|
||||
const out = await fetchAllRows<Row>(pagedQuery({ 0: rows }))
|
||||
expect(out).toHaveLength(3)
|
||||
expect(out.map((r) => r.id)).toEqual(['0', '1', '2'])
|
||||
})
|
||||
|
||||
it('paginates across multiple pages and concatenates in order', async () => {
|
||||
const page1 = makeRows(0, PAGE_SIZE) // full page → fetch continues
|
||||
const page2 = makeRows(PAGE_SIZE, 5) // partial page → stop
|
||||
const out = await fetchAllRows<Row>(pagedQuery({ 0: page1, [PAGE_SIZE]: page2 }))
|
||||
expect(out).toHaveLength(PAGE_SIZE + 5)
|
||||
expect(out[0].id).toBe('0')
|
||||
expect(out[out.length - 1].id).toBe(String(PAGE_SIZE + 4))
|
||||
})
|
||||
|
||||
it('throws when the query returns an error', async () => {
|
||||
await expect(
|
||||
fetchAllRows<Row>(() => Promise.resolve({ data: null, error: { message: 'boom' } })),
|
||||
).rejects.toThrow('boom')
|
||||
})
|
||||
|
||||
it('returns [] when the first page is empty', async () => {
|
||||
const out = await fetchAllRows<Row>(pagedQuery({ 0: [] }))
|
||||
expect(out).toEqual([])
|
||||
})
|
||||
|
||||
// ── The regression-critical behaviour: an unstable cross-page order ──
|
||||
// (a query missing a stable .order()) can return the same row on two
|
||||
// pages. This is the mechanism behind the doubled-balance bugs (#790/#791).
|
||||
|
||||
it('dedupeBy drops a row duplicated across page boundaries (keeps first)', async () => {
|
||||
const page1 = makeRows(0, PAGE_SIZE) // ids 0..999
|
||||
// Unstable order: page 2 re-serves id "999" (already on page 1) plus a new id.
|
||||
const page2: Row[] = [
|
||||
{ id: '999', value: 1 },
|
||||
{ id: '1000', value: 1 },
|
||||
]
|
||||
const out = await fetchAllRows<Row>(
|
||||
pagedQuery({ 0: page1, [PAGE_SIZE]: page2 }),
|
||||
{ dedupeBy: (r) => r.id },
|
||||
)
|
||||
// 1001 unique ids (0..1000), the duplicate "999" removed → no doubling.
|
||||
expect(out).toHaveLength(PAGE_SIZE + 1)
|
||||
const ids = out.map((r) => r.id)
|
||||
expect(ids.filter((id) => id === '999')).toHaveLength(1)
|
||||
expect(new Set(ids).size).toBe(out.length)
|
||||
})
|
||||
|
||||
it('without dedupeBy, cross-page duplicates pass through (unsafe default)', async () => {
|
||||
const page1 = makeRows(0, PAGE_SIZE)
|
||||
const page2: Row[] = [{ id: '999', value: 1 }]
|
||||
const out = await fetchAllRows<Row>(pagedQuery({ 0: page1, [PAGE_SIZE]: page2 }))
|
||||
expect(out).toHaveLength(PAGE_SIZE + 1)
|
||||
expect(out.map((r) => r.id).filter((id) => id === '999')).toHaveLength(2)
|
||||
})
|
||||
|
||||
it('dedupeBy is a no-op for a single page (no cross-page duplicates possible)', async () => {
|
||||
const rows: Row[] = [
|
||||
{ id: 'a' },
|
||||
{ id: 'b' },
|
||||
{ id: 'a' }, // an intra-page repeat is left untouched — single page is trusted
|
||||
]
|
||||
const out = await fetchAllRows<Row>(pagedQuery({ 0: rows }), { dedupeBy: (r) => r.id })
|
||||
expect(out).toHaveLength(3)
|
||||
})
|
||||
})
|
||||
@@ -1,11 +1,41 @@
|
||||
import { createLogger } from '@/lib/logger'
|
||||
|
||||
const log = createLogger('fetch-all')
|
||||
|
||||
const PAGE_SIZE = 1000
|
||||
|
||||
export interface FetchAllRowsOptions<T> {
|
||||
/**
|
||||
* Stable de-duplication key. When supplied AND more than one page was
|
||||
* fetched, rows are de-duplicated by this key after all pages are collected,
|
||||
* and a warn is logged if any duplicates were dropped.
|
||||
*
|
||||
* This is a safety net, NOT the fix: PostgREST `.range()` paging is only
|
||||
* correct when the underlying query has a stable TOTAL order (see the
|
||||
* ordering invariant below). If a duplicate is ever observed here it means a
|
||||
* caller's query is missing that `.order()` — the warn surfaces the
|
||||
* regression in logs instead of letting it silently double financial totals.
|
||||
* Note this only catches *duplicates*; *skipped* rows can only be prevented
|
||||
* by ordering on a unique column at the call site.
|
||||
*/
|
||||
dedupeBy?: (row: T) => string | number
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetches all rows from a Supabase query by paginating through results.
|
||||
* Overcomes PostgREST's default 1000-row limit.
|
||||
*
|
||||
* **Ordering invariant:** any query that can return more than `PAGE_SIZE` rows
|
||||
* MUST `.order()` on a unique column (e.g. the table's `id` PK). Postgres
|
||||
* returns rows in an undefined order that can differ between the two `.range()`
|
||||
* requests, so without a stable total order, rows on a page boundary are
|
||||
* silently DUPLICATED and/or SKIPPED across pages. For aggregating reports
|
||||
* (general ledger, trial balance, grundbok) that means doubled or missing
|
||||
* balances. Order is purely for paging stability — callers that need a
|
||||
* different display order should re-sort after fetching.
|
||||
*
|
||||
* The callback receives `{ from, to }` range values — append `.range(from, to)`
|
||||
* to your query builder:
|
||||
* to your query builder, AFTER a stable `.order()`:
|
||||
*
|
||||
* ```ts
|
||||
* const accounts = await fetchAllRows(({ from, to }) =>
|
||||
@@ -13,27 +43,62 @@ const PAGE_SIZE = 1000
|
||||
* .from('chart_of_accounts')
|
||||
* .select('account_number, account_name')
|
||||
* .eq('company_id', companyId)
|
||||
* .order('account_number', { ascending: true }) // stable total order
|
||||
* .range(from, to)
|
||||
* )
|
||||
* ```
|
||||
*
|
||||
* Pass `{ dedupeBy }` as defense-in-depth for queries where a missing/regressed
|
||||
* order would corrupt money:
|
||||
*
|
||||
* ```ts
|
||||
* const lines = await fetchAllRows(
|
||||
* ({ from, to }) => q.order('id').range(from, to),
|
||||
* { dedupeBy: (r) => r.id },
|
||||
* )
|
||||
* ```
|
||||
*/
|
||||
export async function fetchAllRows<T>(
|
||||
queryFn: (range: { from: number; to: number }) => PromiseLike<{
|
||||
data: T[] | null
|
||||
error: { message: string } | null
|
||||
}>
|
||||
}>,
|
||||
options?: FetchAllRowsOptions<T>
|
||||
): Promise<T[]> {
|
||||
const allRows: T[] = []
|
||||
let from = 0
|
||||
let pages = 0
|
||||
|
||||
while (true) {
|
||||
const { data, error } = await queryFn({ from, to: from + PAGE_SIZE - 1 })
|
||||
if (error) throw new Error(error.message)
|
||||
if (!data || data.length === 0) break
|
||||
allRows.push(...data)
|
||||
pages += 1
|
||||
if (data.length < PAGE_SIZE) break
|
||||
from += PAGE_SIZE
|
||||
}
|
||||
|
||||
// Duplicates are only possible across page boundaries, so single-page results
|
||||
// never need the dedup pass.
|
||||
if (options?.dedupeBy && pages > 1) {
|
||||
const seen = new Set<string | number>()
|
||||
const deduped: T[] = []
|
||||
for (const row of allRows) {
|
||||
const key = options.dedupeBy(row)
|
||||
if (seen.has(key)) continue
|
||||
seen.add(key)
|
||||
deduped.push(row)
|
||||
}
|
||||
const dropped = allRows.length - deduped.length
|
||||
if (dropped > 0) {
|
||||
log.warn(
|
||||
'fetchAllRows dropped duplicate rows across pages — a paginated query is missing a stable .order() on a unique column',
|
||||
{ dropped, total: allRows.length, pages }
|
||||
)
|
||||
return deduped
|
||||
}
|
||||
}
|
||||
|
||||
return allRows
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user