fix(mail): stop Gmail refusing the search, and stop calling that "hittade inget" (#1521)

Pressing Leta produced mails=25, documents=0 on a real two-mailbox run.
Nothing was found because nothing was searched: every request came back
429 "Too many concurrent requests for user".

Two bugs, and the second is the one that matters.

The search fanned out with Promise.all over every message id at once, one
Gmail request per message, per connection. Gmail enforces a per-user
concurrency ceiling as well as a daily quota, and this sailed past it long
before any volume worth worrying about. It now runs through a pool of five
per connection, which is comfortably under and still finishes a page of
results in a couple of round trips.

The catch turned each refusal into an empty array, with a comment saying
one mailbox's failure must not become the company's. Right instinct, wrong
consequence: an empty array is also what an empty mailbox returns, and the
manual hunt loop stops on fetched === 0 because that is its signal for
"the mailboxes hold nothing more for what is open". So a rate-limited
search told the user their receipts do not exist, and stopped looking.

searchFailureCount() now separates "could not look" from "nothing there".
The run route reports it, and the loop treats a pass with failures as
failed rather than finished, so pressing again is the obvious next move
instead of a pointless one.

This is the failure this feature exists to catch, happening inside the
feature: silence that reads as an answer.

Restoring the unbounded fan-out fails one test; removing the failure
counter fails three.

Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

DECISIONS.md

@@ -863,3 +863,4 @@ One line per decision: `[YYYY-MM-DD] <decision>: <why>`. Appended by agents and
 [2026-08-10] Staging DB reconcile (metjnjrhvujscngnpzdv): the tracker had skipped everything from 20260721101500 to 2026-08-10 (105 local-only versions) while 25 rows existed only remotely. Renamed 10 remote rows to their repo versions (same name, MCP apply-time version drift: sandbox-cleanup consolidation, shopify, tax-depreciation, JEL index), deleted 7 superseded sandbox-iteration rows with no local file, and left 8 rows from unmerged branches (white-label brands/teams, vacation columns, agent-atom product tier) untouched since their content is deliberately live for the byra rigs. Older seed_agent_atom_bodies files register version-only: each seed is a full idempotent upsert with a version guard, so only the newest seed's content needs to run.
 [2026-08-11] suggest-booking derives the proposed kontering on demand rather than storing it on the inbox row or computing it in the receipt hunt: a stored proposal goes stale against a corrected amount, a re-matched transaction or a template the company taught itself since, and the nightly hunt is already at its 300 s ceiling for a proposal most rows never open. It composes the existing evaluateMappingRules -> buildTransactionEntryLines chain rather than a second one, so the shown lines cannot drift from the posted lines. It withholds the proposal entirely on a foreign-currency row that matched via the mapping_rules branch: mapping-engine.ts buildResult computes VAT from the transaction's own currency while every other line is SEK (its own NOTE tracks this), which understates ingaende moms by the exchange rate and still balances, so nothing downstream catches it. Guarding the surface was chosen over fixing buildResult in this PR because that changes posted VAT amounts across every caller; the counterparty and static-template paths already convert correctly and are not withheld.
 [2026-08-11] Agent skills for the API ship as generated artifacts, not authored docs: skills/accounted-api/ is CI-checked output (apiskill:check) of scripts/api-skill/generate.ts, rendered from the same lib/api/v1 registry that serves the API and its OpenAPI spec, so the installable skill cannot drift from the server. Edit scripts/api-skill/overlays/ or the registry, never the output. The per-operation renderer is the portable tool inside skills/openapi-to-skill/ (the generic spec-to-skill generator): our own skill dogfoods it. Skills live in top-level skills/ because that is the directory `npx skills add erp-mafia/accounted` scans; the OpenAPI generator was extended to emit requestBody + path parameters (previously response-only) rather than teaching the skill generator to read Zod directly, so every spec consumer benefits, not just the skill.
+[2026-08-11] Gmail search fans out with a bounded pool (5 per connection) instead of Promise.all over every message id: Gmail enforces a per-user CONCURRENCY ceiling, not just a daily quota, and answers 429 "Too many concurrent requests for user" well below this app's volume. A real two-connection run returned mails=25 documents=0 purely from 429s. The catch in searchOne turned every refusal into an empty result, which is indistinguishable from an empty mailbox, and the manual hunt loop stops on fetched===0 as its "nothing left to find" signal, so the user was told their receipts do not exist by a search that never ran. searchFailureCount() now separates "could not look" from "nothing there", the run route returns it, and the loop treats a pass with failures as failed rather than finished.

app/api/receipt-hunt/run/route.ts

@@ -80,11 +80,20 @@ export const POST = withRouteContext('receipt_hunt.run', async (_request, ctx) =
   })
 
   const searched = result.mail?.searched ?? 0
+  const searchFailures = result.mail?.searchFailures ?? 0
+  if (searchFailures > 0) {
+    log.warn('manual receipt hunt: some mailboxes refused the search', {
+      companyId,
+      runId,
+      searchFailures,
+    })
+  }
   log.info('manual receipt hunt finished', {
     companyId,
     runId,
     searched,
     fetched: result.mail?.ingested ?? 0,
+    searchFailures,
     proposed: result.proposed,
   })
 
@@ -96,6 +105,10 @@ export const POST = withRouteContext('receipt_hunt.run', async (_request, ctx) =
       fetched: result.mail?.ingested ?? 0,
       proposed: result.proposed,
       remaining: Math.max(0, result.candidates - searched),
+      // Non-zero means a mailbox could not be read. Zero fetched then means
+      // "we could not look", not "there is nothing there", and the caller must
+      // neither say the second nor treat the run as finished.
+      searchFailures,
     },
   })
 })

components/extensions/general/use-receipt-hunt.ts

@@ -27,6 +27,8 @@ export interface HuntResult {
   proposed: number
   remaining: number
   failed?: boolean
+  /** Connections that refused the search. See the stop condition below. */
+  searchFailures?: number
 }
 
 export interface HuntProgress {
@@ -79,6 +81,15 @@ export function useReceiptHunt(onPass?: () => void) {
         onPass?.()
 
         // Nothing new this pass: the mailboxes have no more for what is open.
+        //
+        // Unless a mailbox refused to be read, in which case zero fetched says
+        // nothing about what is in there. Stopping on it, and reporting it as
+        // "hittade inget", would tell the user their receipts do not exist
+        // because Gmail was busy. Treat it as a failure and let them retry.
+        if ((body.data.searchFailures ?? 0) > 0) {
+          setResult({ ...body.data, fetched, proposed, failed: true })
+          return
+        }
         if (body.data.fetched === 0) {
           setResult({ ...body.data, fetched, proposed })
           return

extensions/general/mail/lib/__tests__/search-concurrency.test.ts

@@ -0,0 +1,118 @@
+/**
+ * Gmail refuses before any quota is near.
+ *
+ * The search fanned out one request per message id at once. Gmail answers 429
+ * "Too many concurrent requests for user" to that, and the catch turned the
+ * refusal into an empty result — which is indistinguishable from a mailbox that
+ * genuinely holds nothing. A real run against two connections produced
+ * `mails=25 documents=0`, and the client loop read the zero as "nothing left to
+ * find" and stopped. The user was told there were no receipts by a search that
+ * never happened.
+ *
+ * Two things must hold: the fan-out stays under the ceiling, and a refusal is
+ * distinguishable from an empty mailbox.
+ */
+import { describe, it, expect, vi, beforeEach } from 'vitest'
+
+const searchMessageIds = vi.fn()
+const getMessageSummary = vi.fn()
+const getAccessToken = vi.fn()
+const listActiveConnections = vi.fn()
+const touchSearched = vi.fn()
+
+vi.mock('@/lib/auth/api-keys', () => ({ createServiceClientNoCookies: () => ({}) }))
+vi.mock('../gmail-client', () => ({
+  searchMessageIds: (...a: unknown[]) => searchMessageIds(...a),
+  getMessageSummary: (...a: unknown[]) => getMessageSummary(...a),
+  fetchAttachmentBytes: vi.fn(),
+  clearMessageCache: vi.fn(),
+  describeAttachment: vi.fn(),
+}))
+vi.mock('../google-oauth', () => ({ isGoogleMailConfigured: () => true }))
+vi.mock('../connections', () => ({
+  getAccessToken: (...a: unknown[]) => getAccessToken(...a),
+  listActiveConnections: (...a: unknown[]) => listActiveConnections(...a),
+  touchSearched: (...a: unknown[]) => touchSearched(...a),
+}))
+
+const { GmailSearchService } = await import('../search-service')
+
+function connection(id: string) {
+  return { id, email_address: `${id}@example.test`, provider: 'gmail', status: 'active' }
+}
+
+beforeEach(() => {
+  vi.clearAllMocks()
+  getAccessToken.mockResolvedValue('token')
+  listActiveConnections.mockResolvedValue([connection('c1')])
+  touchSearched.mockResolvedValue(undefined)
+})
+
+describe('GmailSearchService.search', () => {
+  it('never has more than a handful of summary requests in flight', async () => {
+    // The ceiling is Gmail's, not ours: exceeding it fails the whole search.
+    let inFlight = 0
+    let peak = 0
+    searchMessageIds.mockResolvedValue(Array.from({ length: 40 }, (_, i) => `m${i}`))
+    getMessageSummary.mockImplementation(async () => {
+      inFlight++
+      peak = Math.max(peak, inFlight)
+      await new Promise((r) => setTimeout(r, 1))
+      inFlight--
+      return { messageId: 'm', subject: 'Kvitto', from: 'a@b.c' }
+    })
+
+    const svc = new GmailSearchService()
+    await svc.search('company-1', { merchant: 'x', amount: 1, currency: 'SEK', date: '2026-08-01' })
+
+    expect(getMessageSummary).toHaveBeenCalledTimes(40)
+    expect(peak).toBeLessThanOrEqual(5)
+  })
+
+  it('reports a refused search instead of passing it off as an empty mailbox', async () => {
+    searchMessageIds.mockRejectedValue(new Error('Gmail 429: Too many concurrent requests for user.'))
+
+    const svc = new GmailSearchService()
+    const out = await svc.search('company-1', { merchant: 'x', amount: 1, currency: 'SEK', date: '2026-08-01' })
+
+    // No candidates either way; the count is the only thing that separates
+    // "could not look" from "nothing there".
+    expect(out).toEqual([])
+    expect(svc.searchFailureCount()).toBe(1)
+  })
+
+  it('counts a genuinely empty mailbox as no failure', async () => {
+    searchMessageIds.mockResolvedValue([])
+    const svc = new GmailSearchService()
+    const out = await svc.search('company-1', { merchant: 'x', amount: 1, currency: 'SEK', date: '2026-08-01' })
+    expect(out).toEqual([])
+    expect(svc.searchFailureCount()).toBe(0)
+  })
+
+  it('lets one refused mailbox shrink the search without hiding the others', async () => {
+    listActiveConnections.mockResolvedValue([connection('c1'), connection('c2')])
+    let call = 0
+    searchMessageIds.mockImplementation(async () => {
+      call++
+      if (call === 1) throw new Error('Gmail 429')
+      return ['m1']
+    })
+    getMessageSummary.mockResolvedValue({ messageId: 'm1', subject: 'Kvitto', from: 'a@b.c' })
+
+    const svc = new GmailSearchService()
+    const out = await svc.search('company-1', { merchant: 'x', amount: 1, currency: 'SEK', date: '2026-08-01' })
+
+    expect(out.length).toBe(1)
+    expect(svc.searchFailureCount()).toBe(1)
+  })
+
+  it('starts each search from zero failures', async () => {
+    searchMessageIds.mockRejectedValueOnce(new Error('Gmail 429')).mockResolvedValue([])
+    const svc = new GmailSearchService()
+    const q = { merchant: 'x', amount: 1, currency: 'SEK', date: '2026-08-01' }
+    await svc.search('company-1', q)
+    expect(svc.searchFailureCount()).toBe(1)
+    await svc.search('company-1', q)
+    expect(svc.searchFailureCount()).toBe(0)
+  })
+})

extensions/general/mail/lib/search-service.ts

@@ -43,7 +43,45 @@ function canonicalOrigin(): string {
   return process.env.NEXT_PUBLIC_APP_URL?.trim() || 'http://localhost:3000'
 }
 
+/**
+ * How many message summaries to pull at once, per connection.
+ *
+ * Gmail enforces a per-user concurrency ceiling, not just a daily quota, and
+ * answers 429 "Too many concurrent requests for user" well below any volume
+ * this app generates. Fanning out over every id at once reliably tripped it and
+ * returned an empty search, which is indistinguishable from a mailbox holding
+ * nothing. Five is comfortably under the ceiling and still finishes a page of
+ * results in a couple of round trips.
+ */
+const GMAIL_SUMMARY_CONCURRENCY = 5
+
+/** Map with a bounded worker pool, preserving input order. */
+async function mapWithConcurrency<T, R>(
+  items: readonly T[],
+  limit: number,
+  fn: (item: T) => Promise<R>,
+): Promise<R[]> {
+  const out = new Array<R>(items.length)
+  let next = 0
+  const workers = Array.from({ length: Math.min(limit, items.length) }, async () => {
+    for (;;) {
+      const i = next++
+      if (i >= items.length) return
+      out[i] = await fn(items[i])
+    }
+  })
+  await Promise.all(workers)
+  return out
+}
+
 export class GmailSearchService implements MailSearchService {
+  /**
+   * Connections whose search threw during the last search() call. A refused
+   * mailbox returns no candidates, exactly like an empty one, so without this
+   * the run reports "nothing found" about a mailbox it never managed to read.
+   */
+  private failures = 0
+
   isConfigured(): boolean {
     return isGoogleMailConfigured()
   }
@@ -60,6 +98,7 @@ export class GmailSearchService implements MailSearchService {
   async search(companyId: string, query: MailSearchQuery): Promise<MailCandidate[]> {
     if (!this.isConfigured()) return []
 
+    this.failures = 0
     const supabase = createServiceClientNoCookies()
     const connections = await listActiveConnections(supabase, companyId)
     if (connections.length === 0) return []
@@ -68,6 +107,8 @@ export class GmailSearchService implements MailSearchService {
 
     // Mailboxes are searched in parallel: the work is read-only, so there is
     // nothing to serialise, and one slow account should not delay the rest.
+    // The per-message fan-out inside searchOne is throttled, which is where
+    // Gmail's per-user concurrency ceiling actually bites.
     const perConnection = await Promise.all(
       connections.map((connection) => this.searchOne(supabase, connection, q, query.limit)),
     )
@@ -89,17 +130,24 @@ export class GmailSearchService implements MailSearchService {
       const ids = await searchMessageIds(accessToken, q, limit)
       if (ids.length === 0) return []
 
-      const summaries = await Promise.all(
-        ids.map((id) =>
-          getMessageSummary(accessToken, id, connection.id, connection.email_address),
-        ),
+      // One request per message, but not all at once. Gmail answers 429
+      // "Too many concurrent requests for user" long before any daily quota is
+      // near, and a whole search can come back empty because of it. The failure
+      // used to look exactly like an empty mailbox, so the honest fix is to
+      // stop provoking it rather than to report it more loudly.
+      const summaries = await mapWithConcurrency(ids, GMAIL_SUMMARY_CONCURRENCY, (id) =>
+        getMessageSummary(accessToken, id, connection.id, connection.email_address),
       )
       await touchSearched(supabase, connection.id)
 
       // Cheap pre-filter before anything expensive looks at these.
       return summaries.filter((c) => looksLikeReceipt(c.subject, c.from))
     } catch (error) {
-      // Never let one mailbox's failure surface as the company's failure.
+      // Never let one mailbox's failure surface as the company's failure: the
+      // other mailboxes still have answers. But a refused search is not an
+      // empty one, and the caller has to be able to tell them apart, or
+      // "hittade inget" gets said about a mailbox nobody managed to read.
+      this.failures += 1
       log.warn('gmail search failed for connection', {
         connectionId: connection.id,
         error: error instanceof Error ? error.message : String(error),
@@ -108,6 +156,11 @@ export class GmailSearchService implements MailSearchService {
     }
   }
 
+  /** How many connections refused the last search. */
+  searchFailureCount(): number {
+    return this.failures
+  }
+
   async fetchAttachment(
     connectionId: string,
     messageId: string,

lib/mail-search/service.ts

@@ -104,6 +104,15 @@ export interface MailSearchService {
    * and must not outlive the run that read it, so the caller says when that is.
    */
   releaseCache?(): void
+  /**
+   * How many connections refused the last search.
+   *
+   * A refused mailbox yields no candidates, exactly like an empty one. Without
+   * a way to tell them apart, a run that Gmail rate-limited reports "found
+   * nothing" and the caller stops looking, which is the worst possible answer:
+   * it is wrong, and it sounds final.
+   */
+  searchFailureCount?(): number
 }
 
 class NoopMailSearchService implements MailSearchService {

lib/receipt-hunt/hunt.ts

@@ -128,6 +128,14 @@ export interface MailHuntSummary {
   withCandidates: number
   /** Receipts actually fetched and filed, ready for the amount match. */
   ingested: number
+  /**
+   * Connections that refused a search during this run.
+   *
+   * Non-zero means "we could not read a mailbox", which is not the same as
+   * "the mailbox held nothing". The caller must not report the second when the
+   * first happened, and must not treat the run as finished.
+   */
+  searchFailures: number
   candidates: Array<{
     /** Merchant the model resolved the bank descriptor to. */
     merchant: string
@@ -606,7 +614,7 @@ async function harvestReceiptsFromMail(
   maxMails: number,
 ): Promise<MailHuntSummary> {
   const service = getMailSearchService()
-  const summary: MailHuntSummary = { searched: 0, withCandidates: 0, ingested: 0, candidates: [] }
+  const summary: MailHuntSummary = { searched: 0, withCandidates: 0, ingested: 0, searchFailures: 0, candidates: [] }
   if (!service.isConfigured() || purchases.length === 0) return summary
 
   // Salary and tax runs are a company's largest outgoing rows, so without this
@@ -639,6 +647,9 @@ async function harvestReceiptsFromMail(
       useDateWindow: false,
       limit: MAX_CANDIDATES_PER_MERCHANT,
     })
+    // Accumulated across every merchant searched in this run: one refusal is
+    // enough to make "found nothing" a lie.
+    summary.searchFailures += service.searchFailureCount?.() ?? 0
     // The same mail answers several purchases from one supplier; it is read once.
     for (const c of found) {
       if (!byMessage.has(c.messageId)) byMessage.set(c.messageId, c)