task-0364 Telegram Bug Intake

• Fingerprint: dfa5d2de37a3586d
• Telegram Update ID: 609161123
• Telegram Message ID: 143
• Chat: Phil (6907249628)
• Sender: Phil
• Date: 2026-09-25T09:32:41-06:00

Reporter Message

Forwarded from Telegram user Kathleen (@Kat_kjk)

I tried to open AVST 432 COGs for August (not even YTD)  and got this error. Is there something I need to do differently?

Telegram Attachments

The monitor saved Telegram photos, documents, and other media as task attachments. The task board shows image thumbnails and file links in its Attachments panel.

• photo.jpg: image/jpeg, 9728 bytes, photo, saved. Saved at .agents/tasks/artifacts/attachments/task-0364/photo.jpg.

Triage Checklist

• Identify the affected workflow, branch/location, user role, and concrete records or timestamps mentioned in the report.
• Review saved Telegram media artifacts and mention relevant files in the plan.
• Check whether this duplicates an existing dispatcher task before planning new work.
• Pull fresh logs or database context only as needed, using local-only access details outside this artifact.
• Decide whether the result should be product-code work, support/data cleanup, ops follow-up, duplicate, or no-op.

Investigation Findings

The screenshot is the Financial Transactions result guard introduced by task 0275. Production request logs confirm the reported request was:

• company code AVST;
• account 432;
• 1–31 August 2026;
• period-based dates;
• no YTD and no entity filter.

The production read-only replica contains 26,252 distinct account-432 journal entries for that exact filter. The result joins return the same 26,252 journal IDs; the account-range, chart-of-account, and department joins do not multiply this result. The 10,000-row sentinel is therefore behaving as implemented rather than reporting a false join count.

The unusual volume is concentrated in Logistics freight costing:

Those 25,060 Logistics rows came from 75 logistics invoices. One older logistics invoice affected 1,475 underlying transactions and generated 10,325 account-432 rows when its PO association was repeatedly reverted and reapplied on 7 August. The journal IDs are distinct, but the deeper invoice-level investigation below shows this was an overlapping-save replay storm, not normal invoice detail. It was produced by the existing Logistics costing callbacks rather than by the Financial Transactions SELECT.

August is an outlier: AVST account 432 had 2,581–8,640 rows in each month from January through July 2026, then 26,252 in August. A single day, 7 August, contains 18,070 rows. Narrowing to that day therefore still exceeds the safety ceiling.

Existing Design Context

Task 0275 intentionally chose a 10,000-row complete-result ceiling after unbounded requests used roughly 21–28 GB of Passenger memory. That task also recorded, but did not build, an approved keyset/cursor pagination contract with a 2,500-row maximum page. Task 0198 documents that Logistics PO edits intentionally use destroy/recreate so freight is reverted and reapplied. That makes a naive in-place association update or historical journal deletion unsafe, but it does not make repeated concurrent callback execution correct.

Deeper Invoice-Level Investigation

Invoice and actor

The outlier is SLC Logistics invoice INV-1148, now stored as 1148 (internal Logistics id 21638):

The attribution is not based on an IP address alone. Both invoice versions, every association version, every affected item version, and all 20,650 financial-journal create versions carry Christine Wellington's authenticated PaperTrail identity. They also share the same workstation and browser fingerprint. No other authenticated user appears under that workstation fingerprint that day.

What Christine was trying to do

The only deliberate invoice-field change was:

INV-1148 → 1148

There is no audit event changing 1148 back to INV-1148, changing the amount, moving the invoice to another PO, or altering the 100% split.

The AP records explain the rename:

Under the application behavior in effect on 7 August, Logistics found its AP partner by invoice number and vendor. Renaming the Logistics record to 1148 made it match the surviving AP record instead of the void INV-1148 record.

There is also strong operational context. During the two hours immediately before this rename, Christine deleted eleven old Logistics records, mostly tariff-labelled records. Minutes later, task 0198 was opened to address old Logistics/AP cash-flow orphans and renamed invoice mismatches. These facts strongly support a legacy-data reconciliation purpose. The audit trail cannot prove who asked her to do it or her subjective intent; Christine would need to confirm that detail.

Why seven passes occurred

The evidence shows three executions of the same Save, not seven intentional edits. Each execution submitted the same desired invoice number, the same PO, and the same 100% allocation.

Rails timestamps are stored in UTC below and converted to MDT for readability:

The roughly four-minute spacing matches the time required for one request to walk the historical PO data. The frontend does not automatically retry this request, so the most likely explanation is that Save appeared stuck and the same change was submitted again, possibly after reopening the dialog or from another tab. The old request logs are no longer retained, so the exact click/reopen sequence is an inference rather than a proven fact. What is proven is that the requests overlap and request the same final values; they are not intentional apply/undo operations.

The single outlier Logistics invoice did not have 10,325 independent details. It had 1,475 affected historical inventory transactions. On 7 August, the application ran the same freight traversal seven times:

Every account-432 row has a matching account-118-0100 side, so the callbacks created 20,650 balanced journal rows in total. Passes 2–7 cancel in three exact pairs. The remaining financial effect is pass 1, the initial reversal.

Primary-database version timestamps map those seven passes to association lifecycle events. A save destroyed the old association and created a replacement. A second save then destroyed that replacement and created the association that remains today. Before that second save's freight application completed, a stale overlapping request destroyed the already replaced association again. Its later duplicate association was created and then destroyed by duplicate collapse, causing two more full traversals.

The item state confirms this was not harmless audit noise. All 222 receipt items changed freight exactly seven times, alternated between exactly two values, and finished in the same state as pass 1—the reverted state. None finished in the pass-4 applied state associated with the PO link that still exists. The surviving link and the freight values therefore disagree. The journals remain balanced between accounts, but the freight classification and item values are likely wrong. This is a strong inference from exact state/timestamp matching; any correction still requires an accounting reconciliation.

The code permits this outcome:

• the invoice update action destroys and recreates every matching PO association on every save, even when the PO and split percentage did not change;
• after_create applies freight and before_destroy reverts it by walking all affected PO, receipt, inventory, and historical detail records;
• duplicate collapse destroys an extra association and therefore runs another full reversal;
• there is no server-side per-invoice lock, idempotency boundary, or unique database constraint for the association; and
• the outer invoice/association transaction and the financial-journal transaction use different databases, so they do not form one atomic commit.

The browser disables its Save button while one request is in flight, but that only guards one dialog instance. It cannot serialize retries, multiple tabs, or multiple clients. The overlapping database timestamps demonstrate that concurrent saves occurred despite the browser guard.

This pattern is not unique to the one invoice. In a read-only aggregate for August and September, 20 of 166 Logistics invoice sources had more account-432 rows than underlying transaction IDs; four had at least three effective passes. Those sources are candidates for reconciliation, not proof that every repeated row is erroneous.

Recommendation

Do not raise the 10,000-row guard and do not alter production data yet. Reclassify this as an urgent, high-risk financial-integrity fix before treating pagination as the only answer.

The engineering plan should serialize updates per invoice, reload association state inside that boundary, and make an unchanged PO/split save a no-op. Cost application and reversal should become one explicit, idempotent operation rather than an incidental effect of destroy/create and duplicate collapse. Concurrency coverage must prove that overlapping identical saves produce one logical transition and leave the association, item freight, and balanced journals consistent. A uniqueness constraint may be useful, but it is not sufficient by itself because stale saves and cross-database partial completion still need handling.

Before any correction, produce a read-only reconciliation inventory and have accounting approve the expected state and compensating entries. For invoice 1148, explicitly reconcile the PO association, all affected PO/receipt/inventory/invoice-item freight values, and the account-432/account-118-0100 effect. Historical journal deletion or rewriting is not recommended.

Use a staged prevention strategy:

1. Immediate containment: serialize updates per Logistics invoice, reload current associations after acquiring the lock, reject a concurrent save with a clear "already updating" response, and make an unchanged PO/split payload a true no-op. A header-only invoice-number correction must generate zero freight callbacks.
2. Database guardrail: add a unique association constraint for a Logistics invoice, order type, and order identity. Treat it as defense-in-depth, not the main fix.
3. Explicit costing operation: remove freight posting from generic create/destroy callbacks and run one explicit before/after allocation command. Give each command a durable operation ID and make financial posting idempotent so a retry resumes or returns the prior result instead of posting again.
4. Cross-database recovery: because purchasing/item state and financial journals are stored in different databases, record durable operation state and retry/compensate safely rather than assuming nested ActiveRecord transactions commit atomically.
5. Long-running UX: do large historical recalculations as a tracked background job, show progress, and prevent another edit to that invoice until completion. Keep an immediate client-side re-entry guard, but do not rely on the browser for correctness.
6. Monitoring: record operation ID, invoice, before/after allocation, actor, duration, and rows affected. Alert when the same invoice/transaction receives repeated freight postings or an operation exceeds an expected row/runtime threshold.

Required regression proof includes a header-only edit, two and three simultaneous identical saves, a real split/PO change, duplicate payload rows, and injected failure between primary-database item updates and financial posting. Each case must prove the final association, item freight, and financial journals agree and that a retry cannot create an additional logical adjustment.

Bounded cursor pagination remains worthwhile after this fix: removing six extra passes from this invoice alone would still leave the August account query above 10,000 rows. Retain the task-0275 cursor contract as a separate follow-on, including clear page-only totals and the broad-query safety boundary.