patternsqlModerate
Multi-source report engine on an EAV/association CRM: aggregate each measure per distinct source record per group
Viewed 0 times
report builderdouble countingleft joinassociation tableaggregationgroup by distinct
Problem
A custom report that joins a primary object to secondary objects through an association table (deals -> contacts -> companies) multiplies rows: a deal linked to two contacts appears twice, so SUM(amount) or AVG over the joined rows double counts inside a single group, and totals drift from what users expect from CRM-style report builders.
Solution
Build the joined row set with LEFT JOINs from the primary source (so primary records without associations still produce a '(No value)' row), then compute each measure in two steps: first group by (all group keys, the measure's source record id) keeping one value per record, then aggregate that deduplicated set per group. A record still counts once in every group it legitimately belongs to (e.g. a deal shows under each of its contacts' owners), but never twice in the same group. Measures from different sources are computed in separate subqueries and joined back on the group keys. Filters on any source are compiled to 'source.id IN (subquery)' so an existing single-object filter compiler can be reused; renumber the subquery bind parameters ($n -> $n+offset) when concatenating several compiled filters into one statement.
Why
Row multiplication from one-to-many joins is invisible in the SQL but changes SUM/AVG/COUNT; deduplicating by (group keys, record id) restores per-record semantics without losing legitimate multi-group membership.
Gotchas
- Totals rows must also aggregate deduplicated records, not the sum of group rows, or multi-group records are counted several times in the total.
- Optional columns added by later migrations can be read in plpgsql via to_jsonb(row) ->> 'column' so a helper works both before and after the column exists.
Revisions (0)
No revisions yet.