gotchapythonMajor
A new collection channel entering a normalized time series mid-history creates fake spikes; bare channel names slip past trailing-separator LIKE filters
Viewed 0 times
Python 3.10+, SQLite FTS5
channel filterLIKE patternstep changenew data sourcenormalized seriesfalse spiketerm listtrending searches
macoslinux
Error Messages
Problem
A social-signal system stores rows from many channels in one posts table, tagging each row's channel in a single column ("reddit-sub-name", "bluesky:search:<brand>", "pinterest:search", "google:trending"). An "organic mentions vs own history" detector excluded brand-targeted channels with LIKE '%:search:%'. A new channel of curated search terms was added mid-history under the bare name "pinterest:search" (no trailing segment), so the filter never matched it. Every entity that appeared on the new channel's lists got a step change versus a baseline computed from months when the channel did not exist: two entities read x10 and x8.5 "above their usual" on ONE real post each plus ~26 list rows. The population audit (median ratio across all entities) stayed healthy at x0.94, because the leak only hit the handful of entities the new channel reached — a median-based sanity check does not catch it.
Solution
1) Never rely on a LIKE pattern with a trailing separator to classify channels; keep an explicit list of channel names in one shared constant and build the SQL fragment from it (AND col NOT IN ('pinterest:search','google:trending',...)), used by the numerator, the corpus denominator and the receipts query alike. 2) Treat "lists of terms" (trending searches, related searches, front pages) as separate surfaces, never as conversation counts. 3) When a channel starts mid-history, its rows cannot join any series whose baseline predates it — exclude it from that series, or give it its own baseline once it has enough history. 4) Diagnose by pulling the receipts: if an entity reads x10 but the human-written posts behind it are one or two, group the hits by channel for that window. 5) Add a test that inserts one real post plus one row per list channel and asserts the organic count is 1 and the corpus is 1 while the raw "everything" count still sees all rows.
Why
Classification by string pattern encodes an assumption about the naming scheme (every targeted channel has a trailing ':<target>' segment); the first channel named differently breaks the assumption silently, and a series normalized against its own past has no way to know the input mix changed.
Gotchas
- A median-across-entities sanity check passes while a few entities are badly wrong; check the entities that fire, not just the population.
- The corpus denominator needs the same exclusion, or every other entity's rate drifts down by the new channel's share of rows.
- Keep the list channels readable elsewhere (a themes/surfaces board may legitimately consume them); exclude them from talk counts only.
Code Snippets
One shared constant feeding every query that must ignore term-list channels
LIST_CHANNELS = ("pinterest:search", "google:trending", "tiktok:trending",
"hackernews:frontpage", "producthunt:daily")
def not_a_list_sql(col: str = "p.subreddit") -> str:
quoted = ", ".join(f"'{c}'" for c in LIST_CHANNELS)
return f"AND {col} NOT IN ({quoted})"
# numerator, denominator and receipts all append not_a_list_sql(); a test
# inserts one real post + one row per list channel and asserts organic == 1.Context
Any pipeline that unions heterogeneous sources into one table and judges entities against their own history: social listening, mention counting, anomaly detection on event counts.
Revisions (0)
No revisions yet.