Brand mention matching in social captions: hashtag tails, plurals, apostrophes and the false positives each fix creates

A word-bounded phrase matcher (Aho-Corasick or regex) over social-media captions misses brands glued into hashtags (#brandhaul, #brandsandals), plurals, curly apostrophes from phone keyboards (brand’s), unaccented spellings, and 3-letter names dropped by a minimum-length filter. The naive fixes (match any hashtag that starts with the brand, fold accents in captions) create large false-positive counts.

Measure on raw captions (not on a set already filtered by the old matcher, which hides the misses). Build a WHITELIST of hashtag tails from data: tails used by many distinct brands across categories (haul, finds, review, edit, clips, cosplay, partner, unboxing, gameplay, life, tv, digits...). At match time accept brand+tail only if: (1) tail is whitelisted or 1-4 digits; (2) brand+tail is not itself a dictionary word; (3) the hashtag cannot be read as another dictionary word + a whitelisted tail (e.g. "fallout"+"fits" vs "fall"+"outfits"); (4) common-word brands written as '#word' get no tails. Normalize only apostrophes in captions; add accent-free and NFD variants to PHRASES instead of folding captions. Add plural variants only when neither the base nor the plural is a dictionary word and no other brand owns that exact form. Glue '+' as 'plus' ("disney+" -> "disneyplus", never "disney"). Count once per owner as well as per brand, because parent entries often bundle sub-brands. Validate by running old and new matchers side by side on millions of raw captions and reviewing the top gained and lost tokens.

Open prefix matching turns short brand tokens into prefixes of unrelated words (#motivation, #handmade, #madrid, #babygirl). Accent folding of captions maps foreign words onto brand names (über -> uber). Bare brand names that are ordinary words or Spanish words (fila, moderna, persona, hers) dominate their counts. pyarrow 21's utf8_normalize("NFC") returned decomposed text in testing, so vectorized normalization silently broke accented matches.