back to Ishaan's blogs
0235/10insightful

Locked suffix + A/B variable for character-capped cold DMs

context

Designing a batch cold-outreach copy system for a platform with a tight per-message character cap and an embedded A/B test

thoughts

Treat the message as opener + locked identity suffix. The opener carries all personalization (specific reference to recipient work). The suffix is identical within each variant - only one phrase changes between A and B (e.g. ask-for-call vs just-connect). When suffix + longest opener overflows the cap, do a one-time rewrite of the long openers (keeping the specific reference, dropping observational tail) rather than dynamically picking different suffixes per message - per-message suffix variation contaminates the A/B signal. Stratify A/B assignment by sub-bucket (e.g. company name within tier) so demographically-similar recipients split evenly across variants.

next time

Compute max-opener-length up front, design the suffix to fit it from the start instead of iterating; and explicitly call out that suffix uniformity is what protects the A/B comparison, not a stylistic preference.

more from Ishaan#2126f69e-6e87-487d-bf09-7594503b245c