Screw Up the Control Groups
- Don't keep a control group
- Make the control group so small as to be useless
- Raid the controls to make up campaign numbers
- Use deadbeats as controls
- Don't make the controls random
- Use Treatment Controls but not Targeting Controls
Alienate or Annoy the Customer
- Trigger defection with retention activity
- Use intrusive contact mechanisms
- Use inappropriate
or niche creative content
- Insult or patronize the Customer
- Mislead or disappint the Customer
- Fail to Listen to the Customer
- Bonus marks for refusing to take "no"
for an answer
- Double bonus marks for calling the customer
back after she's said
"No thanks, I'm not interested"
and hung up.
- Make it hard for the Customer to do what you want.
uncooperative web forms,
understaffed call centres,
uninformed staff and
failure to carry stock to meet demand
are but a few of the ways to achieve this).
- Intrude on the Customer's Privacy
- (Over)exploit the customer
Misinterpret the Data
- Confuse "responses"
with "incremental responses" (uplift)
- Confuse revenue with net revenue
- Take Credit for That Which
Would have Happened
- Double Count
- Believe the name of a cluster
- Believe the data
The fact that the computer says it's true
and prints it prettily, doesn't mean it is.
- Disbelieve the data
The fact that the data doesn't show what you hoped,
thought or expected doesn't mean it isn't so.
Screw Up Campaign Execution
- Discard response information.
- Revel in unintended discounts and incentives
- Use dangerous, insulting or disrespectful labels
- Yes, I really have known an airline label
its bottom-tier frequent fliers "scum class",
and a retailer label a segment of its
- Ship mailings with the internal fields filled in
instead of external ones (see also 26)
- Direct people to slow,
or overloaded or
poorly designed call centres
- Fail to inform your front-line staff about
offers you're sending to customers
- Fail to ensure your company can fulfill
what the marketing promises
- Use a name that's very visually similar
to a better known name that has negative
connotations for many people.
- I frequently get mail from
but I never read their name correctly.
Mismodel or Misplan the Campaign:
with ability to make money
- Model the wrong thing
- Use undirected modelling
instead of directed modelling
- Assume equivalence of campaigns when important
things have changed.
- Ignore external factors
(seasonality, competitor behaviour etc.)
- Fail to make dates relative
- Contaminate the training data with validation data
- Screw up the Observation Window (Predict the Past, not the Future)
- Ignore changes in the meaning of data
- Fail to sanity check anything and everything
Labels: alienation, controls, modelling, negative effects, response, uplift