Experiment 004, copy
What is your reply rate, divided by what?
One of our campaigns has a 3.3% reply rate. It also has a 0.6% reply rate. Both are true.
Same campaign, two reply rates
Our sending tool shows one campaign at 0.6% on its campaign list and 3.3% on the campaign's own page. Same campaign, same day, more than five times apart. Neither screen is wrong. They answer two questions nobody states.
The two questions
Divided by what? The emails you sent, or the people you contacted. A sequence sends each person several emails, so the same replies give a much smaller rate per email than per person.
Are auto replies in it? Out of office messages and auto responders land in the same inbox as real replies. Some screens count them, some do not, and most do not say which.
One example, four true answers
Say you email 100 people, three emails each, so 300 emails. 5 people reply, and 3 of them are out of office.
| Divided by people (100) | Divided by emails (300) | |
|---|---|---|
| Auto replies in (5) | 5% | 1.7% |
| Auto replies out (2) | 2% | 0.7% |
All four are true. Counting auto replies is not wrong; not saying so is. And even counted the same way, two campaigns differ in targeting and offer, so one reply rate next to another tells you very little.
What we do now
Every readout we give, in a weekly report or on a call, has one shape:
| Line | Always shown as |
|---|---|
| Replies | total, human, automatic, positive |
| Bounces | count and share of contacted |
| Replies by the recipient's mail provider | count and share of contacted |
Always divided by people contacted, and always with the auto replies shown as their own line.
Try this
Next time someone quotes a reply rate, ask two things: divided by what, and are auto replies in it?