How would you show a 10 percent lift if you were us?
We work in an industry with thin margins. Like supermodels, the margins are only getting thinner. A 10% lift is a big, big deal. It lets you buy traffic you couldn’t afford before. It lets you grow. How would you show it to our market?
Here are some things to think about…
Some viewers are sophisticated business people but not all of them.
Everyone can see their number of sales per day. Some people can track revenue and lifetime value.
It’s unlikely that they are able see standard deviations and calculate statistical significance. Performance is judged more by sight. Typical daily sales variations are up to 50%. So the lift is effectively hidden from view. It can’t be “eyeballed”
There are no off-the-shelf a/b split test tools. They have to be set up.
The most visible metric is cost. But even this isn’t clear. For example, people operating merchant accounts rarely calculate per transaction fees for attempts into an effective processing rate percentage.
So how would you show a 10% lift?
One way is to simply show the lift compared with the baseline. A baseline is what would have happened if the system did not adapt to each user. The lift is the blue line, the baseline is the thick grey line.
It looks like this.
Another way is to show how the lift eliminated the rate. Each percentage increase is a decrease in effective rate. That would look like this. In this case the rate is the blue line and the baseline rate is the thick grey line. It represents the normal processing rate. The blue line would be the falling rate. (Yes, it’s a mirror image of the chart above).
Other ideas? How would you do it? Drop in the comment section to share your ideas.