(This is a capstone bringing together analytics and leadership at the close of the 2016 Vendo Partner Conference.)
We want to share a few thoughts about analytics and leadership as a capstone for the Vendo Partner Conference. A capstone is the last piece you put on an arch. It connects the two sides. We’re going to talk about our experience with AI that relates to both leadership and analytics, the two tracks of the Vendo Partner Conference.
What started us on this journey towards AI was a story.
Imagine you are in a medieval market. It’s around sunset. The market’s dusty. The men selling their wares are starting to close up for the day. You walk up to a stall. He’s selling oranges and almonds. There are no prices because he can’t write and you can’t read.
The shopkeeper sizes you up. You’re wearing nice clothes. It’s the end of the day so he knows you are running out of options. He starts to ask you questions. “Where are you from?” “What do you do?” He watches you squeeze the oranges.
He’s gathering information to give you a price. It’s a one to one sales process. He wants to give you a price that gets you to buy and spend as much as possible. He’s calculating probabilities of Conversion and Lifetime Value in his head.
That’s the way it has been for most of human history. It’s still is that way in many places.
We’ve had that story in mind as we’ve thought about the barriers to one to one selling. We have a vision of moving towards one to one selling because it makes the most sense. Well, dollars and sense. It generates the most sales and the most revenue and keeps your accounts active within risk limits.
It generates sales you wouldn’t have made because the price was too high for the initial or even the rebill.
It creates revenue you wouldn’t have made because you under-charged a shopper who was happy to pay more.
It let’s good transactions in and keeps bad ones out.
When we started out we had the wrong team for AI. It was a team designed to do normal analytics. One of our partners at Vendo has a brother who used to work in the CIA. He was a spy. He had two jobs. One was to stop nuclear weapons from getting in the hands of terrorists. His other job was to report to congress so that they could make decisions. He liked the first job and he was good at it. No terrorists blew up any cities while he was there.
He didn’t like his other job. There’s a reason that congress has a 7% approval rating, or said another way, 93% disapproval rating. They earn it. Part of the problem that congress faces isn’t really its own fault. They are asked to make extremely complex decisions. Imagine a congressman’s day. He’s going from a hearing on farming policy to fund raising to celebrating his local basketball team to talking with reporters to campaigning to a closed door hearing on covert operations against terrorists trying to get nukes. And he’s expected to make the right decision in each situation. It’s impossible.
We realized that information overload was a problem for us, too. We weren’t going to be able to review and analyze the data and make the right decision every time. There were too many situations. Too much data.
An average client of ours with an average site has around 100,000 visitors a day. That’s more than one per second. We can’t possibly figure out which transactions are good and which will chargeback or which price is going to convert best and generate the highest lifetime value for each one of them. Imagine each different shopper coming to the site. It’s happening right now.
How are they similar? How are they different? How did a risk decision or price you gave someone similar six weeks ago perform? Should you set your risk tighter or looser? Should you price higher or lower?
To figure out the answer we needed a new set Key Performance Indicators (KPIs). Andrew spoke about this in the session on Key Metrics. For risk it had to be how many risky transactions we could locate in a smaller and smaller percentages of transactions. For pricing, sales wouldn’t work because a lower price always generates more sales. Lifetime Value didn’t work because a high price would always give you higher LTV. We needed to combine them and focus on revenue per shopper. This lead us to combine both into Revenue per Click, or RPC.
It’s obviously impossible for the human brain to play a valuable role here. It’s like the congressman spending thirty minutes on terrorists at the end of a long day of appointments. He’s probably not going to add very much to the conversation. If he takes the right decision he was lucky. That’s how we felt as we looked the tasks in front of us. We literally had no idea. And we weren’t going to have any ideas that were actionable.
So we had a team that could give us advice but we couldn’t really act on it.
Charles shared his experience with re-organizing teams. It’s one of the biggest challenges a leader can face. It brings together strategy, people and execution. Those are the three areas a leader needs to focus on each day. And it’s full of risk.
We needed a different team. One that could build a system that would act on its own. Today there isn’t a single person on our analytics team with more than four years inside the company. The tools they are working with are new. Many of those tools didn’t exist or had not proven themselves valuable until recently.
Our analytical team members need both collaboration and solitude. Working remotely – even just from home here in Barcelona – is key for them to develop ideas during uninterrupted blocks of time. Julius spoke about criteria he uses for deciding on which roles can be remote in the “remote working” session.
They needed new ways of meeting together. Sean spoke about meetings that are really, in essence, about meetings. We were embarrassed to discover how quickly our meetings could devolve into massive wastes of time. Too many people, too unfocused. This lead us to shift more and more to what Buddy described in the session on collaboration tools as a Basecamp centered organization. Now everything we have planned is in Basecamp with due dates and names attached for each task. Agendas and minutes are there, too, for anyone who wants to see them.
Another part of deciding that we were ready to develop AI was to make sure we met other fundamental requirements.
Did we have enough data?
Did we have the computational power? Could we do it economically?
If we didn’t have either of these two then the team wouldn’t matter.
Fortunately we have data from large, medium and small companies across the industry. We host the pre-join page and apply our AI for pricing and risk there. Because we host this page we get a ton of data. We’re ranked 8,000 on Alexa. That may not sound like much but those are all shoppers who’ve reached the end of a tour and are making a purchase decision. If by chance you aren’t familiar with Alexa, it’s like golf, a lower score is better. Matt and Fadi have a property that’s ranked 60.
We have data on hundreds of millions of shoppers. And we’ve got data over time. We can see the effects of our decisions over years.
Computational power is a moving target. Moore’s law is still in effect. We’ve had to completely upgrade our systems several times. We’re constantly adding and improving and taking down old systems.
So those are the three keys to moving closer to our vision of one to one selling: teams, data and tools. Over the last four years we’ve had to work with limitations in each area.
We didn’t have the right people. We had to build the team. Thierry talked about the progression of analytics teams as companies grow in the “Analytical Teams” session. Not everyone we hired was a good fit. That’s how we learned to give them detailed tests at the beginning. Some of the people we hired have biomedical backgrounds. The guy who built Facebook’s analytical team is now working on finding cures for diseases. There’s a lot of cross over between the two areas. Look there for your next analytical hire.
The data we had at the beginning was practically useless. A friend of mine named Luca is a statistician. In addition to his research he works with the EU to do election monitoring around the world. When you see on the news that some country has had elections and “international observers determined that the results were free and fair” then chances are that he was there. One of his mantra’s is that “90% of statistics is getting clean data.” That’s been true in our experience. We weren’t collecting the right data. Or we were collecting data that didn’t matter, that just created noise. Or we were limited in how we used the data. We created hierarchies that didn’t make sense. Matt talked about the frustration of not being able to get the data you need in the session on analytical tools.
Here’s a challenge we had: Rank order the importance of these variables and their interactions…
New v. Returning
There were more variables, too, but just imagine how you would rank the relative importance of each one and interactions that can include all six. Not only is it difficult to imagine how you would do it. The variables change in importance constantly. At 02:00 in the morning local time that is by far the most important variable. At other times of day the actual time is relatively less important.
Laurent talked in the testing session about tools that go beyond human interaction and into machines taking both the decision and action. Tools that close the loop.
Next we had challenges with our computational power. We had to work around limitations. There would be a way of calculating that seemed perfect but would use all the computing power in the known universe. Since we couldn’t get access to that much power we had to look at other ways of calculating. We also have to look at costs. We make a fraction of what the end user spends and we have to run a profitable company. We can’t over spend to generate results. These are real constraints that we have to factor into our decision making. Google is famous for creating their own cheap servers to power their search engine. We’ve also created custom servers configurations to enable us to efficiently process mountains of data.
Next we needed to tell the world about it. We’re using AI throughout our company. We consider ourselves cyborgs, part human and part machine. It’s here in pricing, its there in risk and it’s heading into other areas. It’s our present and our future. How do we talk about it?
We chose pricing because it is the clearest, simplest example. Now, because of Visa’s decision to lower risk thresholds our AI approach to risk is extremely relevant.
As we get closer to one to one selling we’ll continue to face challenges in all of these areas. I look forward to working with each of you to embrace this future. It will require us to be increasingly effective leaders and analysts.
Let’s do it together.