My friend Ilona recently started her PhD research at London School of Economics. Last week she was sitting in her office when she overheard someone in the next room being congratulated…for winning the Nobel Prize in Economics! It turned out Michael Kremer, one of this year's winners, was visiting LSE that day. The first he heard of it was when the head of LSE's economics department found and congratulated him. An acquaintance had tried to congratulate him via Skype earlier in the day, but he thought it was a scam and rejected the call.
The prize was awarded to three dynamic and impressive economists – Esther Duflo, Abhijit Banerjee and Kremer – working on one of the world's most important problems: relieving global poverty. Duflo is only the second woman to receive the award as well as the youngest ever winner. All three awardees are notable for going out into the field and spending their whole careers on an issue that is much harder than what many economists do: staying in their office and working out theoretical insights on a blackboard.
Two of the approaches used by Duflo, Banerjee and Kremer are of particular interest to us in market research.
The first is behavioural economics. Duflo and Banerjee's book Poor Economics outlines some of the behavioural and psychological traps that can keep people in poverty, and how carefully designed nudges can help them escape. If you keep up with behavioural science you'll recognise many of the terms – commitment devices, hyperbolic discounting and anchoring for example.
The second is the RCT or randomized control trial. In an RCT, a new approach or incentive is randomly assigned to 50% of a population, with the other 50% as control group. Afterwards, the outcomes or behaviour of each group are measured. If the test group outperforms the control group, the intervention is judged a success.
This is a great way to test new policies for alleviating poverty in Malawi, but not only that. The same principle is used in A/B testing of your website or email marketing. And at Irrational Agency we have started to use RCTs in market research, most recently to test TV ads on three randomized groups for a US barbecue brand. We measured which one would buy the most product after seeing the ad, allowing us to get a true behaviour measure instead of relying on claimed appeal.
However, sometimes it isn't possible to run a randomized test on a real population. A recent example from a client of ours in the pharmaceutical industry: they wanted to try some new communications tactics, but it would have taken months and cost millions of dollars to get legal approval to release each version and reissue all their marketing collateral. So we designed a new approach: the Simulated RCT.
How a Simulated RCT works
In a simulated RCT, we recruit a sample of consumers and apply the same randomization principle as a real RCT: divide them into different groups and give a different marketing message to each one.
Then we test the behaviour of each group in a simulated shopping test (or in this case, a simulated doctor-patient consultation). The group who most increases purchasing of the test product, or changes their behaviour in the desired way (such as prescribing a particular medicine) is deemed the successful group. The communications message seen by that group is the winner of the test.
Not only can we identify the most effective communications message, but we can determine how much it outperforms the other test messages and the control group.
Other clients who have used this approach include an energy drinks company testing key drivers of consumption, an insurance company measuring the effectiveness of different marketing touchpoints, and a whisky company exploring how to maximise their returns from e-commerce.
Simulated RCTs are not suitable for every type of research, of course. And there's lots to discuss about both RCTs and Simulated RCTs, such as how best to generalise the results to other situations. We designed the Simulated RCT to address these questions as well as possible, for example by measuring not just which stimulus works best, but why it works (using our System 3 analysis method).
How can you benefit?
If you'd like to take some inspiration from the Nobel Prize winners making an impact against global poverty and hunger, why not try a cost-effective and accurate Simulated RCT on your next research question? We're donating 10% of all our profits from Simulated RCTs in the next year to Evidence Action, a charity that uses RCTs to work out the most effective ways to spend its donors' money around the world to save the most lives.
Simulated RCTs accurately predict the future behaviour of your consumers in response to the marketing strategies you will introduce. Helping your stakeholders make reliable, informed decisions based on actual evidence.
Irrational Agency: new insight into the future from behavioural science