“Having a ‘test and learn’ culture” is something that we hear often from charities when we ask them about their plans around digital transformation. However, many would admit that despite having this as a goal, they have not managed to get this in place (or even make a start on doing so).
So what is a test and learn culture, what’s needed for success and what can you do to embed one into your charity?
What is a ‘test and learn’ culture?
Test and learn is an approach that involves testing ideas in a small-scale way – for example, with small budgets and/or small groups of people – to predict impact and make learnings, before committing more money and resources.
Test and learn cultures are created when this approach is adopted within an organisation and becomes central to the way in which everything is understood, planned and implemented. This can include areas related to people and processes or marketing and sales.
There are three key components that are typical indicators of a test and learn culture:
- It has been recognised that measuring things in a scientific way is beneficial
- People embrace failure and refine rapidly
- Innovation and collaboration are encouraged
Test and learn can be applied to several different areas in organisations but work especially well when it comes to digital marketing and fundraising activity. This makes it perfect for charities who are looking to transform how they engage supporters and improve their conversions.
Why is having one important?
As the name suggests, a test and learn culture is centred around the idea of testing your ideas or hypothesis, as opposed to simply rolling them out. So why should you test things?
- Even if you are confident something is going to work, testing allows you to try to understand and forecast how much of an improvement can be made.
- Not all charities are the same. Therefore, you need to test what works with your supporters and audiences, as this could be different to what others have found.
- You can learn from smaller test spends on activity before committing to larger budgets and resources – this is critical for charities (regardless of size), as it means you are maximising the budgets.
- It provides people with the chance to fail and not be criticised, instead learnings are made from anything that does not go to plan. This can naturally make people more adventurous and innovative.
- It provides the evidence and data to help make decisions, reduce internal debates and remove long meetings with opiniated stakeholders.
- It promotes teamworking and collaboration within a culture. People from different teams and departments work together more – devising ideas, testing, measuring and refining.
Test and learn in action
Diabetes UK, who we have been working with for around three years now, are a great example of a charity that is dedicated to embedding a test and learn culture. We’ve helped them optimise their website for conversions and to help improve the user experience. Over time we have collaborated with their teams to run user research and A/B tests on their website, to learn more about their audience, and what matters to them.
Andrew Russell, Head of Individual Giving said:
“We wanted to adopt more of a test and learn approach, because we knew that it could be used (among other things) to help improve the performance of our website – a key initiative for Diabetes UK. As a result of this, we have not only seen it produce several key performance results, but we have also better developed our own internal understanding of how it can benefit the charity. Without it, we would have undoubtedly spent more time and money, without seeing the same levels of return.”
In 2021 we worked closely with Diabetes UK to improve how their membership offering was presented on the website. Our initial analysis of the membership offering showed there was room for improvement in how it was presented to users. Through design workshops and close collaboration, we created an alternative membership page to A/B test against the existing page.
The new design was simpler than the existing page and did a better job of calling out the Membership benefits. Having gained insight from previous tests and user research it meant that we were able to better match the page layout and content to user needs.
The A/B test proved to be successful, with a 21% increase in members signing up via the page. The improved conversion rate also means that the charity can spend more money on advertising membership, with the confidence that they will get a good ROI.
Not only does the increase in members mean an increase in ongoing revenue for the charity, it also means that they are able to help more people living with Diabetes in the UK. And there’s been the added benefit of them being able to use the learnings to inform their future page design for key fundraising pages and other primary pages across the website.
Eight ways to create a test and learn culture in your charity
In our experience of working with charities (and organisations in other sectors), we’ve created this set of essential things you need to have in place:
- Give people the space and time to innovate
- Embrace the right type of failure
- Question opinions and the source of evidence
- Peer review ideas and evidence
- Frame any proposed change as a “test”
- Have the right people, with the right resource solutions
- Test and measure…always!
- Create a feedback loop.
It might well be the case your charity does some of these things already without realising it. Try taking each of these in turn, then assessing what level of them you have in place right now, then start to work out how you can improve in the areas you need to.
Want to learn more about digital transformation? Take a look at these other blogs:
- What is ‘digital transformation’ anyway?
- Is digital transformation now the key to sustainable fundraising?
Banner Image: Francesco Liotti on Unsplash