Decisions based on personal opinions, speculation, or hearsay can often lead to disaster. An objective test can help when you need the best solution. One easy and effective test used in the digital world is the AB test. It is literally a test where a participant selects one of two options. This article will help you formulate a solid plan for your test. There are plenty of fancy tools you can use. Yet without a solid plan, your test will only set you up for failure. Given the cost of some tests, that is a loss you want to avoid.
Identify one problem
After one or two tests, you will become addicted to this. Knowing which decision is better is exhilarating. Finally, you have numbers that say one design is a clear winner. Just be sure to focus your efforts on one thing at a time. Multi-variant testing is another form of testing and is far more complex. Create small projects that test one aspect of the site or a page. Collect your results. Then change that one thing. Keeping it simple allows you to make regular incremental changes to your digital platforms. A clear sign for your leadership to continue investing in your team’s skills.
Pick a method
There are two things you need to consider when testing a design. The first is if you conduct the in-person and remote. The second is to conduct the test on a live or test environment. Given these choices, there are four possible combinations. The scenario I advise for most churches is in-person sessions on a test environment. Recruit a mix of existing members and unchurched friends. Then test using printed designs, on-screen mockups, or prototypes. Testing with remote audiences and live websites is typically more expensive. Plus you do not get any personal connection with the people using your site.
Select a tool
If you search for testing tools, you might be a bit overwhelmed. There are many online testing tools that allow you to test images or full-blown websites. Companies like Amazon have the luxury of testing an idea with a small percentage of their visitors. Most likely you have limited traffic and budgets. Use a script and printed pages to conduct your test. A user can say if they like a layout more. They might find a link more easily. Or they will better understand a navigation piece because it uses less “churchy” words. Regardless, select an online or offline tool that best suits your needs and budget.
Test without bias
The easiest way to skew your results is to influence your questions. This is most easily done when conducting an in-person test. Body language alone can tell if you are happy with an answer. Do not script your questions to lead anyone to a result you want. Remember to seek what options resonate with your audience. Keep your personal preferences out of the mix.
Measure Results
The great thing about these tests is that you will get numbers. Hopefully, you also get some dialog. Ask your testers why they chose one design over another. Remind them that there is no wrong answer, and you will get more active feedback. Record those results with either a webcam or audio recorder. It is often shocking for a designer to hear their decisions were wrong. Make it less about the team’s personal opinions. Continue to focus on the feedback your visitors give you.
Action Item
I know I covered a lot of material here. Do not shy away from AB testing. This could be as easy as showing two designs on a clipboard after Sunday service. Evaluating your website is a great next step to figure out what projects should come first. Do not settle for having a mediocre website. Strive to create the best experiences for your visitors, and your digital presence will certainly grow.
Photo courtesy of Rohan Baumann
