What is Google Optimize?
Google Optimize is a new tool that allows you to easily build A/B and Multivariate tests with an inbuilt page editor. It uses Analytics to gather data and show results on specific goals like bounce rates, session views and more.
A/B testing lets you test multiple versions of the same web page and find the one that works best for your users. In A/B testing, Variant A is the original and Variants B through N each contain at least one element that is different from the original. A multivariate test allows you to test two or more elements on a page simultaneously to see which combination gives the best outcome.
Why use Google Optimize?
- Google Optimize allows A/B testing which previously was very difficult to set up in Google Analytics, and it also allows Multivariate testing eg. A1/A2, B1/B2, C1/C2
- You can measure tests against multiple metrics simultaneously
- It can remove the need for web developer time
- It's free to use, unlike commercial alternatives like Optimizely which do roughly the same thing
- It allows you to easily create new variants using the Visual Editor without worrying about code
- Targeting allows the user to determine the percentage of visitors that take part in the experiment and assign a percentage to each variant
- Integrates well with Google Analytics to provide accurate data
The Purpose of this Blog Post
Details of running experiment
My role within Zodiac Media is Web Design - working with wireframes, mock ups, creating high fidelity designs and many other brand assets for our clients. In the past I have done user testing face-to-face with just a pen, paper, and some images, so using Optimize is a new experience for me.
The main goal is to test Optimize from a non-developer perspective to find out whether it truly is a tool that’s usable without web development knowledge.
What we plan on testing
There are many possibilities for a test – you could change the colour of a Call-To-Action (CTA – an action you want a user to complete on the site, like clicking a link or submitting a web form), change the homepage banner image, change the text of a button, simply move an element from one place to another, and much more. We planned on testing the banner region of our website, focusing on placement of a CTA and its purpose.
Original, Variant B, Variant C (from left to right)
Attempt 1
After deciding what to test I began creating the A/B test, which was fairly straight forward to do. Within the Create Experiment panel, you can name the experiment, add a test page URL, and choose the type of experiment (A/B, Multivariate or Redirect). After creating a new experiment you are able to set objectives to help determine how your variants perform against your original.
Aside from configuring your experiment objectives, you are also able to choose the percentage of visitors to target at 100%, meaning that every visitor would be included in this experiment. Our test consisted of an original and 2 variants, so we split the weighting evenly at 33.3% each.
After a couple days we noticed a responsive style issue where viewing the page on smaller devices would cause overlapping elements, resulting in a broken layout, so we aborted this experiment and tried again.
Attempt 2
For our second attempt, we put some development time into adjusting the existing styles of the variants to hopefully fix the layout issues. We kept the same experiment settings.
Unfortunately, we aborted a second time due to a recurrence of the responsive issues we were having with the first attempt.
Attempt 3
With the third attempt, more development time was spent on changing the style of the header from ‘fixed height’ to allow the document to ‘flow’ and resolve the issue with smaller devices.
Results
After running for one month, the results were as follows:
Experiment Results
As you can see, both CTA variants outperformed the original, with the Get In Touch variant producing the most conversions.
Conclusion
Optimize looks slick and is free to use so is a potentially great tool for website owners to utilise. It is not as easy as it seems though, as some development time is needed to avoid responsive gotchas, and tests have to be small in scale to minimise risk.
If you have a website that is not responsive and your focus is primarily on a desktop layout, then testing with Optimize can be easy for someone with no web design knowledge. If your website is responsive then you may need some assistance from a web developer to ensure that the variants maintain a good structure. This is because any changes that affect the layout can cause issues when viewed on other devices, and Optimize will not allow the editing of individual resolutions.
Setting up a test is easy, the options are clearly labelled and the Visual Editor does a good job in providing enough options to create your wireframes.
It integrates very well with Google Analytics, which means you can import your objectives over, but for Optimize to give a fair and true result, you need to give enough time for each variant to receive enough visits.