- Types Of Tests: Synchronous – Asynchronous, Sequential Testing, A/B/…/N, Testing Multivariate, …
- Testing: You’ll Get Page Conversion Data (Not Element By Element) – Better For Low Traffic Sites
- Testing Some Rules: Isolate Individual Elements If You Don’t Isolate Elements, DO NOT attribute the lift or loss to any single element
- Multivariate Testing You’ll Get Element By Element Data Don’t Run If Less Than 100k Pageviews Per Month
- TESTING HELPS GUIDE DECISIONS AND IMPROVE THE USER EXPERIENCE WHILE BOOSTING PROFITS
- NOT EVERYONE CAN OR SHOULD BE SPLIT TESTING
- NOT BEING ABLE TO TEST DOESN’T MEAN NOT BEING ABLE TO OPTIMIZE
- THERE ARE PLENTY OF WAYS TO BOOST CONVERSIONS OUTSIDE OF SPLIT TESTING
Part 1 Can I Actually Run A Test
- Testing Eligibility
• ~100 Conversion Per Variation
• Run for at least a full week
• Not run longer than 8 weeks
• Test can reach 95% confidence in this time frame
- Problem: Not Enough Conversions
Solution: Run Longer
Secondarily: If Outside Of 8 Week Testing Window – Don’t Test It
- VARIABLE NUMBER ISN’T THE ONLY THING THAT DICTATES TIME!
- THE PERCENT LIFT IS A HUGE TIMING FACTOR
- In order for this test to get statistical results, it would require a 10% lift
- AND SO IS THE CONFIDENCE RATE. But we’ll talk more about that later
- Does Your Page Get Enough Daily Conversions? YES / No / Can’t Test – Sorry!
Is There A Big Enough Lift Potential?* YES / No / This Is A Testable Page / Can’t Test – Sorry!
*With An Appropriate Confidence Rate/Index
Part 2 Should I Actually Run That Test
- THE CONVERSION HIERARCHY
- Functional: Does the page work and do what it’s supposed to?
- Accessible: Can people of all skill levels on any device use the site? What are the barriers on the site?
- Usable: Can people understand your site, i.e., is it user friendly?
- Intuitive: Does the site have a logical flow? Does the offer match the source?
- Persuasive: Is your site convincing people to buy or take an action?
- “For every test you run, there are an infinite number of other tests that you could have run” – Justindamus 120 AD
Part 3 How To Optimize If I Can’t/Shouldn’t Test
- You Have A Few Options
• Evaluate Your Offer
• On-Site Retargeting
• Changes Based On Qualitative Research
- EVALUATE YOUR OFFER
- It’s not a traffic problem, it’s an offer problem
- It’s not a testing problem, it’s an offer problem
- You might have to head back to the drawing board completely. Does your product/service provide real value?
- You also might be going after the wrong traffic source. Identify if your ideal customer is ‘hanging out’ on the traffic medium
- TRY ON SITE RETARGETING!
- What is On Site Retargeting? Evaluating current interactions and providing a relevant offer based on these interactions.
- PAGE CHANGES BASED ON OPTIMIZATION RESEARCH
- Here are a few things to try
• Click/Scroll Tracking
• Usability Testing
• User Surveys
• Session Recordings
- CLICK/SCROLL TRACKING
- People who actually scroll to the bottom click
- Not enough people are scrolling that far down ~15% of all clicks on this page are on the nav bar
- Try a shorter variation to get people to scroll to the CTA. Add multiple CTA buttons at areas where scrolling is highest. Hide the nav bar on this page
- USER SURVEYS
• Was it was easy to find what you were looking for?
• Were the words/vocab used to define categories/sub-categories clear to you?
• Do you have any suggestions to improve our website navigation?
• Does our website look credible to you?
• Is our pricing clear?
• Is there anything else you’d like to know before signing up with us?
• Will you shop with us again? Why/why not?
• Do you think the form has any confusing/unnecessary input fields? QUESTION IDEAS
- SESSION RECORDINGS
- Shows individual recordings of a user interacting with your site. Unlike heatmaps, this isn’t aggregated data
- Really blow their mind with a real life user recording with audio
TRICKS TO TEST WHEN YOU DON’T HAVE THE TRAFFIC
- USE MICRO CONVERSIONS AS INDICATORS
- Test Length Minimum Conversions
- LOWER THE CONFIDENCE THRESHOLD
- SPLIT TESTING IS NOT OPTIMIZATION…
- IT’S AN OPTIMIZATION TOOL
- SPLIT TESTING CAN ACTUALLY HURT A COMPANY
- SO PRACTICE SAFE TESTING WITH A PAGE THAT MATTERS