• Types Of Tests: Synchronous – Asynchronous, Sequential Testing, A/B/…/N, Testing Multivariate, …
  • Testing: You’ll Get Page Conversion Data (Not Element By Element) – Better For Low Traffic Sites
  • Testing Some Rules: Isolate Individual Elements If You Don’t Isolate Elements, DO NOT attribute the lift or loss to any single element
  • Multivariate Testing You’ll Get Element By Element Data Don’t Run If Less Than 100k Pageviews Per Month
  • TESTING HELPS GUIDE DECISIONS AND IMPROVE THE USER EXPERIENCE WHILE BOOSTING PROFITS
  • NOT EVERYONE CAN OR SHOULD BE SPLIT TESTING
  • NOT BEING ABLE TO TEST DOESN’T MEAN NOT BEING ABLE TO OPTIMIZE
  • THERE ARE PLENTY OF WAYS TO BOOST CONVERSIONS OUTSIDE OF SPLIT TESTING

Part 1 Can I Actually Run A Test

  • Testing Eligibility
    • ~100 Conversion Per Variation
    • Run for at least a full week
    • Not run longer than 8 weeks
    • Test can reach 95% confidence in this time frame
  • Problem: Not Enough Conversions
    Solution: Run Longer
    Secondarily: If Outside Of 8 Week Testing Window – Don’t Test It
  • VARIABLE NUMBER ISN’T THE ONLY THING THAT DICTATES TIME!
  • THE PERCENT LIFT IS A HUGE TIMING FACTOR
  • In order for this test to get statistical results, it would require a 10% lift
  • AND SO IS THE CONFIDENCE RATE. But we’ll talk more about that later
  • Does Your Page Get Enough Daily Conversions? YES / No / Can’t Test – Sorry!
    Is There A Big Enough Lift Potential?* YES / No / This Is A Testable Page / Can’t Test – Sorry!
    *With An Appropriate Confidence Rate/Index

Part 2 Should I Actually Run That Test

  • THE CONVERSION HIERARCHY
  • Functional: Does the page work and do what it’s supposed to?
  • Accessible: Can people of all skill levels on any device use the site? What are the barriers on the site?
  • Usable: Can people understand your site, i.e., is it user friendly?
  • Intuitive: Does the site have a logical flow? Does the offer match the source?
  • Persuasive: Is your site convincing people to buy or take an action?
  • “For every test you run, there are an infinite number of other tests that you could have run” – Justindamus 120 AD

Part 3 How To Optimize If I Can’t/Shouldn’t Test

  • You Have A Few Options
    • Evaluate Your Offer
    • On-Site Retargeting
    • Changes Based On Qualitative Research
  • EVALUATE YOUR OFFER
  • It’s not a traffic problem, it’s an offer problem
  • It’s not a testing problem, it’s an offer problem
  • You might have to head back to the drawing board completely. Does your product/service provide real value?
  • You also might be going after the wrong traffic source. Identify if your ideal customer is ‘hanging out’ on the traffic medium
  • TRY ON SITE RETARGETING!
  • What is On Site Retargeting? Evaluating current interactions and providing a relevant offer based on these interactions.
  • PAGE CHANGES BASED ON OPTIMIZATION RESEARCH
  • Here are a few things to try
    • Click/Scroll Tracking
    • Usability Testing
    • User Surveys
    • Session Recordings
  • CLICK/SCROLL TRACKING
  • People who actually scroll to the bottom click
  • Not enough people are scrolling that far down ~15% of all clicks on this page are on the nav bar
  • Try a shorter variation to get people to scroll to the CTA. Add multiple CTA buttons at areas where scrolling is highest. Hide the nav bar on this page
  • USER SURVEYS
    • Was it was easy to find what you were looking for?
    • Were the words/vocab used to define categories/sub-categories clear to you?
    • Do you have any suggestions to improve our website navigation?
    • Does our website look credible to you?
    • Is our pricing clear?
    • Is there anything else you’d like to know before signing up with us?
    • Will you shop with us again? Why/why not?
    • Do you think the form has any confusing/unnecessary input fields? QUESTION IDEAS
  • SESSION RECORDINGS
  • Shows individual recordings of a user interacting with your site. Unlike heatmaps, this isn’t aggregated data
  • Really blow their mind with a real life user recording with audio

TRICKS TO TEST WHEN YOU DON’T HAVE THE TRAFFIC

  • USE MICRO CONVERSIONS AS INDICATORS
  • Test Length Minimum Conversions
  • LOWER THE CONFIDENCE THRESHOLD

 REMEMBER!

  • SPLIT TESTING IS NOT OPTIMIZATION…
  • IT’S AN OPTIMIZATION TOOL
  • SPLIT TESTING CAN ACTUALLY HURT A COMPANY
  • SO PRACTICE SAFE TESTING WITH A PAGE THAT MATTERS