Hack Google Analytics for Multivariate testing?

Question

I want to implement MVT (multivariate testing) in my site, and wondered if it's possible to use Google Analytics and some server side code to do this, instead of relying on expensive solutions.

Is there a suggested way to do it?

One idea is to have the server-side code randomly draw a tuple with the element alternatives that will appear, and to set one of the GATC codes (e.g. campaign content) to its string representation. For example, the code draws option #2 for the text and #3 for the image, and sets the campaign content to be 'text2_image3'. It's later easy to analyze the results in Excel or other tools.

Will it work? Any better solutions?

Answer 1

https://github.com/danmaz74/ABalytics is a decent starting point if you'd like to write your own JS

Answer 2

If you wanted to go with a homemade solution instead of existing services (e.g. Optimizely, experimently, or convert) I'd suggest the following:

1. Generate the code

   Your server side code generates the recipes (layouts) for the user based on whatever metrics you want . This is commonly just a % of traffic split. Don't forget to include the unaltered version to compare your data to.

2. Give it an identifier

   When your server side code generates the recipes, it should include the a specific value for a custom variable for that page. This unique value should include an identifier to each recipe displayed on the page. E.g. if you have menu A, headline D, and body image G you'd have A:D:G. If you had menu C, headline E, body image I, you'd have C:E:I.

   The type of custom variable depends on your test. Session level would allow you to see how that specific recipe affected the user within a visit. A more standard method would be to set it to a visitor level would allow your recipes to have attribution on returning visits.

3. Run the test & analyze the results

   From there, run the test, gather your data. Once you feel you have statistically significant results, you can stop the test and start to fully analyze your data.

Notes

• The names of the recipes should be unique. For instance, I used A - C for the menu, D - F for the headline, and G - I for the body image. This allows you to create segments based off of that recipe.
• Don't forget to include the control recipe IDs in your test!
• If you're running multiple tests (even if not at the same time), I'd strongly suggest prefixing the custom variable value with the test name / identifier.

This is essentially what Google Experiments does with A/B testing on their backend.

Answer 3

I have found this project, which, while being a bit old seems interesting: https://github.com/jamesyu/cohorts