Multivariate Testing with Optimizely

Question

I've just started using Optimizely, and really love it. However, I need to do a multivariate test. Has anyone come up with a way, hack or otherwise, to do MVT in Optimizely? Manually creating 25-100 variations isn't my idea of fun. Alternatively, maybe there's some way to automate variation creation?

Answer 1

EDIT: Optimizely has long since supported multivariate testing (since at least late 2011). Below suggested workarounds are no longer necessary.

Original answer kept for posterity and in case its helpful for other testing platforms that might not yet support MVT.

---

So, I've been dealing with this same problem, and AFAIK there are 2 solutions.

1. Just run separate experiments for each variable, concurrently. In other words, if your two variables are, say, the size of headlines and the language on buttons, just treat those as separate experiments. As long as they don't conflict with each other in terms of the DOM (be sure your selectors are very specific, and try to use IDs instead of something like $j('p').eq(44); since one experiment could change which that refers to for the other.

2. You could generate all of the permutations manually. This does not scale well, but for 2 or 3 variations, it could be ok. I scripted a very simple multivariate generator. I haven't tested it extensively, but, it will allow you to select sections and different number variations for each section, and it will generate all of the combinations of those values. Try it here: http://htmlto.com/optimizely/

Answer 2

I am a new Optimizely user, but it is my impression that it now has a multivariate testing feature. (At very least, they have an option labelled "multivariate".)