A/B Testing for PhoneGap Application

Question

I've tried using Google Mobile Analytics for a phonegap application. I know that Google Analytics supports Context Experiments, but I couldn't find anything about Google Mobile Analytics doing the same.

My project needs to do A/B Testing and I've been looking around if this is possible using PhoneGap since all my assets/css/js will be shipped together in the appstore and will not be obtained via a remote server.

A workaround I'm thinking about is to create 2 different html versions of one screen and ship all of these in one app.

Is this a wise move? If not, is there a better way?

Answer 1

Since you are using Phonegap, you can pretty much use any JS library out there.

Making 2 versions of each HTML file will get cumbersome and a harder to manage. IMO, it is essentially doubling the work for yourself. Instead, I would recommend using a JS A/B testing library. Here are a few:

1. Genetify
2. Optimizely -- paid service
3. Visual Website Optimizer -- another paid service

These will allow you to tweak variables in a Javascript file to reflect changes in your templates.

If your app requires the user to be connected at all times, I believe you might be able to tweak changes remotely without having the user update the app. If you go with the 2 HTML files approach, you will have to ship an update with each update.

Answer 2

Any A/B general web testing library cannot be used for a phonegap app for two main reasons:

1. You cannot use a service like Optimizely to configure your experiment because you app is not online like a website. Optimizely needs to browse the website and load it in its user interface
2. The A/B testing framework needs to handle the case when the device is not connected to the internet

Arise.io (an A/B testing framework for mobile) supports a phonegap version of its framework: https://arise.readthedocs.org/en/latest/phonegap.html

DISCLAMER: I am the founder of Arise.io

Answer 3

I've created a framework called CordovaUniversalAppServer that will do what you want. You would however need to host your assets via a remote server, but the solution does work offline (assets are cached in the app). It basically allows your server to host different versions of your Cordova bundle, and the actual app will check your server for the right assets upon loading, and save these for offline use. If the device is offline when the app opens, the cached assets are used.

So you can have a version 'A' of your app and a version 'B' of your app hosted at the same time via your server. Some devices will run version B, some devices will run version 'A', depending on how they are configured.