Successful mobile app makers learn what works. And one way they learn is by repeatedly testing and optimizing.
No matter if you have a live app or you’re about to launch, mobile app A/B testing tools can give you the information to retain, engage, and onboard your mobile app users properly.
Ever wondered if your users would buy more in-app purchases if the “Buy” button was blue? or If your users prefer the blue background in your game versus the red background? You need to A/B test to find out.
Now fair warning, a lot of these A/B testing tools have similar features. So choose something that fits your budget with the features you want.
Splitforce (Free – Paid) *Recommended* – Splitforce helps you test everything from your button colors to game physics. I like that Splitforce makes it easy to send out the winning A/B test variation with one click. This makes improving conversions simple and quick. Splitforce also supports localization which is a big deal if you have a localized app (which you should). Splitforce claims it’s easy to install. You just copy and paste the code and in 5 minutes you’re done with setup. Splitforce is available on the iOS, Android, and Unity platform.
Airlock (Free)- Airlock is a free open source A/B testing tool for mobile apps made by Facebook. Airlock seems great in theory but for the life of me I can’t find the direct link to use the thing. If you can find it let me know and I’ll update the article with that information. I also looked around for reviews from others using it and couldn’t really find anything. At least you know this option exists?!
Amazon A/B Testing (Free) – Amazon’s A/B testing tool is essential if you want a cheap mobile backend with your A/B testing tool. Admittedly, Amazon’s mobile app A/B testing tool seems targeted to app makers with lots of users. But I see no reason a small team couldn’t use them as well. In typical Amazon style the UI is sparse but functional. If you don’t mind a plain user interface, try out Amazon’s tool.It’s available for iOS, Android, and of course the Amazon apps. So if you’re looking for Windows and mobile web support, look elsewhere.
Arise (Free – Paid) - Arise comes with all the typical A/B testing features. But with Arise you can test a segment of your users or all of them. I wouldn’t recommend this tool if you don’t have a technical marketing team. Arise is the A/B testing tool for developers in my opinion. There are simple things you can’t do with Arise without coding. Arise is great for teams that think things can be done faster with a few lines of code. Arise is available on iOS, Android, and Phonegap.
You’ve made it this far. Why not subscribe?
Clutch.io (Free) - Clutch.io has the basic features you need in an A/B testing tool. You can track goals and create data-driven tests. Even though Clutch.io hasn’t been updated in about a year, teams still use the tool with good reason. It’s open source so you can customize it for your needs, all for free. If you need a cheap, simple starting point to customize your own mobile app A/B testing tool, go with Clutch.io. Clutch.io is available for iOS and Android only.
Optimimo (Free – Paid) – Optimimo is another A/B testing tool that has a lot of features. Optimimo’s heatmaps, navigation flow tool, and usablity surveys make their tool feature-rich. Optimimo also has advanced filtering features and real-time metrics. Real-time metrics are important to app development teams that need quick improvements in revenue and conversions. Optimimo is available for Android only.
Pathmapp (Paid) – Like many of the other tools Pathmapp helps you make real-time app changes. But you’ll need to code unless your changes can be made with Interface Builder. Either way, this is a tool made for developers, not mobile app product managers and mobile marketers. And like Amazon’s A/B testing tool, Pathmapp allows you to test multiple variants, instead of just two. One thing to remember is that multi-variant testing is only really important to app makers with lots of users. If you try to test too many variants with just a few users, you won’t get the answers you’re looking for. Pathmapp is available only for iOS apps.
Best.ly (Paid) – Bestly is another mobile A/B testing tool with the typical feature set. Bestly has analytics, multi-variate testing, and the ability to make real-time changes. They also have one line copy and paste code install like many of the others. One difference with Bestly is that they have a feature flagging feature. From what I could tell, only one other mobile A/B testing tool on this list has this important feature. Bestly is available for the iOS and Android platforms.
Kii (Free – Paid) – Kii is actually a mobile app developer suite, not just an A/B testing tool. It comes with a mobile app backend, geolocation tools, and mobile app analytics. The good news is Kii has enough features to compete with the other A/B testing tools. They have all of the typical A/B testing tool features plus a one-click feature that lets you push out the optimal version of your app instantly. If you just want A/B testing, pick some other tool. If you think you need all the bells and whistles, go with Kii. Kii is available for iOS, Android, and Unity apps.
You’ve made it this far. Why not subscribe?
Switchboard (Free) - Switchboard is another free and open source mobile A/B testing tool. You can segment users based on UDID and is still usable for users when they are offline. The offline feature is actually very important, since you want users to see updates even when they don’t have an Internet connection. Just like Clutch.io, Switchboard is not updated as frequently as we’d like. But it’s still totally usable and a great starting point if you want to build your own solution.
Hercules (Free) - Hercules is simple and seems to be targeting the non-developer crowd. Hercules has in-browser drag and drop mode that helps non-developers make changes without coding. Hercules is currently in an open beta so join now while it’s free. Hercules is available for the iOS platform only.
Leanplum (Free – Paid) – Leanplum’s A/B testing tool seems like an app marketers dream. It has filters, segments, and a nice dashboard full of every analytics you could possibly need. Leanplum’s interface also seems a bit easier to use. It’s more a WYSIWYG A/B testing tool. This is ideal for teams with mobile product managers and mobile app marketers, because they need quick results.This is difficult to do when you always need the development team involved to make a simple test. The best part about Leanplum is it’s availablity on every platform. Leanplum is available on iOS, Android, Unity, HTML 5, and even has a REST API.
Taplytics (Free – Paid) – Taplytics is an A/B testing tool that lets you make changes live. This is a big deal for mobile developers because many platforms don’t allow for fast iterations. Even though Taplyitcs has a free plan, their higher plans are surprisingly cheap for what you get. Unfortunately, based on the screenshots and video walkthrough it doesn’t seem to be the easiest app to use. But I’ll leave that up to you and your team to decide. Basic installation only requires you to copy and paste one line of code. The Taplytics team has put together some great documentation to help you get started. Taplytics is available for iOS only.
Apptimize (Free – Paid) – Like Taplytics, Apptimize gives you the power to make changes in real-time. So you can make new experiments anytime you want without resubmitting to the App Store. Apptimize is more than A/B testing. You get really powerful feature flagging features that help you roll out new features to certain users. You can even hide the feature for certain user groups during your A/B test. This is really helpful when you’re testing which features users actually want or which features bring in revenue. Now, Apptimize is not as app marketer or product manager friendly as some others, but as they say on their site ‘if you can code it you can test it.’ If you don’t mind coding to do everything, use Apptimize and test your little heart out! Apptimize is available on iOS and Android.
Vessel (Paid)- Vessel is another A/B testing tool that makes it easy to change your app live. But the difference with Vessel is that you get real-time reporting which helps you make decisions quickly. Vessel has extra features like revenue tracking, funnels, and session tracking to make any mobile app marketer or product manager happy. Vessel is available on iOS, Android, Windows, HTML 5, Phonegap, and Titanium. So anyone that wants to A/B test on mobile can with Vessel.
Optimizely – Optimizely is the most “famous” A/B testing tool on the list, but they didn’t have a tool for mobile apps until now. If you’ve ever used Optimizely for your web app, you can imagine how easy their mobile A/B testing tool will be to use. Currently, Optimizely’s A/B testing tool is in closed beta and only looks available for iOS apps. So you have to apply and have certain qualifications to be the first to use it.
Fliptest.io -Fliptest is another mobile A/B testing tool in beta right now. From the little information available on their site, Fliptest helps you make real-time app changes, gives you a user-friendly backend, and can be used by non-developers. Fliptest also won’t require major code changes like many of the others. You’ll only need to copy and paste a line of code to get it working. Fliptest is only available on iOS.
Liquid - Liquid is also in beta. So there isn’t much to talk about yet. But from what I can see they have user segmentation and real-time app updates. Based on the images it looks clean and simple to use. But you have to sign up to really see if this is the tool for you.
VWO – VWO is also in beta. VWO has been a long solution for websites but have now introduce a mobile app A/B testing tool. So far on their site I don’t see exactly which platforms they’re available for. But sign up and be one of the first to test.
Again, the best way to chose an A/B testing tool for your mobile app is to just pick the one that’s simple and easy to use. There are a lot of considerations to keep in mind when choosing an A/B testing mobile app tool:
- How easy is this mobile A/B testing tool to install?
- How quickly can I run an A/B testing experiment in my app?
- How many features can you afford on your budget?
- Which A/B testing tool will provide the most accurate test results?
- How will my A/B testing tool be used by the mobile app marketers and mobile product managers on the team (not necessarily the mobile app developers on the team)?
- How much support will I get (or do I need) to use this A/B testing tool?
We will keep adding to this list. If you feel any of your favorite mobile app A/B testing tools weren’t mentioned in this article, feel free to let me know at [email protected]