How can we help?

Overview

A/B tests and multivariate tests allow you to perform tests to determine which variations lead to the highest number of email acquisitions. This article discusses the differences between A/B tests and multivariate tests.

A/B tests

• You can test design or copy variations. This is accomplished by starting with one lightbox and then creating variations.
• Web forms can integrate with your analytics system so that you can see how your AB tests are impacting other metrics such as page views, revenue, etc.
• You can test two variations at a time (i.e. A vs B)

After an A/B test has completed, the better performing lightbox will not automatically be selected for display. Instead, the primary lightbox will be displayed. By default, the original lightbox is the primary. To display a different variation, you can set a lightbox as the primary.

Multivariate tests

• You can test conditions, widget type, and copy/color. For example, you can test exit intent lightbox vs. three second timer slider vs. static inline form. For multivariate tests you need to create different lightboxes, not just a variation.
• Unlike AB testing, you can test an unlimited number of lightboxes.
• Important to note: It might take a long time to get results if the conditions are very different. For example, the three second slider might show up more often than an exit intent lightbox and therefore it will stop displaying the three second timer once it hits the impression target.
• You can pick a traffic percentage. This is helpful when you have something that’s already working. You keep that going, duplicate it, then use that duplicated version in a multivariate test for only 30% of your traffic. This way only a slight portion of your traffic sees the test.
• Web forms can integrate with your analytics system so that you can see how your multivariate tests impact other data points such as pageviews and revenue.

To test dynamic lightboxes against static lightboxes, you can use multivariate tests.