Multivariate Testing or Multivariable Testing

Multivariate testing or Multivariable testing is a technique for testing hypotheses on complex multi-variable systems, especially used in testing market perceptions.

In internet marketing, multivariate testing is a process by which more than one component of a website may be tested in a live environment. It can be thought of in simple terms as numerous A/B tests performed on one page at the same time. A/B tests are usually performed to determine the better of two content variations, multivariate testing can theoretically test the effectiveness of limitless combinations. The only limits on the number of combinations and the number of variables in a multivariate test are the amount of time it will take to get a statistically valid sample of visitors and computational power.

However, multivariable (or multivariate) testing allows you to test many changes simultaneously – five, ten, or even twenty. You'll still get accurate results, without having to increase your total sample size, and you will be able to identify the impact of each individual change.

Multivariate testing is usually employed in order to ascertain which content or creative variation produces the best improvement in the defined goals of a website, whether that be user registrations or successful completion of a checkout process (that is, conversion rate). Dramatic increases can be seen through testing different copy text, form layouts and even landing page images and background colors. However, not all elements produce the same increase in conversions, and by looking at the results from different tests, it is possible to identify those elements that consistently tend to produce the greatest increase in conversions.

Multivariate testing is currently an area of high growth in internet marketing as it helps website owners to ensure that they are getting the most from the visitors arriving at their site. Areas such as search engine optimization and pay per click advertising bring visitors to a site and have been extensively used by many organizations but multivariate testing allows internet marketeers to ensure that visitors are being shown the right offers, content and layout to convert them to sale, registration or the desired action once they arrive at the website.

There are two principal approaches used to achieve multivariate testing on websites.

being Page Tagging; a process where the website creator inserts Javascript into the site to inject content variants and monitor visitor response. Page tagging typically tracks what a visitor viewed on the website and for how long that visitor remained on the site together with any click or conversion related actions performed. Page tagging usually needs to be done by a technical team and typically cannot be accomplished by a web marketer. Later refinements on this method allow for a single common tag to be deployed across all pages, reducing deployment time and removing the need for re-deployment between tests.

Companies known to employ a tag based method of multivariate testing are: Avenseo, Conversion Works, Adobe, Business Intelligence Group GmbH (B.I.G.), Amadesa, DIVOLUTION, Maxymiser, Google Website Optimizer, Optimost, Vertster and Autonomy Corporation

The second principal approach used does not require page tagging. By establishing a DNS-proxy or hosting within a website's own data center, it is possible to intercept and process all web traffic to and from the site undergoing testing, insert variants and monitor visitor response. In this case, all logic sits server rather than browser-side and after initial DNS changes are made, no further technical involvement is required from the website point of view. SiteSpect is known to employ this method of implementation.

Multivariate testing can also be applied to email body content and mobile web pages.

Multivariable Testing offers two important benefits:

1.Accelerated Learning Curve
A single A/B split test can teach you something important about a key website page. Now imagine how much you can learn, within the same timeframe, when you look at the performance impact of ten or twenty distinct changes.

2.Breakthrough Thinking

When you can test only one change at a time, you are under pressure to think of a "good" change... something you think has a high likelihood of delivering improved results. This can lead to cautious thinking. However, with multivariable testing you can test as many changes as you like. This takes the pressure off and gives you enormous creative latitude, opening the door to breakthrough ideas you might otherwise never have tested.

When to test

If you are wondering if you should be conducting multivariate testing on your website, then you probably should be.

More than any other effort to optimize your site metrics, multivariate testing will increase the effectiveness of your marketing messages, calls to action, and drive higher levels of response within prospects and customers alike.

No matter if your goals are increased product revenue for an eCommerce retailer, increased engagement for a services business or content publisher, or more qualified online prospects for lead generation, multivariate testing with Trinity Insight will help your online business become more profitable online.

Multivariate testing is a fundamental component within any online marketing or eCommerce strategy. Because user attention on the web is fleeting and the opportunity to make a favorable impression is limited, it is paramount that you optimize your website with varied forms of content.

What things to test

So you are ready to take the testing plunge. You have executive buy-in, your team is excited, and you are ready to incorporate multivariate testing within your online business.

Now comes the fun, albeit challenging, aspect of multivariate testing – the process of identifying pages on the website to be tested and conceptualizing testing variations that will help you reach your online goals.

Trinity Insight analysts work with your stakeholders to make this process painless. By incorporating multivariate testing best practices, Trinity Insight is able to effectively launch a testing program with little or no help from your corporate IT department.

As a first step, our analysts first identify high value areas of the site through a detailed review of your web analytics data. This process helps you and your team gain intelligence into the most important pages of the site and how these pages are currently performing.

At this step, Trinity analysts will recommend an initial sub-set of pages and variables for testing. We develop a testing strategy for either a 3 or 6 month duration and provide this documentation to our clients prior to launch of a multivariate testing engagement.

This strategy documentation outlines the core metrics we are looking to optimize (ex. conversion rate), the pages for which we will be testing (ex. furniture category page), the page elements that are being tested (ex. primary promotional spot), and the variables that will be tested (ex. Free Shipping promo/$100.00 off all leather sofas/Free stain protection with all sofas).

No matter if you are an eCommerce store looking to be more sophisticated with your merchandising or a lead generation enterprise looking to extract more value from your visitors, multivariate testing with Trinity Insight will help your business excel online.

How it works

The testing plan is ready. Now it's time to start executing the creative and technology components that define a multivariate testing project.

Trinity Insight manages all of these efforts for you. Our designers provide the marketing and promotional spots that serve as the testing variations. Trinity designers are talented eCommerce veterans that create user experiences that engage and build your brand.

Our technology and marketing analysts handle all of the complex multivariate testing tasks, from page tagging, interface review, and reporting. We follow the test results from start to finish and provide you with the intelligence and recommendations that the test provides.

Simply put, there is no better way to maximize the effectiveness of your website or eCommerce store.

Thanks & Regards ,

Prashant Vadher | QA Engineer


Dennis van der Heijden said...

Hi Prashant, Thanks for the article. I want to invite you to try its a new multivariate testing tool that has behavioral targeting in it and is focused on small medium business. Love to have you get a go at it...

Post a Comment

Design by Prashant Vadher