A/B Test Product Recommendation Campaigns
Plans: All Plans Platforms: Shopify
AfterShip Personalization’s A/B testing feature allows you to split your website visitors into two groups and display each with a different product recommendation strategy. You can determine which strategy is performing better and which needs improvement by analyzing the key metrics, including conversions and average order value (AOV).
You can A/B test your product recommendation strategy for the home page, product, cart, checkout, thank you, and post-purchase page.
You can enable version A with product recommendations and disable version B without product recommendations to show the merits of product recommendations on various key checkpoint webpages.
You can enable version A and version B to compare two different product recommendation strategies and determine which one is performing better.
In this article, we will show you:
Set up A/B testing for multiple scenarios
Additional considerations
Data accuracy
A/B testing is available for all the supported scenarios in AfterShip Personalization: home page, product, cart, cart popup, checkout, thank you, and post-purchase page.
For every scenario, the setup process is the same. Let’s take a look at the example of the home page.
Go to Smart upsell widgets in the AfterShip Personalization admin.
Click Get started. Select a pre or post-purchase scenario.You can edit an existing campaign (widget) or create a new one. To create a new one, select Create widget.
Product recommendations widget editor will open.
Edit the product recommendations settings based on your requirements. These settings will be used for version A.
Configure settings with ease with detailed guides on each here.
Click Create A/B test on the upper-right hand side.
On the popup dialog, select whether you want to create a copy of version A and edit or create version B from scratch and hit Continue.
Configure the traffic volume for A/B testing. Drag the bar left or right to adjust the display percentage of each version.
By default, the traffic would be distributed equally amongst both versions.
Click Continue.
The editor that will now open will be for version B. Edit the product recommendation settings as per your requirements and click Save.
Switch between both versions by clicking the downward arrow in the upper-right-hand corner of the page.
Enable both versions if you want to show each version to the customers based on your traffic configuration.
You do not have to install the widget for both versions on the Shopify theme editor separately. The editor code for each will remain the same.
The preview window will always be open for the enabled version first. If version A is disabled, a preview for version B will display on the preview page.
After accumulating enough samples, you can check the progress of each version on the product recommendations dashboard and access which one is performing better.
The traffic distribution division is based on each browsing session without a tracking cookie, meaning consumers can only see one version during one browsing session.
The accuracy of traffic distribution is significantly dependent on the traffic volume. Our research shows this accuracy is much higher when the traffic amount exceeds 1000.
Here’s the data indicating how data accuracy enriches with the traffic volume.
For any further questions or help, please contact our chat support team.
Overview
AfterShip Personalization’s A/B testing feature allows you to split your website visitors into two groups and display each with a different product recommendation strategy. You can determine which strategy is performing better and which needs improvement by analyzing the key metrics, including conversions and average order value (AOV).
You can A/B test your product recommendation strategy for the home page, product, cart, checkout, thank you, and post-purchase page.
You can enable version A with product recommendations and disable version B without product recommendations to show the merits of product recommendations on various key checkpoint webpages.
You can enable version A and version B to compare two different product recommendation strategies and determine which one is performing better.
What you’ll learn
In this article, we will show you:
Set up A/B testing for multiple scenarios
Additional considerations
Data accuracy
Set up A/B testing for multiple scenarios
A/B testing is available for all the supported scenarios in AfterShip Personalization: home page, product, cart, cart popup, checkout, thank you, and post-purchase page.
For every scenario, the setup process is the same. Let’s take a look at the example of the home page.
Go to Smart upsell widgets in the AfterShip Personalization admin.
Click Get started. Select a pre or post-purchase scenario.You can edit an existing campaign (widget) or create a new one. To create a new one, select Create widget.
Product recommendations widget editor will open.
Edit the product recommendations settings based on your requirements. These settings will be used for version A.
Configure settings with ease with detailed guides on each here.
Click Create A/B test on the upper-right hand side.
On the popup dialog, select whether you want to create a copy of version A and edit or create version B from scratch and hit Continue.
Configure the traffic volume for A/B testing. Drag the bar left or right to adjust the display percentage of each version.
By default, the traffic would be distributed equally amongst both versions.
Click Continue.
The editor that will now open will be for version B. Edit the product recommendation settings as per your requirements and click Save.
Switch between both versions by clicking the downward arrow in the upper-right-hand corner of the page.
Additional considerations
Enable both versions if you want to show each version to the customers based on your traffic configuration.
You do not have to install the widget for both versions on the Shopify theme editor separately. The editor code for each will remain the same.
The preview window will always be open for the enabled version first. If version A is disabled, a preview for version B will display on the preview page.
After accumulating enough samples, you can check the progress of each version on the product recommendations dashboard and access which one is performing better.
Data accuracy
The traffic distribution division is based on each browsing session without a tracking cookie, meaning consumers can only see one version during one browsing session.
The accuracy of traffic distribution is significantly dependent on the traffic volume. Our research shows this accuracy is much higher when the traffic amount exceeds 1000.
Here’s the data indicating how data accuracy enriches with the traffic volume.
| Traffic amount | Accuracy (1:1) | |
|---|---|---|
| Version A | Version B | |
| 100 | 60% | 40% |
| 1000 | 53% | 47% |
| 10000 | 51% | 49% |
| 100000 | 50% | 50% |
For any further questions or help, please contact our chat support team.
Updated on: 13/06/2024
Thank you!