Skip to content
  • There are no suggestions because the search field is empty.

Sessions vs. Sessions with Impressions in Theme Tests

In a Theme Test, Sessions and Sessions with Impressions (SWI) may show different numbers, even though the test runs across the storefront and users are assigned to one of the tested themes.

Since Theme Tests run across all storefront pages and users, these metrics should generally show similar results. However, differences can occur, primarily due to checkout activity.

Sessions

Sessions measures sessions associated with the test, including activity that may occur outside the initial storefront experience.

For example, a user may visit the storefront and complete a purchase hours or days later, or start a new session directly on the checkout page.

Sessions with Impressions (SWI)

Sessions with Impressions counts sessions where a user visits the storefront and sees the tested theme.

If the user makes a purchase, it is only counted as a conversion in SWI if the purchase happens during that same session.

Why Do the Numbers Differ?

The main reason is that Theme Tests do not run on checkout pages.

As a result, certain checkout sessions or transactions may be included in Sessions or Users reporting but not in Sessions with Impressions.

Examples include:

  • Direct checkout access: A user opens checkout through a direct link without visiting the storefront during that session.
  • Delayed purchases: A user visits the storefront but completes their purchase hours or days later in a separate session.
  • Existing checkout tabs: A user returns to an open checkout tab and completes a purchase in a later session.

In these scenarios, the activity may be attributed to the broader Sessions or Users report but not to SWI.

Which Metric Should You Use?

  • Sessions with Impressions (SWI): To measure the immediate impact of the tested theme within the same session.
  • Sessions and Users: To measure overall activity and conversions attributed to the test, including those occurring in later sessions.

Having Trouble?

We’ve got your back.
Email us at support@visually.io or reach out on chat for help with setup or troubleshooting.