Skip to main content

A/B Testing

A/B testing lets you measure the impact of switching to Marqo by running it side by side with your existing search implementation. You split storefront traffic between two variants - control (your current search) and marqo (Marqo search) - and compare engagement, conversion, and revenue metrics in the Analytics dashboard.

How It Works​

  1. Assign variants: When a user visits your storefront, they are assigned to a variant of the experiment, either control or marqo. The assignment is sticky: the same user gets the same variant on every page load and return visit.
  2. Route search requests: For control users, send queries to your existing search backend as normal. For marqo users, send queries to Marqo.
  3. Compare results: The Pixel tags all events (searches, clicks, add-to-carts, purchases) with the variant. Use the Analytics dashboard to compare metrics and determine which backend delivers better outcomes.

Setup​

Prerequisites​

Step 1: Assign Users to Variants​

The Pixel provides a built-in method that divides users into equal buckets using the most persistent identifier available (user ID > session ID > IP address, falling back to a random assignment). Call it early in your page lifecycle, right after the Pixel loads:

const variant = window.MARQO_PIXEL.chooseVariant(["control", "marqo"]);

This returns "control" or "marqo". The assignment is deterministic: the same user gets the same variant on every page load and return visit, with no extra storage logic needed on your side.

The Pixel automatically tags all subsequent events with the chosen variant.

Optional: Rolling Out to a Subset of Traffic​

By default, every visitor is placed in one of the two variants. If you'd like to start with a smaller slice of traffic — for example, to sanity-check the integration on a small audience before opening it up — you can pass a percentage between 1 and 100:

const variant = window.MARQO_PIXEL.chooseVariant(["control", "marqo"], {
  percentage: 10, // 5% control, 5% marqo, 90% unassigned
});

Users outside the experiment receive undefined and are not tagged on events, so they don't contribute to either group's metrics. As you raise the percentage over time, users who were already in the experiment stay on the variant they were first assigned to.

We recommend running the experiment at 100% (a full 50/50 split) wherever possible. Larger samples reach statistical significance faster, so you can make a confident decision in days rather than weeks. If you need to phase the rollout, ramp up to 50/50 as quickly as your team is comfortable with.

Step 2: Route Search Requests​

Use the variant to decide which search backend handles each request:

const variant = window.MARQO_PIXEL.chooseVariant(["control", "marqo"]);

async function search(query) {
  if (variant === "marqo") {
    // Send the query to Marqo
    const response = await fetch(
      `https://ecom.marqo-ep.ai/api/v1/search?q=${encodeURIComponent(query)}`,
      { headers: { "x-marqo-index-id": `${indexId}` } }
    );
    return response.json();
  }

  // Default: send the query to your existing search backend
  return existingSearch(query);
}

Step 3: Compare Results​

The Analytics API and Analytics dashboard break down all metrics by variant. Compare key indicators between control and marqo:

MetricWhat it tells you
Click-through rateAre users finding results more relevant?
Add-to-cart rateAre results driving stronger purchase intent?
Conversion rateAre more sessions ending in a purchase?
Revenue per sessionIs overall revenue increasing?

Best Practices​

  • Make sure both variants use the Pixel: the Pixel must be loaded for all users, including the control group, so you have complete data for comparison.
  • Avoid other changes during the test: don't redesign the results page or change merchandising rules mid-test, as this makes it harder to attribute differences to the search backend.

Using Your Own A/B Testing Tools​

If you already have your own A/B testing or analytics tools in place, discuss with your Marqo account manager to help you integrate them with Marqo and ensure everything works smoothly.