Experiments & attribution: prove a sort lifts sales

A better sort should mean more sales — but “should” isn’t proof. QSortby lets you run two versions side by side and measure what each actually earned, so you ship the version the data supports instead of a hunch. This is the honest answer to “which sort maximizes conversion value”: you don’t guess, you run the experiment.

Two ways QSortby shows lift

• Per collection — Attribution. A sorted collection page is credited like any other surface: the orders it was in the path of show up under Attribution as Collection, and the Sort projects tab in Analytics splits that by the config version that was live at the time. That’s the read for a single collection.
• Best Sellers block & For You feed — Experiments. For the storefront feeds, the Experiments page runs a live A/B test between two versions and reports the funnel with a significance verdict. That’s what the rest of this guide covers.

Run an A/B experiment

Left menu → Experiments → start a new experiment. What you pick depends on the surface, because the two surfaces test different things:

• Best Sellers block — you’re testing one sales window against another. A is the window the block is configured with; B is the control window you nominate.
• For You block — you’re testing one personalize logic against another, head to head. You need two saved logics before this option does anything; the Start button stays disabled until you pick two different ones.

Then set a traffic split — 50/50, 70/30 or 90/10. An even split reaches a verdict fastest. One running experiment per surface.

Visitors are split deterministically: the same shopper always sees the same version for the life of the test, so the comparison stays clean.

Read the funnel

For each version, QSortby tracks what it can honestly see from the storefront:

1. Impressions — cards that came into view. Best Sellers only. QSortby deliberately doesn’t count impressions inside the For You block: feeding “was shown” back into a personalized ranking would make top products rank higher just for having been shown.
2. Click-through rate (CTR) — clicks into a product from the feed, over those impressions.
3. Add-to-cart — the step before purchase. Counted on both surfaces.
4. Influenced revenue — orders attributed back to the feed a shopper engaged with, per version.

Comparing A and B across these tells you not just whether more people clicked, but whether the sort moved shoppers all the way to a purchase.

Let significance call the winner

A version can look ahead by chance when traffic is low. The verdict is computed on CTR — clicks ÷ impressions — because that’s the highest-volume signal both versions stamp. QSortby stays on “not enough data” until each version has at least 300 impressions, then either calls a winner at 95% confidence or reports inconclusive. Ship the variant once it’s a significant winner; keep running if it’s still too close to call.

Attribution: the revenue a sort influenced

Left menu → Attribution shows the orders a QSortby feed helped drive and the influenced revenue behind them, broken down by surface and variant. It’s the money view that sits underneath the experiment funnel.

Below it, a second table breaks the For You and Best Sellers blocks down by position — clicks and add-to-carts per slot, so you can see whether engagement concentrates near the top where it should. That one is a shape check on your sort, not a revenue view: no money is attributed per position, and impressions are left out on purpose, since the For You block doesn’t log them and any rate built on them would read as a misleading 0%.

A simple workflow

1. Pick a feed you want to improve.
2. Try a signal that fits it — see the How QSortby ranks guides — e.g. Best converting or Trending.
3. Run it as an experiment: a window against a window on Best Sellers, a logic against a logic on For You.
4. On Best Sellers, wait for significance, then read CTR → add-to-cart → influenced revenue. On For You, skip straight to clicks → add-to-cart → influenced revenue.
5. Ship the winner; then test the next idea. Small, proven steps compound.