All posts
AI & automationOctober 11, 20267 min read

AI personalization: do you have enough data yet?

Recommendations, segments and dynamic copy all have a data threshold. Here is the arithmetic for yours, and what to do until you clear it.

The short version

  • Detecting a 2.0% to 2.4% conversion improvement takes roughly 21,000 sessions per variant, so most small stores cannot verify whether personalization worked.
  • Catalog size makes recommendation data sparser, not richer: 60 products means 1,770 possible pairs that a few hundred orders cannot fill.
  • Segment email works far earlier than onsite personalization, and is where the returns are for a small store.
  • A vendor dashboard reporting 'influenced revenue' is counting click-throughs, not lift. Ask for a holdout.

Three products wearing one name

Type ai personalization ecommerce into a search box and every result promises the same thing: each shopper sees the store that converts them. Behind the phrase sit three different products with three different data requirements, and conflating them is why so many small stores buy something that does nothing.

Recommendations decide what to show — rails like "customers also bought", "you might like", "complete the look".

Segmentation decides who to message — lapsed buyers, one-time buyers, high-value customers, cart abandoners.

Dynamic content decides what to say — swapping a headline, a hero image, a currency, a shipping promise based on who is looking.

They become available at wildly different points in a store's life. One of them works on day one. One of them needs a scale most stores do not have and will not have this year. Knowing which is which is the whole decision.

The number that decides most of it

Personalization is a claim that a tailored experience beats a sensible default. To act on that claim you have to be able to tell the difference, and telling the difference takes events.

Here is the standard calculation, which you can redo in any free sample-size calculator. Suppose your store converts at 2.0% and you want to detect an improvement to 2.4% — a relative lift of twenty percent, which would be a genuinely good result. At 95% confidence and 80% power, you need roughly 21,000 sessions per variant. Call it 42,000 sessions to answer one question.

At 3,000 sessions a month, that is more than a year of traffic for a single test. The season will have changed, your product mix will have changed, and the result will be unusable.

This is not an argument against personalization. It is an argument against believing your personalization, which is a different and much more expensive mistake. Below a few tens of thousands of monthly sessions you are not running experiments, you are making design decisions. Make them on judgment, cheaply, and stop paying for tools that imply measurement you cannot do.

The second number: how sparse your data really is

Recommendations have their own threshold, and it is easy to compute for your own store.

Say you carry 60 products. The number of possible product pairs is 60 × 59 ÷ 2 = 1,770. Now say you have had 500 orders and the average basket is 1.4 items, so roughly 200 orders contained two or more products. Those 200 orders can contribute at most a few hundred observed pairs, spread across 1,770 possible ones.

Which means most pairs have never been seen together, and the pairs that have were seen once. A co-purchase model built on that returns your three bestsellers for every product, dressed up as a recommendation. Shoppers notice. It reads as a store that is not paying attention.

Collaborative filtering — the "people like you also bought" family — is worse, because it needs density across both customers and products at once.

What each technique actually needs

TechniqueWhat it needs before it beats a defaultWhat to do until then
Recently viewedNothing. Works on day oneJust turn it on; it is the best value in personalization
Bestsellers and trendingA few dozen ordersHard baseline to beat; keep it even after you have more
Hand-picked cross-sellsYour own product knowledgeTwo or three per product, chosen by you, outperform a sparse model
"Customers also bought"Thousands of multi-item orders relative to catalog sizeUse the hand-picked list
Collaborative filteringDense history across many customers and productsCategory and attribute filters give shoppers the same control
Geo and currency adaptationThe visitor's country, which you already haveRule-based, works immediately, genuinely raises conversion
Segment email and messagingA few hundred customers with order historyThree segments beat twelve: one-time, repeat, lapsed
Onsite dynamic copy testsTens of thousands of sessions per variantWrite one good version and move on
Predicted lifetime value or churnA year or more of repeat historyCount orders per customer in a spreadsheet
Personalized pricingDo notIt damages trust, screenshots travel, and disclosure rules are tightening

The pattern in that table is the useful part. The techniques that need the least data are also the ones that produce most of the available gain, and the ones vendors demo hardest are the ones that need the most.

The other three things that decide it

Does the shopper come back? Personalization needs to recognize someone across sessions. If most of your orders are first-time guest checkouts on mobile, there is no profile to personalize against. Count the share of orders from returning customers first. Under about a fifth, skip onsite personalization entirely and put the effort into the post-purchase email sequence, where you do have identity.

Does your catalog have substitutes and complements? A store with 25 products in one category has almost nothing to recommend — everything is either the same thing or irrelevant. A store with accessories, consumables, refills, or multi-item use cases has real structure to exploit, and even a simple rule set works well.

Can you see the same person twice? Logged-out traffic, consent banners, multiple devices, and in-app browsers all break identity. Before buying anything, check how much of your traffic is even identifiable. The answer is usually lower than the dashboard implies.

The measurement trap

Every recommendation tool ships with a dashboard reporting the revenue it influenced. Read how that number is built before you renew anything.

Almost always it counts sessions that touched the widget and then converted. That is not lift — it is a tally of purchases that would mostly have happened anyway, because the people clicking a recommendation rail are the people who were already shopping. A tool measuring its own contribution with click attribution will never report a bad number.

The only honest measure is a holdout: a share of traffic that never sees the feature, compared over the same period. If a vendor cannot run a holdout, treat their reported figures as marketing rather than evidence.

When the obvious answer is wrong

"We're too small for any of this." Mostly true for onsite recommendations, mostly false for email. Segmenting a few hundred customers into one-time, repeat, and lapsed, and writing three different messages, is available far earlier than anything else here and is where the returns are. AI for email marketing covers that side.

"We have a big catalog, so recommendations will work." Catalog size makes sparsity worse, not better. More products means more pairs to observe with the same number of orders. Big catalog plus modest traffic is the worst case for a model and the best case for curation.

"We have lots of traffic, so we're ready." Traffic is necessary and not sufficient. A single-product store with a hundred thousand monthly sessions has nothing to recommend. What it has is a landing-page test it can finally run properly.

"Our AOV is high, so the upside is big." High-value, low-volume stores — furniture, commissions, bespoke work — should not automate this at all. At twenty orders a month, a personal email from a human about the exact thing the customer was looking at beats any model, and it is one of the few advantages a small business genuinely has.

"Personalization is a marketing feature." The personalization that pays at small scale is operational: showing accurate stock, the right currency, a real delivery estimate for their address, and the payment methods used in their country. None of it involves a model, and all of it is noticed.

Where mAutomate fits, honestly

Two things are worth stating plainly rather than implying.

mAutomate does not include a product recommendation engine, onsite content testing, or predicted lifetime value. If you are shopping specifically for a recommendation widget, that is not what this is.

What it does include is the layer above: the AI Marketing Suite plans, generates, schedules, and publishes campaigns across social, ads, and SEO with performance in one place, and Business Operations covers inventory, sales, and analytics. One genuinely useful piece of adaptation happens at signup — the country you sign up from sets your currency and selling region, so prices appear in the money your customers actually use.

And as with everything else, the AI drafts and you approve. A campaign aimed at the wrong segment is a mistake a model can make silently; it cannot make it silently here.

The short version

If you have under roughly 20,000 monthly sessions, personalization for you means: recently viewed, bestsellers, two hand-picked cross-sells per product, correct currency, and three email segments. That stack takes an afternoon, costs almost nothing, and is close to the ceiling of what your data can support.

Revisit the rest when your traffic can tell you whether it worked.

Questions people ask

How much traffic do I need before personalization is measurable?

Detecting a conversion improvement from 2.0% to 2.4% at standard confidence and power takes roughly 21,000 sessions per variant, so about 42,000 sessions to answer one question. Below a few tens of thousands of sessions a month you can still make design choices, but you cannot verify that any of them helped.

Why do product recommendations look wrong on a small store?

With 60 products there are 1,770 possible product pairs, and a few hundred multi-item orders cannot populate them. The model falls back on whatever is generally popular, so every product shows the same three bestsellers. Two or three cross-sells you pick yourself beat that until order volume rises substantially.

What personalization actually works on day one?

A recently-viewed rail, a bestsellers row, two cross-sells chosen by you for each product, prices in the visitor's currency, and three email segments covering one-time, repeat and lapsed customers. None of it needs a model, and none of it needs data you do not already hold.

Does mAutomate include a product recommendation engine?

It does not. There is no recommendation widget, onsite content testing, or predicted lifetime value in it. What it covers is the layer above: generating, scheduling and publishing campaigns across social, ads and SEO, plus inventory, sales and analytics — with your approval before anything goes live.

  • Personalization
  • Analytics
  • AI & automation

Ready to start your store?

Let the AI build, market, and run it — you ship the product.

Get started free