There is a very tempting way to use AI when something goes wrong in an ecommerce business:
“My conversion rate dropped. Analyze my data and tell me why.”
It sounds reasonable.
Give the model the numbers. Let it look at the funnel. Maybe upload some customer reviews, analytics exports, or screenshots.
Then wait for the answer.
The problem is that this often produces something that looks like analysis but isn't really a diagnosis.
You get a list:
- Maybe traffic quality changed.
- Maybe the product page isn't convincing enough.
- Maybe pricing is too high.
- Maybe mobile UX is causing friction.
- Maybe there is a technical issue.
- Maybe customers don't trust the store.
- Maybe the offer isn't competitive.
- Maybe checkout is broken.
- Maybe seasonality is involved.
Technically, none of these explanations may be wrong.
But a list of possible explanations is not the same thing as finding an explanation.
Good diagnosis gets narrower.
And this is where I think AI is being used incorrectly in ecommerce analysis.
AI Is Very Good at Looking for Evidence
Large language models are surprisingly useful when the problem is not simply calculating a metric, but examining a large amount of messy evidence.
Customer reviews.
Support conversations.
Survey responses.
Reddit discussions.
Ad comments.
Search queries.
Product feedback.
Analytics exports.
Session notes.
These sources contain context that doesn't fit neatly into a dashboard.
AI can help you find recurring patterns, contradictions, unusual cases, and relationships that would take a human much longer to identify.
But there is an important distinction:
Finding evidence is not the same as deciding what the evidence means.
That distinction becomes especially important when diagnosing an ecommerce problem.
Start With a Question, Not an AI Prompt
Suppose your store's conversion rate falls from 2.1% to 1.4%.
The lazy way to use AI is:
“Why did conversion fall?”
That's an extremely broad question.
The model has almost unlimited possible explanations available to it.
Instead, you might begin with something more specific:
“Traffic volume is roughly unchanged, but conversion fell. Did the composition of traffic change?”
Now the investigation has a direction.
You can ask AI to compare:
- new vs. returning visitors
- organic vs. paid traffic
- campaign groups
- geographic sources
- device types
- landing pages
- high-intent vs. low-intent entry points
Maybe the data shows that total traffic barely changed, but the proportion of returning visitors fell significantly.
That doesn't prove the cause.
But it changes the question.
Now you can investigate why the traffic mix changed.
This is what I mean by hypothesis-driven diagnosis.
You don't ask AI to explain the entire business.
You ask it to investigate one plausible explanation.
The Difference Between “What Happened?” and “Why?”
This sounds like a small distinction, but it changes how you analyze data.
Imagine you see:
Conversion rate: 2.3% → 1.5%
That's an observation.
You might then discover:
Organic traffic: down 30%
That's another observation.
Then:
Paid traffic: up 28%
Another observation.
Then perhaps:
Paid traffic has a lower conversion rate than organic traffic.
Now you have a possible explanation for the overall decline.
But even that isn't necessarily the final answer.
Why did paid traffic increase?
Was there a campaign change?
Did the targeting change?
Did the creative change?
Did the landing page change?
Did the audience become broader?
Did the product itself become less relevant to the people being acquired?
The investigation keeps getting narrower.
This is much more useful than asking an AI model to jump directly from:
“Conversion rate fell.”
to:
“Here are 15 reasons why.”
Ecommerce Data Often Needs a Second Source
Another reason I don't like asking AI for a diagnosis from one dataset is that ecommerce data is full of proxies.
A conversion rate is a proxy.
Traffic is a proxy.
Add-to-cart rate is a proxy.
Bounce rate is a proxy.
Revenue is a business outcome, but even revenue doesn't tell you exactly what happened inside the business.
The underlying customer behavior is much more complicated.
This is why I prefer cross-validation.
Suppose you suspect that customers became less convinced about the product.
You could ask AI to investigate customer evidence:
- What objections appear more frequently?
- Are customers asking different questions?
- Are price concerns increasing?
- Are customers confused about the product's use?
- Are they comparing alternatives more often?
- Are expectations changing?
Then you can compare that with behavioral data.
Perhaps product-page engagement changed.
Perhaps add-to-cart declined.
Perhaps checkout behavior stayed normal once customers reached the cart.
Now the two sources are telling you something together.
Neither source necessarily proves the diagnosis.
But the explanation becomes more credible when independent evidence points in the same direction.
This Is Where AI Becomes Much More Interesting
Most discussions about AI ecommerce analysis focus on speed.
Upload the CSV.
Ask a question.
Get a chart.
Generate a summary.
That's useful, but it isn't the interesting part to me.
The more interesting capability is that AI can help connect different types of evidence.
A human might see:
“Conversion rate is down.”
Then look at Google Analytics.
Then look at Shopify.
Then look at a few reviews.
Then maybe check some Reddit threads.
Then read customer support messages.
Each source is usually examined separately.
AI can help you work across these sources.
For example:
“I'm investigating whether customers have become more hesitant about the product's value. Compare recent customer conversations with the previous period. Look specifically for changes in price-related objections, perceived usefulness, comparisons with alternatives, and language suggesting higher expectations. Don't give me recommendations yet. Give me the evidence that supports or contradicts this hypothesis.”
That is a very different task from:
“Analyze my customer feedback.”
The second prompt asks for a summary.
The first asks for an investigation.
That's an important difference.
AI Should Be Allowed to Disagree With Your Hypothesis
There is another part of this process that matters.
The purpose of a hypothesis is not to prove yourself right.
It is to make the investigation more focused.
Suppose you believe:
“Customers think the product is too expensive.”
You ask AI to examine recent customer conversations.
Instead, it finds that customers mention the price frequently, but most of those customers still purchase.
That's valuable.
Because “expensive” does not necessarily mean “unwilling to buy.”
Someone might say:
“It's more expensive than I expected, but I think it's worth it.”
That is very different from:
“It's too expensive, so I'm not buying it.”
The word expensive alone doesn't tell you which situation you're dealing with.
Context does.
And this is one reason simple sentiment analysis is often insufficient for ecommerce diagnosis.
A negative-sounding sentence can coexist with a purchase.
A positive-sounding sentence can coexist with abandonment.
The important question isn't simply:
“Is this comment positive or negative?”
It is:
“What does this statement tell us about the customer's decision?”
Don't Give AI the Entire Store and Ask for Answers
There is a tendency to think that more data automatically produces better AI analysis.
I'm not convinced.
If you give an AI model:
- every analytics report
- every customer review
- every support ticket
- every ad
- every product page
- every campaign
- every social comment
and ask:
“Tell me what's wrong.”
you haven't necessarily created a better investigation.
You've created a much larger search space.
The model can now produce an impressive-looking report containing dozens of observations.
But you still have the original problem:
Which one matters?
This is why I prefer a narrower workflow.
1. Observe
Something changed.
2. Form a hypothesis
Choose a small number of plausible explanations.
3. Ask AI to investigate
Give it the evidence relevant to that hypothesis.
4. Look for supporting and contradictory evidence
Don't only ask AI to confirm what you already believe.
5. Cross-check with another source
Customer evidence, behavioral data, technical data, or another independent signal.
6. Make the business judgment
This part belongs to the human.
The result should be a narrower explanation with evidence behind it.
Not a longer list.
The Human Still Has to Decide What Matters
This is probably the most important limitation of AI-assisted diagnosis.
AI can find patterns.
It can compare periods.
It can cluster language.
It can identify anomalies.
It can connect evidence.
It can suggest relationships worth investigating.
But the business decision is still yours.
You know things that may not exist in the dataset.
You know that a major product change happened three weeks ago.
You know that a supplier changed.
You know that the company deliberately started targeting a new audience.
You know that a promotion ended.
You know that a competitor launched something important.
You know that a particular customer segment is strategically important even though it represents only a small percentage of revenue.
That context changes the interpretation of the numbers.
So I don't see AI replacing the person doing ecommerce analysis.
I see it changing what that person can investigate.
The Real Advantage for Smaller Ecommerce Businesses
Large ecommerce companies can afford analysts, researchers, data scientists, CRO specialists, and customer research teams.
A small store usually cannot.
That creates an interesting asymmetry.
The small business may have less structured data, but it can still have enormous amounts of unstructured customer evidence.
Hundreds of reviews.
Thousands of comments.
Customer emails.
Support conversations.
Social discussions.
Product questions.
Competitor comparisons.
The problem is that nobody has time to read all of it carefully.
This is one area where AI can genuinely change the economics of analysis.
Not because AI magically knows why customers behave the way they do.
But because it can make previously impractical investigation possible.
A small merchant can ask much more specific questions of their customer evidence than they could before.
And that's potentially much more valuable than another dashboard.
Don't Ask AI for the Diagnosis
The next time something changes in your ecommerce business, try a different approach.
Don't start with:
“AI, tell me what's wrong.”
Start with:
“Here is what changed. Here are the explanations I currently consider plausible. Investigate this one first. Show me the evidence that supports it, the evidence that contradicts it, and what additional evidence would help distinguish it from the alternatives.”
That changes AI's role.
It is no longer pretending to be the person who knows the answer.
It becomes an investigation partner.
And I think that's a much more useful way to think about AI for ecommerce diagnosis.
The goal isn't to get AI to produce the smartest-sounding explanation.
The goal is to use AI to move from:
something changed → possible explanations → evidence → narrower explanation → human judgment.
That's what diagnosis should look like.
And importantly, the final answer should get smaller as the evidence gets better.