One of the most frustrating things about ecommerce analytics is opening three different dashboards and finding three different answers.
Shopify says one thing.
GA4 says something else.
Meta says something else again.
Google Ads has its own version.
Then someone asks:
“Which number is correct?”
I think that's often the wrong first question.
Because these systems are not necessarily measuring the same thing.
And if you treat them as if they are, you can end up trying to fix a data discrepancy that isn't actually a business problem.
Worse, you may make a business decision based on whichever number happens to support the story you already believe.
That's much more dangerous.
Different Numbers Don't Automatically Mean Bad Data
Imagine a store has:
- 10,000 sessions in one analytics system
- 8,700 sessions in another
- 9,400 attributed sessions in an ad platform
- 120 orders in Shopify
At first glance, this looks broken.
Maybe the tracking is wrong.
Maybe the analytics setup is wrong.
Maybe the ad platform is inflating conversions.
Maybe Shopify is undercounting something.
There are certainly situations where tracking is broken.
But before concluding that, you need to understand what each system is actually trying to measure.
A platform may have a different definition of:
- session
- user
- conversion
- attribution
- purchase
- reporting window
- traffic source
- timezone
It may also use different rules for cookies, consent, ad blockers, attribution windows, or event processing.
So the fact that two systems disagree doesn't tell you which system is wrong.
It tells you that something about the measurement systems is different.
That's the beginning of an investigation, not the end.
The Dangerous Question Is: “Which Dashboard Should I Trust?”
This question sounds practical.
It isn't always.
Suppose Shopify reports 100 orders and your advertising platform reports 130 purchases.
You could immediately say:
“The ad platform is wrong. Shopify is the source of truth.”
Sometimes that will be a reasonable conclusion.
But if you're investigating advertising performance, Shopify may not answer the same question that the ad platform is trying to answer.
The ad platform might be asking:
“How many conversions can be attributed to this advertising system under our attribution rules?”
Shopify might be asking:
“How many completed orders occurred in this store?”
Those are different questions.
Both numbers can be internally consistent while still being different.
The real question is:
What decision are you trying to make?
If you're deciding whether an order actually happened, your order system is obviously more relevant.
If you're evaluating how an advertising platform attributes conversions, its attribution data has a different purpose.
The mistake is treating both numbers as interchangeable.
Measurement Is Not the Business
This is something I think ecommerce operators sometimes forget.
The dashboard is not the business.
The business is what actually happened with customers, products, money, and operations.
Analytics systems are ways of observing parts of that business.
They're useful precisely because the business itself is difficult to observe directly.
But every measurement system introduces some kind of boundary.
You don't see the customer directly.
You see a recorded event.
You don't see “intent.”
You see actions that may be related to intent.
You don't see “trust.”
You see behaviors that might indicate increasing or decreasing trust.
You don't see “customer quality.”
You see patterns in the customers who eventually convert.
This distinction becomes particularly important when data sources disagree.
You are not choosing between competing versions of reality.
You're comparing different observations of reality.
Start by Asking What Changed
Suppose your store suddenly appears to have a conversion problem.
Shopify says conversion fell from 2.0% to 1.4%.
GA4 shows a much smaller decline.
Meta reports that purchase volume is almost unchanged.
Which number do you use?
Don't choose one yet.
First ask:
What changed in the underlying business?
For example:
- Did actual orders decline?
- Did revenue decline?
- Did average order value change?
- Did the number of visitors change?
- Did traffic sources change?
- Did the customer mix change?
- Did the tracking implementation change?
- Did consent behavior change?
- Did the checkout process change?
- Did attribution settings change?
The answer to those questions determines what the discrepancy means.
If actual orders are stable but reported conversion rates changed dramatically, you may have a measurement problem.
If actual orders declined too, then you have a business problem that still needs diagnosis.
Those are very different situations.
Don't Let a Data Disagreement Become the Diagnosis
This happens surprisingly easily.
A merchant notices:
“GA4 and Shopify don't match.”
Then spends hours trying to make them match.
Eventually the dashboards look cleaner.
But the original business problem hasn't been solved.
Imagine actual sales dropped 30%.
You discover that Shopify and GA4 disagree by 15%.
You fix the tracking discrepancy.
Now they are only 5% apart.
Great.
But sales are still down 30%.
The tracking issue was real.
It just wasn't necessarily the reason the business declined.
This is an important diagnostic distinction:
A measurement problem can exist at the same time as a business problem.
One does not automatically explain the other.
This Is Where Cross-Validation Matters
When I investigate an ecommerce problem, I don't want one metric to convince me.
I want different pieces of evidence to point in the same direction.
Suppose orders really declined.
You might then investigate:
Behavioral evidence
Did fewer visitors reach product pages?
Did add-to-cart behavior change?
Did checkout initiation change?
Customer evidence
Are customers expressing new objections?
Are they asking different questions?
Are they complaining about something that wasn't previously important?
Technical evidence
Did the site, checkout, payment system, theme, or tracking change?
Traffic evidence
Did the source or composition of visitors change?
Business evidence
Did pricing, inventory, shipping, promotion, product availability, or positioning change?
You don't need every possible dataset.
You need the datasets that can distinguish between your leading hypotheses.
That's a much more efficient way to investigate.
AI Can Help Here — But Not by Picking the “Correct” Number
This is a useful place for AI.
Not:
“Shopify says 2%. GA4 says 1.6%. Which one is correct?”
That's too simplistic.
Instead:
“These systems report different conversion rates. Here are their definitions, reporting periods, and available dimensions. Identify the differences that could explain the discrepancy. Separate measurement differences from potential tracking errors. Then tell me which additional evidence would help distinguish between them.”
Now AI has a much better job.
It can compare:
- metric definitions
- event structures
- time windows
- dimensions
- attribution logic
- missing events
- duplicated events
- unusual changes
- relationships between datasets
It can also help identify contradictions that deserve human investigation.
For example:
“Orders stayed stable, but GA4 purchases dropped sharply.”
That is a very different signal from:
“Orders dropped sharply, and both GA4 and Shopify purchases declined.”
The first makes tracking or measurement a stronger hypothesis.
The second makes an actual business change more plausible.
AI can help surface this distinction quickly.
But again, it doesn't make the final judgment.
Don't Use AI to Reconcile Everything Automatically
There is another trap here.
Because AI is very good at finding patterns, it is tempting to give it every dataset and ask:
“Reconcile these numbers and give me the final answer.”
I wouldn't do that blindly.
You need to decide what you're actually trying to establish.
For example:
Question 1: Did sales actually decline?
Your order and revenue data are central.
Question 2: Did the measured traffic change?
Your analytics systems become more relevant.
Question 3: Did advertising performance change?
Ad-platform attribution and downstream business outcomes both matter.
Question 4: Did customers behave differently?
Behavioral and customer evidence matter more.
These questions may use overlapping data, but they aren't the same question.
The mistake is expecting one universal metric to answer all of them.
Sometimes the “Wrong” Number Is Still Useful
This is probably the counterintuitive part.
A number doesn't have to be perfectly comparable to be useful.
Suppose Meta reports a large increase in attributed purchases while Shopify orders remain flat.
That discrepancy itself is information.
It tells you to investigate:
- attribution changes
- campaign changes
- audience changes
- reporting windows
- tracking implementation
- duplicate or missing events
- differences in conversion definitions
The disagreement is a signal.
You don't necessarily need to eliminate the disagreement immediately.
You need to understand it.
That's a very different mindset.
A Good Diagnosis Should Survive More Than One Dataset
Let's say your initial hypothesis is:
“The conversion decline is caused by lower-quality traffic.”
You investigate the traffic data.
You find that traffic composition did change.
Good.
But don't stop there.
Now look for independent evidence.
Did the new visitors behave differently?
Did product-page engagement change?
Did add-to-cart behavior change?
Did customer conversations indicate weaker purchase intent?
Did the change coincide with a campaign or targeting change?
If several independent signals point in the same direction, your hypothesis becomes stronger.
If they don't, that's equally useful.
Maybe traffic composition changed, but customer behavior didn't.
Then perhaps traffic isn't the main explanation.
This is what cross-validation gives you.
It doesn't guarantee certainty.
It reduces the chance that you're building a business conclusion around one misleading signal.
The Goal Isn't Perfect Data
I don't think a small ecommerce business needs every analytics system to agree perfectly.
That would be a strange goal.
The goal is to know enough about your measurement systems that you can tell the difference between:
a measurement problem
and
a business problem.
And when there is a business problem, you want enough evidence to narrow down what might actually be happening.
Perfect dashboards don't give you that.
Better questions do.
Don't Pick the Number You Like
When three systems disagree, there is a very human temptation to choose the number that makes the story easiest.
If Shopify looks good, trust Shopify.
If Meta looks good, trust Meta.
If GA4 looks bad, blame GA4.
If GA4 looks good, blame Shopify.
That's not analysis.
That's confirmation bias wearing an analytics costume.
Instead, ask:
What exactly is each system measuring?
Then:
What business question am I trying to answer?
Then:
Which evidence can distinguish between the explanations I am considering?
And finally:
Do independent sources support the same conclusion?
That's a much better foundation for ecommerce diagnosis.
Because the purpose of analytics isn't to find the dashboard with the number you believe.
It is to get closer to what actually happened.
And sometimes, the disagreement between the numbers is itself one of the things you need to understand.