Book the exclusive demo launch to experience the OWOX BI analytics

How to check statistical significance of calculation results?

Mathematically significance and reliability can be checked by measuring confidence interval.

Confidence interval — is a range of sample where the true value of the parameter belongs to with some confidence level.
In attribution calculation results, step probability is the parameter.

Confidence interval = z*SQRT((p*(1-p))/n),
where z — confidence level. For 80% confidence level it's 1.28, for 90% — 1.64, 95% — 1.96, 99% — 2.58.
     p — step probability,
     n — count of previous step sessions.

Example

Lets measure confidence interval with 90% confidence level for a step where were 590 visits. There were 1,000 visits on the previous step, i.e. step probability is 59%.

Confidence interval = 1.64*SQRT((0.59*(1-0.59))/1000) = 2.55%.
That means, in 90% of cases the step probability is 59% ±2.55%, i.e. it's in range from 56.45% to 61.55%.

We consider calculation results to be statistically significant if the confidence interval value is not more than 1/10 of step probability value.
In the example below, confidence interval value below 5.9% would be acceptable.

Was this article helpful?
1 out of 1 found this helpful
Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.