Mathematically significance and reliability can be checked by measuring confidence interval.
Confidence interval — is a range of sample where the true value of the parameter belongs to with some confidence level.
In attribution calculation results, step probability is the parameter.
Confidence interval = z*SQRT((p*(1-p))/n)
,
where z
— confidence level. For 80% confidence level it's 1.28, for 90% — 1.64, 95% — 1.96, 99% — 2.58.
p
— step probability,
n
— count of previous step sessions.
Example
Lets measure confidence interval with 90% confidence level for a step where were 590 visits. There were 1,000 visits on the previous step, i.e. step probability is 59%.
Confidence interval = 1.64*SQRT((0.59*(1-0.59))/1000) = 2.55%
.
That means, in 90% of cases the step probability is 59% ±2.55%, i.e. it's in range from 56.45% to 61.55%.
We consider calculation results to be statistically significant if the confidence interval value is not more than 1/10 of step probability value.
In the example below, confidence interval value below 5.9% would be acceptable.
0 Comments