Cookies

We use cookies to ensure that we give you the best experience on our website. You can change your cookie settings at any time. Otherwise, we'll assume you're OK to continue.

Durham University

School of Education

Research Projects

Publication details

Gorard, S. (2014). Confidence intervals, missing data and imputation: a salutary illustration. International Journal of Research in Educational Methodology 5(3): 693-698.

Author(s) from Durham

Abstract

This paper confirms that confidence intervals are not a generally useful measure or estimate of anything in practice. CIs are recursive in definition and reversed in logic, meaning that they are widely misunderstood. Perhaps most importantly, they should not be used with cases that do not form a complete and true random sample from a known population – the latter is a key premise underlying their calculation. This means that, whatever their merits, CIs should not be used in the vast majority of real-life social science analyses. The second part of the paper illustrates the dangers of ignoring this premise, perhaps on some purported pragmatic grounds. Using 100 simulations of a sample of 100 integers from a uniform population with members in the range 0 to 9, it shows that CIs are very misleading as soon as there is deviation from randomness. For example, when 5% of the cases in each sample are deleted a reported 95% CI would be no better than a 66% CI in reality. If 10% of the lowest score cases are replaced with the achieved mean for the sample, then a reported 95% CI would be more like a 43% CI in reality. In addition, the simulation shows that the mean and standard deviation for any sample are correlated (an issue of linked scale). This illustrates that using the sample standard deviation as an estimate for the SD of the sampling distribution in order to try and assess whether the sample mean is close to the mean of the sampling distribution will simply make matters worse. The best and only available estimate of the sampling distribution mean, in practice, is the sample mean.

School of Education