Publication details for Prof Stephen Gorard

Gorard, S. (2019). Significance testing with incompletely randomised cases cannot possibly work. International Journal of Science and Research Methodology 11(2): 42-51.

• Publication type: Journal Article
• ISSN/ISBN: 2454-2008
• Further publication details on publisher web site
• Durham Research Online (DRO) - may include full text

Author(s) from Durham

• Prof Stephen Gorard

Abstract

This brief paper illustrates why the use of significance testing
cannot possibly work with incompletely randomised cases.
The first section reminds readers of the logical argument of
“denying the consequence”, and the fallacy of trying to affirm
the consequence, of a set of premises. The second section
extends the argument of the denying the consequence to the
weaker situation where there is uncertainty, and the third
shows that this weaker situation is the „logical‟ basis for the
practice of significance testing when analysing data. The
fourth section looks at how the same argument becomes a
fallacy when conducting significance tests with incompletely
randomised or non-random cases. The final section
summarises the implications for analysts, and for their future
analyses and reporting.