Research Methods in Education

Scale of data	One sample	Two samples		More than two samples
		Independent	Related	Independent	Related
Nominal	Binomial	Fisher exact test	McNemar	Chi-square ( c 2 ) k-samples test	Cochran Q
	Chi-square ( c 2 ) one-sample test	Chi-square ( c 2 ) two-samples test
Ordinal	Kolmogorov-Smirnov one-sample test	Mann-Whitney U test	Wilcoxon matched pairs test	Kruskal-Wallis test	Friedman test
		Kolmogorov-Smirnov test	Sign test	Ordinal regression analysis
		Wald-Wolfowitz
		Spearman rho
		Ordinal regression analysis
Interval and ratio	t-test	t-test	t-test for paired samples	One-way ANOVA	Repeated measures ANOVA
		Pearson product moment correlation		Two-way ANOVA
				Tukey hsd test
				Scheffé test

Box 26.3: Types of statistical tests for four scales of data

	Nominal	Ordinal	Interval and ratio
Measures of association	Tetrachoric correlation	Spearman’s rho	Pearson product-moment correlation
	Point biserial correlation	Kendall rank order correlation
	Phi coefficient	Kendall partial rank correlation
	Cramer’s V
Measures of difference	Chi-square	Mann-Whitney U test	t-test for two independent samples
	McNemar	Kruskal-Wallis	t-test for two related samples
	Cochran Q	Wilcoxon matched pairs	One-way ANOVA
	Binomial test	Friedman two-way analysis of variance	Two-way ANOVA for more
		Wald-Wolfowitz test	Tukey hsd test
		Kolmogorov-Smirnov test	Scheffé test
Measures of linear relationship between independent and dependent variables		Ordinal regression analysis	Linear regression
			Multiple regression
Identifying underlying factors, data reduction			Factor analysis
			Elementary linkage analysis

Box 26.4: Choosing statistical tests for parametric and non-parametric data

Box 26.5: Statistics available for different types of data

Data type

Legitimate statistics

Points to observe/questions/examples

Nominal

Mode (the score achieved by the greatest number of people)

Frequencies

Chi-square ( c 2 ) (a statistic that charts the difference between statistically expected and actual scores)

Is there a clear ‘front runner’ that receives the highest score with low scoring on other categories, or is the modal score only narrowly leading the other categories? Are there two scores which are vying for the highest score – a bi-modal score?

Which are the highest/lowest frequencies? Is the distribution even across categories?

Are differences between scores caused by chance/accident or are they statistically significant, i.e. not simply caused by chance?

Ordinal

Mode

Median (the score gained by the middle person in a ranked group of people or, if there is an even number of cases, the score which is midway between the highest score obtained in the lower half of the cases and the lowest score obtained in the higher half of the cases).

Frequencies

Chi-square ( c 2 )

Spearman rank order correlation (a statistic to measure the degree of association between two ordinal variables)

Mann-Whitney U-test (a statistic to measure any significant difference between two independent samples)

Kruskal-Wallis analysis of variance (a statistic to measure any significant differences between three or more independent samples)

Which score on a rating scale is the most frequent?

What is the score of the middle person in a list of scores?

Do responses tend to cluster around one or two categories of a rating scale? Are the responses skewed towards one end of a rating scale (e.g. ‘strongly agree’)? Do the responses pattern themselves consistently across the sample? Are the frequencies generally high or generally low (i.e. whether respondents tend to feel strongly about an issue)? Is there a clustering of responses around the central categories of a rating scale (the central tendency, respondents not wishing to appear to be too extreme)?

Are the frequencies of one set of nominal variables (e.g. sex) significantly related to a set of ordinal variables?

Do the results from one rating scale correlate with the results from another rating scale? Do the rank order positions for one variable correlate with the rank order positions for another variable?

Is there a significant difference in the results of a rating scale for two independent samples (e.g. males and females)?

Is there a significant difference between three or more nominal variables (e.g. membership of political parties) and the results of a rating scale?

Interval and ratio

Mode
Mean
Frequencies
Median
Chi-square ( c 2 )
Standard deviation (a measure of the dispersal of scores)

z-scores (a statistic to convert scores from different scales, i.e. with different means and standard deviations, to a common scale, i.e. with the same mean and standard deviation, enabling different scores to be compared fairly)

Pearson product moment correlation (a statistic to measure the degree of association between two interval or ratio variables)

t-tests (a statistic to measure the difference between the means of one sample on two separate occasions or between two samples on one occasion)

Analysis of variance (a statistic to ascertain whether two or more means differ significantly)

What is the average score for this group?

Are the scores on a parametric test evenly distributed? Do scores cluster closely around the mean? Are scores widely spread around the mean? Are scores dispersed evenly? Are one or two extreme scores (‘outliers’) exerting a disproportionate influence on what are otherwise closely clustered scores?

How do the scores obtained by students on a test which was marked out of 20 compare to the scores by the same students on a test which was marked out of 50?

Is there a correlation between one set of interval data (e.g. test scores for one examination) and another set of interval data (e.g. test scores on another examination)?

Are the control and experimental groups matched in their mean scores on a parametric test? Is there a significant different between the pre-test and post-test scores of a sample group?

Are the differences in the means between test results of three groups statistically significant?

Box 26.6: Assumptions of statistical tests

TEST	ASSUMPTIONS
Mean	Data are normally distributed, with no outliers
Mode	There are few values, and few scores, occurring which have a similar frequency
Median	There are many ordinal values
Chi-square	Data are categorical (nominal); Randomly sampled population; Mutually independent categories; Data are discrete (i.e. no decimal places between data points); 80% of all the cells in a crosstabulation contain 5 or more cases;
Kolmogorov-Smirnov	The underlying distribution is continuous; Data are nominal;
t-test and Analysis of Variance	Population is normally distributed; Sample is selected randomly from the population; Each case is independent of the other; The groups to be compared are nominal, and the comparison is made using interval and ratio data; The sets of data to be compared are normally distributed (the bell-shaped Gaussian curve of distribution); The sets of scores have approximately equal variances, or the square of the standard deviation is known; The data are interval or ratio.
Wilcoxon Test	The data are ordinal; The samples are related
Mann-Whitney and Kruskal-Wallis	The groups to be compared are nominal, and the comparison is made using ordinal data; The populations from which the samples are drawn have similar distributions; Samples are drawn randomly; Samples are independent of each other;
Spearman rank order correlation	The data are ordinal;
Pearson correlation	The data are interval and ratio;
Regression (simple and multiple)	Assumptions underlying regression techniques: The data derive from a random or probability sample; The data are interval or ratio (unless ordinal regression is used); Outliers are removed; There is a linear relationship between the independent and dependent variables; The dependent variable is normally distributed (the bell-shaped Gaussian curve of distribution); The residuals for the dependent variable (the differences between calculated and observed scores) are approximately normally distributed; Collinearity is removed (where one independent variable is an exact or very close correlate of another);
Factor analysis	The data are interval or ratio; The data are normally distributed; Outliers have been removed; The sample size should not be less than 100-150 persons; There should be at least five cases for each variable; The relationships between the variables should be linear; The data must be capable of being factored.