Electronic Proceedings of the Twenty-eighth Annual International Conference on Technology in Collegiate Mathematics

Atlanta, Georgia, March 10-13, 2016

Paper A004

This is an electronic reprint, reproduced by permission of Pearson Education Inc. Originally appeared in the Proceedings of the Twenty-eighth Annual International Conference on Technology in Collegiate Mathematics, ISBN 013480029X, Copyright (C) 2017 by Pearson Education, Inc.


Testing Unfairness - Mathematics and Simulations

G. Donald Allen


Texas A&M University


list of all papers by this author

Dianne Goldsby


Texas A&M University


list of all papers by this author

Sandra Nite


Texas A&M University


list of all papers by this author

Click to access this paper: paper.pdf

ABSTRACT

This paper examines some novel issues of testing unfairness from perspectives of the student, teacher, but mostly the test itself. The literature usually examines testing unfairness between various groups, considering mainly group demographics. We consider cognitively differentiated groups and test difficulty to examine how groups perform on a particular test, both having defined distributions of ability and difficulty, respectively. Even controlling the statistical parameters of the test and/or the group, there can be vast differences in outcomes, i.e. exam averages. This holds even for high stakes testing, leading to the reality grades are almost always 'adjusted' to conform to testing expectations, i.e. we curve exam scores. In this section, we consider yet another aspect of unfairness, and that is with the numbers embedded in the test itself, and how it affects class grades with respect to class parameters. The two principle parameters about the student are the average ability and the variation of those abilities. For the test, the two principle parameters are the average item difficulty and its variation. However, we need also consider the actual distributions of the student abilities and the test difficulties. These must be co-mingled in an intuitive manner. Various types of distributions of these populations and difficulties are considered. In addition, we need to model the actual densities of student abilities and item difficulties given the numbers of students and test items. For tests given to a relatively small number of students, there are other considerations such as the need to use simulations. We discuss the sensitivity of grades to parameters, distribution of grades, the effect of a small number of test takers, and the relative variation in standardized tests.

Keyword(s): assessment