What is test reliability and why does it matter?

Definition of reliability

                Reliability is the relationship between a person’s true score/ability and the observed score and is a key component in test development. Now one might thing that these will be perfectly correlated. However it is not going to be the case think back to a time when you were taking a test and you knew the material well but the score you achieved was well below because you had some life stressor happening. Or you can think of the time when you did better on a test than you should have. These both demonstrate how there is a difference between your true score which is defined as actual ability versus your obtained score which takes into account errors both positive and negative. Therefore reliability is when a survey or an assessment shows a high degree of correlation between the true score and observed score.

Why does reliability matter?

                People have stated to me why does it matter if my test is not reliable as long as it demonstrates criterion validity that’s all I care about. To some extent they are correct. However, validity is directly limited by the amount of internal reliability a test shows. Therefore if a test developer does not care about reliability and only goes after validity, they will be limiting their maximum validity in doing so. Further and on a more fundamental level, reliability of a survey means that the results will be consistent. In as much as if a person were to take the assessment or the survey again their results would be very similar.

                If an exam were to be unreliable using it for hiring selection could lead to unreliable results that are inconsistent.  For each time an applicant takes the exam there could wild variability of test scores and other factors besides the exam will play a heavy role in the final score. Confounding factors can include things such as race, gender, religion, time of day etc… One wants to avoid these as much as possible in order to ensure that the assessment that was developed was done so properly and measures the intended topic consistently.

If you like this please check out our other blogs!