Conditional Precision of Measurement for Test Scores: Are Conditional Standard Errors Sufficient?

This inquiry is focused on three indicators of the precision of measurement—conditional on fixed values of θ, the latent variable of item response theory (IRT). The indicators that are compared are (1) The traditional, conditional standard errors, σ ( e X | θ ) = CSEM; (2) the IRT-based conditional standard errors, σ irt ( e X | θ ) = C irt SEM = 1 / I ( θ , X ) (where I ( θ , X ) is the IRT score information function); and (3) a new conditional reliability coefficient, ρ ( X , X ′ | θ ) . These indicators of conditional precision are shown to be functionally related to one another. The IRT-based, conditional CSEM, C irt SEM , and the conditional reliability, ρ ( X , X ′ | θ ) , involve an estimate of the conditional true variance, σ 2 ( t X | θ ) , which is shown to be approximately equal to the numerator of the score information function. It is argued—and illustrated with an example—that the traditional, conditional standard error, CSEM, is not sufficient for determining conditional score precision when used as the lone indicator of precision; hence, the portions of a score distribution, where scores are most-and-least precise, can be misidentified