Order

Bock’s nominal response model (NRM) is sometimes used to identify the empirical order of response categories in polytomous items but its application tags many items as having disordered categories. Disorderly estimated categories may not reflect a true characteristic of the items but, rather, a numerically best-fitting solution possibly equivalent to other solutions with orderly estimated categories. To investigate this possibility, an order-constrained variant of the NRM was developed that enforces the preassumed order of categories on parameter estimates, for a comparison of its outcomes with those of the original unconstrained NRM. For items with ordered categories, order-constrained and unconstrained solutions should account for the data equally well even if the latter solution estimated disordered categories for some items; for items with truly disordered categories, the unconstrained solution should outperform the order-constrained solution. Criteria for this comparative analysis are defined and their utility is tested in several simulation studies with items of diverse characteristics, including ordered and disordered categories. The results demonstrate that a comparison of order-constrained and unconstrained calibrations on such criteria provides the evidence needed to determine whether category disorder estimated on some items by the original unconstrained form of the NRM is authentic or spurious. Applications of this method to assess category order in existing data sets are presented and practical implications are discussed