Using video-cases to assess student reflection: Development and validation of an instrument

Fourth and fifth year undergraduate medical students were shown two interactive video-cases and asked to reflect on this experience, guided by six standard questions. The quality of students’ reflections were scored using a specially developed Student Assessment of Reflection Scoring rubric (StARS？). Reflection scores were analyzed concerning interrater reliability and ability to discriminate between students. Further, the intra-rater reliability and case specificity were estimated by means of a generalizability study with rating and case scenario as facets.Reflection scores of 270 students ranged widely and interrater reliability was acceptable (Krippendorff’s alpha？=？0.88). The generalizability study suggested 3 or 4 cases were needed to obtain reliable ratings from 4th year students and？≥？6 cases from 5th year students.Use of StARS？ to assess student reflections triggered by standardized video-cases had acceptable discriminative ability and reliability. We offer this practical method for assessing reflection summatively, and providing formative feedback in training situations.