Testing Mark

Many social identities (e.g., race, ethnicity) are measured using mark-all-that-apply (MATA) questions because they allow survey respondents to account for the multiple, nonexclusive ways in which they identify themselves. We test the use of MATA measures of sexual orientation and gender identity and compare them with forced choice (FC), an alternative format using a series of yes-or-no questions. Respondents, including an oversample of lesbian, gay, bisexual, and queer (LGBQ) individuals, participated in a 2 × 3 factorial survey experiment. For the first factor, we hypothesize that respondents randomly assigned to FC will report a higher count of identities than those assigned to MATA. For the second factor, we hypothesize that increased topic salience will help LGBQ respondents in particular to overcome poor question design. Findings suggest that MATA and FC measure comparably when question writing best practices are followed, but topic salience can yield higher data quality when poorly formatted questions are used