In order to improve the understanding of media violence effects, it is crucial to extend knowledge about factors that threaten the validity of such effects in empirical research. Research artifacts can be expected when participants are (a) aware of a scientist’s hypothesis, (b) motivated to confirm or disconfirm the hypothesis, and (c) capable of manipulating their responses in line with their motivation. Based on social identity theory (SIT) and self-categorization theory (SCT), we assumed that identifying with the social group of video game players would provide a motivation to disconfirm the “violent video games increase aggression” hypothesis. We further assumed that the use of nontransparent aggression measures and cover stories would prevent research artifacts. Our results showed that highly identified (compared to lowly identified) players of video games reported less aggression on a transparent aggression measure but not on a nontransparent aggression measure. However, providing participants with a cover story did not prevent hypothesis awareness nor eliminate hypothesis-disconfirming response patterns. These results provide empirical support for the ideas that (a) motivational factors may contribute to a biased estimation of media violence effects, (b) cover stories may not always be effective, and (c) the use of nontransparent aggression measures can provide a valid methodological approach for avoiding biases in media effects research.
References
[1]
Strohmetz, D.B. Research artifacts and the social psychology of psychological experiments. Soc. Personal. Psychol. Compass 2008, 2, 861–877, doi:10.1111/j.1751-9004.2007.00072.x.
[2]
Orne, M.T. On the social psychology of the psychological experiment: With particular reference to demand characteristics and their implications. Am. Psychol. 1962, 17, 776–783, doi:10.1037/h0043424.
[3]
Nichols, A.L.; Maner, J.K. The good-subject effect: Investigating participant demand characteristics. J. Gen. Psychol. 2008, 135, 151–165, doi:10.3200/GENP.135.2.151-166.
[4]
Rosenthal, R. Experimenter Effects in Behavioral Research; Appleton-Century-Crofts: East Norwalk, CT, USA, 1966.
[5]
Rosenthal, R. Interpersonal expectancy effects: A 30-year perspective. Curr. Dir. Psychol. Sci. 1994, 3, 176–179.
[6]
Rosenzweig, S. The experimental situation as a psychological problem. Psychol. Rev. 1933, 40, 337–354, doi:10.1037/h0074916.
[7]
Weber, S.J.; Cook, T.D. Subject effects in laboratory research: An examination of subject roles, demand characteristics, and valid inference. Psychol. Bull. 1972, 77, 273–295, doi:10.1037/h0032351.
[8]
Horowitz, I.A.; Rothschild, B.H. Conformity as a function of deception and role playing. J. Pers. Soc. Psychol. 1970, 14, 224–226, doi:10.1037/h0028851.
[9]
Asch, S.E. Effects of Group Pressure upon the Modification and Distortion of Judgments. In Groups, Leadership, and Men; Guetzkow, H., Ed.; Carnegie Press: Pittsburgh, PA, USA, 1951.
[10]
Roethlisberger, F.J.; Dickson, W.J.; Wright, H.A. Management and the Worker: An Account of a Research Program Conducted by the Western Electric Company, Hawthorne Works, Chicago; Harvard University Press: Cambridge, MA, USA, 1939.
[11]
Rosenthal, R.; Rosnow, R.L. Artifact in Behavioral Research; Academic Press: New York, NY, USA, 1969.
[12]
Rosnow, R.L.; Aiken, L.S. Mediation of artifacts in behavioral research. J. Exp. Soc. Psychol. 1973, 9, 181–201, doi:10.1016/0022-1031(73)90009-7.
[13]
Strohmetz, D.B.; Rosnow, R.L. A Mediational Model of Research Artifacts. In Probability in Theory-Building: Experimental and Nonexperimental Approaches to Scientific Research in Psychology; Brzezinski, J., Ed.; Editions Rodopi: Amsterdam, The Netherlands, 1994; pp. 177–196.
[14]
Carlopio, J.; Adair, J.G.; Lindsay, R.C.L.; Spinner, B. Avoiding Artifact in the Search for Bias: The Importance of Assessing Subjects’ Perceptions of the Experiment. J. Pers. Soc. Psychol. 1983, 44, 693–701, doi:10.1037/0022-3514.44.4.693.
[15]
Rosenberg, M.J. The conditions and consequences of evaluation apprehension. In Artifact in Behavioral Research; Rosenthal, R., Rosnow, R.L., Eds.; Academic Press: New York, NY, USA, 1969; pp. 279–349.
[16]
Rosnow, R.L.; Goodstadt, B.E.; Suls, J.M.; Gitter, A.G. More on the social psychology of the experiment: When compliance turns to self-defense. J. Pers. Soc. Psychol. 1973, 27, 337–343, doi:10.1037/h0034943.
[17]
Brehm, J.W. A Theory of Psychological Reactance; Academic Press: New York, NY, USA, 1966.
[18]
Miron, A.M.; Brehm, J.W. Reactance theory: 40 years later. Zeitschrift für Sozialpsychologie 2006, 37, 9–18, doi:10.1024/0044-3514.37.1.9.
[19]
Viswesvaran, C.; Ones, D.S. Meta-Analyses of Fakability Estimates: Implications for Personality Measurement. Educ. Psychol. Meas. 1999, 59, 197–210, doi:10.1177/00131649921969802.
[20]
Ziegler, M.; Schmidt-Atzert, L.; Bühner, M.; Krumm, S. Fakability of different measurement methods for achievement motivation: Questionnaire, semi-projective, and objective. Psychol. Sci. 2007, 49, 291–307.
[21]
Podlesny, J.; Raskin, D.R. Physiological Measures and the Detection of Deception. Psychol. Bull. 1977, 84, 782–799, doi:10.1037/0033-2909.84.4.782.
[22]
Anderson, C.A.; Dill, K.E. Video games and aggressive thoughts, feelings, and behavior in the laboratory and in life. J. Pers. Soc. Psychol. 2000, 78, 772–790, doi:10.1037/0022-3514.78.4.772.
[23]
Bushman, B.; Anderson, C.A. Comfortably Numb: Desensitizing Effects of Violent Media on Helping Others. Psychol. Sci. 2009, 20, 273–277, doi:10.1111/j.1467-9280.2009.02287.x.
[24]
Carnagey, N.L.; Anderson, C.A. The Effects of Reward and Punishment in Violent Video Games on Aggressive Affect, Cognition, and Behavior. Psychol. Sci. 2005, 16, 882–889, doi:10.1111/j.1467-9280.2005.01632.x.
[25]
Levy, L.H. Awareness, learning, and the beneficent subject as expert witness. J. Pers. Soc. Psychol. 1967, 6, 365–370, doi:10.1037/h0024716.
[26]
Gallo, P.S.; Smith, S.; Mumford, S. Effects of deceiving subjects upon experimental results. J. Soc. Psychol. 1973, 89, 99–107, doi:10.1080/00224545.1973.9922573.
[27]
Golding, S.L.; Lichtenstein, E. Confession of awareness and prior knowledge of deception as a function of interview set and approval motivation. J. Pers. Soc. Psychol. 1970, 14, 213–223, doi:10.1037/h0028853.
[28]
Barlett, C.; Anderson, C. Violent Video Games and Public Policy. In Wie wir Spielen, Was Wir Werden: Computerspiele in unserer Gesellschaft. (In German); Bevc, T., Zapf, H., Eds.; UVK Verlagsgesellschaft: Konstanz, Germany, 2009; pp. 227–240.
[29]
Tajfel, H. Social identity and intergroup behaviour. Soc. Sci. Inform. 1974, 13, 65–93, doi:10.1177/053901847401300204.
[30]
Turner, J.; Hogg, M.; Oakes, P.; Reicher, S.; Wetherell, M. Rediscovering the Social Group: A Self-Categorization Theory; Blackwell: Oxford, UK, 1987.
[31]
Scharrer, E. Virtual Violence: Gender and Aggression in Video Game Advertisements. Mass Comm. Soc. 2004, 7, 393–412.
[32]
Williams, D.; Yee, N.; Caplan, S.E. Who plays, how much, and why? Debunking the stereotypical gamer profile. J. Comput. -Mediat. Comm. 2008, 13, 993–1018.
[33]
Gamespot. Available online: http://www.gamespot.com/ (accessed on 10 September 2013).
[34]
Thurm, F. Wir sind keine Amokl？ufer! Available online: http://www.zeit.de/online/2009/31/Killerspiele-Demo (accessed on 9 September 2009).
[35]
Nauroth, P.; Gollwitzer, M.; Bender, J.; Rothmund, T. Gamers against Science: The Case of the Violent Video Games Debate. Eur. J. Soc. Psychol. 2013. submitted for publication.
[36]
Ellemers, N.; Spears, R.; Doosje, B. Self and social identity. Annu. Rev. Psychol. 2002, 53, 161–186, doi:10.1146/annurev.psych.53.100901.135228.
[37]
Branscombe, N.R.; Ellemers, N.; Spears, R.; Doosje, B. The Context and Content of Social Identity Threat. In Social Identity: Context, Commitment, Content; Ellemers, N., Spears, R., Doosje, B., Eds.; Blackwell Science: Oxford, UK, 1999; pp. 35–58.
[38]
Branscombe, N.R.; Wann, D.L. Physiological arousal and reactions to outgroup members during competitions that implicate an important social identity. Aggress. Behav. 1992, 18, 85–93.
[39]
Lilienfeld, S.O.; Wood, J.M.; Garb, H.N. The scientific status of projective techniques. Psychol. Sci. Publ. Interest 2000, 1, 27–66.
[40]
Greenwald, A.G.; McGhee, D.E.; Schwartz, J.L. Measuring individual differences in implicit cognition: The implicit association test. J. Pers. Soc. Psychol. 1998, 74, 1464–1480, doi:10.1037/0022-3514.74.6.1464.
[41]
Williams, J.M.; Mathews, A.; MacLeod, C. The emotional Stroop task and psychopathology. Psychol. Bull. 1996, 120, 3–24, doi:10.1037/0033-2909.120.1.3.
[42]
Carnagey, N.L.; Anderson, C.A.; Bushman, B.J. The effect of video game violence on physiological desensitization to real-life violence. J. Exp. Soc. Psychol. 2007, 43, 489–496, doi:10.1016/j.jesp.2006.05.003.
[43]
Uhlmann, E.; Swanson, J. Exposure to violent video games increases automatic aggressiveness. J. Adolesc. 2004, 27, 41–52.
[44]
Glock, S.; Kneer, J. Game Over? The Impact of Knowledge about Violent Digital Games on the Activation of Aggression-Related Concepts. J. Media Psychol. 2009, 21, 151–160.
[45]
Anderson, C.A.; Carnagey, N.L.; Eubanks, J. Exposure to violent media: The effects of songs with violent lyrics on aggressive thoughts and feelings. J. Pers. Soc. Psychol. 2003, 84, 960–971, doi:10.1037/0022-3514.84.5.960.
[46]
Anderson, C.A.; Gentile, D.A.; Buckley, K.E. Violent Video Game Effects on Children and Adolescents: Theory, Research, and Public Policy; Oxford University Press: New York, NY, USA, 2007.
[47]
Gentile, D.A.; Lynch, P.J.; Linder, J.R.; Walsh, D.A. The effects of violent video game habits on adolescent hostility, aggressive behaviors, and school performance. J. Adolesc. 2004, 27, 5–22, doi:10.1016/j.adolescence.2003.10.002.
[48]
Fleming, M.J.; Rickwood, D.J. Effects of Violent Versus Nonviolent Video Games on Children’s Arousal, Aggressive Mood, and Positive Mood. J. Appl. Soc. Psychol. 2001, 31, 2047–2071, doi:10.1111/j.1559-1816.2001.tb00163.x.
[49]
Anderson, C.A.; Carnagey, N.L.; Flanagan, M.; Benjamin, A.J.; Eubanks, J.; Valentine, J.C. Violent Video Games: Specific Effects of Violent Content on Aggressive Thoughts and Behavior. In Advances in Experimental Social Psychology; Zanna, M.P., Ed.; Elsevier Academic Press: San Diego, CA, USA, 2004; Volume 36, pp. 199–249.
[50]
Simmons, J.P.; Nelson, L.D.; Simonsohn, U. False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychol. Sci. 2011, 22, 1359–1366, doi:10.1177/0956797611417632.
[51]
We recruited exclusively male participants, because many female participants in our previously conducted experiments reported difficulties while handling the controls of a video game.
[52]
Little’s MCAR test revealed that the missing values were missing completely at random, χ2(14, N = 100) = 12.55, p = 0.56. Based on guidelines for dealing with missing data, data were imputed using the expectation-maximization (EM) algorithm, and all analyses were run with and without imputation. Analyses with imputed data produced the same results as analyses on data with listwise deletion. Consequently, all analyses are reported for data with listwise deletion.
[53]
Tabachnick, B.G.; Fidell, L.S. Using Multivariate Statistics, 5th ed. ed.; Allyn & Bacon: Boston, MA, USA, 2007.
[54]
Little, R.J.A. A Test of Missing Completely at Random for Multivariate Data With Missing Values. J. Am. Stat. Assoc. 1988, 83, 1198–1202.
[55]
Luhtanen, R.; Crocker, J. A collective self-esteem scale: Self-evaluation of one’s social identity. Pers. Soc. Psychol. Bull. 1992, 18, 302–318.
[56]
Call of Duty. version green pepper; Computer software, Activision, Los Angeles, CA, USA, 2003.
[57]
Kelava, A.; Moosbrugger, H. Deskriptivstatistische Evaluation von Items (Itemanalyse) und Testwertverteilungen. In Testtheorie und Fragebogenkonstruktion. (in German); Moosbrugger, H., Kelava, A., Eds.; Springer: Heidelberg, Germany, 2007; pp. 73–98.
[58]
It might be argued that in the “no cover story” condition, highly identified video game players would question the ostensible purpose of the nontransparent measure. If this was the case, then both the transparent and the nontransparent measure should be negatively related to identification in the “no cover story” condition. This, however, was not the case: Whereas identification was negatively correlated with the transparent measure (r = ？0.22, p = 0.13), it was not correlated with the nontransparent measure (r = 0.05, p = 0.74).
[59]
Cohen, J.; Cohen, P.; West, S.G.; Aiken, L.S. Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences, 3rd ed. ed.; Lawrence Erlbaum Associates Publishers: Mahwah, NJ, USA, 2003.
[60]
Aiken, L.S.; West, S.G. Multiple regression: Testing and Interpreting Interactions; Sage Publications: Newbury Park, CA, USA, 1991.
[61]
Straits, B.C.; Wuebben, P.L.; Crowle, A.J. Confession of Prior Knowledge about Experimental Procedures as a Function of Evaluation Apprehension and Commitment. Soc. Sci. Res. 1982, 11, 227–244, doi:10.1016/0049-089X(82)90010-2.
[62]
De Houwer, J. What Are Implicit Measures and Why Are We Using Them. In The Handbook of Implicit Cognition and Addiction; Wiers, R.W., Stacy, A.W., Eds.; Sage Publishers: Thousand Oaks, CA, 2006; pp. 11–28.
[63]
Fazio, R.H.; Olson, M.A. Implicit measures in social cognition research: Their meaning and use. Annu. Rev. Psychol. 2003, 54, 297–327, doi:10.1146/annurev.psych.54.101601.145225.
[64]
Anderson, C.A.; Shibuya, A.; Ihori, N.; Swing, E.L.; Bushman, B.J.; Sakamoto, A.; Rothstein, H.R.; Saleem, M. Violent video game effects on aggression, empathy, and prosocial behavior in Eastern and Western countries: A meta-analytic review. Psychol. Bull. 2010, 136, 151–173, doi:10.1037/a0018251.
[65]
Cook, T.D.; Bean, J.R.; Calder, B.J.; Frey, R.; Krovetz, M.L.; Reisman, S.R. Demand characteristics and three conceptions of the frequently deceived subject. J. Pers. Soc. Psychol. 1970, 14, 185–194.
[66]
Hong, S.M. Hong’s Psychological Reactance Scale: A further factor analytic validation. Psychol. Rep. 1992, 70, 512–514, doi:10.2466/pr0.1992.70.2.512.
[67]
Hong, S.M.; Faedda, S. Refinement of the Hong Psychological Reactance Scale. Educ. Psychol. Meas. 1996, 56, 173–182, doi:10.1177/0013164496056001014.
[68]
Leonardelli, G.J.; Brewer, M.B. Minority and majority discrimination: When and why. J. Exp. Soc. Psychol. 2001, 37, 468–485, doi:10.1006/jesp.2001.1475.
[69]
Hyman, H. Interviewing in Social Research; University of Chicago Press: Chicago, IL, USA, 1954.
