Many countries use national-level surveys to capture student opinions about their university experiences. It is necessary to interpret survey results in an appropriate context to inform decision-making at many levels. To provide context to national survey outcomes, we describe patterns in the ratings of science and engineering subjects from the UK’s National Student Survey (NSS). New, robust statistical models describe relationships between the Overall Satisfaction’ rating and the preceding 21 core survey questions. Subjects exhibited consistent differences and ratings of “Teaching”, “Organisation” and “Support” were thematic predictors of “Overall Satisfaction” and the best single predictor was “The course was well designed and running smoothly”. General levels of satisfaction with feedback were low, but questions about feedback were ultimately the weakest predictors of “Overall Satisfaction”. The UK’s universities affiliated groupings revealed that more traditional “1994” and “Russell” groups over-performed in a model using the core 21 survey questions to predict “Overall Satisfaction”, in contrast to the under-performing newer universities in the Million+ and Alliance groups. Findings contribute to the debate about “level playing fields” for the interpretation of survey outcomes worldwide in terms of differences between subjects, institutional types and the questionnaire items.
References
[1]
Chalmers, D.; Lee, K.; Walker, B. International and National Quality Teaching and Learning Performance Models Currently in Use, 2008. Available online: http://www.catl.uwa.edu.au/projects/tqi (accessed on 23 February 2013).
[2]
Hewson, P. Preliminary Analysis of the National Student SurveyMSOR Connections Vol 11, No 1 Spring Term 2011. Available online: http://www-new1.heacademy.ac.uk./documents/subjects/msor/Hewson.pdf (accessed on 23 February 2013).
[3]
O’Neill, M.; Palmer, A. Cognitive dissonance and the stability of service quality perceptions. J. Serv. Market. 2004, 18, 433–449, doi:10.1108/08876040410557221.
[4]
Fielding, A.H.; Dunleavy, P.J.; Langan, A.M. Effective use of the UK’s National Student (Satisfaction) Survey (NSS) data in science and engineering subjects. J. Furth. High. Educ. 2010, 33, 347–368, doi:10.1080/0309877X.2010.484054.
Garcia-Aracil, A. European graduates’ level of satisfaction with higher education. High. Educ. 2009, 57, 1–21, doi:10.1007/s10734-008-9121-9.
[7]
Ramsden, P. A performance indicator of teaching quality in higher education: The course experience questionnaire. Stud. High. Educ. 1991, 16, 129–150, doi:10.1080/03075079112331382944.
[8]
Richardson, J.T.E. Instruments for obtaining student feedback: A review of the literature. Assess. Eval. High. Educ. 2005, 30, 387–415, doi:10.1080/02602930500099193.
[9]
Gibbs, G. Dimensions of Quality; Higher Education Academy: York, UK, 2010.
[10]
Curtis, P. Student survey is bland and worthless, claims academic. Educ. Guard. 2008. Available online: http://www.guardian.co.uk/education/2008/jun/12/highereducation.uk2 (accessed on 23 February 2013).
[11]
Brown, S. Bringing about positive change in the higher education student experience: A case study. Qual. Assur. Educ. 2011, 19, 195–207, doi:10.1108/09684881111158027.
[12]
Ainley, J.; Long, M. The Course Experience Survey 1992 Graduates; Australian Government Publishing Service: Canberra, Australia, 1994.
[13]
Brennan, J.; Williams, R.; Brighton, R.; Moon, N.; Richardson, J.; Rindl, J. Collecting and Using Student Feedback on Quality and Standards of Learning and Teaching in Higher Education; Higher Education Funding Council for England: London, UK, 2003. Available online: www.hefce.ac.uk/Pubs/rdreports/2003/rd08_03/rd08_03.pdf (accessed on 23 February 2013).
[14]
Surridge, P. Interpreting the National Student Survey Data Using Multi-Level Modelling. A Non-Technical Guide; The Higher Education Academy: York, UK, 2008. Available online: http://www.heacademy.ac.uk./York/documents/ourwork/research/NSS_interpreting_data_nontechnical_guide.pdf (accessed on 23 February 2013).
[15]
Thomas, E.H.; Galambos, N. What satisfies students? Mining student-opinion data with regression and decision tree analysis. Res. High. Educ. 2004, 45, 251–269, doi:10.1023/B:RIHE.0000019589.79439.6e.
[16]
Breiman, L. Random forests. Mach. Learn. 2001, 45, 5–32, doi:10.1023/A:1010933404324.
[17]
Breiman, L. Statistical modelling: The two cultures. Stat. Sci. 2001, 16, 199–215, doi:10.1214/ss/1009213726.
[18]
Breiman, L. Bagging predictors. Mach. Learn. 1996, 24, 123–140.
[19]
Breiman, L.; Cutler, A. Interface Workshop–April 2004. 2004. Available online: http://stat-www.berkeley.edu/users/breiman/RandomForests/interface04.pdf (accessed on 23 February 2013).
[20]
Crawley, M.J. The R Book; John Wiley and Sons: Chichester, UK, 2007.
[21]
Marsh, H.W.; Cheng, J.H.S. Dimensionality, Multilevel Structure, and Differentiation at the Level of University and Discipline: Preliminary Results, 2008. Available online: http://www.heacademy.ac.uk/resources/detail/ourwork/research/NSS_herb_marsh (accessed on 23 February 2013).
[22]
Dean, A.A. Enhancing the Learning Experience of Students in Higher Education: Key Findings from the NSS Survey (2008 to 2010) for University Business Schools in England. Report to the Higher Education Academy, Business, Accountancy and Finance Network, 2010. Available online: http://www-new1.heacademy.ac.uk/business/news/detail/2011/nss-survey-report (accessed on 23 February 2013).
[23]
Crawford, K.; Hagyard, A.; Saunders, G. Creative Analysis of NSS Data and Collaborative Research to Inform Good Practice in Assessment Feedback. SWAP Project Output: Higher Education Academy Social Policy and Project Work, 2010. Available online: http://www.swap.ac.uk/docs/projects/nss_report.pdf (accessed on 23 February 2013).
[24]
Boehler, M.L.; Rogers, D.A.; Schwind, C.J.; Mayforth, R.; Quin, J.; Williams, R.G.; Dunnington, G. An investigation of medical student reactions to feedback: a randomised controlled trial. Med. Educ. 2006, 40, 746–749, doi:10.1111/j.1365-2929.2006.02503.x.
[25]
Richardson, J.T.E.; Slater, J.B.; Wilson, J. The national student survey: Development, findings and implications. Stud. High. Educ. 2007, 32, 557–580, doi:10.1080/03075070701573757.
[26]
Emery, C.; Kramer, T.; Tian, R. Customers vs. products: Adopting an effective approach to business students. Qual. Assur. Educ. 2001, 9, 110–115, doi:10.1108/09684880110389681.
[27]
Buckley, A. Making It Count: Reflecting on the National Student Survey in the Process of Enhancement; Higher Education Academy: York, UK, 2012.
