Dissertation
A neural and behavioural analysis of speaking test rater assessment of familiar and unfamiliar pronunciation in two presentation modes: audio-only and audio-visual
University of the Sunshine Coast, Queensland
Doctor of Philosophy, University of the Sunshine Coast, Queensland
2022
DOI:
https://doi.org/10.25907/00676
Abstract
In standardized high-stakes English language tests, pronunciation is rated as a component of speech proficiency. In test rubrics the rating of pronunciation is often related to listener effort (e.g., IELTS, TOEFL) and it is partly conditional on the familiarity of the rater with the candidate’s interlanguage phonology (Carey et al., 2011; Winke et al., 2013; Browne & Fulcher, 2016). Phonological familiarity has been long-established as a variable that contributes positively to second-language perception (Gass & Varonis, 1984; Kennedy & Trofimovich, 2008; Bradlow & Bent, 2008), which in turn poses a threat to speaking test validity as it is a test construct-irrelevant feature of the rating process. There is emerging evidence in the literature suggesting that the mode of test-taker presentation (audio-only versus audio-visual) and other test construct-irrelevant variables – the rater’s gender, personality traits, and mono/multilingual status – could also potentially affect the rating process, and it is not known how, and to what extent, these different variables interact to impact pronunciation rating. This thesis investigates the interaction effects of these variables through two quasi-experimental studies in which the independent variables are manipulated experimentally, but participants were not randomly assigned to conditions or orders of conditions. Instead, rater participants in both studies were selected purposively so they were rating NNS speaker speech samples with interlanguage phonology that was systematically categorised to be either familiar or unfamiliar to them. The first study was an exploratory fMRI neurolinguistic study to find evidence of the perceptual substrates of raters’ phonological familiarity while rating second language pronunciation. The findings of this study then informed the design of a second behavioral study that investigated trained raters’ assessment of familiar and unfamiliar interlanguage phonologies in two presentation modes: audio-only and audio-visual to experimentally model the presentation modes of the TOEFL iBT (audio-only and no socio-cultural information provided to the rater) and IELTS (audio-visual with socio-cultural information provided). The results show that the interlanguage phonology familiarity and the mode of presentation interact in rating bias. Further, the results show that presentation mode has an ameliorating effect on familiarity bias, as bias was only present in the audio-only presentation mode but not in the audio-visual, while other rater-related variables only seem to have a small effect on the scores.
Details
- Title
- A neural and behavioural analysis of speaking test rater assessment of familiar and unfamiliar pronunciation in two presentation modes: audio-only and audio-visual
- Authors
- Stefan Szocs - University of the Sunshine Coast, Queensland, School of Education and Tertiary Access
- Contributors
- Michael Carey (Principal Supervisor) - University of the Sunshine Coast, Queensland, School of Education and Tertiary AccessJim Lagopoulos (Co-Supervisor) - University of the Sunshine Coast, Queensland, Thompson InstituteKathryn Broadhouse (Co-Supervisor) - University of the Sunshine Coast, Queensland, School of Health - Nursing
- Awarding institution
- University of the Sunshine Coast, Queensland
- Degree awarded
- Doctor of Philosophy
- Publisher
- University of the Sunshine Coast, Queensland
- DOI
- 10.25907/00676
- Organisation Unit
- Indigenous and Transcultural Research Centre; School of Education and Tertiary Access; University of the Sunshine Coast, Queensland; Engage Research Lab
- Language
- English
- Record Identifier
- 99664998802621
- Output Type
- Dissertation
Metrics
83 File views/ downloads
253 Record Views