Files and links (1)
carey-szocs-2023-revisiting-raters-accent-familiarity-in-speaking-tests-evidence-that-presentation-mode-interacts-with363.73 kB
Published VersionCC BY V4.0, Open Access
Journal article
Revisiting raters' accent familiarity in speaking tests: Evidence that presentation mode interacts with accent familiarity to variably affect comprehensibility ratings
Language Testing, Vol.41(2), pp.290-315
2024
Appears in UniSC Supported Open Access Outputs
Abstract
This controlled experimental study investigated the interaction of variables associated with rating the pronunciation component of high-stakes English language speaking tests such as IELTS and TOEFL iBT. One hundred experienced raters who were all either familiar or unfamiliar with Brazilian-accented English or Papua New Guinean Tok Pisin accented-English, respectively, were presented with speech samples in audio-only or audiovisual mode. Two-way ordinal regression with post-hoc pairwise comparisons found that the presentation mode interacted significantly with accent familiarity to increase comprehensibility ratings (χ² = 88.005, df = 3, p < .0001), with presentation mode having the stronger effect in the interaction than accent familiarity (χ² = 59.328, df = 1, p < .0001). Based on odds ratios, raters were significantly more likely to score comprehensibility higher when the presentation mode was audiovisual (compared to audio-only) for both the unfamiliar (91% more likely) and familiar speakers (92.3% more likely). The results suggest that semi-direct speaking tests using audio-only or audiovisual modes of presentation should be evaluated through research to ascertain how accent familiarity and presentation mode interact to variably affect comprehensibility ratings. Such research may be beneficial to investigate the virtual modes of speaking test delivery that have emerged post-COVID-19.
Details
- Title
- Revisiting raters' accent familiarity in speaking tests: Evidence that presentation mode interacts with accent familiarity to variably affect comprehensibility ratings
- Authors
- Michael D Carey (Corresponding Author) - University of the Sunshine Coast, Queensland, School of Education and Tertiary AccessStefan Szocs (Author) - University of the Sunshine Coast, Queensland, School of Education and Tertiary Access
- Publication details
- Language Testing, Vol.41(2), pp.290-315
- Publisher
- Sage Publications Ltd.
- Date published
- 2024
- DOI
- 10.1177/02655322231200808
- ISSN
- 1477-0946; 0265-5322
- Copyright note
- © The Author(s) 2023. This article is distributed under the terms of the Creative Commons Attribution 4.0 Lficense (https://creativecommons.org/licenses/by/4.0/) which permits any use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage).
- Organisation Unit
- Indigenous and Transcultural Research Centre; School of Education and Tertiary Access; Engage Research Lab
- Language
- English
- Record Identifier
- 99746998802621
- Output Type
- Journal article
Metrics
6 File views/ downloads
76 Record Views