Does a rater’s familiarity with a candidate’s pronunciation affect the rating in oral proficiency interviews?

Michael D Carey; R H Mannell; Peter K Dunn

doi:10.1177/0265532210393704

Back

Does a rater’s familiarity with a candidate’s pronunciation affect the rating in oral proficiency interviews?

Journal article

Open access

Peer reviewed

Does a rater’s familiarity with a candidate’s pronunciation affect the rating in oral proficiency interviews?

Michael D Carey, R H Mannell and Peter K Dunn

Language Testing, Vol.28(2), pp.201-219

2011

DOI: https://doi.org/10.1177/0265532210393704

Files and links (2)

pdf

PDF - Author Accepted Version333.39 kBDownload View

Accepted VersionPDF - Author Accepted Version Open Access

url

https://doi.org/10.1177/0265532210393704View

Published Version

Abstract

accent familiarity

oral proficiency interview

pronunciation criteria

rater training

speaking test

This study investigated factors that could affect inter-examiner reliability in the pronunciation assessment component of speaking tests. We hypothesized that the rating of pronunciation is susceptible to variation in assessment due to the amount of exposure examiners have to non-native English accents. An inter-rater variability analysis was conducted on the English pronunciation ratings of three test candidate interlanguages: Chinese, Korean, and Indian English. Pronunciation was rated by 99 International English Language Testing System (IELTS) examiners across five geographically dispersed test centres. The examiners had either prolonged exposure, or no, or little exposure to the interlanguage of the candidates. A significant proportion of examiners rated pronunciation higher when they had prolonged exposure, and lower when they had no, or little, exposure to the candidates' interlanguage. The location of the test centre also had a significant effect on the pronunciation rating, independent of the familiarity variable, with a significant proportion of NNS raters scoring candidates from their home country higher than those who were not. It is recommended that interlanguage phonology familiarity should be considered in the design of speaking tests and rater training and that research is required into test centre bias and the phonological judgements and awareness of OPI raters.

Details

Title: Does a rater’s familiarity with a candidate’s pronunciation affect the rating in oral proficiency interviews?
Authors: Michael D Carey (Author) - University of the Sunshine Coast - Faculty of Science, Health and Education
R H Mannell (Author) - Macquarie University
Peter K Dunn (Author) - University of the Sunshine Coast - Faculty of Science, Health and Education
Publication details: Language Testing, Vol.28(2), pp.201-219
Publisher: Sage Publications Ltd.
Date published: 2011
DOI: 10.1177/0265532210393704
ISSN: 0265-5322
Copyright note: Copyright © 2011 The Author. The author's accepted version is reproduced here in accordance with the publisher's copyright policy. The definitive version is available at: http://dx.doi.org/10.1177/0265532210393704
Organisation Unit: School of Education - Legacy; Indigenous and Transcultural Research Centre; School of Education and Tertiary Access; University of the Sunshine Coast, Queensland; School of Health and Sport Sciences - Legacy; School of Science, Technology and Engineering; Engage Research Lab
Language: English
Record Identifier: 99449792502621
Output Type: Journal article

Metrics

422 File views/ downloads

1223 Record Views

73 Times Cited - Web of Science

See more details

InCites Highlights

These are selected metrics from InCites Benchmarking & Analytics tool, related to this output

Collaboration types: Domestic collaboration
Web Of Science research areas: Language & Linguistics; Linguistics