ChatGPT versus engineering education assessment: a multidisciplinary and multi-institutional benchmarking and analysis of this generative artificial intelligence tool to investigate assessment integrity

Sasha Nikolic; Scott Daniel; Rezwanul Haque; Marina Belkina; Ghulam Hassan; Sarah Grundy; Sarah Lyden; Peter Neal; Caz Sandison

doi:10.1080/03043797.2023.2213169

Back

ChatGPT versus engineering education assessment: a multidisciplinary and multi-institutional benchmarking and analysis of this generative artificial intelligence tool to investigate assessment integrity

Journal article

Open access

Peer reviewed

ChatGPT versus engineering education assessment: a multidisciplinary and multi-institutional benchmarking and analysis of this generative artificial intelligence tool to investigate assessment integrity

Sasha Nikolic, Scott Daniel, Rezwanul Haque, Marina Belkina, Ghulam Hassan, Sarah Grundy, Sarah Lyden, Peter Neal and Caz Sandison

European Journal of Engineering Education, Vol.48(4), pp.559-614

2023

DOI: https://doi.org/10.1080/03043797.2023.2213169

Files and links (2)

pdf

ChatGPT versus engineering education assessment a multidisciplinary and multi-institutional benchmarking and analysis of this generative artificial intelligence tool to investigate assessment integrity4.71 MBDownload View

Published VersionCC BY V4.0, Open Access

url

https://doi.org/10.1080/03043797.2023.2213169View

Published Version

Related material

Is related to

ChatGPT vs engineering assessment

Abstract

artificial intelligence (AI)

assessment

ChatGPT

engineering education

GPT-3

integrity

ChatGPT, a sophisticated online chatbot, sent shockwaves through many sectors once reports filtered through that it could pass exams. In higher education, it has raised many questions about the authenticity of assessment and challenges in detecting plagiarism. Amongst the resulting frenetic hubbub, hints of potential opportunities in how ChatGPT could support learning and the development of critical thinking have also emerged. In this paper, we examine how ChatGPT may affect assessment in engineering education by exploring ChatGPT responses to existing assessment prompts from ten subjects across seven Australian universities. We explore the strengths and weaknesses of current assessment practice and discuss opportunities on how ChatGPT can be used to facilitate learning. As artificial intelligence is rapidly improving, this analysis sets a benchmark for ChatGPT’s performance as of early 2023 in responding to engineering education assessment prompts. ChatGPT did pass some subjects and excelled with some assessment types. Findings suggest that changes in current practice are needed, as typically with little modification to the input prompts, ChatGPT could generate passable responses to many of the assessments, and it is only going to get better as future versions are trained on larger data sets.

Details

Title: ChatGPT versus engineering education assessment: a multidisciplinary and multi-institutional benchmarking and analysis of this generative artificial intelligence tool to investigate assessment integrity
Authors: Sasha Nikolic (Corresponding Author) - University of Wollongong
Scott Daniel (Author) - University of Technology Sydney
Rezwanul Haque (Author) - University of the Sunshine Coast, Queensland, School of Science, Technology and Engineering
Marina Belkina (Author) - Western Sydney University
Ghulam Hassan (Author) - The University of Western Australia
Sarah Grundy (Author) - UNSW Australia
Sarah Lyden (Author) - University of Tasmania
Peter Neal (Author) - UNSW Australia
Caz Sandison (Author) - University of Wollongong
Publication details: European Journal of Engineering Education, Vol.48(4), pp.559-614
Publisher: Taylor & Francis
Date published: 2023
DOI: 10.1080/03043797.2023.2213169
ISSN: 1469-5898; 0304-3797
Copyright note: © 2023 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/),which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The terms on which thisarticle has been published allow the posting of the Accepted Manuscript in a repository by the author(s) or with their consent.
Organisation Unit: School of Science, Technology and Engineering
Language: English
Record Identifier: 99728398602621
Output Type: Journal article

Metrics

29 File views/ downloads

203 Record Views

129 Times Cited - Web of Science

See more details

InCites Highlights

These are selected metrics from InCites Benchmarking & Analytics tool, related to this output

Collaboration types: Domestic collaboration
Web Of Science research areas: Education & Educational Research