ChatGPT, Copilot, Gemini, SciSpace and Wolfram versus higher education assessments: an updated multi-institutional study of the academic integrity impacts of Generative Artificial Intelligence (GenAI) on assessment, teaching and learning in engineering

Sasha Nikolic; Carolyn Sandison; Rezwanul Haque; Scott Arthur Daniel; Sarah Grundy; Marina Belkina; Sarah Lyden; Ghulam Mubashar Hassan; Peter Neal

doi:10.1080/22054952.2024.2372154

Back

ChatGPT, Copilot, Gemini, SciSpace and Wolfram versus higher education assessments: an updated multi-institutional study of the academic integrity impacts of Generative Artificial Intelligence (GenAI) on assessment, teaching and learning in engineering

Journal article

Open access

Peer reviewed

ChatGPT, Copilot, Gemini, SciSpace and Wolfram versus higher education assessments: an updated multi-institutional study of the academic integrity impacts of Generative Artificial Intelligence (GenAI) on assessment, teaching and learning in engineering

Sasha Nikolic, Carolyn Sandison, Rezwanul Haque, Scott Arthur Daniel, Sarah Grundy, Marina Belkina, Sarah Lyden, Ghulam Mubashar Hassan and Peter Neal

Australasian Journal of Engineering Education, Vol.29(2), pp.126-153

2024

DOI: https://doi.org/10.1080/22054952.2024.2372154

Files and links (2)

pdf

ChatGPT Copilot Gemini SciSpace and Wolfram versus higher education assessments an updated multi-institutional study of the academic integrity imp4.28 MBDownload View

Published VersionCC BY-NC-ND V4.0, Open Access

pdf

teen_a_2372154_sm5079465.15 kBDownload View

Supplementary Material (supplemental)CC BY V4.0, Open Access

Abstract

assessment

academic integrity

chatGPT

cheating

education

generative artificial intelligence (GenAI)

More than a year has passed since reports of ChatGPT-3.5’s capability to pass exams sent shockwaves through education circles. These initial concerns led to a multi-institutional and multi-disciplinary study to assess the performance of Generative Artificial Intelligence (GenAI) against assessment tasks used across 10 engineering subjects, showcasing the capability of GenAI. Assessment types included online quiz, numerical, oral, visual, programming and writing (experimentation, project, reflection and critical thinking, and research). Twelve months later, the study was repeated using new and updated tools ChatGPT-4, Copilot, Gemini, SciSpace and Wolfram. The updated study investigated the performance and capability differences, identifying the best tool for each assessment type. The findings show that increased performance and features can only heighten academic integrity concerns. While cheating concerns are central, opportunities to integrate GenAI to enhance teaching and learning are possible. While each GenAI tool had specific strengths and weaknesses, ChatGPT-4 was well-rounded. A GenAI Assessment Security and Opportunity Matrix is presented to provide the community practical guidance on managing assessment integrity risks and integration opportunities to enhance learning.

Details

Title: ChatGPT, Copilot, Gemini, SciSpace and Wolfram versus higher education assessments: an updated multi-institutional study of the academic integrity impacts of Generative Artificial Intelligence (GenAI) on assessment, teaching and learning in engineering
Authors: Sasha Nikolic (Corresponding Author) - University of Wollongong
Carolyn Sandison - University of Wollongong
Rezwanul Haque - University of the Sunshine Coast, Queensland, School of Science, Technology and Engineering
Scott Arthur Daniel - University of Technology Sydney
Sarah Grundy - UNSW Sydney
Marina Belkina - Western Sydney University
Sarah Lyden - University of Tasmania
Ghulam Mubashar Hassan - The University of Western Australia
Peter Neal - UNSW Australia
Publication details: Australasian Journal of Engineering Education, Vol.29(2), pp.126-153
Publisher: Taylor & Francis Australasia
Date published: 2024
DOI: 10.1080/22054952.2024.2372154
ISSN: 1325-4340; 1324-5821
Copyright note: © 2024 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group. This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or built upon in any way. The terms on which this article has been published allow the posting of the Accepted Manuscript in a repository by the author(s) or with their consent.
Grant note: The funds are provided by the Australian Council of Engineering Deans (ACED) and the Australasian Association of Engineering Education (AAEE).
Organisation Unit: School of Science, Technology and Engineering
Language: English
Record Identifier: 991047798702621
Output Type: Journal article

Metrics

4 File views/ downloads

80 Record Views

See more details