Testing the Reliability and Validity of Net-HARMS: A New Systems-Based Risk Assessment Method in HFE

Adam Hulme; Neville A Stanton; Guy H Walker; Patrick Waterson; Paul M Salmon

doi:10.1007/978-3-030-74602-5_51

Back

Testing the Reliability and Validity of Net-HARMS: A New Systems-Based Risk Assessment Method in HFE

Conference paper

Peer reviewed

Testing the Reliability and Validity of Net-HARMS: A New Systems-Based Risk Assessment Method in HFE

Adam Hulme, Neville A Stanton, Guy H Walker, Patrick Waterson and Paul M Salmon

Proceedings of the 21st Congress of the International Ergonomics Association (IEA 2021), pp.354-362

Congress of the International Ergonomics Association, 21st (Vancouver, Canada, 13-Jun-2021 - 18-Jun-2021)

Lecture Notes in Networks and Systems, 219, Springer International Publishing

2021

DOI: https://doi.org/10.1007/978-3-030-74602-5_51

Files and links (1)

url

https://doi.org/10.1016/10.1007/978-3-030-74602-5_51View

Published Version

Abstract

Signal detection theory

Reliability and validity

Risk assessment

Net-HARMS

There is growing interest in the use of systems-based risk assessment (RA) methods in human factors and ergonomics (HFE). Despite this, there has been a lack of formal reliability and validity testing undertaken to determine whether systems-based RA methods have the capacity to reliably and accurately identify potential risks within complex systems. The purpose of this study was to test the intra-rater reliability (within subject stability) and criterion-related validity (‘gold standard’ performance) of the Networked Hazard Analysis and Risk Management System (Net-HARMS). Net-HARMS is a new and innovative systems-based RA method that supports analysts with the identification of emergent risks. Emergent risks represent new risks that are created when risks from across a complex system interact with one another. Reliability and validity measures for Net-HARMS were obtained using the Signal Detection Theory (SDT) paradigm. Matthews Correlation Coefficient (MCC) was used to analyze the complete SDT data to measure the strength of the correlation between risks. Findings indicate a weak to moderate level of reliability and validity for Net-HARMS based on the MCC score. The results suggest that there is merit to the continued use of Net-HARMS following a series of methodological recommendations that aim to enhance the reliability and validity of future applications.

Details

Title: Testing the Reliability and Validity of Net-HARMS: A New Systems-Based Risk Assessment Method in HFE
Authors: Adam Hulme (Corresponding Author) - University of the Sunshine Coast, Queensland, Centre for Human Factors and Sociotechnical Systems
Neville A Stanton (Author) - University of the Sunshine Coast, Queensland, Centre for Human Factors and Sociotechnical Systems
Guy H Walker (Author) - Heriot-Watt University
Patrick Waterson (Author) - Loughborough University
Paul M Salmon (Author) - University of the Sunshine Coast, Queensland, Centre for Human Factors and Sociotechnical Systems
Contributors: Nancy L Black (Editor) - Université de Moncton
W Patrick Neumann (Editor) - Toronto Metropolitan University
Ian Noy (Editor)
Publication details: Proceedings of the 21st Congress of the International Ergonomics Association (IEA 2021), pp.354-362
Conference details: Congress of the International Ergonomics Association, 21st (Vancouver, Canada, 13-Jun-2021 - 18-Jun-2021)
Series: Lecture Notes in Networks and Systems; 219
Publisher: Springer International Publishing
DOI: 10.1007/978-3-030-74602-5_51; 10.1007/978-3-030-74602-5
ISSN: 2367-3389; 2367-3389
ISBN: 9783030746025
Organisation Unit: School of Law and Society; University of the Sunshine Coast, Queensland; Centre for Human Factors and Sociotechnical Systems
Language: English
Record Identifier: 99530008902621
Output Type: Conference paper

Metrics

38 Record Views

See more details