Testing the reliability and validity of risk assessment methods in Human Factors and Ergonomics

Adam Hulme; Neville A Stanton; Guy H Walker; Patrick Waterson; Paul M Salmon

doi:10.1080/00140139.2021.1962969

Back

Testing the reliability and validity of risk assessment methods in Human Factors and Ergonomics

Journal article

Peer reviewed

Testing the reliability and validity of risk assessment methods in Human Factors and Ergonomics

Adam Hulme, Neville A Stanton, Guy H Walker, Patrick Waterson and Paul M Salmon

Ergonomics, Vol.65(3), pp.407-428

2022

DOI: https://doi.org/10.1080/00140139.2021.1962969

PMID: 34328389

Files and links (1)

url

https://doi.org/10.1080/00140139.2021.1962969View

Published Version

Related material

Abstract

EAST-BL

Net-HARMS

reliability validity

Risk assessment

STPA

There is growing interest in the use of systems-based risk assessment methods in Human Factors and Ergonomics (HFE). The purpose of this study was to test the intra-rater reliability and criterion-referenced concurrent validity of three systems-based risk assessment approaches: (i) the Systems-Theoretic Process Analysis (STPA) method; (ii) the Event Analysis of Systemic Teamwork Broken Links (EAST-BL) method; and, (iii) the Network Hazard Analysis and Risk Management System (Net-HARMS) method. Reliability and validity measures were obtained using the Signal Detection Theory (SDT) paradigm. Whilst STPA identified the highest number of risks, the findings indicate a weak to moderate level of reliability and validity for STPA, EAST-BL and Net-HARMS. There were no statistically significant differences between the methods across analyses. The results suggest that there is merit to the continued use of systems-based risk assessment methods following a series of methodological extensions that aim to enhance the reliability and validity of future applications. Practitioner summary The three risk assessment methods produced weak to moderate levels of stability and accuracy regarding their capability to predict risks. There is a pressing need to further test the reliability and validity of safety methods in Human Factors and Ergonomics.

Details

Title: Testing the reliability and validity of risk assessment methods in Human Factors and Ergonomics
Authors: Adam Hulme (Author) - University of the Sunshine Coast, Queensland, Centre for Human Factors and Systems Science
Neville A Stanton (Author) - University of the Sunshine Coast, Queensland, Centre for Human Factors and Systems Science
Guy H Walker (Author) - University of the Sunshine Coast, Queensland, Centre for Human Factors and Systems Science
Patrick Waterson (Author) - Loughborough University
Paul M Salmon (Author) - University of the Sunshine Coast, Queensland, Centre for Human Factors and Systems Science
Publication details: Ergonomics, Vol.65(3), pp.407-428
Publisher: Taylor & Francis
Date published: 2022
DOI: 10.1080/00140139.2021.1962969
ISSN: 1366-5847
PMID: 34328389
Grants: NextGen Accident Prevention: a new theory and toolkit for safer systems, DP180100806, Australian Research Council (Australia, Canberra) - ARC
Organisation Unit: Centre for Human Factors and Systems Science; University of the Sunshine Coast, Queensland; School of Law and Society
Language: English
Record Identifier: 99569208602621
Output Type: Journal article

Metrics

33 Record Views

30 Times Cited - Web of Science

InCites Highlights

These are selected metrics from InCites Benchmarking & Analytics tool, related to this output

Collaboration types: Domestic collaboration; International collaboration
Web Of Science research areas: Engineering, Industrial; Ergonomics; Psychology; Psychology, Applied

UN Sustainable Development Goals (SDGs)

This output has contributed to the advancement of the following goals:

Source: SDGs from InCites