Files and links (2)
Patterns of drinking alcohol and intentions to binge drink among medical students in Vietnam230.75 kB
Accepted VersionCC BY V4.0, Open Access
Journal article
Predicting alcohol dependence treatment outcomes: a prospective comparative study of clinical psychologists versus ‘trained’ machine learning models
Addiction, Vol.115(11), pp.2164-2175
2020
PMID: 32150316
Abstract
Background and aims:
Clinical staff are typically poor at predicting alcohol dependence treatment outcomes. Machine learning (ML) offers the potential to model complex clinical data more effectively. This study tested the predictive accuracy of ML algorithms demonstrated to be effective in predicting alcohol dependence outcomes, compared with clinical judgement and traditional linear regression.
Design:
Prospective study. ML models were trained on 1016 previously treated patients (training‐set) who attended a hospital‐based alcohol and drug clinic. ML models (n = 27), clinical psychologists (n = 10) and a ‘traditional’ logistic regression model (n = 1) predicted treatment outcome during the initial treatment session of an alcohol dependence programme.
Setting:
A 12‐week cognitive behavioural therapy (CBT)‐based abstinence programme for alcohol dependence in a hospital‐based alcohol and drug clinic in Australia.
Participants:
Prospective predictions were made for 220 new patients (test‐set; 70.91% male, mean age = 35.78 years, standard deviation = 9.19). Sixty‐nine (31.36%) patients successfully completed treatment.
Measurements:
Treatment success was the primary outcome variable. The cross‐validated training‐set accuracy of ML models was used to determine optimal parameters for selecting models for prospective prediction. Accuracy, sensitivity, specificity, area under the receiver operator curve (AUC), Brier score and calibration curves were calculated and compared across predictions.
Findings:
The mean aggregate accuracy of the ML models (63.06%) was higher than the mean accuracy of psychologist predictions (56.36%). The most accurate ML model achieved 70% accuracy, as did logistic regression. Both were more accurate than psychologists (P < 0.05) and had superior calibration. The high specificity for the selected ML (79%) and logistic regression (90%) meant they were significantly (P < 0.001) more effective than psychologists (50%) at correctly identifying patients whose treatment was unsuccessful. For ML and logistic regression, high specificity came at the expense of sensitivity (26 and 31%, respectively), resulting in poor prediction of successful patients.
Conclusions:
Machine learning models and logistic regression appear to be more accurate than psychologists at predicting treatment outcomes in an abstinence programme for alcohol dependence, but sensitivity is low.
Details
- Title
- Predicting alcohol dependence treatment outcomes: a prospective comparative study of clinical psychologists versus ‘trained’ machine learning models
- Authors
- Martyn Symons (Author) - The University of Western AustraliaGerald F. X Feeney (Author) - Princess Alexandra HospitalMarcus R Gallagher (Author) - The University of QueenslandRoss Young (Author) - Queensland University of TechnologyJason P Connor (Corresponding Author) - The University of Queensland
- Publication details
- Addiction, Vol.115(11), pp.2164-2175; 12
- Publisher
- Wiley-Blackwell Publishing Ltd.
- Date published
- 2020
- DOI
- 10.1111/add.15038
- ISSN
- 1360-0443; 0965-2140
- PMID
- 32150316
- Copyright note
- Copyright (c) 2020. This is the pre-peer reviewed version of the following article: Symons, M., Feeney, G. F. X., Gallagher, M. R., Young, R. MD., and Connor, J. P. (2020) Predicting alcohol dependence treatment outcomes: a prospective comparative study of clinical psychologists versus ‘trained’ machine learning models. Addiction, 115: 2164– 2175. https://doi.org/10.1111/add.15038, which has been published in final form at https://doi.org/10.1111/add.15038. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Use of Self-Archived Versions.
- Organisation Unit
- University of the Sunshine Coast, Queensland; Office of the Deputy Vice-Chancellor (Research and Innovation)
- Language
- English
- Record Identifier
- 99550993902621
- Output Type
- Journal article
Metrics
84 File views/ downloads
29 Record Views
InCites Highlights
These are selected metrics from InCites Benchmarking & Analytics tool, related to this output
- Collaboration types
- Domestic collaboration
- Web Of Science research areas
- Psychiatry
- Substance Abuse
UN Sustainable Development Goals (SDGs)
This output has contributed to the advancement of the following goals:
Source: InCites