Files and links (1)
On why we lack confidence in some signal-detection-based analyses of confidence2.18 MB
Accepted VersionCC BY-NC-ND V4.0, Embargoed Access, Embargo ends: 06-Jun-2025
Journal article
On why we lack confidence in some signal-detection-based analyses of confidence
Consciousness and Cognition, Vol.113, pp.1-16
2023
PMID: 37295196
Abstract
Signal-detection theory (SDT) is one of the most popular frameworks for analyzing data from studies of human behavior – including investigations of confidence. SDT-based analyses of confidence deliver both standard estimates of sensitivity (d’), and a second estimate informed by high-confidence decisions – meta d’. The extent to which meta d’ estimates fall short of d’ estimates is regarded as a measure of metacognitive inefficiency, quantifying the contamination of confidence by additional noise. These analyses rely on a key but questionable assumption – that repeated exposures to an input will evoke a normally-shaped distribution of perceptual experiences (the normality assumption). Here we show, via analyses inspired by an experiment and modelling, that when distributions of experience do not conform with the normality assumption, meta d’ can be systematically underestimated relative to d'. Our data highlight that SDT-based analyses of confidence do not provide a ground truth measure of human metacognitive inefficiency. We explain why deviance from the normality assumption is especially a problem for some popular SDT-based analyses of confidence, in contrast to other analyses inspired by the SDT framework, which are more robust to violations of the normality assumption.
Details
- Title
- On why we lack confidence in some signal-detection-based analyses of confidence
- Authors
- Derek H. Arnold (Corresponding Author) - University of QueenslandAlan Johnston - University of NottinghamJoshua Adie - University of CanberraKielan Yarrow - City, University of London
- Publication details
- Consciousness and Cognition, Vol.113, pp.1-16
- Publisher
- Academic Press
- Date published
- 2023
- DOI
- 10.1016/j.concog.2023.103532
- ISSN
- 1090-2376
- PMID
- 37295196
- Data Availability
- Data will be made available on request.
- Organisation Unit
- School of Health - Psychology
- Language
- English
- Record Identifier
- 991101937102621
- Output Type
- Journal article
Metrics
1 Record Views
InCites Highlights
These are selected metrics from InCites Benchmarking & Analytics tool, related to this output
- Collaboration types
- Domestic collaboration
- International collaboration
- Web Of Science research areas
- Psychology, Experimental