“We Don’t Have Time to Read the Entire Document to Evaluate the Quality” – The Role of Metadata in Quality Evaluation of Unstructured Textual Data

Hui  Zhou; Tianwa Chen; Gianluca Demartini; Marta Indulska; Shazia Sadiq

Back

“We Don’t Have Time to Read the Entire Document to Evaluate the Quality” – The Role of Metadata in Quality Evaluation of Unstructured Textual Data

Conference paper

Open access

Peer reviewed

“We Don’t Have Time to Read the Entire Document to Evaluate the Quality” – The Role of Metadata in Quality Evaluation of Unstructured Textual Data

Hui Zhou, Tianwa Chen, Gianluca Demartini, Marta Indulska and Shazia Sadiq

ACIS 2025 Proceedings, pp.1-16

Australasian Conference on Information Systems (ACIS), 36th (Sunshine Coast, Australia, 01-Dec-2025–05-Dec-2025)

Australasian Association for Information Systems (AAIS)

2025

Files and links (1)

pdf

_We Don_t Have Time to Read the Entire Document to Evaluatethe Q564.32 kBDownload View

Published Version Open Access CC BY-NC V4.0

Abstract

Metadata

Unstructured textual data

Data quality evaluation

Semi-structured interview

Qualitative analysis

Unstructured information artefacts, such as social media posts, news articles, and company reports, are vital for knowledge workers but often suffer from uncertain quality, making evaluation challenging. Metadata can serve as a valuable aid in assessing data quality, yet little research explores how users engage with it. This study investigates user interaction with metadata when evaluating large PDF documents. Through qualitative analysis, we examine user behaviours, cognitive processes, and trust dynamics in metadata-driven assessments. Our findings reveal how users explore, verify, and utilize metadata to assess quality, highlighting its alignment with dimensions such as relevance, completeness, and timeliness. We identify key factors influencing trust in metadata and its role as a cognitive aid in decision-making. Our results contribute to a deeper understanding of metadata’s impact on data quality assessment and inform the design of user-centred systems that support more effective decision-making in unstructured data environments.

Details

Title: “We Don’t Have Time to Read the Entire Document to Evaluate the Quality” – The Role of Metadata in Quality Evaluation of Unstructured Textual Data
Authors: Hui Zhou - The University of Queensland
Tianwa Chen - The University of Queensland
Gianluca Demartini - The University of Queensland
Marta Indulska - The University of Queensland
Shazia Sadiq - The University of Queensland
Publication details: ACIS 2025 Proceedings, pp.1-16
Conference details: Australasian Conference on Information Systems (ACIS), 36th (Sunshine Coast, Australia, 01-Dec-2025–05-Dec-2025)
Publisher: Australasian Association for Information Systems (AAIS)
Date published: 2025
Copyright note: © 2025 Hui Zhou, Tianwa Chen, Gianluca Demartini, Marta Indulska, Shazia Sadiq. This is an open-access article licensed under a Creative Commons Attribution-Non-Commercial 4.0 International License, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ACIS are credited.
Grants: PBIAS: A Principled Approach to Data Bias Management in Data Pipelines, FT240100022, Australian Research Council (Australia, Canberra) - ARC
ARC Training Centre for Information Resilience, IC200100022, Australian Research Council (Australia, Canberra) - ARC
Organisation Unit: School of Science, Technology and Engineering
Language: English
Record Identifier: 991246799702621
Output Type: Conference paper

Metrics

1 Record Views