Situation Awareness in AI-based Technologies and Multimodal Systems: Architectures, Challenges and Applications

Jieli Chen; Kah Phooi Seng; Jeremy Smith; Li Minn Ang

doi:10.1109/ACCESS.2024.3416370

Back

Situation Awareness in AI-based Technologies and Multimodal Systems: Architectures, Challenges and Applications

Journal article

Open access

Peer reviewed

Situation Awareness in AI-based Technologies and Multimodal Systems: Architectures, Challenges and Applications

Jieli Chen, Kah Phooi Seng, Jeremy Smith and Li Minn Ang

IEEE Access, Vol.12, pp.88779-88818

2024

DOI: https://doi.org/10.1109/ACCESS.2024.3416370

Files and links (1)

pdf

Situation_Awareness_in_AI-Based_Technologies_and_Multimodal_Systems_Architectures_Challenges_and_Applications3.48 MBDownload View

Published VersionCC BY-NC-ND V4.0, Open Access

Abstract

Artificial intelligence

situation awareness

deep learning

machine learning

reinforcement learning

multimodal fusion

Situation Awareness (SA) is a process of sensing, understanding and predicting the environment and is an important component in complex systems. The reception of information from the environment tends to be continuous and of a multimodal nature. AI technologies provide a more efficient and robust support by subdividing the different stages of SA objectives into tasks such as data fusion, representation, classification, and prediction. This paper provides an overview of AI and multimodal methods used to build, enhance and evaluate SA in a variety of environments and applications. Emphasis is placed on enhancing perceptual integrity and persistence. Research indicates that the integration of artificial intelligence and multimodal approaches has significantly enhanced perception and comprehension in complex systems. However, there remains a research gap in projecting future situations and effectively fusing multimodal information. This paper summarizes some of the use cases and lessons learned where AI and multimodal techniques have been used to deliver SA. Future perspectives and challenges are proposed, including more comprehensive predictions, greater interpretability, and more advanced visual information.

Details

Title: Situation Awareness in AI-based Technologies and Multimodal Systems: Architectures, Challenges and Applications
Authors: Jieli Chen - Xi’an Jiaotong-Liverpool University
Kah Phooi Seng - University of the Sunshine Coast, Queensland, School of Science, Technology and Engineering
Jeremy Smith - University of Liverpool
Li Minn Ang - University of the Sunshine Coast, Queensland, School of Science, Technology and Engineering
Publication details: IEEE Access, Vol.12, pp.88779-88818
Publisher: Institute of Electrical and Electronics Engineers
Date published: 2024
DOI: 10.1109/ACCESS.2024.3416370
ISSN: 2169-3536
Copyright note: (c) 2024 The Authors. This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
Organisation Unit: School of Science, Technology and Engineering
Language: English
Record Identifier: 991043598002621
Output Type: Journal article

Metrics

568 File views/ downloads

30 Record Views