Logo image
A novel Sigmexd Activation Function to enhance Physics-Informed Neural Networks (PINNs) for differential equations with a perspective on the role of activation functions in PINNs
Journal article   Open access   Peer reviewed

A novel Sigmexd Activation Function to enhance Physics-Informed Neural Networks (PINNs) for differential equations with a perspective on the role of activation functions in PINNs

Manoj Kurukulasuriya, Chanaka Batuwatta-Gamage, Chaminda Karunasena, Hyogu Jeong, Kalani Ranathunga, Charith Rathnayaka and YuanTong Gu
Neurocomputing, Vol.666, pp.1-25
2026
pdf
1-s2.0-S092523122503036X-main29.24 MBDownloadView
Published Version Open Access CC BY V4.0

Abstract

Activation Functions Activation Selection Physics-Informed Neural Networks (PINNs) Sigmexd Activation Function Solving Differential Equations
Physics-Informed Neural Networks (PINNs) is a rapidly evolving machine learning technique that leverages deep learning strengths to solve physical equations. In PINNs, Activation Functions (AFs) are key to capturing nonlinearities of complex real-world problems. In addition, the inherent randomness of neural-network learning makes ‘seeding’ an important consideration for ensuring reproducibility and consistency. In this background, this investigation focuses on enhancing PINN activation to achieve reliable solutions across diverse differential equations. The key objectives include: a) Evaluating the role of different AFs and seeding arrangements in PINNs; b) Proposing a novel AF specifically suitable for PINN-based computational modeling; and c) Providing preliminary recommendations on PINN activation and seeding selection. In achieving these objectives, nine different differential-equation setups have been considered with three Ordinary Differential Equations (ODEs) and six Partial Differential Equations (PDEs). Six different AFs, including the newly proposed Sigmexd AF and a range of widely-utilized AFs, have been comprehensively compared with respect to PINNs. The Sigmexd AF demonstrates superior performance, achieving up to five-fold improvements compared to the next-best AF. The practical impact of Sigmexd has been clearly demonstrated, as a versatile AF that can consistently deliver high levels of accuracy across a wide range of linear and nonlinear differential equations of varying orders. The analyses further reveal an interesting interaction between seeding arrangements and AFs. A preliminary activation selection setup is recommended for PINN-based modeling, offering foundational-level guidance for optimal activation and seeding. To the best of the authors’ knowledge, this represents the first systematic approach of its kind.

Details

Metrics

5 Record Views

InCites Highlights

These are selected metrics from InCites Benchmarking & Analytics tool, related to this output

Collaboration types
Domestic collaboration
International collaboration
Web Of Science research areas
Computer Science, Artificial Intelligence
Logo image