Positive-Unlabeled Learning using Random Forests via Recursive Greedy Risk Minimization

Jonathan Wilton; Abigail Koay; Ryan K. L. Ko; Miao Xu; Nan Ye

Back

Positive-Unlabeled Learning using Random Forests via Recursive Greedy Risk Minimization

Conference paper

Peer reviewed

Positive-Unlabeled Learning using Random Forests via Recursive Greedy Risk Minimization

Jonathan Wilton, Abigail Koay, Ryan K. L. Ko, Miao Xu and Nan Ye

Advances in Neural Information Processing Systems, Vol.35, pp.24060-24071

Neural Information Processing Systems (NeurIPS) Conference, 36th (New Orleans, United States, 28-Nov-2022–09-Dec-2022)

Curran Associates, Inc.

2022

Files and links (1)

url

https://proceedings.neurips.cc/paper_files/paper/2022/file/98257285340854262185500e59bc0f28-Paper-Conference.pdfView

Published Version Open

Abstract

Computer Science - Learning

Statistics - Machine Learning

The need to learn from positive and unlabeled data, or PU learning, arises in many applications and has attracted increasing interest. While random forests are known to perform well on many tasks with positive and negative data, recent PU algorithms are generally based on deep neural networks, and the potential of tree-based PU learning is under-explored. In this paper, we propose new random forest algorithms for PU-learning. Key to our approach is a new interpretation of decision tree algorithms for positive and negative data as recursive greedy risk minimization algorithms. We extend this perspective to the PU setting to develop new decision tree learning algorithms that directly minimizes PU-data based estimators for the expected risk. This allows us to develop an efficient PU random forest algorithm, PU extra trees. Our approach features three desirable properties: it is robust to the choice of the loss function in the sense that various loss functions lead to the same decision trees; it requires little hyperparameter tuning as compared to neural network based PU learning; it supports a feature importance that directly measures a feature's contribution to risk minimization. Our algorithms demonstrate strong performance on several datasets. Our code is available at https://github.com/puetpaper/PUExtraTrees.

Details

Title: Positive-Unlabeled Learning using Random Forests via Recursive Greedy Risk Minimization
Authors: Jonathan Wilton (Corresponding Author) - The University of Queensland
Abigail Koay (Author) - The University of Queensland
Ryan K. L. Ko (Author) - The University of Queensland
Miao Xu (Author) - The University of Queensland
Nan Ye (Author) - The University of Queensland
Publication details: Advances in Neural Information Processing Systems, Vol.35, pp.24060-24071
Conference details: Neural Information Processing Systems (NeurIPS) Conference, 36th (New Orleans, United States, 28-Nov-2022–09-Dec-2022)
Publisher: Curran Associates, Inc.
Date published: 2022
Grant note: This work was funded by The University of Queensland Cyber Security Seed Funding
Organisation Unit: Healthy Ageing Research Cluster; School of Science, Technology and Engineering
Language: English
Record Identifier: 991216851402621
Output Type: Conference paper

Metrics

1 Record Views