English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Tree ensembles such as Random Forests have achieved impressive empiricalsuccess across a wide variety of applications. To understand how these modelsmake predictions, people routinely turn to feature importance measurescalculated from tree ensembles. It has long been known that Mean DecreaseImpurity (MDI), one of the most widely used measures of feature importance,incorrectly assigns high importance to noisy features, leading to systematicbias in feature selection. In this paper, we address the feature selection biasof MDI from both theoretical and methodological perspectives. Based on theoriginal definition of MDI by Breiman et al. for a single tree, we derive atight non-asymptotic bound on the expected bias of MDI importance of noisyfeatures, showing that deep trees have higher (expected) feature selection biasthan shallow ones. However, it is not clear how to reduce the bias of MDI usingits existing analytical expression. We derive a new analytical expression forMDI, and based on this new expression, we are able to propose a debiased MDIfeature importance measure using out-of-bag samples, called MDI-oob. For boththe simulated data and a genomic ChIP dataset, MDI-oob achievesstate-of-the-art performance in feature selection from Random Forests for bothdeep and shallow trees.

A Debiased MDI Feature Importance Measure for Random Forests