Feature Importance Ranking Via Out Of Bag Permutation

via “feature importance ranking via out-of-bag permutation”

* 🏆 2001: [A fast and elitist multiobjective genetic algorithm (NSGA-II)](https://ieeexplore.ieee.org/abstract/document/996017)

Unique: Uses out-of-bag samples (data naturally held out during bootstrap training) to compute importance without requiring a separate validation set, and measures importance via prediction accuracy drop rather than split-based Gini/entropy metrics — this approach captures feature interactions and is more robust to feature scaling

vs others: More computationally efficient than SHAP for tabular data and does not require retraining, while being more interpretable than gradient-based feature importance because it directly measures prediction impact

via “feature importance ranking via impurity reduction”

* 🏆 1989: [A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition (HMM)](https://ieeexplore.ieee.org/abstract/document/18626)

Unique: CART's impurity-reduction-based importance is computationally efficient (O(n_nodes)) and directly tied to the tree's decision logic, making it interpretable. Unlike permutation importance (which requires retraining) or SHAP values (which require complex game-theoretic calculations), it is built into the tree structure itself.

vs others: Faster to compute than permutation importance or SHAP; more directly interpretable than model-agnostic methods because it reflects actual splits; less robust to feature correlations than permutation importance, which accounts for feature interactions