Bayesian Error Estimation And Model Selection In Sparse Logistic Regression
Heikki Huttunen, Tapio Manninen, Jussi Tohka

Abstract:
Regularized logistic regression models have recently become an important
classification tool for high dimensional problems due to their sparseness
and embedded feature selection property of the L1 penalty.
However, the degree of sparseness is determined by a regularization parameter
lambda, whose selection is typically done by cross validation.
In this paper we study the applicability of a recently proposed Bayesian error
estimation approach for the selection of a proper model along the regularization
path. The model selection by the new Bayesian error estimator is experimentally shown to improve the classification accuracy in small sample-size situations, and is able to avoid the excess variability inherent to traditional cross validation approaches.