J. Japan Statist. Soc., Vol. 34 (No. 2), pp. 207-227, 2004

Logistic Discrimination based on regularized local likelihood method

Yoshisuke Nonaka and Sadanori Konishi

Abstract. We consider the problem of constructing a nonlinear discriminant procedure, using a regularized local likelihood method. The local likelihood method is effective for analyzing data with complex structure and applicable to discriminant analysis within the framework of logistic regression. The stability of the local likelihood estimators, however, is not guaranteed in the case that the structure of the system is quite complex. Instability of the local likelihood estimators may affect the construction of the discriminant boundary region. In order to overcome this difficulty, we propose the regularized local likelihood method which unites local likelihood and regularization.
A crucial issue in constructing nonlinear discriminant models is a choice of smoothing parameter and regularization parameter. In order to evaluate constructed models estimated by the regularized local likelihood method, we derive a model selection criterion from an information-theoretic point of view. We apply our discriminant procedure to real data. The results show that our technique performs well in the sense of minimizing the test error, and that clear improvements are achieved by employing the regularization.

Key words and phrases: Discriminant analysis, local likelihood, logistic regression, model selection, regularization.

[Full text] (PDF 664 KB)