Subspace Information Criterion for Nonquadratic Regularizers-Model Selection for Sparse Regressors  

Oleh:

Tsuda, K. ; Sugiyama, M. ; Miller, K.-R.

Jenis: Article from Journal - ilmiah internasional
Dalam koleksi: IEEE Transactions on Neural Networks vol. 13 no. 1 (2002), page 70-80.
Topik: models and modeling; subspace information; non quadratic; sparse regressors

Ketersediaan

Perpustakaan Pusat (Semanggi) Nomor Panggil: II36.6 Non-tandon: 1 (dapat dipinjam: 0) Tandon: tidak ada

Lihat Detail Induk

Isi artikel

Nonquadratic regularizers, in particular the l(1) norm regularizer can yield sparse solutions that generalize well. In this work we propose the generalized subspace information criterion (GSIC) that allows to predict the generalization error for this useful family of regularizers. We show that under some technical assumptions GSIC is an asymptotically unbiased estimator of the generalization error. GSIC is demonstrated to have a good performance in experiments with the l1 norm regularizer as we compare with the network information criterion (NIC) and cross - validation in relatively large sample cases. However in the small sample case, GSIC tends to fail to capture the optimal model due to its large variance. Therefore, also a biased version of GSIC is introduced,which achieves reliable model selection in the relevant and challenging scenario of high - dimensional data and few samples.

Opini Anda

Klik untuk menuliskan opini Anda tentang koleksi ini!

Kembali