General Statistical Inference for Discrete and Mixed Spaces by An Approximate Application of the Meximum Entropy Principle  

Oleh:

Miller, D. J. ; Yan, Lian

Jenis: Article from Journal - ilmiah internasional
Dalam koleksi: IEEE Transactions on Neural Networks vol. 11 no. 3 (2000), page 558-573.
Topik: entropy; statistical; discrete; application; maximum entropy

Ketersediaan

Perpustakaan Pusat (Semanggi) Nomor Panggil: II36.4 Non-tandon: 1 (dapat dipinjam: 0) Tandon: tidak ada

Lihat Detail Induk

Isi artikel

We propose a method for learning a general statistical inference engine, operating on discrete and mixed discrete / continuous feature spaces. Such a model allows inference on any of the discrete features, given values for the remaining features. Applications are, e. g., to medical diagnosis with multiple possible diseases, fault diagnosis, information retrieval, and imputation in databases. Bayesian networks (BN s) are versatile tools that possess this inference capability. However, BNs require explicit specification of conditional independencies, which may be difficult to assess given limited data. Alternatively, Cheeseman (1983) proposed finding the maximum entropy (ME) joint probability mass function (pmf) consistent with arbitrary lower order probability constraints. This approach is in principle powerful and does not require explicit expression of conditional independence. However, until now the huge learning complexity has severely limited the use of this approach. Here we propose an approximate ME method, which also encodes arbitrary low - order constraints but while retaining quite tractable learning. Our method uses a restriction of joint pmf support (during learning) to a subset of the feature space. Results on the University of California - Irvine repository reveal performance gains over several BN approaches and over multilayer perceptrons.

Opini Anda

Klik untuk menuliskan opini Anda tentang koleksi ini!

Kembali