A Theoretical Comparison of Batch-Mode, On-Line, Cyclic, and Almost-Cyclic Learning  

Oleh:

Heskes, T. ; Wigerinck, W.

Jenis: Article from Journal - ilmiah internasional
Dalam koleksi: IEEE Transactions on Neural Networks vol. 7 no. 4 (1996), page 919-925.
Topik: COMPARISON; theoretical comparison; batch - mode; cylic; learning

Ketersediaan

Perpustakaan Pusat (Semanggi) Nomor Panggil: II36.1 Non-tandon: 1 (dapat dipinjam: 0) Tandon: tidak ada

Lihat Detail Induk

Isi artikel

We study and compare different neural network learning strategies : batch - mode learning, online learning, cyclic learning, and almost - cyclic learning. Incremental learning strategies require less storage capacity than batch - mode learning. However, due to the arbitrariness in the presentation order of the training patterns, incremental learning is a stochastic process ; whereas batch - mode learning is deterministic. In zeroth order, i. e., as the learning parameter ? tends to zero, all learning strategies approximate the same ordinary differential equation for convenience referred to as the “ideal behaviour”. Using stochastic methods valid for small learning parameters ?, we derive differential equations describing the evolution of the lowest - order deviations from this ideal behaviour. We compute how the asymptotic misadjustment, measuring the average asymptotic distance from a stable fixed point of the ideal behavior, scales as a function of the learning parameter and the number of training patterns. Knowing the asymptotic misadjustment, we calculate the typical number of learning steps necessary to generate a weight within order &epsiv ; of this fixed point, both with fixed and time - dependent learning parameters. We conclude that almost - cyclic learning (learning with random cycles) is a better alternative for batch - mode learning than cyclic learning (learning with a fixed cycle).

Opini Anda

Klik untuk menuliskan opini Anda tentang koleksi ini!

Kembali