Neural Models of Bayesian Belief Propagation Rajesh ... - Washington

More documents

Recommendations

Info

256 11 Neural Models of Bayesian Belief Propagation Rajesh P. N. Rao inference method known as particle filtering. The model is similar to our approach in that inference involves message passing between different levels, but whereas the particle-filtering method assumes continuous random variables, our approach uses discrete random variables. The latter choice allows a concrete model for neural representation and processing of probabilities, while it is unclear how a biologically plausible network of neurons can implement the different components of the particle filtering algorithm. The hierarchical model for attention described in section 11.4.3 bears some similarities to a recent Bayesian model proposed by Yu and Dayan [47] (see also [7, 32]). Yu and Dayan use a five-layer neural architecture and a log probability encoding scheme as in [30] to model reaction time effects and multiplicative response modulation. Their model, however, does not use an intermediate representation to factor input images into separate feature and location attributes. It therefore cannot explain effects such as the influence of attention on neighboring unattended locations [4]. A number of other neural models exist for attention, e.g., models by Grossberg and colleagues [13, 14], that are much more detailed in specifying how various components of the model fit with cortical architecture and circuitry. The approach presented in this chapter may be viewed as a first step toward bridging the gap between detailed neural models and more abstract Bayesian theories of perception. 11.5.2 Open Problems and Future Challenges An important open problem not addressed in this chapter is learning and adaptation. How are the various conditional probability distributions in a graphical model learned by a network implementing Bayesian inference? For instance, in the case of the HMM model used in section 11.4.1, how can the transition probabilities between states from one time step to the next be learned? Can well-known biologically plausible learning rules such as Hebbian learning or the Bienenstock-Cooper-Munro (BCM) rule (e.g., see [5]) be used to learn conditional probabilities? What are the implications of spike-timing dependent plasticity (STDP) and short-term plasticity on probabilistic representations in neural populations? A second open question is the use of spikes in probabilistic representations. The models described above were based directly or indirectly on instantaneous firing rates. Even the noisy spiking model proposed in section 11.3.3 can be regarded as encoding posterior probabilities in terms of instantaneous firing rates. Spikes in this model are used only as a mechanism for communicating information about firing rate over long distances. An intriguing alternate possibility that is worth exploring is whether probability distributions can be encoded using spike timing-based codes. Such codes may be intimately linked to timing-based learning mechanisms such as STDP. Another interesting issue is how the dendritic nonlinearities known to exist in cortical neurons could be exploited to implement belief propagation as in equation (11.19). This could be studied systematically with a biophysical compartmental model of a cortical neuron by varying the distribution and densities
11.5 Discussion 257 of various ionic channels along the dendrites. Finally, this chapter explored neural implementations of Bayesian inference in only two simple graphical models (HMMs and a three-level hierarchical model). Neuroanatomical data gathered over the past several decades provide a rich set of clues regarding the types of graphical models implicit in brain structure. For instance, the fact that visual processing in the primate brain involves two hierarchical but interconnected pathways devoted to spatial and object vision (the “what” and “where” streams) [24] suggests a multilevel graphical model wherein the input image is factored into progressively complex sets of object features and their transformations. Similarly, the existence of multimodal areas in the inferotemporal cortex suggests graphical models that incorporate a common modality-independent representation at the highest level that is causally related to modality-dependent representations at lower levels. Exploring such graphical models that are inspired by neurobiology could not only shed new light on brain function but also furnish novel architectures for solving fundamental problems in machine vision and robotics. Acknowledgments This work was supported by grants from the ONR Adaptive Neural Systems program, NSF, NGA, the Sloan Foundation, and the Packard Foundation. References [1] Anderson CH (1995) Unifying perspectives on neuronal codes and processing. In 19th International Workshop on Condensed Matter Theories. Caracas, Venezuela. [2] Anderson CH, Van Essen DC (1994) Neurobiological computational systems. In Zurada JM, Marks II RJ, Robinson CJ, eds., Computational Intelligence: Imitating Life, pages 213–222, New York: IEEE Press. [3] Carpenter RHS, Williams MLL (1995) Neural computation of log likelihood in control of saccadic eye movements. Nature, 377:59–62. [4] Connor CE, Preddie DC, Gallant JL, Van Essen DC (1997) Spatial attention effects in macaque area V4. Journal of Neuroscience, 17:3201–3214. [5] Dayan P, Abbott LF (2001) Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems. Cambridge, MA: MIT Press. [6] Dayan P, Hinton G, Neal R, Zemel R (1995) The Helmholtz machine. Neural Computation, 7:889–904. [7] Dayan P, Zemel R (1999) Statistical models and sensory attention. In Willshaw D, Murray A, eds., Proceedings of the International Conference on Artificial Neural Networks (ICANN), pages 1017–1022, London: IEEE Press. [8] Deneve S (2005) Bayesian inference in spiking neurons. In Saul LK, Weiss Y, Bottou L, eds., Advances in Neural Information Processing Systems 17, pages 353–360, Cambridge, MA: MIT Press. [9] Eliasmith C, Anderson CH (2003) Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems. Cambridge, MA: MIT Press.
Page 1 and 2: 11 Neural Models of Bayesian Belief
Page 3 and 4: 11.2 Bayesian Inference through Bel
Page 5 and 6: 11.2 Bayesian Inference through Bel
Page 7 and 8: 11.3 Neural Implementations of Beli
Page 9 and 10: 11.3 Neural Implementations of Beli
Page 11 and 12: 11.4 Results 245 deviation σ. Figu
Page 13 and 14: 11.4 Results 247 Nonlinear Spiking
Page 15 and 16: 11.4 Results 249 again leaky-integr
Page 17 and 18: 11.4 Results 251 a high value for P
Page 19 and 20: 11.5 Discussion 253 an evidence acc
Page 21: 11.5 Discussion 255 the neural popu
Page 25 and 26: 11.5 Discussion 259 [28] Pouget A,
Page 27 and 28: 11.5 Discussion 261 50% 100% D 0.05
Page 29 and 30: 11.5 Discussion 263 Normalized Firi
Page 31 and 32: Index acausal, 122 attention, 101 a
Page 33: Index 319 regularization, 11 revers

Neural Models of Bayesian Belief Propagation Rajesh ... - Washington

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?