Storyboard

Multisensory fusion combines information from multiple channels (vision + proprioception + vestibular, for example) to produce more accurate estimates of the state of the world than any single channel. The optimal Bayesian framework states that optimal fusion weights each signal inversely proportional to its variance.

The precision gain with N identical sensors is N: two sensors with ² each produce ²/2 when merged. This is the basis of multisensory integration: even if two modalities measure the same thing with different precision, it is always optimal to combine them. The human nervous system implements near-optimal fusion in visuo-haptic position estimation tasks.

The problem of causality is crucial: do the signals of two modalities come from the same source (common cause) or from different sources? Fusion is only beneficial if C = 1 (common cause). The causal Bayesian model estimates P(C=1|d,d) and weights between full fusion (C=1) and independent processing (C=0). This model explains ventriloquism (spatially displaced common cause illusion) and its decomposition with great disparities.

The McGurk effect is the best-known demonstration of obligatory audio-visual fusion: hearing /ba/ while watching /ga/ be articulated produces the perception of /da/. The Bayesian model shows that the percept is the phoneme that maximizes the joint audio-visual verisimilitude: neither of the two signals alone is sufficient to explain the result, and the fusion is more consistent than either individual channel.

The Kalman gain K_fus = P/(P+R) determines how much to trust the model prediction (P) versus the new measurement (R): if R is small (accurate sensor), K_fus 1 (trust the sensor); if P is small (accurate model), K_fus 0 (trust the prediction). This adaptive weighting is the central principle of all inertial navigation systems and is implemented by the cerebellum in sensorimotor coordination.

>Model

ID:('ky', 587)

Description

ID:(0, 587)