Miyawaki, Y., Uchida, H., Yamashita, O., Sato, M. ., Morito, Y., Tanabe, H. C., Sadato, N., ... Kamitani, Y. (January 01, 2008). Visual Image Reconstruction from Human Brain Activity using a Combination of Multiscale Local Image Decoders. Neuron Cambridge Ma-, 60, 5, 915-929.
[...]
In the present study, we attempted to reconstruct visual images defined by binary contrast patterns consisting of 10 × 10 square patches (Figure 1). Given fMRI signals, we modeled a reconstruction image by a linear combination of local image bases (Olshausen and Field, 1996).
[...]
We have shown that contrast-defined arbitrary visual images can be reconstructed from fMRI signals of the human visual cortex on a single trial basis. By combining the outputs of local decoders that predicted local contrasts of multiple scales, we were able to reconstruct a large variety of images (2100 possible images) using only several hundred random images to train the reconstruction model. Analyses revealed that both the multivoxel and the multiscale aspects of our method were essential to achieve the high accuracy. Our automatic method for identifying relevant neural signals uncovered information represented in correlated activity patterns, going beyond mere exploitation of known functional anatomy.
Although our primary purpose was to reconstruct visual images from brain activity, we also performed image identification analysis to quantify the accuracy (Figure 3). Analysis showed that nearly 100% correct identification was possible with a hundred image candidates and that >10% performance could be achieved even with image sets of 107.4–1010.8 using 6 block-averaged data, and with image sets of 105.8–108.5 using 2 single-volume data.
[...]
However, experiments have shown that neural and behavioral responses to a localized visual stimulus are affected by surrounding stimuli. Such phenomena known as contextual effects (Kapadia et al., 1995, Meng et al., 2005, Sasaki and Watanabe, 2004, Zipser et al., 1996) could compromise the linearity assumption. However, the random patterns rarely contain specific configurations inducing contextual effects enough to bias the training of the local decoders. Thus, the influences from contextual effects may be negligible, and predictions from local decoders are largely based on fMRI signals corresponding to the local state of the visual stimulus.
[...]
In the present study, we attempted to reconstruct visual images defined by binary contrast patterns consisting of 10 × 10 square patches (Figure 1). Given fMRI signals, we modeled a reconstruction image by a linear combination of local image bases (Olshausen and Field, 1996).
[...]
We have shown that contrast-defined arbitrary visual images can be reconstructed from fMRI signals of the human visual cortex on a single trial basis. By combining the outputs of local decoders that predicted local contrasts of multiple scales, we were able to reconstruct a large variety of images (2100 possible images) using only several hundred random images to train the reconstruction model. Analyses revealed that both the multivoxel and the multiscale aspects of our method were essential to achieve the high accuracy. Our automatic method for identifying relevant neural signals uncovered information represented in correlated activity patterns, going beyond mere exploitation of known functional anatomy.
Although our primary purpose was to reconstruct visual images from brain activity, we also performed image identification analysis to quantify the accuracy (Figure 3). Analysis showed that nearly 100% correct identification was possible with a hundred image candidates and that >10% performance could be achieved even with image sets of 107.4–1010.8 using 6 block-averaged data, and with image sets of 105.8–108.5 using 2 single-volume data.
[...]
However, experiments have shown that neural and behavioral responses to a localized visual stimulus are affected by surrounding stimuli. Such phenomena known as contextual effects (Kapadia et al., 1995, Meng et al., 2005, Sasaki and Watanabe, 2004, Zipser et al., 1996) could compromise the linearity assumption. However, the random patterns rarely contain specific configurations inducing contextual effects enough to bias the training of the local decoders. Thus, the influences from contextual effects may be negligible, and predictions from local decoders are largely based on fMRI signals corresponding to the local state of the visual stimulus.
No comments:
Post a Comment