Beauchamp:Stimuli

From OpenWetWare
Brain picture
Beauchamp Lab




Stimuli from Ozker et al

Here are the speech stimuli from

  1. Ozker et al., "Converging Evidence from Electrocorticography and BOLD fMRI for a Sharp Functional Boundary in Superior Temporal Gyrus Related to Multisensory Speech Processing". Frontiers in Human Neuroscience (Under Review)
  2. Ozker M, Schepers IM, Magnotti JF, Yoshor D, Beauchamp MS. 2017. A Double Dissociation between Anterior and Posterior Superior Temporal Gyrus for Processing Audiovisual Speech Demonstrated by Electrocorticography. J Cogn Neurosci 29: 1044-60

The stimuli are adapted from the Hoosier Audiovisual Multi-Talker Database (Sheffert, Lachs & Hernandez,1996).

Click here to download archive of stimuli

McGurk and Control Audiovisual Speech Syllables

The McGurk-MacDonald effect is an audiovisual illusion. McGurk stimuli consist of an auditory syllable and an incongruent visual syllable that produce the percept of a different syllable (e.g. auditory "ba" + visual "ga" perceived as "da"). Other incongruent stimuli (e.g. auditory "ga" + visual "ba") do not produce the effect. The archive contains movies that demonstrate the effect. If you use the stimuli in a scientific publication, please cite the appropriate reference in Beauchamp:Publications. Note that some of the videos in the archive require a particular video codec that is freely available in the VLC player, available from http://www.videolan.org/vlc/


Click here to download archive of different McGurk and control syllable stimuli from Nath, AR and Beauchamp, MS. A Neural Basis for Interindividual Differences in the McGurk Effect, a Multisensory Speech Illusion. Neuroimage. 2012 59: 781-787. Click here to download the PDF.

To see the stimuli without downloading the archive, you may view them on YouTube

  1. Auditory "ba" + Visual "ga" --> AV "da" http://www.youtube.com/watch?v=WK3T7LWIkP8
  2. Auditory "pa" + Visual "ka" --> AV "ta" http://www.youtube.com/watch?v=An5vvn-gcwA

Here are the McGurk and control stimuli from Experiment 2 in Basu Mallick D, Magnotti JF, Beauchamp MS. Variability and stability in the McGurk effect: contributions of participants, stimuli, time, and response type. Psychonomic Bulletin and Review. (2015) 22:1299–1307. DOI 10.3758/s13423-015-0817-4 Click here to download the PDF.


Localizers

AV recordings of Aesop's Fables used as localizer stimuli from Nath et al., 2012.

  1. https://www.dropbox.com/s/olhfw8vfcueyp8j/AesopsFables.zip?dl=0

Please contact MSB if the link is broken.

Single-syllable audiovisual words

Please cite

  1. Nath, AR and Beauchamp, MS. A Neural Basis for Interindividual Differences in the McGurk Effect, a Multisensory Speech Illusion. Neuroimage. 2012 59:781-787.
  2. Nath, AR, Fava EE and Beauchamp, MS. Neural Correlates of Interindividual Differences in Children's Audiovisual Speech Perception. Journal of Neuroscience. 2011 Sept 28;31(39)13963-13971.

The following is from the methods section of Nath and Beauchamp (2012):

Word stimuli for the localizer were selected from two hundred single-syllable words from the MRC Psycholinguistic Database with Brown verbal frequency of 20 to 200, imageability rating greater than 100, age of acquisition less than 7 years and Kucera-Francis written frequency greater than 80 (Wilson, M., 1988. The MRC Psycholinguistic Database: Machine Readable Dictionary, Version 2. Behav. Res. Methods Instrum. Comput. 20, 6–11.). The duration of the words ranged from 0.5 to 0.7 seconds. The total length of each video clip ranged from 1.1 to 1.8 seconds in order to start and end each video with the speaker in a neutral, mouth-closed position and to include all mouth movements from mouth opening to closing.

Here is a link to a ZIP file containing all of the words. Please contact MSB if the link is broken. https://dl.dropboxusercontent.com/u/70081110/MRC_Words_1_syl_words.zip

Unisensory Visual Eye and Mouth Stimuli from Zhu et al. (2017)

Here are the eye and mouth movements stimuli from Experiment 1 in Zhu LL, Beauchamp MS. Mouth and Voice: A Relationship between Visual and Auditory Preference in the Human Superior Temporal Sulcus. Journal of Neuroscience 8 March 2017, 37 (10) 2697-2708; DOI: https://doi.org/10.1523/JNEUROSCI.2914-16.2017. Click here to download the PDF. The stimuli were created using the H264 codec.

Unisensory Stimuli

Auditory stimuli

The stimuli were presented with a grey background on the monitor. If you would like to use any of the following stimuli please cite as: Basu Mallick, D., Magnotti, J.F. & Beauchamp, M.S. (2015, in press). Variability and stability in the McGurk effect: contributions of participants, stimuli, time, and response type.

Male Speaker

Female Speaker

McGurk Stimuli Available on YouTube

McGurk stimulus: Auditory "ba" + Visual "ga" --> AV "da" http://www.youtube.com/watch?v=WK3T7LWIkP8

McGurk stimulus: Auditory "pa" + Visual "ka" --> AV "ta" http://www.youtube.com/watch?v=An5vvn-gcwA

Bearded guy: http://www.youtube.com/watch?v=aFPtc8BVdJk

Brain Rules Book: http://www.youtube.com/watch?v=I1XWDOwH47Y&NR=1

German guy: http://www.youtube.com/watch?v=rIWrnJH2jAY

Sentence level McGurk guy: http://www.youtube.com/watch?v=DsdyE491KcM

BBC McGurk guy: http://www.youtube.com/watch?v=ypd5txtGdGw&feature=more_related

Curly-haired girl: http://www.youtube.com/watch?v=9aAmTNdkEPw&feature=related

Shelves guy: http://www.youtube.com/watch?v=lo-iBiEiLxs&feature=related

Brian Sawyer: http://www.youtube.com/watch?v=DPlxRtdzLIA&feature=more_related

Blue and Yellow Stripe Guy: http://www.youtube.com/watch?v=0bgY6AxeBeU&NR=1

Dk-haired Bearded guy: http://www.youtube.com/watch?v=5Lq26mgFpOc&feature=more_related

Guy with Glasses - words: http://www.youtube.com/watch?v=BgDhafI9n1I&feature=related

Notes on Stimulus Creation

When showed on a video projector (such as for giving a talk), some of Audrey's stimuli are very dark so that the mouth is invisible. To fix this, use iMovie to brighten the clips. The following parameters were used: 

 exposure/brightness/contrast/saturation
 156%/ 53%/ 57%/ 200%

Alternately, use the Auto button in iMovie. This sets the following 

 Levels (left slider) = 0   Levels (Right slider) = 70%


The new stimuli we recorded were edited on Final Cut Pro, cut to 2 seconds in length with the onset of the sound at the mid-time point. They were exported as .mov files.

Retrieved from "https://openwetware.org/mediawiki/index.php?title=Beauchamp:Stimuli&oldid=1038698"