cneuromod dataset is called HCP test-retest (
hcptrt), because participants repeated 15 times the functional localizers developed by the Human Connectome Project, for a total of approximately 10 hours of functional data per subject. The protocol consisted of seven tasks, described below (text adapted from the HCP protocol). Before each task, participants were given detailed instructions and examples, as well as a practice run. A session was typically composed either of two repetitions of the HCP localizers, or one resting-state run and one HCP localizer. The eprime scripts for preparation and presentation of the stimuli can be found in the HCP database. Stimuli and e-prime scripst were provided by the Human Connectome Project, U-Minn Consortium (Principal Investigators: David Van Essen and Kamil Ugurbil; 1U54MH091657) funded by the 16 NIH Institutes and Centers that support the NIH Blueprint for Neuroscience Research, and by the McDonnell Center for Systems Neuroscience at Washington University. Note that in the
cneuromod DataLad, functional runs are named
func_sub-<participant>_ses-<sess>_task-<task>_run-<run>, where the
<participant> tag includes
sub-05. For each functional run, a companion file
_events.tsv contains the timing and type of events presented to the subject. Session tags
002 etc, and the number and composition of sessions vary from subject to subject. The
<task> tags are
relational. Tasks that were repeated twice have separate
<run> tags (
gambling duration: approximately 4 minutes. Participants were asked to guess whether a hidden number (represented by a “?” during 1500ms) was above or below 5 (Delgado et al. 2000). They indicated their choice using a button press, and were then shown the actual number. If they guessed correctly they were told they won money (+$1.00,
win trial), if they guessed incorrectly they were told they lost money (-$0.50,
loss trial), and if the number was exactly 5 they were told that they neither won or lost money ($0,
neutral trial). Note that no money was actually given to the participants and, as such, this task may not be an accurate reproduction of the HCP protocol. The conditions were presented in blocks of 8 trials of type
win trials pseudo randomly interleaved with either 1
neutral and 1
loss trial, 2
neutral trials, or 2
loss trials) or of type
loss trials pseudo-randomly interleaved with either 1
neutral and 1
win trial, 2
neutral trials, or 2
win trials). There were four blocks per run (2
reward and 2
punishment), and two runs in total.
motor duration: approximately 5 minutes. This task was adapted from (Buckner et al. 2011; Yeo et al. 2011). Participants were presented a visual cue, and were asked to either tap their left or right fingers (event types
right_hand, resp.), squeeze their left or right toes (event types
right_foot, resp.), or move their tongue to map motor area (event type
tongue). Each movement lasted 12 seconds, and in total there were 13 blocks, with 2 of
tongue movements, 4 of hand movements (2
right_hand and 2
left_hand), and 4 of foot movements (2
right_foot and 2
left_foot), and three 15 second fixation blocks where participants were instructed not to move anything. There were two runs in total, and 13 blocks per run.
language duration: approximately 5 minutes. Participants were presented with two types of events. During
story events, participants listened to an auditory story (5-9 sentences, about 20 seconds), followed by a two-alternative forced-choice question. During
math events, they listened to a math problem (addition and subtraction only, varies in length), and were instructed to push a button to select the first or the second answer as being correct. The task was adaptive so that for every correct answer the level of difficulty increased. The math task was designed this way to maintain the same level of difficulty between participants. There were 2 runs, each with 4
story and 4
math blocks, interleaved.
relational duration: approximately 5 minutes. Participants were shown 6 different shapes filled with 1 of 6 different textures (Smith et al. 2007). There were two conditions: relations processing (event type
relational), and control matching condition (event type
control). In the
relational events, 2 pairs of objects were presented on the screen, with one pair at the top of the screen, and the other pair at the bottom. Participants were instructed to decide what dimension differed in the top pair (shape or texture), and then decide if the bottom pair differed, or not, on the same dimension (i.e. if the top pair differed in shape, did the bottom pair also differ in shape). Their answers were recorded by one of two button presses: “a” differ on same dimension; “b” don’t differ on same dimension. In the
control events, participants were shown two objects at the top of the screen, and one object at the bottom of the screen, with a word in the middle of the screen (either “shape” or “texture”).They were told to decide whether the bottom object matched either of the top two objects on that dimension (i.e., if the word is “shape”, did the bottom object have the same shape as either of the top two objects). Participants responded “yes” or “no” using the button box. For the
relational condition, the stimuli were presented for 3500 ms, with a 500 ms ITI, and there were four trials per block. In the
controlcondition, stimuli were presented for 2800 ms, with a 400 ms ITI, and there were 5 trials per block. In total there were two runs, each with three
relational blocks, three
control blocks and three 16-second fixation blocks.
emotion duration: approximately 4 minutes. Participants were shown triads of faces (event type
face) or shapes (event type
shape), and were asked to decide which of the shapes at the bottom of the screen matches the target face/ shape at the top of the screen (adapted from Smith et al. 2007). Faces had either an angry or fearful expression. Faces, and shapes were presented in three blocks of 6 trials (3
face and 3
shape), with each trial lasting 2 seconds, followed by a 1 second inter-stimulus interval. Each block was preceded by a 3000 ms task cue (“shape” or “face”), so that each block was 21 seconds long, including the cue. In total there were two runs, three
face blocks and three
shape blocks, with 8 seconds of fixation at the end of each run.
wm duration: approximately 5 minutes. There were two subtasks: a category specific representation, and a working memory task. Participants were presented with blocks of either places, tools, faces, and body parts. Within each run, all 4 types of stimuli were presented in block, with each block being labelled as a 2-back task (participants needed to indicate if they saw the same image two images back), or a version of a 0-back task (participants were shown a target at the start of the trial and they needed to indicate if the image that they were seeing matched the target). There were thus 8 different event types
<stim> was one of
<back> was one of
2back. Each image was presented for 2 seconds, followed by a 500 ms ITI. Stimuli were presented for 2 seconds, followed by a 500 ms inter-task interval. Each of the 2 runs included 8 event types with 10 trials per type, as well as 4 fixations blocks (15 secs).
restingstate duration: 15 minutes. In every other session, one resting-state fMRI run was acquired, giving 5 runs per participant. Participants were asked to have their eye open, be looking at fixation cross in the middle of the screen and be instructed to not fall asleep. A total of five resting-state fMRI runs were acquired per subject.
This dataset includes about 10 hours of functional data for all 6 participants. The python & psychopy scripts for preparation and presentation of the clips can be found in
src/tasks/video.py of the following github repository.
vid002 etc, and the number and composition of sessions varied from subject to subject. The
<task> tags used in DataLad corresponded to each movie (
hiddenfigures). Each movie was cut into roughly ten minutes segments (tags
02, etc) presented in a separate run. Exact cutting points were manually selected to not interrupt the narrative flow. Fade out to a black screen was added at the end of each clip, and with a few seconds overlap between the end of a clip and the beginning of the next clip. The movie segments can be found under
movie10/stimuli/<movie>/<movie>_seg<seg>.mkv, and the functional runs are named
func_sub-<participant>_ses-<sess>_task-<movie>_run-<seg>, where the
<participant> tag ranges from
sub-06. A companion file
_events.tsv contains the timing and type of conditions presented to the subject.
The participants watched the following movies (cogatlas):
bournesupremacy: The Bourne supremacy. Duration ~100 minutes.
wolfofwallstreet: The wolf of wall street. Duration ~170 minutes.
hiddenfigures: Hidden figures. Duration ~120 minutes. This movie was presented twice, for a total duration of ~240 minutes.
life: Life Disc one of four: “Challenges of life, reptiles and amphibian mammals”. DVD set was narrated by David Attenborough Duration, and lasted ~50 minutes. This movie was presented twice, for a total duration of ~100 minutes.