Datasets

hcptrt

This cneuromod dataset is called HCP test-retest (hcptrt), because participants repeated 15 times the functional localizers developed by the Human Connectome Project, for a total of approximately 10 hours of functional data per subject. The protocol consisted of seven tasks, described below (text adapted from the HCP protocol). Before each task, participants were given detailed instructions and examples, as well as a practice run. A session was typically composed either of two repetitions of the HCP localizers, or one resting-state run and one HCP localizer. The eprime scripts for preparation and presentation of the stimuli can be found in the HCP database. Stimuli and e-prime scripst were provided by the Human Connectome Project, U-Minn Consortium (Principal Investigators: David Van Essen and Kamil Ugurbil; 1U54MH091657) funded by the 16 NIH Institutes and Centers that support the NIH Blueprint for Neuroscience Research, and by the McDonnell Center for Systems Neuroscience at Washington University. Note that in the cneuromod DataLad, functional runs are named func_sub-<participant>_ses-<sess>_task-<task>_run-<run>, where the <participant> tag includes sub-01, sub-02, sub-03 and sub-05. For each functional run, a companion file _events.tsv contains the timing and type of events presented to the subject. Session tags <sess> are 001, 002 etc, and the number and composition of sessions vary from subject to subject. The <task> tags are restingstate, gambling, motor, social, wm, emotion, language and relational. Tasks that were repeated twice have separate <run> tags (01, 02).

Gambling

gambling duration: approximately 4 minutes. Participants were asked to guess whether a hidden number (represented by a “?” during 1500ms) was above or below 5 (Delgado et al. 2000). They indicated their choice using a button press, and were then shown the actual number. If they guessed correctly they were told they won money (+$1.00, win trial), if they guessed incorrectly they were told they lost money (-$0.50, loss trial), and if the number was exactly 5 they were told that they neither won or lost money ($0, neutral trial). Note that no money was actually given to the participants and, as such, this task may not be an accurate reproduction of the HCP protocol. The conditions were presented in blocks of 8 trials of type reward (6 win trials pseudo randomly interleaved with either 1 neutral and 1 loss trial, 2 neutral trials, or 2 loss trials) or of type punishment (6 loss trials pseudo-randomly interleaved with either 1 neutral and 1 win trial, 2 neutral trials, or 2 win trials). There were four blocks per run (2 reward and 2 punishment), and two runs in total.

Motor

motor duration: approximately 5 minutes. This task was adapted from (Buckner et al. 2011; Yeo et al. 2011). Participants were presented a visual cue, and were asked to either tap their left or right fingers (event types left_hand and right_hand, resp.), squeeze their left or right toes (event types left_foot and right_foot, resp.), or move their tongue to map motor area (event type tongue). Each movement lasted 12 seconds, and in total there were 13 blocks, with 2 of tongue movements, 4 of hand movements (2 right_hand and 2 left_hand), and 4 of foot movements (2 right_foot and 2 left_foot), and three 15 second fixation blocks where participants were instructed not to move anything. There were two runs in total, and 13 blocks per run.

Language processing

language duration: approximately 5 minutes. Participants were presented with two types of events. During story events, participants listened to an auditory story (5-9 sentences, about 20 seconds), followed by a two-alternative forced-choice question. During math events, they listened to a math problem (addition and subtraction only, varies in length), and were instructed to push a button to select the first or the second answer as being correct. The task was adaptive so that for every correct answer the level of difficulty increased. The math task was designed this way to maintain the same level of difficulty between participants. There were 2 runs, each with 4 story and 4 math blocks, interleaved.

Social cognition

social duration: approximately 5 minutes. Participants were presented with short video clips (20 seconds) of objects (squares, circles, triangles) that either interacted in some way (event type mental), or moved randomly on the screen (event type random) (Castelli et al. 2000; Wheatley et al. 2007). Following each clip, participants were asked to judge whether the objects had a “Mental interaction” (an interaction that appeared as if the shapes were taking into account each other’s feelings and thoughts), whether the were “Not Sure”, or if there was “No interaction”. Button presses were used to record their responses. In each of the two runs, participants viewed 5 mental videos and 5 random videos, and had 5 fixation blocks of 15 seconds each.

Relational processing

relational duration: approximately 5 minutes. Participants were shown 6 different shapes filled with 1 of 6 different textures (Smith et al. 2007). There were two conditions: relations processing (event type relational), and control matching condition (event type control). In the relational events, 2 pairs of objects were presented on the screen, with one pair at the top of the screen, and the other pair at the bottom. Participants were instructed to decide what dimension differed in the top pair (shape or texture), and then decide if the bottom pair differed, or not, on the same dimension (i.e. if the top pair differed in shape, did the bottom pair also differ in shape). Their answers were recorded by one of two button presses: “a” differ on same dimension; “b” don’t differ on same dimension. In the control events, participants were shown two objects at the top of the screen, and one object at the bottom of the screen, with a word in the middle of the screen (either “shape” or “texture”).They were told to decide whether the bottom object matched either of the top two objects on that dimension (i.e., if the word is “shape”, did the bottom object have the same shape as either of the top two objects). Participants responded “yes” or “no” using the button box. For the relational condition, the stimuli were presented for 3500 ms, with a 500 ms ITI, and there were four trials per block. In the controlcondition, stimuli were presented for 2800 ms, with a 400 ms ITI, and there were 5 trials per block. In total there were two runs, each with three relational blocks, three control blocks and three 16-second fixation blocks.

Emotion processing

emotion duration: approximately 4 minutes. Participants were shown triads of faces (event type face) or shapes (event type shape), and were asked to decide which of the shapes at the bottom of the screen matches the target face/ shape at the top of the screen (adapted from Smith et al. 2007). Faces had either an angry or fearful expression. Faces, and shapes were presented in three blocks of 6 trials (3 face and 3 shape), with each trial lasting 2 seconds, followed by a 1 second inter-stimulus interval. Each block was preceded by a 3000 ms task cue (“shape” or “face”), so that each block was 21 seconds long, including the cue. In total there were two runs, three face blocks and three shape blocks, with 8 seconds of fixation at the end of each run.

Working memory

wm duration: approximately 5 minutes. There were two subtasks: a category specific representation, and a working memory task. Participants were presented with blocks of either places, tools, faces, and body parts. Within each run, all 4 types of stimuli were presented in block, with each block being labelled as a 2-back task (participants needed to indicate if they saw the same image two images back), or a version of a 0-back task (participants were shown a target at the start of the trial and they needed to indicate if the image that they were seeing matched the target). There were thus 8 different event types <stim>_<back>, where <stim> was one of place, tools, face or body, and <back> was one of 0back or 2back. Each image was presented for 2 seconds, followed by a 500 ms ITI. Stimuli were presented for 2 seconds, followed by a 500 ms inter-task interval. Each of the 2 runs included 8 event types with 10 trials per type, as well as 4 fixations blocks (15 secs).

Resting state

restingstate duration: 15 minutes. In every other session, one resting-state fMRI run was acquired, giving 5 runs per participant. Participants were asked to have their eye open, be looking at fixation cross in the middle of the screen and be instructed to not fall asleep. A total of five resting-state fMRI runs were acquired per subject.

movie10

This dataset includes about 10 hours of functional data for all 6 participants. The python & psychopy scripts for preparation and presentation of the clips can be found in src/tasks/video.py of the following github repository. Session tags <sess> were vid001, vid002 etc, and the number and composition of sessions varied from subject to subject. The <task> tags used in DataLad corresponded to each movie (bournesupremacy, wolfofwallstreet, life, hiddenfigures). Each movie was cut into roughly ten minutes segments (tags <seg> in 01, 02, etc) presented in a separate run. Exact cutting points were manually selected to not interrupt the narrative flow. Fade out to a black screen was added at the end of each clip, and with a few seconds overlap between the end of a clip and the beginning of the next clip. The movie segments can be found under movie10/stimuli/<movie>/<movie>_seg<seg>.mkv, and the functional runs are named func_sub-<participant>_ses-<sess>_task-<movie>_run-<seg>, where the <participant> tag ranges from sub-01 to sub-06. A companion file _events.tsv contains the timing and type of conditions presented to the subject.

The participants watched the following movies (cogatlas):

  • <task> name bournesupremacy: The Bourne supremacy. Duration ~100 minutes.
  • <task> name wolfofwallstreet: The wolf of wall street. Duration ~170 minutes.
  • <task> name hiddenfigures: Hidden figures. Duration ~120 minutes. This movie was presented twice, for a total duration of ~240 minutes.
  • <task> name life: Life Disc one of four: “Challenges of life, reptiles and amphibian mammals”. DVD set was narrated by David Attenborough Duration, and lasted ~50 minutes. This movie was presented twice, for a total duration of ~100 minutes.