Image decoding

Name: image
Category: cognitive decoding
Dataset: Gifford2022Large (THINGS-EEG2)
Objective: Retrieval
Split: Predefined

Usage

neuralbench eeg image
Show config.yaml
# Copyright (c) Meta Platforms, Inc. and affiliates.
# All rights reserved.
#
# This source code is licensed under the license found in the
# LICENSE file in the root directory of this source tree.

data:
  study:
    source:
      name: Gifford2022Large
    split:
      name: PredefinedSplit
      test_split_query: null
      col_name: split
      valid_split_by: timeline
      valid_split_ratio: 0.2
      valid_random_state: 33
  neuro.baseline: [0.0, 0.2]
  target:
    name: HuggingFaceImage
    model_name: facebook/dinov2-giant
    layers: 0.6667
    token_aggregation: mean
    imsize: 518
    aggregation: trigger
    infra:
      cluster: auto
      keep_in_ram: false
      timeout_min: 180
      gpus_per_node: 1
      cpus_per_task: 10
      min_samples_per_job: 64
  trigger_event_type: Image
  start: -0.2
  duration: 1.0
  summary_columns: [category, filepath]
brain_model_output_size: &brain_model_output_size 1536
trainer_config.monitor: val/batch_top5_acc
trainer_config.mode: max
loss:
  name: ClipLoss
  norm_kind: y
  temperature: false
  symmetric: false
metrics: !!python/name:neuralbench.defaults.metrics.retrieval_metrics
test_full_retrieval_metrics: !!python/name:neuralbench.defaults.metrics.test_full_retrieval_metrics

Description

The image decoding task involves decoding visual stimuli from EEG recordings [Benchetrit2023]. In this task, we use the Gifford2022Large dataset [Gifford2022Large], which contains EEG data recorded while subjects viewed images from the THINGS database, a large-scale collection of naturalistic object images [Hebart2019]. The goal is to retrieve the presented image based on the EEG signals and a fixed pretrained image feature extractor.

Dataset Notes

  • We use [Gifford2022Large] for evaluation because test images were recorded in separate runs, limiting the potential impact of temporal correlations on decoding performance.

Additional Datasets

The following additional EEG image-decoding datasets can also be used with this task:

  • Xu2024Alljoined (Alljoined1) – 8 participants, 64-channel EEG at 512 Hz, viewing static images from the Natural Scenes Dataset (NSD) [Xu2024Alljoined].

  • Xu2025Alljoined (Alljoined-1.6M) – 20 participants viewing static images in EEG [Xu2025Alljoined]. Source: Hugging Face.

  • Grootswagers2022Human (THINGS-EEG1) – 50 participants, 64-channel EEG at 1000 Hz, viewing rapid serial visual presentation (RSVP) streams covering all 1,854 THINGS object concepts; uses the dataset’s predefined train/test split [Grootswagers2022].

To run with an alternate dataset:

neuralbench eeg image --dataset xu2024alljoined

References

[Benchetrit2023]

Benchetrit, Yohann, Hubert Banville, and Jean-Rémi King. “Brain decoding: toward real-time reconstruction of visual perception.” arXiv preprint arXiv:2310.19812 (2023).

[Gifford2022Large] (1,2)

Gifford, Alessandro T., et al. “A large and rich EEG dataset for modeling human visual object recognition.” NeuroImage 264 (2022): 119754.

[Hebart2019]

Hebart, Martin N., et al. “THINGS: A database of 1,854 object concepts and more than 26,000 naturalistic object images.” PloS one 14.10 (2019): e0223792.

[Xu2024Alljoined]

Xu, Jonathan, et al. “Alljoined – A dataset for EEG-to-Image decoding.” arXiv preprint arXiv:2404.05553 (2024).

[Xu2025Alljoined]

Xu, Jonathan, et al. “Alljoined-1.6M: A Million-Trial EEG-Image Dataset for Evaluating Affordable Brain-Computer Interfaces.” arXiv preprint arXiv:2508.18571 (2025).

[Grootswagers2022]

Grootswagers, Tijl, et al. “Human EEG recordings for 1,854 concepts presented in rapid serial visual presentation streams.” Scientific Data 9.1 (2022): 3.