← Back to API reference

neuraltrain.models.fmri_mlp.FmriMlp

pydantic model neuraltrain.models.fmri_mlp.FmriMlp

Residual MLP for fMRI decoding, adapted from MindEye [1].

Parameters:

• hidden (int) – Hidden dimension of the residual MLP blocks.

• n_blocks (int) – Number of residual MLP blocks.

• norm_type ({"ln", "bn"}) – Normalization layer type: "ln" for LayerNorm, "bn" for BatchNorm.

• act_first (bool) – If True, apply activation before normalization in each block.

• n_repetition_times (int) – Number of fMRI repetition times (TRs) expected in the input.

• time_agg ({"in_mean", "in_linear", "out_mean", "out_linear"}) – Temporal aggregation strategy. "in_*" aggregates before the MLP; "out_*" aggregates after.

• use_tr_embeds (bool) – Add learned per-TR embeddings to the input features.

• tr_embed_dim (int) – Dimensionality of each TR embedding.

• use_tr_layer (bool) – Use a grouped 1-D convolution (one group per TR) as the first projection instead of a shared nn.Linear.

• out_dim (int or None) – Explicit output dimension. Overridden by n_outputs at build time.

• subject_layers_config (SubjectLayers or None) – If set, prepend a per-subject linear projection.

• subject_layers_dim ({"input", "hidden"}) – Dimension used for the subject-layer projection.

• output_head_config (Mlp, dict[str, Mlp], or None) – Optional output head(s) appended after the MLP backbone.

References

[1]

Scotti, Paul, et al. “Reconstructing the mind’s eye: fMRI-to-image with contrastive learning and diffusion priors.” NeurIPS 2024.

Fields:

• act_first (bool)

• hidden (int)

• n_blocks (int)

• n_repetition_times (int)

• norm_type (str)

• out_dim (int | None)

• output_head_config (neuraltrain.models.common.Mlp | dict[str, neuraltrain.models.common.Mlp] | None)

• subject_layers_config (neuraltrain.models.common.SubjectLayers | None)

• subject_layers_dim (Literal['input', 'hidden'])

• time_agg (Literal['in_mean', 'in_linear', 'out_mean', 'out_linear'])

• tr_embed_dim (int)

• use_tr_embeds (bool)

• use_tr_layer (bool)

field hidden: int = 4096

field n_blocks: int = 4

field norm_type: str = 'ln'

field act_first: bool = False

field n_repetition_times: int = 1

field time_agg: Literal['in_mean', 'in_linear', 'out_mean', 'out_linear'] = 'out_linear'

field use_tr_embeds: bool = False

field tr_embed_dim: int = 16

field use_tr_layer: bool = False

field out_dim: int | None = None

field subject_layers_config: SubjectLayers | None = None

field subject_layers_dim: Literal['input', 'hidden'] = 'hidden'

field output_head_config: Mlp | dict[str, Mlp] | None = None

build(n_in_channels: int, n_outputs: int | None) → Module

← Back to API reference