← Back to API reference

neuralbench.check_model

neuralbench.check_model(model: Module, device: str, task: str | list[str], *, dataset: str | list[str] | None = None, overrides: Mapping[str, Any] | None = None, batch_size: int = 2) → DataFrame

Push synthetic batches of a selection’s shapes through model.

One instance runs the whole selection, so it has to survive every window length and montage width in it. Reads only YAML, so it takes seconds and needs no downloaded data – the alternative being to discover a shape bug an hour into a real run.

Returns one row per (task, window, channel count) with status either "ok" or the exception that would have surfaced during the run. Raises instead when forward cannot accept channel positions at all, since that is a property of the model rather than of any one task.

The temporal width is computed from the task window and sampling rate and may be off by a sample, and the channel counts are probes rather than the widths a dataset will actually emit. So this catches a model that cannot handle a task’s shapes at all, not an exact match with the dataloader. Likewise the synthetic positions are random and the ch_names given to a model that asks for them are 10-05 names: enough to exercise the shapes, not a montage any dataset has.

← Back to API reference