API Reference

Core

BaseModel	Base pydantic model with extra=forbid and nicer print
Step	Base class for composable pipeline nodes.
Chain	A sequence of processing steps executed in order.
Frequency	Sampling rate in Hz, with helpers for second/sample conversion.
TimedArray	Numpy array annotated with time metadata.

Studies

Study	Interface to an external dataset: loads events from raw recordings.
SpecialLoader	Loader for special methods that need to be serialized and called later.
StudyInfo	Records expected dataset characteristics for testing and validation.

Events

Action	Event indicating the participant performed (or imagined) an action.
Artifact	Artifact or noise event in neural recordings.
Audio	Audio event corresponding to a WAV file.
Background	'Background' activity, i.e. no specific epilepsy or arousal event happening during this time.
BaseDataEvent	Base class for events whose data needs to be read from a file.
BaseSplittableEvent	Base class for dynamic events (audio and video) which can be read in parts.
BaseText	Base class for text-based events.
CategoricalEvent	Base class for categorical events.
Eeg	Electroencephalography (EEG) recording event.
Emg	Electromyography (EMG) recording event.
Event	Base class for all event types.
EyeState	Eye state event indicating whether eyes are open or closed.
Fmri	Functional MRI (fMRI) recording event.
Fnirs	Functional Near-Infrared Spectroscopy (fNIRS) recording event.
Ieeg	Intracranial EEG (iEEG) event.
Image	Image event with optional caption.
Keystroke	Keystroke event with associated text.
Meg	Magnetoencephalography (MEG) recording event.
MneRaw	Brain recording saved as MNE Raw object.
Phoneme	Single phoneme event.
Seizure	Seizure event with specific seizure type classification.
Sentence	Single sentence text event with optional context information.
SleepStage	Sleep stage event following AASM manual classification [sleep1].
Spikes	Spikes recording event saved as HDF5 object.
Stimulus	General stimulus presentation event identified by a code.
Text	General text event, possibly containing multiple sentences.
Video	Video event with support for partial loading.
Word	Single word event with optional sentence information.

Transforms

AlignEvents	Creates timelines where events (eg: Meg, FMRI) are aligned to a trigger (eg: Image / Word)
ConfigureEventLoader	Modifies loading parameters for events with dynamic filepaths, i.e., URIs and JSON SpecialLoader jsons.
CreateColumn	Create a new column with a default value, and optionally update selected rows via a query.
QueryEvents	Filter events based on a pandas query, with auto-generated index columns.
RemoveMissing	Remove events of specified type(s) that have missing or empty values in a given field.
SelectIdx	Select a subset of events based on unique values in a column.

EnsureTexts	Create Text events from Words if not already present.
AddConcatenationContext	Adds contextual information to events by concatenating previous events of the same type.
AddContextToWords	Add a context field to the events dataframe, for each word event, by concatenating the sentence fields.
AddPartOfSpeech	Add Part-Of-Speech (POS) tags to events.
AddPhonemes	Add phoneme information to events.
AddSentenceToWords	Adds sentence-level information to word events based on Text rows.
AddSummary	Generate concise summaries for Text events using a pretrained language model.

AssignKSplits	Assign k splits to events.
AssignSentenceSplit	Assign a train/val/test split to each sentence in a deterministic fashion.
AssignWordSplitAndContext	Sequence of 3 standard operations: add sentences, assign split, add context.
SklearnSplit	Perform train/val/test split using sklearn's train_test_split.

ExtractAudioFromVideo

Extract audio tracks from Video events and add them as separate Audio events.

ChunkEvents

This functions chunks long events (audio or video) into shorter events.

Extractors — Base

BaseExtractor	Base class for extracting features from events within a Segment.
BaseStatic	Base class for extractors that produce one feature vector per event.

Extractors — General

EventDetector	Extracts time-aligned extractors from event annotations.
EventField	Extractor which extracts an int or float attribute from an event.
LabelEncoder	Encode a given field from an event, e.g. to be used as a label.
Pulse	Constant-one extractor — returns a single 1.0 scalar for every event.

Extractors — Neuro

ChannelPositions	Channel positions in 2D or 3D, extracted from a Raw object's mne.Info.
EegExtractor	EEG feature extractor.
EmgExtractor	EMG feature extractor.
FmriExtractor	fMRI feature extraction with optional projection, signal cleaning, and caching to a NumPy memmap.
FnirsExtractor	Functional Near-Infrared Spectroscopy (fNIRS) feature extractor.
HrfConvolve	Convolve the output of an extractor by the Hemodynamic Response Function.
IeegExtractor	Intracranial EEG feature extractor.
MegExtractor	MEG feature extractor.
MneRaw	Feature extractor for raw MNE data files.
SpikesExtractor	Feature extractor for spike data stored in HDF5/NWB files.

Extractors — Audio

HuggingFaceAudio	Base class for extracting audio features from Hugging Face models.
MelSpectrum	Compute the Mel spectrogram representation of an audio waveform.
SeamlessM4T	Extract speech embeddings using the pretrained Seamless M4T model from Hugging Face.
SonarAudio	Extract deep audio embeddings from waveforms using the Sonar speech encoder.
SpeechEnvelope	Extract the acoustic amplitude envelope from audio waveforms.
Wav2Vec	Extract speech embeddings using a pretrained Wav2Vec 2.0 model from Hugging Face.
Wav2VecBert	Extract speech embeddings using the pretrained Wav2Vec2-BERT model from Hugging Face.
Whisper	Extract speech embeddings using the pretrained Whisper model from Hugging Face.

Extractors — Text

HuggingFaceText	Get embeddings from HuggingFace language models.
SentenceTransformer	Get embeddings from SentenceTransformers: https://huggingface.co/sentence-transformers.
SonarEmbedding	Get embeddings from sonar: https://arxiv.org/abs/2308.11466
SpacyEmbedding	Get word embedding from spacy.
TfidfEmbedding	Get TF-IDF embeddings for Sentence events.
WordFrequency	Get word frequency from wordfreq package.
WordLength	Get word length.

Extractors — Image

ColorHistogram	Color histogram.
HOG	Histogram of oriented gradients (Dalal & Triggs, 2005).
HuggingFaceImage	Compute image embeddings using transformer-based models obtained through HuggingFace API.
LBP	Local Binary Pattern (LBP).
RFFT2D	(Cropped) 2D Fourier spectrum of an image of real values.

Extractors — Video

HuggingFaceVideo

Extract video features using a HuggingFace transformer model.

Extractors — Meta

AggregatedExtractor	Aggregate multiple extractors along the specified dimension.
CroppedExtractor	Crop a extractor to a given offset and duration.
ExtractorPCA	Applies a PCA to another extractor's data The underlying extractor is first computed through the prepare method, and then the current extractor applies the PCA on it.
HuggingFacePCA	Applies a PCA to the underlying HuggingFace extractor.
TimeAggregatedExtractor	Remove the time dimension of a dynamic extractor, either by summing/averaging or by selecting the first, middle or last time point.
ToStatic	Crop a extractor by a given offset and duration.

DataLoader

Segment

A time window on a single timeline, backed by a shared event store.

Segmenter	Build a SegmentDataset from an events DataFrame and extractors.
SegmentDataset	Dataset defined through Segment instances and BaseExtractor instances.
Batch	A collection of extracted features for a list of segments.

Splitting

DeterministicSplitter	Hash-based splitter that assigns a deterministic train/val/test split.
chunk_events	Split events into smaller chunks.

SimilaritySplitter

A class used to split events based on similarity clustering of static extractors.