Track 4 – EMG-to-Pose (cross-user regression)

Given 16-channel surface EMG (sEMG) recorded from a wristband during everyday movement, predict the corresponding trajectory of 20 hand-joint angles. The competition tests cross-user generalisation, and adds two more held-out axes on top, so a model has to survive changes in anatomy, wristband placement and kinematic context at once.

• Shift: held-out users, held-out movement stages, and unseen combinations of known users and stages.

• Headline metric: mean absolute angular error (lower is better). The task trains and logs radians; the competition and the paper report the same quantity in degrees.

• Data: emg2pose / NM000281 (193 participants, 25,253 recordings, 370 hours, 29 movement stages, 2 kHz). The hidden evaluation cohort follows the same protocol, pairing 16-channel wrist sEMG with 20 UmeTrack joint angles.

New to NeuralBench? Start here

This page is a track guide, not an introduction to the ecosystem. If you arrived straight from the competition website:

• Challenge overview – what NeuralBench is, how it relates to the competition, and the baseline numbers for all four tracks.

• Installation and the quickstart – get a task running on a 1.5 GB dataset before you download anything large.

• Official Track 4 guide – registration, rules, data access, prizes, leaderboard. Authoritative on every competition matter; this page only covers the code.

• How to Submit a Model.

Note that this is the one track on another device: its tasks live under neuralbench emg, not neuralbench eeg, which also means it is aggregated separately by --plot-cached.

Where to find this task in NeuralBench

The matching task in NeuralBench is Hand pose decoding.

• CLI: neuralbench emg pose

• Default dataset: Salter2024Emg2pose (16-channel sEMG paired with motion-capture hand pose).

• Model: VEMG2Pose, the paper’s regression baseline.

• Target: a dense 20-joint angle trajectory for each 5-s window.

• Test split: the paper scores three held-out sets separately, and this task keeps the user_stage one – unseen combinations of known users and stages – so a single test/mae means something.

• Headline metric key: test/mae (radians; x57.29578 for degrees).

What the config is. A NeuralBench task is one config.yaml, and nothing else: a YAML overlay on neuralbench/defaults/config.yaml naming the study to load, how to split it, what the target is, the loss, and the metrics. Reading it is the fastest way to know exactly what the baseline does.

Show tasks/emg/pose/config.yaml

# Copyright (c) Meta Platforms, Inc. and affiliates.
# All rights reserved.
#
# This source code is licensed under the license found in the
# LICENSE file in the root directory of this source tree.

# emg/pose: surface-EMG -> hand joint-angle trajectories (Salter2024Emg2pose,
# NEMAR NM000281), following the paper's regression setting.

data:
  batch_size: 64
  study:
    source:
      name: Salter2024Emg2pose
    # Recordings shorter than the longest window (VEMG2Pose's 5.895 s) leave the
    # segmenter nothing to cut; filtering here keeps every model on one set.
    drop_recordings_shorter_than_a_window:
      name: QueryEvents
      query: "duration >= 5.895"
    # The paper scores its three test sets separately (Table 4), so a pooled
    # ``test/mae`` matches none of them; ``val`` keeps both of its scenarios.
    keep_held_out_user_stage_test:
      name: QueryEvents
      query: "split != 'test' or generalization == 'user_stage'"
    split:
      name: PredefinedSplit
      event_type: Emg
      test_split_query: null
      col_name: split
      valid_split_by: null
  # No filter / notch / baseline / scaler / clamp: the paper feeds raw 2 kHz
  # EMG straight to the model, same convention as ``emg/typing``.
  neuro:
    =replace=: true
    name: EmgExtractor
    picks: [emg]
    frequency: 2000.0
    filter: null
    notch_filter: null
    baseline: null
    scaler: null
    clamp: null
    infra:
      cluster: !!python/name:neuralbench.config_manager.CLUSTER
      folder: !!python/name:neuralbench.config_manager.CACHE_DIR
      # exca's RAM cache never evicts, so retaining any of these 25232
      # recordings grows past 250 GB and the host OOM-kills the run.
      keep_in_ram: false
      slurm_partition: !!python/name:neuralbench.config_manager.SLURM_PARTITION
      timeout_min: 180
      gpus_per_node: 1
      cpus_per_task: 10
      min_samples_per_job: 200
  target:
    =replace=: true
    # The 20 joint angles are MISC channels in the same BDF as the EMG.
    name: EmgExtractor
    picks: [misc]
    frequency: 2000.0
    filter: null
    notch_filter: null
    baseline: null
    scaler: null
    # Radians, as emg2pose trains and logs them; its Table 4 degrees are a
    # reporting-time x57.29578. Scaling here would stall VEMG2Pose's output_scalar.
    clamp: null
    # Same cache as the neuro pass above: without it, ``=replace=`` drops the
    # default target infra and every job re-reads all 25232 recordings.
    infra:
      cluster: !!python/name:neuralbench.config_manager.CLUSTER
      folder: !!python/name:neuralbench.config_manager.CACHE_DIR
      keep_in_ram: false
      slurm_partition: !!python/name:neuralbench.config_manager.SLURM_PARTITION
      timeout_min: 180
      gpus_per_node: 1
      cpus_per_task: 10
      min_samples_per_job: 200
  # 5-s trajectories, the paper's evaluation length. Models with left context
  # widen ``duration`` alone, so their scored 5-s tails still tile without gaps.
  trigger_event_type: Emg
  start: 0.0
  duration: 5.0
  stride: 5.0
  stride_drop_incomplete: true
  # emg2pose's ``skip_ik_failures``: windows overlapping an IK failure are dropped
  # from every split rather than kept with the failed frames masked out.
  min_finite_target_fraction: 1.0
  summary_columns: [user, stage, side, generalization]
brain_model_output_size: &brain_model_output_size 20
brain_model_config:
  =replace=: true
  name: VEMG2Pose
  kwargs:
    sfreq: 2000.0
trainer_config:
  monitor: val/mae
  mode: min
  strategy: auto
  # emg2pose allows 500 epochs with patience 50; capped to fit SLURM's 2-day
  # limit at roughly 0.3 h/epoch.
  patience: 20
  n_epochs: 100
  gradient_clip_val: 0
  # The Rich bar buffers its writes, leaving the log silent for a whole epoch;
  # per-epoch lines still land without it.
  enable_progress_bar: false
# emg2pose holds the learning rate at 1e-3 (config/experiment/regression_*.yaml)
# where the neuralbench default anneals a 10x smaller one through OneCycleLR.
lightning_optimizer_config:
  =replace=: true
  optimizer:
    name: Adam
    lr: 1.0e-3
  scheduler: null
# emg2pose's RotationAugmentation, over its single 16-electrode band. Upstream
# redraws the offset per window; braindecode's BandRotation draws one per batch.
augmentation:
  probability: 1.0
  num_bands: 1
  electrodes_per_band: 16
  band_offsets: [-1, 0, 1]
loss:
  name: L1Loss
# Not get_regression_metric_configs(20): num_outputs returns one value per
# joint, which Lightning cannot log. The paper also reports the joint average.
metrics:
  - log_name: mae
    name: MeanAbsoluteError
  - log_name: rmse
    name: MeanSquaredError
    kwargs:
      squared: false
  - log_name: r2_score
    name: R2Score

Split and model selection

Split. The emg2pose paper’s own train / validation / test partition is used verbatim: PredefinedSplit reads the split column the study carries from the upstream metadata rather than drawing folds here, so nothing is randomised and valid_split_by is left unset.

The paper scores three held-out scenarios separately – unseen users, unseen movement stages, and unseen combinations of the two – and a pooled test/mae over all three would match none of them. The task therefore keeps only the third at test time, via query: "split != 'test' or generalization == 'user_stage'". Validation keeps both of its scenarios.

So this is not a held-out-user split. Both the users and the movement stages in the test set appear elsewhere in the data; what the model has never seen is a given user performing a given stage. That is also exactly the partition Codabench scores during warm-up, so a test/mae here and a warm-up leaderboard score measure the same thing – after converting radians to the degrees Codabench reports.

Model selection. The checkpoint with the lowest ``val/mae`` is kept – validation mean absolute angular error in radians, the same quantity as the headline test/mae. This track trains longer than the EEG ones: at most 100 epochs, with early stopping after 20 epochs without improvement. (The paper allows 500 epochs with patience 50; the task caps it to fit inside SLURM’s two-day limit.)

How to change it, in increasing order of effort:

• -m <model> and -w <preset> swap the architecture and the adaptation strategy (frozen probe, LoRA, full fine-tuning) without touching any file. -m neuropose and -m sensingdynamics select the paper’s other two regression baselines.

• This task ships no datasets/ variants – emg2pose is the only corpus registered for it – so there is nothing to select with --dataset.

• Anything else – window length, learning rate, the split – is a config edit. From Python, pass dotted keys to evaluate_model() (overrides={"data.duration": 2.0}). From a source checkout (pip install -e), edit config.yaml directly, or add a datasets/*.yaml variant and select it with --dataset. Both routes are described in Adding a New Task.

Reproducing the baseline

emg2pose is served through EEG-Dash, which the base install does not pull, so install it first: pip install 'eegdash>=0.8.2'.

Tip

This task ships no smaller dataset variant, but Salter2024Emg2pose scopes its download to the study query, so --download --debug fetches only the three subjects the debug run uses instead of all 193. Following it with a plain --debug, which builds its own cache in-process, exercises the whole path before you commit to the ~330 GB corpus; --prepare and the full runs need all of it.

Every command below passes -m vemg2pose, --download and --prepare included. That is not decoration: the model config widens data.duration to 5.895 s, adding the 1790 samples of TDS left context the model needs to emit the 10000 frames emg2pose scores. Preparing the cache without it caches 5.0 s windows the training run cannot use.

# 1. Download emg2pose / NM000281 into DATA_DIR: ~330 GB across 76k files,
#    hours over a typical link. One-off per machine, and safe to interrupt
#    and re-run.
neuralbench emg pose -m vemg2pose --download

# 2. Preprocess into CACHE_DIR -- window the 25232 recordings and pair
#    them with the joint-angle trajectories once, so every later run reads
#    the cache instead. ~25 min spread over 128 SLURM jobs. Budget
#    ~440 GB: more than the raw download, because the 20 joint angles are
#    cached as a second 2 kHz pass (~230 GB) alongside the EMG (~185 GB).
neuralbench emg pose -m vemg2pose --prepare

# 3. Sanity check before you queue anything: 2 epochs, a data subset, one
#    seed, always in-process, so progress lands in your terminal. ~1 min
#    on one V100 with the cache warm.
neuralbench emg pose -m vemg2pose --debug

# 4. Full paper regression baseline. ~2.5 h per seed, and the default grid
#    is three seeds (concurrent on SLURM). ``-m neuropose`` is ~4 h.
neuralbench emg pose -m vemg2pose

Step 4 caches the test-metric dictionary under SAVE_DIR – test/mae in radians, so multiply by 57.29578 to compare with the paper’s degrees – and re-running with --plot-cached turns those cached metrics into comparison plots and CSV tables without retraining. On SLURM it returns as soon as the grid is queued, so the numbers appear in the job logs rather than your terminal; see Collecting and plotting your results.

Evaluating a model of your own

A model that lives in your own codebase needs no YAML here: evaluate_model() takes the built instance, wraps it in a probe sized to the task, and returns the scores as a DataFrame.

from neuralbench import check_model, evaluate_model

print(check_model(my_model, "emg", "pose"))  # shapes only, seconds
scores = evaluate_model(my_model, "emg", "pose", name="my-fm", debug=True)

See Evaluating your own model for what forward has to accept, the adaptation presets, and how to fan the runs out to SLURM.

Scope and data handling

NeuralBench implements the paper’s regression_vemg2pose setting. The autoregressive tracking setting, which also conditions on an initial pose and previous predictions, is outside this task’s scope.

The paper split comes from the BIDS scans.tsv, falling back to the upstream emg2pose_metadata.csv on releases whose scans.tsv omits it. BAD_IK events mark intervals without inverse-kinematics labels, and any window overlapping one is dropped from every split; padded recording tails are not segmented into training windows either. Joint angles stay in the radians emg2pose trains on, so the loss and metrics are radians too.

Warning

emg2pose is released under CC-BY-NC-SA-4.0, and UmeTrack under CC-BY-NC-4.0. Both licenses are non-commercial.