Exca

Execution and caching for Python — validate configs, cache results, and scale to Slurm clusters with zero boilerplate.

Get Started   GitHub
pip install exca
Introduction

Learn why Exca exists — config validation, hierarchical experiments, and transparent caching.

Exca - Execution and caching
Tutorials

Step-by-step guides covering TaskInfra, MapInfra, caching, remote computation, and more.

Tutorials
How-to guides

Recipes for common tasks — debugging, job arrays, uid exclusion, versioning, workdir, and discriminated unions.

How-to guide
Explanation

Deep dives into the design philosophy, uid computation, ConfDict, CacheDict, and more.

Explanations
Serialization

How CacheDict stores data, built-in handlers, and how to write your own custom handler.

Serialization
API Reference

Complete API for TaskInfra, MapInfra, SubmitInfra, CacheDict, helpers, and associated classes.

API Reference

Why Exca?

A few features that make experiment management simpler.

Config

Validated configs — Pydantic-powered hierarchical configurations catch errors before you hit the cluster.

Cache

Transparent caching — Filesystem-backed, uid-based caching that Just Works™. Rerun only what changed.

Cluster

Cluster-ready — Submit to Slurm or run locally — switch between the two by changing one field.

Modular

Modular pipelines — Discriminated unions, composable sub-configs, and job arrays for grid-search with ease.

Comparison

Feature

lru_cache

hydra

submitit

stool

exca

RAM cache

✔

✘

✘

✘

✔

File cache

✘

✘

✘

✘

✔

Remote compute

✘

✔

✔

✔

✔

Pure Python

✔

✘

✔

✘

✔

Hierarchical config

✘

✔

✘

✘

✔

Citing#

@misc{exca,
    author = {J. Rapin and J.-R. King},
    title = {{Exca - Execution and caching}},
    year = {2024},
    publisher = {GitHub},
    journal = {GitHub repository},
    howpublished = {\url{https://github.com/facebookresearch/exca}},
}