What is xFormers?

Flexible Transformers, defined by interoperable and optimized building blocks.


xFormers is focused on the following values

  • Field agnostic. This library is not focused on any given field, by design.

  • Composable. Ideally, break all the Transformer inspired models into a block zoo, which allows you to compose reference models but also study ablations or architecture search.

  • Extensible. xFormers aims at being easy to extend locally, so that one can focus on a specific improvement, and easily compare it against the state of the art.

  • Optimized. Reusing building blocks across domains means that engineering efforts can be more valued. And since you cannot improve what you cannot measure, xFormers is benchmark-heavy.

  • Tested. Each and every of the variant in the repo is tested, alone and composed with the other relevant blocks. This happens automatically anytime somebody proposes a new variant through a PR.

  • Crowd Sourced. PRs are really welcome, the state of the art is moving too fast for anything but a crowd sourced effort.