What is xFormers?

Flexible Transformers, defined by interoperable and optimized building blocks.

xFormers is focused on the following values

• Field agnostic. This library is not focused on any given field, by design.

• Composable. Ideally, break all the Transformer inspired models into a block zoo, which allows you to compose reference models but also study ablations or architecture search.

• Extensible. xFormers aims at being easy to extend locally, so that one can focus on a specific improvement, and easily compare it against the state of the art.

• Optimized. Reusing building blocks across domains means that engineering efforts can be more valued. And since you cannot improve what you cannot measure, xFormers is benchmark-heavy.

• Tested. Each and every of the variant in the repo is tested, alone and composed with the other relevant blocks. This happens automatically anytime somebody proposes a new variant through a PR.

• Crowd Sourced. PRs are really welcome, the state of the art is moving too fast for anything but a crowd sourced effort.