What is xFormers?¶
Flexible Transformers, defined by interoperable and optimized building blocks.
xFormers is focused on the following values
Field agnostic. This library is not focused on any given field, by design.
Composable. Ideally, break all the Transformer inspired models into a block zoo, which allows you to compose reference models but also study ablations or architecture search.
Extensible. xFormers aims at being easy to extend locally, so that one can focus on a specific improvement, and easily compare it against the state of the art.
Optimized. Reusing building blocks across domains means that engineering efforts can be more valued. And since you cannot improve what you cannot measure, xFormers is benchmark-heavy.
Tested. Each and every of the variant in the repo is tested, alone and composed with the other relevant blocks. This happens automatically anytime somebody proposes a new variant through a PR.
Crowd Sourced. PRs are really welcome, the state of the art is moving too fast for anything but a crowd sourced effort.