Tutorials¶
- Replace all attentions from an existing ViT model with a sparse equivalent?
- Using BlockSparseAttention
- How to Enable Fused Operations Using AOTAutograd and NVFuser
- Extend the xFormers parts zoo
- I’m only interested in testing out the attention mechanisms that are hosted here
- Building an encoder, comparing to PyTorch
- Building full models
- Using the Reversible block
- Using Triton-based layers
- Hierarchical Transformers
