fairseq2.nn

This module contains various PyTorch modules and related APIs to help with building new model architectures. It follows similar conventions to torch.nn and can be considered an addendum to it.

        classDiagram
  ABC <|-- Embedding
  ABC <|-- IncrementalState
  ABC <|-- InterpolatedPositionEncoder
  ABC <|-- LayerNorm
  ABC <|-- PositionEncoder
  ABC <|-- Projection
  ABC <|-- ResidualConnect
  Embedding <|-- ShardedEmbedding
  Embedding <|-- StandardEmbedding
  Embedding <|-- VocabShardedEmbedding
  InterpolatedPositionEncoder <|-- SinusoidalNdPositionEncoder
  LayerNorm <|-- RMSNorm
  LayerNorm <|-- StandardLayerNorm
  Module <|-- Embedding
  Module <|-- InterpolatedPositionEncoder
  Module <|-- LayerNorm
  Module <|-- LayerStack
  Module <|-- PositionEncoder
  Module <|-- Projection
  Module <|-- ResidualConnect
  PositionEncoder <|-- LearnedPositionEncoder
  PositionEncoder <|-- RotaryEncoder
  PositionEncoder <|-- SinusoidalPositionEncoder
  Projection <|-- ColumnShardedLinear
  Projection <|-- IdentityProjection
  Projection <|-- Linear
  Projection <|-- RowShardedLinear
  Projection <|-- TiedProjection
  ResidualConnect <|-- DropPathResidualConnect
  ResidualConnect <|-- ScaledResidualConnect
  ResidualConnect <|-- StandardResidualConnect
  SinusoidalNdPositionEncoder <|-- Sinusoidal2dPositionEncoder
  SinusoidalNdPositionEncoder <|-- Sinusoidal3dPositionEncoder
    

• Position Encoders