Table of Contents

Shortcuts

Block factory¶

class xformers.factory.block_factory.xFormerEncoderBlock(config: xFormerEncoderConfig, **kwargs)[source]¶

Bases: Module

A vanilla Transformer Encoder block

classmethod from_config(config: xFormerEncoderConfig)[source]¶

static get_reversible_layer(config) → Tuple[Module, Module][source]¶

forward(x: Tensor, att_mask: Optional[Union[Tensor, AttentionMask]] = None, input_mask: Optional[Tensor] = None)[source]¶

training: bool¶

class xformers.factory.block_factory.xFormerDecoderBlock(config: xFormerDecoderConfig, **kwargs)[source]¶

Bases: Module

A vanilla Transformer Decoder block

… note: this implementation is not (yet ?) reversible

training: bool¶

classmethod from_config(config: xFormerDecoderConfig)[source]¶

forward(target: Tensor, memory: Tensor, encoder_att_mask: Optional[Union[Tensor, AttentionMask]] = None, decoder_att_mask: Optional[Union[Tensor, AttentionMask]] = None, input_mask: Optional[Tensor] = None)[source]¶