Feedforward mechanisms

class xformers.components.feedforward.MLP(dim_model: int, dropout: float, activation: Activation, hidden_layer_multiplier: int, bias: bool = True, *args, **kwargs)

Bases: Feedforward

forward(inputs: Tensor) → Tensor

training: bool

class xformers.components.feedforward.Feedforward(dim_model: Optional[int] = None, dropout: Optional[float] = None, activation: Optional[Activation] = None, *args, **kwargs)

Bases: Module

classmethod from_config(config: FeedforwardConfig) → Self

training: bool

xformers.components.feedforward.build_feedforward(config: Union[Dict[str, Any], FeedforwardConfig])

Builds a feedforward from a config.

This assumes a ‘name’ key in the config which is used to determine what attention class to instantiate. For instance, a config {“name”: “my_feedforward”, “foo”: “bar”} will find a class that was registered as “my_feedforward” (see register_feedforward()) and call .from_config on it.

xformers.components.feedforward.register_feedforward(name: str, config: ~typing.Any = <class 'xformers.components.feedforward.base.FeedforwardConfig'>)

Registers a subclass.

This decorator allows xFormers to instantiate a given subclass from a configuration file, even if the class itself is not part of the xFormers library.