Projection Layers

class fairseq2.nn.Projection(input_dim, output_dim)

Bases: Module, ABC

Applies a linear transformation to incoming data.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

abstract forward(x)

Parameters:

x (Tensor) – The input to project. Shape: \((*,H_{inp})\), where \(H_{inp}\) is the input dimensionality.

Returns:

The projected output. Shape: \((*,H_{out})\), where all but the last dimension are the same shape as the input and \(H_{out}\) is the output dimensionality.

Return type:

Tensor

final class fairseq2.nn.Linear(input_dim, output_dim, bias, *, init_fn=None, device=None, dtype=None)

Bases: Projection

Applies a linear transformation to incoming data using weights and bias.

Unless overridden by a subclass, the weights and bias are initialized from \(\mathcal{U}(-\sqrt{k}, \sqrt{k})\), where \(k = \frac{1}{\text{input_dim}}\).

Note

This class is identical to torch.nn.Linear.

Parameters:

• input_dim (int) – The dimensionality of inputs.

• output_dim (int) – The dimensionality of projected outputs.

• bias (bool) – If True, learns an additive bias.

• init_fn (Callable[[Linear], None] | None) – The callable to initialize the weight and bias.

reset_parameters()

forward(x)

Parameters:

x (Tensor) – The input to project. Shape: \((*,H_{inp})\), where \(H_{inp}\) is the input dimensionality.

Returns:

The projected output. Shape: \((*,H_{out})\), where all but the last dimension are the same shape as the input and \(H_{out}\) is the output dimensionality.

Return type:

Tensor

final class fairseq2.nn.TiedProjection(weight, bias)

Bases: Projection

Applies a linear transformation to incoming data using the weights and bias of another Module instance.

Parameters:

• weight (Parameter) – The shared weights.

• bias (Parameter | None) – The shared bias.

forward(x)

Parameters:

x (Tensor) – The input to project. Shape: \((*,H_{inp})\), where \(H_{inp}\) is the input dimensionality.

Returns:

The projected output. Shape: \((*,H_{out})\), where all but the last dimension are the same shape as the input and \(H_{out}\) is the output dimensionality.

Return type:

Tensor