Class TransformerPositionalEncoding

A module to generate the positional encoding for a Transformer of a given input dimension,

$$ \mathrm{PE}_{i, 2z} = \sin \left( \frac{i}{10000^{2z/d}} \right) $$

$$ \mathrm{PE}_{i, 2z + 1} = \cos \left( \frac{i}{10000^{2z/d}} \right) $$

where $i$ is the sequence position, $2z$ and $2z+1$ are the dimensions of the input embedding, and $d$ is the dimensionality of the input embedding.

The multiplicative factors $\frac{1}{10000^{2z/d}}$ are precomputed during object creation as they are constant for all $i$.

The full PE is initially precomputed for all $i$ up to 256 (configurable). If the module is called with a sequence length larger than the initial value, the additional values are also calculated and stored.


  • Module
    • TransformerPositionalEncoding



arguments: any
caller: Function
dim: number
encoding: Tensor
encodingFactors: Tensor
length: number
name: string

Returns the name of the function. Function names are read-only and can not be changed.

prototype: any
sequenceLength: number

The default initSequenceLength if none is supplied in the constructor.

ENCODING_BASE: 10000 = 10000

The base of the exponent in the positional encoding.


  • Determines whether the given value inherits from this function if this function was used as a constructor function.

    A constructor function can control which objects are recognized as its instances by 'instanceof' by overriding this method.


    • value: any

    Returns boolean

  • Calls the function, substituting the specified object for the this value of the function, and the specified array for the arguments of the function.


    • this: Function
    • thisArg: any

      The object to be used as the this object.

    • Optional argArray: any

      A set of arguments to be passed to the function.

    Returns any

  • For a given function, creates a bound function that has the same body as the original function. The this object of the bound function is associated with the specified object, and has the specified initial parameters.


    • this: Function
    • thisArg: any

      An object to which the this keyword can refer inside the new function.

    • Rest ...argArray: any[]

      A list of arguments to be passed to the new function.

    Returns any

  • Calculate positional encodings at a given range of sequence positions.


    a Tensor of calculated positional embeddings with shape [end - start, dim]


    • start: number

      Start of the range to calculate

    • end: number

      End of the range to calculate

    Returns Tensor

  • Calls a method of an object, substituting another object for the current object.


    • this: Function
    • thisArg: any

      The object to be used as the current object.

    • Rest ...argArray: any[]

      A list of arguments to be passed to the method.

    Returns any

  • Returns

    a Tensor of positional embeddings with shape [length, dim], using precomputed values if available


    • sequenceLength: number

      Length of the sequence for which the positional embedding should be calculated

    Returns Tensor

  • Returns a string representation of a function.

    Returns string

Generated using TypeDoc