Number of dimensions of the input embeddings
Number of heads for the multi-head attention
Number of encoder layers
Optional attentionDim: numberNumber of dimensions of the embeddings used in the scaled dot-product attention, or dim if not specified
Optional feedForwardDim: numberNumber of dimensions in the hidden layer of the feed forward network, or dim if not specified
Optional initSequenceLength: numberInitial sequence length that the positional encoding should be computed for, or DEFAULT_SEQUENCE_LENGTH if not specified
Private attentionPrivate depthPrivate dimPrivate feedPrivate headsPrivate layersReadonly lengthReadonly nameReturns the name of the function. Function names are read-only and can not be changed.
Private positionalDetermines whether the given value inherits from this function if this function was used as a constructor function.
A constructor function can control which objects are recognized as its instances by 'instanceof' by overriding this method.
Calls the function, substituting the specified object for the this value of the function, and the specified array for the arguments of the function.
The object to be used as the this object.
Optional argArray: anyA set of arguments to be passed to the function.
For a given function, creates a bound function that has the same body as the original function. The this object of the bound function is associated with the specified object, and has the specified initial parameters.
An object to which the this keyword can refer inside the new function.
Rest ...argArray: any[]A list of arguments to be passed to the new function.
Calls a method of an object, substituting another object for the current object.
The object to be used as the current object.
Rest ...argArray: any[]A list of arguments to be passed to the method.
Generated using TypeDoc
Transformer encoder as described by Vaswani et al containing an arbitrary number of TransformerEncoderLayers.
This module includes the positional encoding, but does not include any initial embedding of an input sequence into vectors (which should have been separately done by e.g. word2vec).