Number of dimensions of the input embeddings
Number of heads for the multi-head attention
Optional attentionDim: numberNumber of dimensions of the embeddings used in the scaled dot-product attention, or dim if not specified
Optional feedForwardDim: numberNumber of dimensions in the hidden layer of the feed forward network, or dim if not specified
Private attentionPrivate dimPrivate feedPrivate ffPrivate ffPrivate headsReadonly lengthPrivate mhaPrivate mhaReadonly nameReturns the name of the function. Function names are read-only and can not be changed.
Determines whether the given value inherits from this function if this function was used as a constructor function.
A constructor function can control which objects are recognized as its instances by 'instanceof' by overriding this method.
Calls the function, substituting the specified object for the this value of the function, and the specified array for the arguments of the function.
The object to be used as the this object.
Optional argArray: anyA set of arguments to be passed to the function.
For a given function, creates a bound function that has the same body as the original function. The this object of the bound function is associated with the specified object, and has the specified initial parameters.
An object to which the this keyword can refer inside the new function.
Rest ...argArray: any[]A list of arguments to be passed to the new function.
Calls a method of an object, substituting another object for the current object.
The object to be used as the current object.
Rest ...argArray: any[]A list of arguments to be passed to the method.
Generated using TypeDoc
A layer of the Transformer encoder, as described by Vaswani et al, consisting of a multi-head attention layer and a fully-connected feed forward network. Both of these use residual connections and are normalised with LayerNorm.