Number of dimensions of the input embeddings
Number of heads for the multi-head attention
Optional attentionDim: numberNumber of dimensions of the further embeddings used by the scaled dot-product attentions, or dim if not specified
Private attentionPrivate attentionPrivate concatPrivate dimPrivate headsPrivate keyReadonly lengthReadonly nameReturns the name of the function. Function names are read-only and can not be changed.
Private queryPrivate valueDetermines whether the given value inherits from this function if this function was used as a constructor function.
A constructor function can control which objects are recognized as its instances by 'instanceof' by overriding this method.
Calls the function, substituting the specified object for the this value of the function, and the specified array for the arguments of the function.
The object to be used as the this object.
Optional argArray: anyA set of arguments to be passed to the function.
For a given function, creates a bound function that has the same body as the original function. The this object of the bound function is associated with the specified object, and has the specified initial parameters.
An object to which the this keyword can refer inside the new function.
Rest ...argArray: any[]A list of arguments to be passed to the new function.
Calls a method of an object, substituting another object for the current object.
The object to be used as the current object.
Rest ...argArray: any[]A list of arguments to be passed to the method.
A Tensor of shape [..., queryTokens, dim]
Tensor of query vectors, shape [..., queryTokens, dim]
Tensor of key vectors, shape [..., keyTokens, dim]
Tensor of value vectors each corresponding to a key, shape [..., keyTokens, dim]
Optional mask: TensorTensor mask of shape [queryTokens, keyTokens] for the TransformerDotProductAttention
Generated using TypeDoc
Multi-head attention mechanism as described by Vaswani et al. The input Tensors are linearly embedded before being passed to scaled dot-product attentions.