TextTokenDecoder

class fairseq2.data.text.TextTokenDecoder[source]

Bases: ABC

Decodes text from tokens or token indices.

abstract __call__(token_indices)[source]

Parameters:: token_indices (Tensor) – The token indices to decode from.
Return type:: str | CString

abstract decode_from_tokens(tokens)[source]

Parameters:: tokens (Sequence[str | CString]) – The tokens to decode from.
Return type:: str | CString