fairseq2.data.text

This module contains text tokenizers and text specific data pipeline operators.

• Text Tokenizers