Evaluation Toolkit

As part of `stopes`, we are building a number of evaluation tools that you can use to evaluate multimodal translation models.

📄️ ALTI+

ALTI+ is a tool for inspecting token contributions in a transformer encoder-decoder model.

📄️ BLASER: an evaluation metric of translation accuracy for speech and text

BLASER leverages a multilingual multimodal encoder to directly encode the speech segments for source input, translation output and (optionally) reference into a shared embedding space and computes a score of the translation quality that can be used as a proxy to human evaluation.