ultravox

Ultravox is a multimodal LLM that can understand text as well as human speech without the need for a separate Audio Speech Recognition (ASR) stage. It extends any open-weight LLM with a multimodal projector that converts audio directly into the high-dimensional space used by LLMs.