Step-by-step on how to use pre-trained AI models with Hugging Face for audio to text?

All answers to this question.

Whisper is arguably the best open-source model for this right now. The transformers library makes it very simple. You should use the WhisperProcessor to handle the feature extraction (like converting audio to log-mel spectrograms) and WhisperForConditionalGeneration for the actual transcription. One thing to note: Whisper expects audio sampled at 16kHz. If your meeting recordings are in 44.1kHz, you must resample them first using the librosa or torchaudio library, otherwise, the transcription quality will be terrible.

   Answered 2025-03-14 by Helen Phillips

Are you planning to run this locally or on a cloud provider like AWS? The "large-v3" model is very accurate but requires a decent GPU with at least 10GB of VRAM to run smoothly.

   Answered 2025-04-22 by Ryan Mitchell

• Ryan, I'm planning to deploy this on an AWS EC2 g4dn instance. It has a T4 GPU, so I think it should handle the medium or large model. My main concern is the cost—do you think the "small" model is enough for clear English recordings to save on compute?

     Commented 2025-04-25 by Larry Roberts

For English-only meetings, the distil-whisper models are amazing. They are much faster and almost as accurate as the original large models.

   Answered 2025-05-15 by Steven Evans

• Steven is right. Distil-whisper is a game changer for cost-saving on AWS. It cuts down inference time by almost 50% without a noticeable drop in transcription quality for clear speech.

     Commented 2025-05-22 by Helen Phillips