Conformer

Conformer-2 is a cutting-edge AI model specifically designed for automatic speech recognition (ASR). Building upon the success of its predecessor, Conformer-1, this advanced model has been trained on an extensive dataset of 1.1 million hours of English audio, leading to remarkable improvements in various aspects of speech recognition.

Focus Areas: The primary goal of Conformer-2 is to enhance the recognition of proper nouns, alphanumerics, and noise robustness. By focusing on these critical areas, the model significantly improves its ability to accurately transcribe spoken content.

Scaling Laws and Training Data: The development of Conformer-2 was guided by the scaling laws proposed in DeepMind’s Chinchilla paper. Understanding the importance of sufficient training data for large language models, Conformer-2 leverages a massive 1.1 million hours of English audio data during its training process.

Ensembling Technique: One of the standout features of Conformer-2 is its adoption of model ensembling. Rather than relying on predictions from a single teacher model, Conformer-2 generates labels from multiple strong teachers. This ensembling technique reduces variance and enhances the model’s performance when dealing with previously unseen data during training.

Improved Speed and Processing: Despite its increased model size, Conformer-2 exhibits improvements in terms of speed compared to Conformer-1. The serving infrastructure has been meticulously optimized, resulting in faster processing times. Conformer-2 achieves up to a 55% reduction in relative processing duration across all audio file durations.

Real-World Performance: In real-world applications, Conformer-2 demonstrates significant enhancements in various user-oriented metrics. Notably, it achieves a 31.7% improvement on alphanumerics, a 6.8% improvement on proper noun error rate, and a 12.0% improvement in noise robustness. These enhancements are attributed to both the vast training data and the use of an ensemble of models.

Ideal for AI Pipelines: The Conformer-2 model proves to be an invaluable component for AI pipelines that focus on generative AI applications using spoken data. Its remarkable speech-to-text transcription capabilities make it a valuable tool for generating accurate transcriptions with exceptional precision and reliability.