ElevenLabs has unveiled Eleven v4 and its low-latency variant, Eleven v4 Turbo: their fastest and most expressive text-to-speech models to date. “AI is making extraordinary strides in science and programming, but in many parts of the economy and society, the way we interact with it will be as important as its ability to reason,” says Mati Staniszewski, cofounder and chief executive of ElevenLabs. “Eleven v4 brings the richness and expressivity that have been missing in AI-powered communication. Thanks to its emotional nuance and contextual understanding, the model will enable new artistic experiences and AI agents across many more markets around the world.”
“Eleven v4 is built on a new architecture that offers a broader range of emotional registers and tighter control,” explains Piotr Dąbkowski, cofounder and chief technology officer of ElevenLabs. “The model deeply understands audio and prosody, which translates into richer interpretations and more natural dialogues. From the outset we designed it for applications that require ultra-low latency, and we’re excited to bring that same expressiveness to conversational agents.”
A voice that interprets
Whereas earlier generations of text-to-speech systems were limited to reading text aloud, Eleven v4 interprets it. The model extracts from the text the tone, pace, character, and context to generate a voice capable of conveying emotions: from urgency and drama to tenderness or humor.
To fine-tune interpretation with precision, users can describe in natural language how a sentence should be spoken or insert tags in the text to indicate emotion, style, sound effects, or manner of speaking. Examples include [laughs] — laughter—, [said angrily in British accent] — said with anger and a British accent—, [light rain] — gentle rain—, or [phone buzzing] — cellphone vibration—.
Eleven v4 offers greater consistency than Eleven v3 and improves the generation of dynamic conversations among multiple speakers. Thanks to a new method for representing vocal identity, it preserves the distinctive traits of each voice across different audio generations, maintaining coherence in conversations, audiobooks, or ads. Additionally, by understanding the context of a complete scene, it generates natural dialogues in which each participant responds to what they just heard, rather than stringing together isolated phrases.
More expressive agents, with real-time responses
Until now, high-quality voice models tended to take longer to generate audio, forcing companies to choose between speed and expressiveness. Many have opted for effective but monotone agents that can feel robotic when dealing with a nervous or upset customer.
Eleven v4 Turbo enables expressive agents capable of keeping conversations agile and fluid. The model averages about 100 milliseconds to start issuing its first response—less than the typical pause between turns in a two-person conversation—and it carries Eleven v4’s emotional richness into real-time interactions.
This combination of speed and expressivity opens new possibilities across numerous sectors: from agents with a warm, soothing voice capable of pronouncing medical terms correctly in healthcare, to video game characters who speak quickly and use colloquialisms to entertain players.
Eleven v4 Turbo is designed to integrate fully with ElevenAgents, ElevenLabs’ platform for conversational agents. The company’s research and engineering teams have optimized both products together, as a single system, to deliver more expressive and reliable agents with shorter response times.
An AI that speaks with the naturalness of every language
Generating a voice that sounds native requires more than simply pronouncing words correctly. Each language has its own rhythms, and each cultural setting has conventions for how to speak and express emotions. These also vary depending on who is speaking and who they are addressing. For example, the way you address an unfamiliar older person in Japan differs from Italy.
Eleven v4 generates expressive, human-like voices in more than 90 languages, compared with the 70 supported by v3. The model infers the context and the character from the text to deliver a more natural interpretation in the chosen language. Voice quality improves across the board, with especially notable gains in Japanese, Mandarin Chinese, and Brazilian Portuguese.
For brands and creators who want to maintain a single voice identity across markets, Eleven v4 lets you switch languages and accents without losing other voice characteristics. So, a voice recorded in Korean can read English text with a native accent, preserving the original tone and timbre.
Identity and data protection
Voice agents participate in some of the most sensitive conversations a company has with its customers: verifying identity, discussing health, or handling finances. Many of the sectors where Eleven v4 and Eleven v4 Turbo can add value—healthcare, financial services, and government—are subject to stringent security and regulatory requirements.
According to ElevenLabs, the models comply with the GDPR, HIPAA, SOC 2, and PCI, and offer data-residency options in the European Union, the United States, Singapore, and India.
Leveraging the benefits of more expressive models also requires proper safeguards to protect identity. To use ElevenLabs’ professional voice-cloning tools, users must pass a voice CAPTCHA: read, within a short window, a dynamically generated sentence to verify their voice. Meanwhile, the No-Go Voices protection system blocks cloning of voices belonging to politicians and other highly public figures.
