Voice AI: Conversational Interface to Workflow

The Evolution of Command-and-Control to Conversation

We have entered into an era of foundational renaissance for voice AI. The interactive voice response (IVR) systems that still route billions of enterprise calls were built on rigid decision trees and touch-tone menus, and the statistical speech engines that followed improved recognition without ever making the interaction feel human. Voice was just a routing layer, not a conversation.

Conversational voice has become dramatically better, evolving from cascading models to S2S models with human latency. Today’s conversations can handle interruption handling, acoustic nuance, and real-time reasoning. Voice is moving from basic set commands into complex work, powering real-time translation, backend workflows, and conversational intelligence.

Over the last few decades, voice technology has iterated over several generations:

Voice has graduated from an application feature into foundational infrastructure:

• Speech to Text: Early STT relied on Hidden Markov Models with rigid pipelines: an acoustic model identified the sound, a dictionary mapped it to a phoneme, and a language model guessed the word. If you had an accent, or if there was background noise, the brittle system collapsed. Modern speech to text became revolutionized by transformer architectures to process audio (i.e. OpenAI's Whisper). Instead of guessing sounds frame-by-frame sequentially, transformers used attention mechanisms to analyze the entire audio sequence and surrounding context simultaneously, making them highly resilient to noise and accents. 
• Text to Speech: In 2022, researchers at Meta and Google created neural audio codecs. These algorithms compress raw, continuous audio down into discrete digital "tokens" similar to how an LLM breaks text into words. By turning sound into tokens, companies could treat speech generation as a standard language modeling problem. Instead of predicting raw physics, the AI predicts the "next sound token. Modern TTS uses in context learning, similar to how text prompts work, by feeding audio clips that generate new speech with the prefix as the style guide, copying the target speaker's timbre, accent, room acoustics, and speaking cadence without any extra model training.

The Modern Stack

Today’s voice landscape has built the foundation. To build these real-time conversational agents, the modern technical architecture is primarily divided into two distinct layers: foundation models that generate the core audio and intelligence, and the orchestration middleware that connects them. 

Closed Source Leaders:‍

The market is currently split between fully integrated end-to-end multimodal models (like OpenAI’s GPT-4o and Google’s Gemini, which process audio natively) and specialized modular models dedicated solely to Speech-to-Text (STT) or Text-to-Speech (TTS).

• ElevenLabs: Originally just a TTS company, ElevenLabs is now a complete conversational platform. Their suite includes Eleven v3 / Flash v2.5 (TTS), Scribe v2 (STT), and a native Conversational AI platform (ConvAI) that handles the orchestration and turn-taking for developers.  
• Deepgram: A heavily developer-focused platform that built its reputation on transcription. Their infrastructure now seamlessly connects Nova-3 (STT) and Aura-2 (TTS) via their Voice Agent API, which is highly optimized for enterprise call center IVR and high-volume routing.  
• Cartesia: Built for extreme low-latency use cases. Cartesia’s platform now includes Sonic 3.5 (TTS), Ink (STT), and their orchestration layer called Line, allowing developers to build byte-level streaming agents that respond in under 100ms.  
• OpenAI: Through the Realtime API (upgraded to Realtime-2 in 2026), OpenAI functions as a massive, unified platform. Instead of stringing models together, developers pass raw audio directly to the multimodal model, handling STT, reasoning, and TTS in a single native loop.  
• Inworld AI: A massive player in 2026 offering a fully integrated proprietary stack. They provide their own Realtime TTS-2, real-time STT, and an "Inworld Router" that seamlessly connects audio to over 220 third-party and first-party LLMs on their own infrastructure.  
• Hume AI: Hume’s core product—the Empathic Voice Interface (EVI)—has actually evolved into a full Speech-to-Speech (S2S) Platform

Open Source Frontier:

The open-weight ecosystem has exploded in 2026, with community models now matching or exceeding proprietary models on leading leaderboards.

• Kyutai Moshi (Open Source): The open-weight pioneer of full-duplex speech-to-speech. Built on the Mimi neural audio codec, it achieves ~200ms latency and can listen and speak simultaneously. It is the go-to architecture for self-hosted conversational agents.
• Breeze TTS 2 (BreezeBlue): Released in August 2026, this 3B parameter transformer is currently a top-ranked open model. It outperforms many closed-source models and offers advanced "Voice Direction" to steer emotion and pace simultaneously.
• Fish Audio S2 Pro / Fish Speech V1.5: Utilizes a highly efficient "Dual-AR" architecture. It excels at zero-shot voice cloning and allows developers to control prosody via natural language inline tags (e.g., [whisper], [laugh]).
• CosyVoice 2: Optimized for speed, this model achieves ultra-low latency (around 150ms) in streaming mode while maintaining high synthesis quality, making it ideal for real-time applications.
• IndexTTS-2 & Microsoft VibeVoice: IndexTTS-2 pushes the boundaries of zero-shot emotional control, while Microsoft's VibeVoice is recognized as a frontier model for raw vocal naturalness

The Orchestration Layer:

To build a voice agent, developers cannot just call APIs sequentially; the latency would be disastrous. The orchestration layer is the middleware that manages WebRTC/telephony transport, streams bytes concurrently, handles endpointing (knowing when the user stopped speaking), and manages interruptions.

• Vapi, Retell AII, Bland AI (Managed Platforms): These act as a unified control plane. A developer can plug in Deepgram to listen, Anthropic's Claude to think, and ElevenLabs to speak. Vapi and Retell manage the complex streaming, turn-taking, and telephony (SIP) connections to ensure the total pipeline remains under the 700ms budget.
• LiveKit & Pipecat (Open Source Frameworks): For teams that want full pipeline control and self-hosted infrastructure, these open-source frameworks provide the WebRTC transport and routing logic necessary to stitch models together without being locked into a managed vendor.
• Deepgram Voice Agent API: Instead of unbundling the stack, Deepgram offers a streamlined Voice Agent API that tightly couples their own STT and TTS models with bring-your-own-LLM support, simplifying architecture for enterprise deployments
• Dograh AI: Positioned in 2026 as the open-source alternative to commercial orchestrators like Vapi. It ships as a complete, self-hostable product with a visual workflow builder, native telephony integration, and the ability to run local models (like Whisper for STT and Kokoro/Chatterbox for TTS) entirely offline without API keys.

Voice Architectures

The Cascaded Pipeline (STT → LLM → TTS)

The conversational voice AI landscape has historically relied on traditional cascaded Speech-to-Text-to-Speech (S2T2S) pipelines to facilitate interaction between humans and machine intelligence. Under this traditional cascaded paradigm, a voice activity detector (VAD) determines when a user has completed their turn, an automatic speech recognition (ASR) engine transcribes the spoken input into text, a large language model (LLM) processes the text to generate a text response, and a text-to-speech (TTS) synthesizer finally converts that text back into spoken audio. Although cascaded systems remain widely used because developers can easily swap, observe, and optimize individual components independently, they introduce massive systemic latency and discard critical non-linguistic dimensions of human communication, including prosody, emotion, laughter, and hesitation

Before the recent paradigm shift, conversational voice AI was cobbled together using three distinct, isolated systems:

1. Speech-to-Text (STT / ASR): A model like OpenAI's original Whisper would listen to your voice and transcribe it into text.
2. The Intelligence Layer (Text LLM): A language model (like GPT-4 or Claude 3) would receive that text, process the logic, and generate a text response.
3. Text-to-Speech (TTS): A synthesizer (like ElevenLabs) would take the text response and generate human-sounding audio.

Pros: Cascading architecture allows for control. The text intermediary gives developers total control. You can inject business logic, route tool calls reliably, and filter inappropriate content before it ever hits the TTS engine. It also allows for modularity, letting teams swap out the LLM for a smarter model tomorrow without touching the speech pipeline.

Cons: This cascaded approach has latency and realism drawbacks. The system has to wait for users to finish speaking before it could transcribe, think, and synthesize a response. Because the LLM only sees text, it cannot hear the user's emotional state, stripping away “paralinguistic” data. As voice gets converted to text, the AI loses tone, breathing, sarcasm, hesitations, and emotional state, resulting in a slightly more robotic interaction.

Native Speech-to-Speech (S2S)

Emerging S2S models use a unified, end-to-end architecture that processes sound waves directly in a single model. It analyzes not just words, but also paralinguistic cues like pitch, tone, pace, and emotion. S2S can natively generate sighs, laughter, or changes in tone because it outputs acoustic tokens, not text. In addition, removing the transcript conversion steps drastically reduces response times to human levels. The race to dominate native audio modeling features several major technology companies and AI labs like Gemini’s Live Series, OpenAI’s Realtime, and Moshi by Kyutai.

The challenge with S2S models is the "Fixed Persona" problem. Because the voice is baked directly into the neural weights, it is historically difficult to customize. You cannot easily swap in a custom brand voice like you can with a modular TTS provider. Without text, it’s also harder to filter audio tokens in real time between the  LLM and TTS engine. 

Pros: Real time latency. Because there are no sequential handoffs, S2S significantly reduces latency and allows for highly natural conversational dynamics, including seamless "barge-in" interruptions.

Cons: Transparency. S2S models are a "black box." If the AI says something inappropriate, debugging is difficult because there is no intermediate transcript to inspect. S2S also suffers from a single point of failure; if the model goes down, the entire system drops.

Full-Duplex Architecture

Traditional voice agents operate in a half-duplex format, meaning they utilize a rigid turn-taking protocol where the agent cannot ingest new microphone input while it is speaking, and the user must wait for the agent to finish its output stream before speaking again. Full duplex systems, by contrast, establish simultaneous, bidirectional audio streams that allow concurrent listening and speaking. In a true full-duplex system, microphone ingestion continues during assistant playback, and the system can react dynamically and reversibly to real-time interruptions, backchanneling, overlapping speech, and floor-taking transitions.

Unlike older turn-based (half-duplex) systems that wait for silence before processing or responding, a full duplex processes incoming and outgoing audio simultaneously in a continuous stream. This mimics a standard telephone call or face-to-face conversation. GPT-Live: OpenAI's conversational voice model uses a full-duplex design to handle simultaneous two-way audio and real-time interruptions. Moshi is an open-source full-duplex spoken dialogue model designed by Kyutai that concurrently streams audio input and output for simultaneous listening and talking.

Pros: True Barge-In: You can interrupt the AI mid-sentence, and it reacts immediately, stopping its output and pivoting naturally.  The model stops relying on acoustic silence to know when the speaker is finished talking as it understands context. The AI can utter natural "uh-huhs" and "hmms" while you speak without stealing the conversational floor.

Cons: Extreme Compute Costs: Keeping a massive neural network constantly listening and predicting tokens frame-by-frame requires immense GPU overhead compared to triggering a model only when a user finishes a sentence.

Newer research models like DuplexSLA are able to address these challenges. For example, DuplexSLA has a dedicated background action running parallel to its speech channel, so it can plan and execute tools while it is actively speaking. In standard voice systems, a tool call destroys the flow of speech. It also synchronizes user audio, assistant audio, and action tokens on a shared 160-millisecond chunk grid. When an interruption occurs, the generative engine can automatically cancel downstream inference and drop the contextual tokens that were not yet synthesized to the client device.

Voice Market Map and Landscape

Where Value Accrues in the Stack

If voice is becoming infrastructure, the investment question is which layer keeps the margin. Our view: value is migrating up and down, away from the middle.

The model layer is bifurcating, not disappearing. The undifferentiated middle — a generic STT or TTS API — is being squeezed from above (the labs bundling voice natively) and below (open weights like Moshi and Qwen3-Omni). But the frontier still captures enormous value: ElevenLabs climbed up-stack into branded enterprise products rather than selling raw inference. A fine-tuned or specialized model is a powerful feature, but rarely a durable company on its own unless it's fused to proprietary data, a language or domain the general models handle badly.

Native S2S represents a breakthrough in modality processing, shifting the foundation of voice AI from discrete text tokens to continuous acoustic representations. Full Duplex represents a breakthrough in interaction mechanics, establishing the networking, stream-concurrency, and turn-taking logic required for bidirectional communication. Models like Google Gemini Live Series, Kyutai Moshi, and OpenAI's GPT-Live integrate both these architectures. Modern systems achieve sub-300ms end-to-end latency, with practical implementations delivering 250-290ms response times and time-to-first-audio as low as 40-95ms, finally matching natural human conversational timing thresholds. Despite the impressive naturalness of these integrated full-duplex S2S models, enterprise adoption remains sparse given the lack of granular orchestration, safety guardrails, and deterministic debugging.

Orchestration captures outsized value today, but it's a transitional pool. The transport engineering is genuinely hard right now, and the traction is real: Vapi reached in under two years on pure usage-based pricing. We're cautious on pure, model-agnostic middleware. Ii's squeezed from both sides: the base-model providers already ship WebRTC / SIP / turn-taking natively (OpenAI's Realtime API), and app builders build their own; the orchestrator's core competency is exactly what the labs are absorbing. Second, the economics are thin: marking up passthrough compute on usage-based pricing is a low-margin utility unless you own the models, which is why Bland runs proprietary self-hosted GPUs, and had to raise prices to cover them. Orchestrators are now moving toward proprietary speech layers and up into verticals.

Durable value concentrates at the two ends of the stack:

• Full-stack infrastructure: Companies that own the model, the pipeline, and the latency and compliance guarantees end-to-end (Bland's self-hosted bet; ElevenLabs pushing down into agents).
• Workflow-owning applications: Apps that own a system of record and price against an outcome (a resolved claim, a booked appointment), building switching costs through deep integration and staying insulated from lab-bundling. This only works for apps that own depth; a thin voice wrapper on a crowded vertical commoditizes.

Market Opportunities to Build

We're a thematic, early-stage firm, and we look for the picks-and-shovels and the applications that get pulled along as a platform shift plays out. Across the voice stack, four layers stand out.

• Full Stack Providers: The base-model race is largely a big-lab and well-capitalized-incumbent game, but the specialized layer is wide open. We're interested in models and runtimes that meaningfully improve realism, latency, and reaction time: expressive and controllable TTS, true full-duplex conversation, on-device and edge inference, and the transport engineering (turn-taking, interruption handling, jitter/packet-loss resilience) that separates a demo from a production call center. The most defensible infra players are pushing toward proprietary speech layers and, in some cases, self-hosted ownership of the full pipeline to control latency, reliability, and data handling end-to-end.
• Data & Languages: Model quality is downstream of data quality, and clean conversational audio is scarce: raw web audio is noisy, duplicated, and demographically skewed. We see opportunity in the data layer: licensed and synthetic conversational datasets, dialect- and accent-aware models, and localization for the large non-English markets where generic models fail today. This is also where accessibility becomes a business, not just a virtue: agents that adapt to slower, fragmented, or non-standard speech unlock users the current generation quietly excludes.
• Voice to Workflow Applications: This is where value ultimately accrues to end customers, and where Montage's core sectors, financial services and healthcare, still lack enterprise adoption. We're drawn to agentic applications that don't just answer questions but take action: verifying insurance eligibility, navigating clinical IVR menus, authenticating callers, booking and rescheduling, updating systems of record. The winners will own a workflow deeply enough to be more than a voice on top of someone else's software. A caution we hold firmly: beware the commoditizing point-solution. In freight brokerage, for example, enterprise buyers are already treating standalone voice agents as interchangeable and signaling a preference for integrated, all-in-one suites. Single-feature voice apps in a crowded vertical are a hard place to build durable value; depth of integration and system-of-record ownership are the antidotes.
• QA, Control & Trust: We think this is the most underrated layer. Traditional scripted QA breaks the moment an agent produces variable, non-linear responses. Creating scalable systems to simulate environments that generate realistic conversational paths, evaluate and observe agents in production, and automatically learn from mistakes will create trust for enterprise buyers. More importantly, voice AI providers that expand beyond monitoring calls and into procedural evaluation across multi-step workflows will win. Finally, compliance is becoming a moat, not just a checkbox. Complying with global and state level regulations, industry certifications like HIPAA, and guaranteeing data sovereignty and auditability is becoming a necessary and durable wedge.

Solving for the Enterprise

Enterprises still run an enormous share of their most consequential work over the phone, and until recently voice AI could only deflect simple FAQs than run a regulated, high-stakes conversation. What has changed is not the underlying speech models, which are rapidly commoditizing, but the architecture around them: orchestration and "harness" layers that manage long, multi-step interactions, real-time supervisory guardrails that validate every turn before it is spoken, and self-improvement loops that learn from outcomes.

At Montage, our view is that the winners in enterprise voice will be defined by this reliability-and-compliance layer plus deep integration into systems of record. That thesis points us squarely at regulated sectors like financial services. Banks, lenders, insurers, and collections operations run millions of high-value calls a year under strict regimes (TCPA, FDCPA/Reg F, state licensing, KYC, the EU AI Act). The very barriers that make these workflows hard: compliance and auditability, integration with core and carrier-grade systems, and the trust required to let an AI speak on a regulated institution's behalf, and why a horizontal chatbot cannot simply walk in. That combination makes regulated financial services both difficult and defensible: the vendor that earns compliance sign-off and embeds into the systems of record captures durable, outcome-based budget rather than a thin, swappable software fee.

The true value creation in regulated sectors lies far beyond automated appointment reminders or simple balance notifications. We are looking at deeply complex, multi-turn use cases that require both domain expertise and emotional intelligence. The opportunity we find most interesting sits well beyond reminder and notification calls, in consequential, systems-integrated workflows:

• Dynamic wealth management assistants that can walk a client through a portfolio rebalancing strategy based on real-time market data
• Debt collection with live negotiation, payment plans, and promise-to-pay
• Card and account servicing such as activation, replacing a physical debit card, dispute and fraud triage, and limit changes;
• Loan servicing and even origination: payment extensions, hardship handling, identity verification, credit pulls, and bank linking — long-horizon flows that can unfold over days.
• Insurance adds FNOL and claims intake, coverage and policy servicing, and renewals
• In healthcare, prior authorization, insurance-eligibility verification, prescription refills, clinical triage, and EHR-integrated scheduling , dynamic clinical triage and complex post-discharge orchestration

The barriers are the moat. Nearly every obstacle that slows deployment is also a defensible wedge for whoever solves it:

• Compliance and auditability are the primary gate. Institutions won't select a model or vendor they can't audit. This involves evaluation of non-deterministic voice agents before and after deployment in a robust, scalable way ‍
• Data residency and on-prem preferences push regulated buyers toward self-hosted and open-weight deployments (JPMorgan keeps its most sensitive workloads off the public cloud); sovereignty is a wedge‍
• Legacy core integration is repeatedly named the single biggest bottleneck. Real-time voice needs real-time data, and batch-processing cores can't provide it.‍
• Procurement cycles of 60–120 days (and 6–12 months for core-platform changes) create an "innovation paradox" that punishes small vendors — so accreditation and compliance-readiness are themselves a go-to-market advantage.‍
• Reliability and control for long, branching, action-taking conversations, where S2S still trails cascading‍
• The realism-vs-guardrails tension. As voices become indistinguishable from humans, the ability to constrain, verify, and audit them becomes harder precisely when it matters more
• ‍Global coverage as models degrade sharply on non-English speech, regional dialects, code-switching (e.g., mixed Hindi-English or Arabic-English), and non-standard speech patterns (older adults, speech impediments)‍

What this tells us to back. The emerging winners are the ones making voice deployable: guardrails that tie every spoken statement back to verified source evidence, real-time deepfake and voice-fraud detection plus step-up authentication, and agent-governance platforms with evaluation, guardrails, and observability built in.

Why Now, and What Would Make Us Wrong

Why now: foundation models crossed the "feels human" threshold, latency dropped below the perceptual floor, and enterprise budgets are moving — Gartner projects $80B in contact-center savings from conversational AI in 2026 alone. The unit economics are visceral: we've seen outbound calling costs fall from ~$45 (human labor) to ~$0.29 per call in real deployments. When a capability gets ~100x cheaper and materially better in the same 18 months, categories get rebuilt.

The central risk of platform absorption. What happens when OpenAI, Google, and the model labs offer this natively? GPT-Live and Gemini Live make parts of the stack a feature, not a company. We are looking for durable startups will win on the things you can't rent — proprietary data and vertical depth, ownership of a system of record and a workflow, distribution into regulated buyers, self-hosted control where sovereignty matters, and trust/compliance infrastructure that the labs are slow to build. Horizontal "voice agent as a thin wrapper" plays are the ones most exposed.

Other risks we're watching: S2S reliability may take longer to mature than the hype implies (keeping cascading stacks and their vendors relevant longer; commoditization compressing margins in crowded verticals; and a deepfake/regulatory backlash that could slow consumer deployment even as it creates the trust-layer opportunity above.

An RFP for Founders

We're actively looking to partner with founders building the next layer of the conversational interface, whether that's a specialized model that beats the clock, the data and localization that make voice work, an application that owns a regulated workflow end-to-end, or the QA, evaluation, and trust infrastructure that lets any of it ship safely. If you're building here, please reach out to connie@montageventures.com.