Mapping emerging Voice AI UI patterns, perceived agent roles, key design decisions and limitations
Radio Rex (1922) was the first voice-activated toy ever sold. Rex the dog lived in a little doghouse held shut by an electromagnet. When you said “Rex,” the sound energy of the letter “x,” around 500 Hz, vibrated the metal bridge and interrupted the current. The electromagnet released and a spring popped the dog out. A simple mechanism, triggered by nothing more than a fixed frequency and it must have seemed almost magical at the time.
This idea barely changed for most of the century that followed. Decades of voice-enabled toys, robots and digital assistants works on the same basic principle Rex did, i.e to detect a sound pattern, match it to intent and then trigger a response. In the last decade, Siri, Alexa, and Google Assistant became few of the dominant voice interfaces and conversation design mostly meant accurately extracting intent from user speech and mapping it to the right outcome.
More recently, rapid gains in performance, cost, reliability, and latency of large language models and speech models have reshaped voice as a medium, unlocking new use-cases and interaction patterns for storytelling, tutoring, productivity, and enterprise workflows nobody could have imagined a few years ago.
This article maps 8 emerging Voice AI UI patterns across voice products, the perceived role of the AI agent within each, and the key design decisions and limitations that shape them.
1. Avatar video/ voice call
This voice experience is rendered as a video call, featuring either an animated character or a hyper-realistic human avatar. Common use cases include language and skills coaching like in Duolingo’s video call with Lily, MasterClass AI call or Speakology where a live avatar makes conversational practice feel more engaging. Or that of AI companions catering to assistance, friendship, or therapy like in Tolans or Replika’s video mode where character presence makes emotional disclosure feel reciprocated and supported.
Agent’s perceived role
The agent can be framed as a lifelike conversational companion. Beyond design for voice, its core strength lies in face-to-face presence like eye contact, lip-sync, and idle micro-expressions all reinforce the sense of being seen and related. In a company reported survey of Tolan users, 85.6% said it helped them get through an emotionally difficult experience, and 72.5% said it helped them manage or improve a relationship.
Key design decisions & limitations
- AI personality design: This pattern relies on either borrowing a personality, like that of a famous celebrity or creating a fictional one. Creating a fictional persona involves shaping how it looks, its expressions, voice and tonality, and even small details like idle behaviour and latency masking. For instance, Tolan casts its companions as cute and colourful aliens, designed to befriend and help humans. In a survey of Tolan users, 85.6% said it helped them get through an emotionally difficult experience, and 72.5% said it helped them manage or improve a relationship.
- Realism vs uncanny valley: Photorealistic experiences like HeyGen LiveAvatar sometimes glitch on latency or show uneven lip-syncing, breaking immersion. This risks tipping users into uncanny valley discomfort. On the other hand, vowel-based mouth-shape mapping using rive is cheaper and great for flat 2D animated personas.
- Moreover, the experience is supported by real-time bidirectional voice, powered by VAD (Voice activity detection)l which lets the user interrupt mid-sentence so the exchange flows more natural.
2. Turn-by-turn
This voice experience uses strict turn-taking with visual states for who’s speaking and who’s listening. Consumer apps like HireVue AI, Final Round AI or Supernova covers use-cases like structured tutoring, interview prep and skills assessment, where the rigid turn structure mirrors real evaluation (answer, evaluate, feedback).
Agent’s perceived role
The agent can be framed as a structured examiner or instructor. Its strength lies in creating a controlled pedagogical or assessment exchange where the user knows exactly when to speak, when to wait and when they are being evaluated.
Key design decisions & limitations
- The absence of a face is crucial: With no animated avatar, attention stays on the turn by turn exchange. The UI stays minimal and the effort goes into voice quality and clean turn handover. Morever, turn signalling must be clear. Color, copy and transition animation must combine so the user never wonders whose turn it is.
- No room for barge-in or overlap: People cannot cut in or talk over each other, so the pattern is limited to narrow use cases where turns are meant to be clean like in structured tutoring or interview prep.
- Show progress or communicate time duration to set the expectation for the usecase because without a visible marker they have no idea whether they are two questions in or nearly done.
3. AI phone call
This voice experience mimics a real phone call covering use cases like customer support automation, lead qualification, appointment booking, or surveys. Ringg AI, Retell, Vapi are some of the B2B SaaS platforms offering these voice agent. More recently, B2C apps like hiRobin or equalAI are solving for new usecases like offering phone assistants for everyday phone calls.
Agent’s perceived role
The agent can be framed as a voice-only service agent, like a support rep, salesperson, or an assistant offering real-time bidirectional voice over telephony with barge-in and natural interrupt handling.
Key design decisions & limitations
- Conversational arc design: The conversation follows a structured arc whose shape depends on the use case. Inbound support runs greeting → intent → resolution → wrap-up, while an outbound sales call has to identify itself and earn permission before it can qualify anyone.
- Personality is carried by words alone: With no face to do any of the work, it lives entirely in word choice, pace, and warmth. Vapi reports outbound conversion improved from 14% to 22% after rewriting an agent to sound like a colleague rather than a corporate script with no change to the underlying flow.
- Callers can’t scroll back: Nothing stays on screen, so anything important like an appointment date, a name or an amount should get confirmed in AI’s next turn. For instance: “Got it, booking your appointment for January 15th.”
- Text anything they’d have to write down: Links, reference numbers, and addresses don’t survive being read aloud. Google’s conversation design recommend offering the caller a channel switch SMS or email, to send and receive important information.
4. Voice-augmented chat
This is a voice modality layered onto a general-purpose text chat, using the same agent, just spoken instead of typed. It’s especially useful when the user is mobile and wants to go hands-free, or needs to dictate long queries. Popular voice experiences include, ChatGPT voice mode, Claude voice mode, Gemini Live.
Agent’s perceived role
The agent can be framed as a general-purpose voice assistant. This pattern typically shows up in two forms. Full-screen voice mode takes over the interaction entirely, the chat thread disappears, and it becomes purely audio. On the other hand, Inline voice sits within the existing chat thread, spoken input or output alongside the text.
Both modalities share the same thread, memory, and identity. This pattern also offers multimodal experience where the agent takes in real-time video, reasons over what it sees, and responds with voice.
Key design decisions & limitations
- Voice mode vs inline voice: Full-screen voice mode which is powerful for hands-free voice only context but hides the thread and could be frustrating when users want to scroll back or reference an earlier message where inline voice mode could be important.
- Push-to-talk (PTT) and voice-activity-detection (VAD) are the two common interaction mechanisms in this pattern. PTT suits noisy rooms, long prompts, and anything where being cut off mid-thought loses real content. VAD suits on-the-go back-and-forth, in situations where looking tapping on button isn’t practical.
- Limits complex output: This pattern limits output like code, tables, artefacts that can’t be spoken through TTS or presented in inline. The pattern either strip them out silently or asks to redirect users back to text chat thus breaking the voice flow.
5. Voice to text
This pattern has no actual voice conversation back and forth. The user speaks into a mic and gets transcribed text inside an input box or an artefact.
The most common everyday use case is quick messaging like texts, chats, notes using dictation using the keyboard mic on iOS or Android. Prompting AI tools works for the same reason, since long instructions are faster spoken than typed, and the model parses them as-is. Even chat assistants like ChatGPT and Claude now support dictation directly inside the text box. More purpose-built voice-to-text apps like Wispr Flow and Spokenly go further, removing filler words and returning formatted prose, casual or formal depending on the context.
Agent’s perceived role and interaction design
There is no perceived agent or character here. This is a purely utility tool, speech to text, with AI sitting silently in the background and no conversation back and forth.
Key design decisions & limitations
- Output accuracy is the whole product: Context, whether formal or casual, proper nouns like regional names, and tonality all have to land for the output to be usable. Get any of them wrong and the user has to go back and fix it by hand. Wispr Flow claims more “ready‑to‑send” text with 90% zero‑edit rate versus 71% for OpenAI, and independent reviews report around 97% accuracy on everyday speech thus more reliable on
- Correcting by voice is slower than typing: Which is why both Wispr Flow and Spokenly ship custom iOS keyboards rather than leaving you to fix things by speaking. A wake phrase for edits is the another option but it adds ambiguity, because the tool can’t reliably tell speech from instruction.
- Output adapts to context: Purpose-built dictation apps produces different outputs depending on where the text is going, casual for WhatsApp, formal for email, the app either infers this or the user sets it.
6. Scripted video + voice
This experience pairs pre-recorded video with voice input. A tutor talks to you on screen, pauses to listen when it’s your turn, then continues based on what you said.The AI doesn’t generate responses, it simply grades the user’s voice input and selects a filmed feedback that matches user input. Say the sentence right and the tutor smiles or nods. Get it wrong and the video jumps to “Try again.” The branches are all pre-shot and the AI just picks which one to play. Speak and Teuida are the two language learning apps I could find that build most clearly on this pattern.
Agent’s perceived role
The agent is framed as a constrained human teacher. Users perceive the person on screen as a real teacher, while the AI sits invisibly in the matching and branching layer.
Key design decisions & limitations
- Filmed footage creates a real environment which puts the user in what feels like a real place, a cafe, an office or somewhere real. Teuida shoots its scenes in first-person POV, like short K-dramas where the tutor talks straight to camera, so the person on screen is addressing you rather than performing for an audience. No other pattern creates that illusion of camera moving through a scene and interacting with a real human which makes the learning experience quite immersive.
- Filming doesn’t scale since every branch is a shoot with real actors, locations, editing. Adding new content or scaling for another language means the cost multiplies with every market. It also means content is slow to fix, since a line that isn’t working takes a reshoot rather than an edit. But this is a temporary constraint as AI video generation improves, you could film once and have the system generate localized variations without re-shooting.
7. Co-pilot transcript
This is an AI that listens to a voice conversation and produces structured artifacts like key decisions, action items, summaries, responsibility assignments (“who said what”), or follow-up emails. Granola, Otter, and Fireflies are some of the popular consumer apps which cover this use case.
Gong and Chorus do the same for sales calls to analyse objections, sentiment, and coachable moments. This pattern can extend to many other usecases like healthcare consultations, legal depositions, and user research, where the structured output of the final artefact is useful.
Agent’s perceived role
The agent here is an observer, not an active participant. The agent can be framed as a smart note-taker or the meeting bot. It listens to the conversation, and assists in structured output.
Key design decisions & limitations
- Active, passive and invisible observer: Granola actively assists during the call, it shows a live transcript, lets you ask questions, and surfaces real-time suggestions. This shapes how the meeting unfolds. Fireflies listens silently and only shows you what it found after. Active assistance can distract or empower depending on meeting culture while passive keeps focus on human conversation. Some products like Cluely.ai go further and operate invisibly in assessment or interview contexts with no visible recording indicator which could be problematic from a transparency standpoint.
- How the agent joins decides how people behave: While products like Granola capture audio silently from your device, Zoom and Fireflies send a visible bot into a live meeting which often makes the other participants awkward or conscious. The choice could have real behavioral consequences as people are quieter and more careful when they see a bot in the participant list than when they don’t know they’re being recorded.
- Automate sharing or keep human-in-the loop: When the AI assembles a meeting summary, products offer choices to share it automatically, or require human approval first. Auto-sharing is fast but risks a mistake going out under your name like a misheard name, an action item that didn’t actually happen.
- Make AI work transparent : Users need to verify accuracy. Visual distinction between human and AI written content, source and screenshot links to transcript moments all let users see the evidence. Without transparency, the summary is either trusted entirely or verified by re-listening.
8. Ambient / wake-word
An ambient device can be a phone, smart speaker, smart glasses, a car system or even a social robot. This pattern dates back to 1961, when IBM demonstrated Shoebox, a machine that could recognise a small vocabulary of spoken numbers and commands.
Earlier voice assistants waited for carefully phrased commands like “Hey Siri,” “Alexa,” followed by exact command word to recognise intent, which put the burden on the user to learn the right commands or serve a narrow use cases. Today’s systems are moving toward conversational, contextual and increasingly agentic interfaces that can understand a messy request, remember context and interact with external tools like calendars, maps, contacts, search indexes, music libraries and smart-home devices.
Agent’s perceived role and interaction design
The agent can be mostly framed as a command executor and not as a companion or thinking partner. Wake-word detection triggers a short listening window, followed by a single utterance and a brief response. There is no session or a thread. Every interaction effectively starts cold, with optional follow-up windows that briefly skip the wake word.
Key design decisions & limitations
- Getting the wake word right is essential: Triggering when nobody spoke makes the device look stupid or missing when someone did is infuriating. Every other decision here is followed after this one.
- There is no persistent UI to teach capability: Users have to already know the wake word and remember what’s possible. Platforms patch this with onboarding, “things to try” cards, proactive suggestions and ambient screen prompts, none of which have worked well.
- Answers have to be short: There is minimal or no UI, so the user can’t re-read what the agent said. Anything longer than a few sentences has to be remembered. Like the phone call pattern, the agent ends up confirming out loud what it heard and what it’s about to do, because saying it back is the only way to check.
Conclusion
For most of the last century, voice interfaces solved the problem of intent matching. Radio Rex listened for a frequency; Siri and Alexa listened for commands. That has improved enormously, and what it opened up are new use cases and experiences like tutors that respond in real time, support calls good enough to mimic a human, meeting summaries accurate enough to act on.
Enabling these experiences requires new design choices. Should the AI have a face on screen or exist only as a voice? How should it sound, and what personality should it carry? Should it interact in real time or stay silent until the conversation is over? Should it be an active participant shaping the exchange or a passive observer? Each choice sets the relationship and expectation. As the medium matures, designing what the agent seems to be will matter as much as designing what it can do.
References:
- LiveKit Blog, 21 Feb 2026- “Turn Detection for Voice Agents: VAD, Endpointing, and Model‑Based Detection.” explains voice‑first agent flows, turn detection, and how audio‑only vs chat‑integrated interactions work.
- Sleek Design. “AI Chatbot Mobile App Design Template | Edit with AI.” Sleek.design, shows and describes a dedicated full‑screen “voice mode” screen alongside a normal chat thread, illustrating the full‑screen vs inline pattern.
- OpenAI. “Voice Activity Detection (VAD).” OpenAI API Docs — Realtime API Guide, defines VAD and explains how it detects when a user starts/stops speaking in real time.
- Voicebot, “Timeline of Voice Assistants” — Chronological overview of voice assistant development from early systems through Siri, Alexa, and Google Assistant.
- Google conversation design guidance — Best practices for designing conversational AI flows, including openings, closings, and handoff patterns.
- Vapi conversation design playbook — Practical guide to voice agent design with real conversion metrics and persona guidance.
- “Over the Anthill” Substack analysis of Granola — Analysis of Granola’s partial autonomy model and human-centered design approach.
- Cognitive Revolution interview with Granola co-founder Sam Stephenson — Discussion of Granola’s system-audio capture, privacy philosophy, and calm design principles.
- Clark Boyd, “The Past, Present and Future of Speech Recognition Technology” — Historical account of speech recognition development from Bell Labs to modern neural networks.
- Sharang Sharma , Conversation design principles — Elements of Conversation system for crafting voice experiences for chat interfaces, voice-first devices, and social robots
- Sharang Sharma, So your AI wants a personality — Emerging personality patterns that drive differentiation in AI products
8 voice AIUX patterns was originally published in UX Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.
