News
AssemblyAI published a tutorial for building a server-side Agora voice agent using Universal-3.6 Pro Realtime, LLM Gateway and Text-to-Speech.
AssemblyAI has published a technical tutorial showing how to build a server-side voice agent for an Agora RTC channel using its speech-to-text, LLM Gateway and text-to-speech stack. The guide is centered on a Python bot that joins an Agora call like any other participant, listens to the caller and speaks replies back into the channel, with barge-in handled in the application loop.
Evidence and context
In the tutorial published on 29 Oct 2026, AssemblyAI says the bot combines Agora’s WebRTC transport with three AssemblyAI layers on one API key: Universal-3.6 Pro Realtime for streaming speech-to-text, LLM Gateway for model responses, and AssemblyAI Text-to-Speech for the spoken reply. The company says the architecture keeps audio routing on Agora while the conversation loop stays in user code.
The example uses 16 kHz, 16-bit, mono PCM end to end. According to the guide, Agora delivers the caller audio at 16 kHz, Universal-3.6 Pro Realtime accepts that rate directly, and the text-to-speech session is opened with sample_rate=16000 and encoding="pcm_s16le", so the bot does not need resampling code. AssemblyAI also says the tutorial is written against agora_python_server_sdk 2.4.10 and requires Python 3.10+.
AssemblyAI’s tutorial says Universal-3.6 Pro Realtime is used with built-in turn detection and reports two benchmark figures for the model: a 5.19% mean word error rate on AssemblyAI’s English voice-agent benchmark and a median 307 ms to final transcript on the Pipecat open benchmark, both as company-reported results. The guide also describes barge-in logic in which a real caller interruption cancels synthesis in progress and clears buffered audio in Agora, while short backchannels such as “mhm” or “yeah” are ignored.
The article also includes pricing references from AssemblyAI’s published materials. In the background material supplied with this request, Universal-3.6 Pro Realtime is listed at $0.45 per audio hour, while AssemblyAI Text-to-Speech is described as a separate service priced at $15 per 1 million input characters, with the cited report dated 29 Oct 2026. Those figures are attributed to AssemblyAI’s published pricing and launch coverage rather than a live check.
The tutorial notes several constraints. It is designed for one caller per bot, follows the first participant who speaks, and does not support Windows natively. For production, AssemblyAI says the server should run on Linux x86-64, while macOS is supported for development and testing. It also warns that an Agora App Certificate, if enabled, requires a valid temporary token for the bot’s channel and UID.
