LiveKit Blog

LiveKit SDK for ESP32: bringing voice AI to embedded devices

Since the launch of the LiveKit Agents framework, we’ve seen developers build voice AI experiences on web pages, mobile apps, and even embedded Linux devices like the Raspberry Pi Zero 2W. But we kept getting asked: can LiveKit run on even smaller microcontrollers like the ESP32? Can you build

Introducing the Grok Voice Agent API in partnership with xAI

Every day, millions of people around the world talk to Grok via first-party apps and in Tesla vehicles. The underlying model that brings Grok to life is a voice-to-voice model which understands the expressive range of human speech, and can generate correspondingly expressive responses; it can laugh and whisper and

Improved end-of-turn model cuts Voice AI interruptions 39%

We're excited to release a new iteration of our transformer-based end-of-turn detection model, v0.4.1-intl, which pushes the boundaries of accuracy and responsiveness. This update focuses on detecting speech completion for structured inputs and better generalization across languages. The latest MultilingualModel has been deployed to agents running

Announcing Agent Builder: a pathway to code

Today we’re introducing Agent Builder, a browser-based experience that lets you create, test, and deploy production-grade LiveKit voice agents in minutes, with zero local setup. These are not prototypes or sandbox toys. Every agent you build in Agent Builder is a real, fully functional LiveKit Agent running on LiveKit

Introducing LiveKit Phone Numbers: zero to ringing in 60 seconds

Today we’re launching LiveKit Phone Numbers, our first-party telephony service that lets you connect voice agents directly to the phone network in minutes, with no third-party SIP trunk required. Purchase US local or toll-free numbers straight from the LiveKit Cloud dashboard or CLI, attach a dispatch rule to your

Streamline troubleshooting with Agent Observability

Your voice agent feels slow. One user reports it interrupted them mid-sentence. Another says it failed to complete a task. You know something went wrong, but figuring out what requires jumping between your LLM provider’s dashboard, your transcription service’s logs, and your own server logs, trying to correlate

Introducing LiveKit Inference: A unified model interface for voice AI

We’re excited to launch LiveKit Inference, a low-latency model gateway, purpose-built for voice AI. With just your LiveKit API key, you can use top-performing speech-to-text (STT), large language model (LLM), and text-to-speech (TTS) models. You no longer have to create and manage multiple provider accounts and integrations. We'

Deploy and scale agents on LiveKit Cloud

Over the past two years, as more teams pushed LiveKit voice agents to production, the same questions kept coming up: * How much CPU and memory do I allocate to my agent pools? * How do I handle sudden traffic spikes? * How can I instrument and optimize performance across sessions? Our Agents

Deploy and run voice and video agents on LiveKit Cloud

Bringing AI avatars to voice agents

Introducing integration with Tavus Video avatars aren't just gimmicks—they've become genuinely useful tools that developers and businesses actually want. We've been hearing this a lot from customers, especially in education, healthcare, mental wellness, and marketing. Everyone seems keen on turning their voice interactions

LiveKit’s Series B: Building the all-in-one platform for voice AI agents

In September 2023, together with OpenAI we unveiled ChatGPT Voice Mode. When OpenAI launched the new feature, we also released LiveKit Agents, an open source framework that made it easy for developers to build their own voice AI agents. Back then Voice AI was not a thing — most investors we

Say hello to improved Cloud analytics

When we originally introduced LiveKit Cloud, we also launched a realtime analytics and telemetry product to give you insights into how your users interacted with your LiveKit applications. Over the past two years, LiveKit Cloud has grown significantly, powering realtime applications ranging from AI assistants to robotic pile drivers to

Using a transformer to improve end of turn detection

One of the hardest problems to solve right now for voice AI applications is end-of-turn detection. In the context of conversational AI, turn detection is the task of determining when a user is done speaking and when an AI model can respond without unintentionally interrupting the user. Currently, the most

Yeah, but can it run DOOM?

This ubiquitous question is what I was asking myself a couple years ago at LiveKit's first hackathon. Could I play DOOM over LiveKit? Potentially ambitious for a 24-hour competition, but I had to solve this rite of passage. So, I partnered with our resident WebRTC expert, Raja, and

OpenAI and LiveKit partner to turn Advanced Voice into an API

LiveKit and OpenAI are partnering to help you build your own apps using the same technology powering ChatGPT’s new Advanced Voice feature.

Towards a future-aligned pricing model

LiveKit Cloud’s pricing model is different from the industry norm. You only pay for the resources you use: 1. Compute: the time your users spend connected to our servers 2. Bandwidth: the data your application transfers over LiveKit’s network For simplicity we chose to embed the cost of

LiveKit's Series A: Infra for the AI computing era

LiveKit has raised $22.5M in additional funding to build infrastructure for realtime voice and video-driven AI applications.

An open source stack for real-time multimodal AI

It feels impossible to keep up with everything happening in AI. There’s seemingly a new breakthrough every week. And even though we won’t have AGI in 2024, it’s interesting to consider what the world might look like when we do. In particular, the LiveKit team has been

A tale of two protocols: comparing WebRTC against HLS for live streaming

Do you remember the last time you watched a WWDC keynote? Did you watch it live? Did you read tweets about something you hadn't seen yet and wonder why? The answer is that Apple live streams these events using HLS, just like most other streaming video services, including

Streaming video from a canvas with WebRTC and React

With WebRTC you can live stream video from a canvas. This post is a step-by-step guide which shows you how. We use LiveKit’s WebRTC stack to build a real-time application for sending canvas video. Check out the full code. A lot of people know WebRTC as the technology that

Stream music over WebRTC using React and WebAudio

With technology built into every modern web browser, you can live stream audio to other people using just a URL. This post is a step-by-step guide which shows you how. We use WebAudio and LiveKit’s WebRTC stack to build a real-time application for listening to music with your friends.

Using WebRTC + React + WebAudio to create spatial audio

Real-time audio is a critical part of our modern digital lives. It enables us to connect with each other no matter where we are in the world. One of the big recent trends in real-time social applications is spatial audio (sometimes called positional audio). 0:00 /0:30 1× While

Mini tutorial: setting up a LiveKit Cloud project

LiveKit is an open-source WebRTC SFU and set of client libraries for all major platforms. Because LiveKit is open source, you can easily run the server yourself. Running a LiveKit server is pretty easy but it's even easier to use LiveKit Cloud and its generous free-tier. Creating a

Live conversations with AI using ChatGPT and WebRTC

Jarvis. Samantha. Joi. HAL. Science fiction has long dreamt of anthropomorphized AI. Between GPT, Claude, Bard and other LLMs it seems like we’re on the precipice of this becoming reality. While we’ve enjoyed exchanging texts with ChatGPT, the LiveKit team thought it would be more fun to see

Decentraland's Catalyst: using WebRTC to interact in the metaverse

Every decentralized platform builder faces this problem: which components should be decentralized, and to what extent? The metaverse project Decentraland tackled this when their original peer-to-peer messaging transport hit performance bottlenecks, which limited their users’ ability to chat and interact in real time. In this post, we’ll walk through

How we built a globally distributed mesh network to scale WebRTC

Any time you’ve used Zoom, Discord, or Twitter Spaces, you’ve interacted with a media server. A media server is responsible for allowing clients to do things like exchange camera or microphone data in real-time. It behaves quite differently from say, an HTTP server. A media server hosts sessions,