LiveKit Blog

Using a transformer to improve end of turn detection

One of the hardest problems to solve right now for voice AI applications is end-of-turn detection. In the context of conversational AI, turn detection is the task of determining when a user is done speaking and when an AI model can respond without unintentionally interrupting the user. Currently, the most

Yeah, but can it run DOOM?

This ubiquitous question is what I was asking myself a couple years ago at LiveKit's first hackathon. Could I play DOOM over LiveKit? Potentially ambitious for a 24-hour competition, but I had to solve this rite of passage. So, I partnered with our resident WebRTC expert, Raja, and

OpenAI and LiveKit partner to turn Advanced Voice into an API

LiveKit and OpenAI are partnering to help you build your own apps using the same technology powering ChatGPT’s new Advanced Voice feature.

Towards a future-aligned pricing model

LiveKit Cloud’s pricing model is different from the industry norm. You only pay for the resources you use: 1. Compute: the time your users spend connected to our servers 2. Bandwidth: the data your application transfers over LiveKit’s network For simplicity we chose to embed the cost of

LiveKit's Series A: Infra for the AI computing era

LiveKit has raised $22.5M in additional funding to build infrastructure for realtime voice and video-driven AI applications.

An open source stack for real-time multimodal AI

It feels impossible to keep up with everything happening in AI. There’s seemingly a new breakthrough every week. And even though we won’t have AGI in 2024, it’s interesting to consider what the world might look like when we do. In particular, the LiveKit team has been

A tale of two protocols: comparing WebRTC against HLS for live streaming

Do you remember the last time you watched a WWDC keynote? Did you watch it live? Did you read tweets about something you hadn't seen yet and wonder why? The answer is that Apple live streams these events using HLS, just like most other streaming video services, including

Streaming video from a canvas with WebRTC and React

With WebRTC you can live stream video from a canvas. This post is a step-by-step guide which shows you how. We use LiveKit’s WebRTC stack to build a real-time application for sending canvas video. Check out the full code. A lot of people know WebRTC as the technology that

Stream music over WebRTC using React and WebAudio

With technology built into every modern web browser, you can live stream audio to other people using just a URL. This post is a step-by-step guide which shows you how. We use WebAudio and LiveKit’s WebRTC stack to build a real-time application for listening to music with your friends.

Using WebRTC + React + WebAudio to create spatial audio

Real-time audio is a critical part of our modern digital lives. It enables us to connect with each other no matter where we are in the world. One of the big recent trends in real-time social applications is spatial audio (sometimes called positional audio). 0:00 /0:30 1× While

Mini tutorial: setting up a LiveKit Cloud project

LiveKit is an open-source WebRTC SFU and set of client libraries for all major platforms. Because LiveKit is open source, you can easily run the server yourself. Running a LiveKit server is pretty easy but it's even easier to use LiveKit Cloud and its generous free-tier. Creating a

Live conversations with AI using ChatGPT and WebRTC

Jarvis. Samantha. Joi. HAL. Science fiction has long dreamt of anthropomorphized AI. Between GPT, Claude, Bard and other LLMs it seems like we’re on the precipice of this becoming reality. While we’ve enjoyed exchanging texts with ChatGPT, the LiveKit team thought it would be more fun to see

Decentraland's Catalyst: using WebRTC to interact in the metaverse

Every decentralized platform builder faces this problem: which components should be decentralized, and to what extent? The metaverse project Decentraland tackled this when their original peer-to-peer messaging transport hit performance bottlenecks, which limited their users’ ability to chat and interact in real time. In this post, we’ll walk through

How we built a globally distributed mesh network to scale WebRTC

Any time you’ve used Zoom, Discord, or Twitter Spaces, you’ve interacted with a media server. A media server is responsible for allowing clients to do things like exchange camera or microphone data in real-time. It behaves quite differently from say, an HTTP server. A media server hosts sessions,

The end of Participant Minute pricing

We made LiveKit so every developer in the world could have access to a modern, end-to-end WebRTC stack for free. The growth of the project and community around it has been as stunning as the things we’ve seen developers build with LiveKit. We believe real-time, multiplayer applications that help

Announcing LiveKit Cloud

I recently spoke with an engineer whose company transforms agricultural vehicles like tractors into self-driving, autonomous robots for tasks like mowing, spraying and weeding. Seriously, this is straight out of Interstellar: Sometimes a farmer needs to take control of the machine—often deployed in a rural area with spotty internet

Cloud 100 Rising Star

We're honored for LiveKit to be named a Cloud 100 Rising Star [https://www.forbes.com/sites/rashishrivastava/2022/08/09/cloud-100-rising-stars-2022-meet-the-newest-cohort-of-cloud-innovators/?sh=324b4d1056dd] ! Each year, Forbes in partnership with Bessemer Venture Partners and Salesforce Ventures, compiles a list of the top 100 private cloud companies in the

LiveKit Community Day

The idea for LiveKit came when David and I tried adding real-time video to a side project. There just wasn't an open source, end-to-end stack that made working with WebRTC simple. Since our launch last July, the outpouring of love from builders and amazing projects using LiveKit has

Universal Egress

WebRTC–a low latency protocol with ubiquitous support across devices–is fantastic for last-mile delivery, but can't address every need a developer has when working with audio and video. An application may want to do things like store a session for future playback, relay a stream to a

Bringing Zoom's end-to-end optimizations to WebRTC

When we started LiveKit [https://github.com/livekit/livekit], our aim was to build an end-to-end, open source WebRTC stack accessible to all. After 20 months and nearly 1000 commits, we're releasing version 1.0 of LiveKit. This also includes 1.0 releases for these client SDKs: * JS

LiveKit coming to React Native!

Increasingly, developers are thinking multi-platform and how to get more done with fewer resources and maintenance overhead. React developers, in particular, naturally consider ReactNative as an entry point into mobile development. Since we launched LiveKit last summer, React Native has consistently been the most requested platform for us to support.

React Core and Components

If you're building with React, our React SDK makes it simple to add real-time audio and video to your app. We provide state management and media rendering utility components, allowing you to implement custom UI components and designs. The same SDK also includes pre-built components for constructing video

Real-time audio and video in the metaverse

In the last couple years, many exciting metaverse projects have started with visions to connect us through immersive, spatial environments; closer to how we connect with others in the physical world. A unifying principle among them is accessibility: a truly open metaverse should run in a browser, agnostic to hardware

Launch Week! 🚀

For the last few months, we’ve been quietly working on a bunch of things. Following in the footsteps of other OSS projects, we’ve lined up a whole week’s worth of announcements! Every day next week at 10am PT, we’ll drop something new. Here’s a preview

The metaverse needs open infrastructure

Pre-pandemic, we did most things offline. 2020 brought a sea change: we worked, shopped, studied, cooked, watched movies, exercised, dated, and even got married over the Internet — in short, we now did most things online by default. While it looked like the Brady Bunch credits, the moment we collectively began