Can LiveKit handle thousands of concurrent AI voice sessions without crashing?

All answers to this question.

Scaling is where LiveKit really pulls ahead. It uses a "Worker" model where your agent logic runs in separate processes. When a user joins a room, the LiveKit server dispatches a "Job" to an available worker. This means you can scale horizontally by just adding more pods in a Kubernetes cluster. Last year, we scaled our customer service bot to handle 500 concurrent AI voice calls across three different regions. The orchestration is handled by the LiveKit Agent Server, so you don't have to write your own load balancer from scratch.

   Answered 2025-02-15 by Deborah Foster

Is it better to use the hosted LiveKit Cloud or self-host the entire stack on AWS to save on costs?

   Answered 2025-02-18 by Christopher Gray

• For a scaling startup, I’d suggest LiveKit Cloud first. Self-hosting WebRTC is a nightmare because you have to manage TURN servers and global latency yourself. The Cloud version takes care of the "SFU" (Selective Forwarding Unit) logic, which is the hardest part. You still run your own AI voice agent logic on your own servers, so you keep control over your proprietary code and data. We made the switch in mid-2024 and it saved us about 30 hours of DevOps work every month while actually improving our global connection reliability.

     Commented 2025-02-20 by Steven Moore

The observability tools they provide make it much easier to track down why a specific AI voice session might be lagging.

   Answered 2025-02-21 by Lisa Bennett

• Spot on, Lisa. Having a dashboard that shows the STT-to-LLM-to-TTS pipeline timing in real-time is a lifesaver when you're trying to optimize your costs and performance.

     Commented 2025-02-23 by Thomas Reed