Hi HN — I’m launching an open-source WhatsApp AI Voice Agent for phone calls.
Tech stack: It runs on VideoSDK for the SIP gateway, bridging WebRTC ↔ SIP under the hood. For the AI side you can plug in whatever stack you prefer (LLM + STT + TTS). The repo includes example configs.
Why open-source? Most WhatsApp/voice AI projects out there are closed or tied to a single vendor. I wanted something people can actually hack on, fork, and extend — whether that’s experimenting with different voices, building domain-specific agents, or integrating with CRMs.
Performance: End-to-end round-trip latency is ~400–600ms in typical setups. With faster STT/TTS backends there’s headroom to improve this.
I’d love feedback on use cases you’d actually want to build with this: customer support lines, personal AI assistants, language tutors, appointment scheduling, etc. Curious what directions the HN crowd would push this in.
Hi HN, I'm excited to share our new open-source project: an AI voice agent specifically designed for call centers. This project aims to streamline customer interactions and reduce the workload on human agents by automating initial call handling.
Imagine using it to manage customer inquiries, handle reservations, or conduct surveys without human intervention. It's a game-changer for businesses looking to improve efficiency.
Key features include:
- Real-time, low-latency voice conversation.
- A cascading pipeline using Deepgram for STT, OpenAI (GPT-4o) for LLM, and ElevenLabs for TTS (customizable).
- Advanced turn detection and voice activity detection (VAD) for smooth, natural conversations.
- Fully open-source and easily customizable.
- Support for Agent2Agent and MCP protocols.
Thanks for the mention. Curious—what challenges are you finding with Pipecat that you're hoping something else (like https://github.com/videosdk-live/agents) might fix?
Always looking to improve based on real gaps devs are facing.
That is one of our goals. If a solution is under 500 lines of Python, you should not need to pay $499 per month for it. We want to lower the barrier for developers and businesses to build their own voice agents.
The same technology can also enable businesses that never had live phone support to offer it affordably. The goal is augmentation and access, not mass replacement.
That risk is real. That’s why we made this open-source to empower smaller businesses to build responsible systems with their own logic, prompts, and escalation paths.
GitHub: https://github.com/videosdk-live/agents
That’s valid. But many people, including elderly users, prefer voice interfaces. Our system can serve those customers without requiring a smartphone or web access.
Fair point. But when implemented properly, these agents can reliably handle narrow, production-grade tasks like appointment reminders or smart call routing.
That is a fantastic idea. We’ve tested it on low-resource hardware. It’s SIP-agnostic and modular, making it ideal for home or SMB setups. We would love to highlight your RPi implementation if you publish it.
GitHub: https://github.com/videosdk-live/agents
You absolutely can. Our framework can answer calls, run speech-to-text, analyze intent, and respond with LLMs, making it a great defensive tool against spam or scam calls.
Valid concern. That’s why we focused on responsible use cases such as appointment bots, IVRs, or call reminders, not spam. Our project is open-source, modular, and designed for ethical, contextual automation.
Hi I am Sagar, We just open-sourced a complete framework to build an AI-powered telephony agent that can handle both inbound and outbound calls—using Python, SIP, and cloud LLMs like OpenAI or Gemini.
You can use it to create smart appointment bots, voice feedback collectors, or even enterprise IVR systems. It’s modular (plug in your SIP provider or AI model), production-ready, and extensible for real-time workflows.
Features include:
SIP & VoIP call handling (Twilio, Plivo, etc)
LLM-integrated AI agent (customizable prompt & tools)
Chatterbox is great for local/private TTS with Resemble AI.
voice agent SDK is broader it's full real-time voice infra with STT, LLM, TTS, memory, and RAG built in. You can plug in Resemble, ElevenLabs, etc., and deploy across web, mobile, and telephony with <80ms latency.
Totally fair. The space moves fast, and it's smart to be skeptical. Here's how VideoSDK Real-Time AI Agents stand out from OpenAI agents SDKs and others:
1. Voice infra included
OpenAI agents handle logic and memory, but they don’t include real-time audio infra.
VideoSDK gives you:
- <80ms global WebRTC latency
- Built-in turn-taking, VAD, and noise suppression
- Real-time voice across web, mobile, IoT, and telephony
2. Fully modular pipeline
No vendor lock-in. Swap STT, LLM, TTS, and avatars. Change models live per user or use case. Want ElevenLabs for tone and OpenAI for reasoning? Easy.
3. Native RAG + memory
Integrated long-term memory and retrieval help reduce hallucinations and keep conversations grounded.
4. Scale-ready
Deploy globally with one click using Agent Cloud or self-host with full control. Built for production use.
If you're building real-time, voice-first agents that need to work across platforms and scale reliably, this is purpose-built for that.
Happy to dive into your use case if you're exploring options.