Open sourceActive

QuiLLMan

Voice chat example app built on Modal using Kyutai Lab's Moshi speech-to-speech model.

Open source page ↗

Field note

What it does

QuiLLMan is a complete voice chat application deployed serverlessly on Modal, with a React frontend and a Moshi websocket backend.

Capabilities

Available capabilities

GPU Auto-scaling Voice Chat Serverless Deployment Opus Audio Compression Bidirectional Websocket Streaming

Tags

Tags

WebSocket Voice AI Speech-to-Speech

Ways to use it

Ways to use it

sdk

python / https://fastapi.tiangolo.com/

sdk

javascript / Not filed

cli

python / Not filed

sdk

javascript / opus-recorder

sdk

javascript / ogg-opus-decoder

Product features

Product features

Addenda

Configuring vLLM for maximum throughput

Deploying vLLM on Modal

FastAPI Server

Serves both frontend and backend, exposed on Modal via @app.asgi_app().

Loading filings from the SEC EDGAR Feed

Moshi Websocket Server

Loads a Moshi model instance and maintains a bidirectional websocket connection with the client.

Opus Audio Compression

Compresses audio across the network using the Opus codec for efficient transmission.

Organizing a batch job on Modal

React Frontend

Static React app using Web Audio API to record microphone audio and play back model responses.

Run LLM inference at maximum throughput

Serving tokens at maximum throughput

Transforming SEC filings for batch processing

Utilities for loading filings from the SEC EDGAR Feed

Utilities for transforming SEC Filings