Jarvis
Iron-Man-style AI assistant that controls your Mac by voice.
Overview
A voice-and-text AI assistant where Claude is the brain, an AppleScript-powered MCP server with 60 macOS tools is the hands, and a React HUD is the face. Say "open YouTube and find the latest Claude videos" — Jarvis plans the steps, runs the right Mac tools, and reports back, with a confirmation gate before anything irreversible.
My role
Solo builder — agent loop, MCP server, voice pipeline, web HUD
Highlights
- 01
60 MCP tools across 18 macOS categories — Chrome, Mail, Telegram, Calendar, Files, Music, Shortcuts, and more
- 02
~1–1.5s voice round-trip — local whisper.cpp STT, always-listening "Jarvis" wake word with barge-in, Claude tokens streamed straight into ElevenLabs TTS
- 03
LiveKit voice agent (Python) — talk to Jarvis from your phone or browser anywhere while actions execute on your Mac
Key features
- Provider-agnostic agent loop streaming step, confirm, and final events over WebSocket
- Wake-word conversation loop — acknowledgment chime, speech that starts while the model is still writing, follow-up window without repeating the wake word
- Long-term memory: durable facts, a searchable activity diary with 14-day recall, sessions that survive restarts
- Iron-Man HUD web UI — voice, chat, dashboard, routines, activity, and memory pages behind a WebGL aurora
- Safety gate — irreversible or outbound tools are held for interactive confirmation; content read from web/mail/messages is treated as untrusted
- Cron-backed routines scheduler with a friendly every-day/weekdays/weekly builder