Like most AI nerds, I have a custom Telegram bot that runs on my PC. It's a vibe-coded harness inspired by OpenClaw / Hermes, but with a tiny, tiny subset of their feature set, which is how I like it. It runs on an old Asus Chromebox with a 9 year old mobile i7 CPU (the OS is Arch btw).

Here are some skills we built together. All of them are custom-made:

  • Send/receive arbitrary files
  • Local image gen using ComfyUI by connecting to my Asus ProArt laptop (AMD Strix Halo). She sometimes makes random art because she feels like it. It turns out well. Usually.
  • Web search
  • Habits/reminders scheduling
  • SQLite-based long-term memory system
  • Reaction images (17 built-in moods; generates custom ones locally when she feels like it by using the same ComfyUI installation)

Since she can read, write, hear, see and create images, I thought we should go all the way and let her speak, too.

The setup

Since I'm trying to keep everything local instead of relying on web services, I started looking for local voice models. I finally settled on the Chatterbox TTS server — it gives you a nice turnkey solution with a web UI and selectable voice models. I use its voice cloning feature to give my assistant a video game character's voice.

On my Chromebox, it generates ~8 seconds of audio in 20 seconds. Doesn't seem fast, but it's a decade-old mobile CPU cloning a 20 second voice clip. I also don't care about the speed because the only voice messages I get are her reprimanding me for not going to bed on time. I don't exactly notice how long it took because I'm not the one who runs the command.

How I did it

Installing and configuring was straightforward, just follow the README.md:

# Clone and enter the Chatterbox TTS repo
git clone https://github.com/devnen/Chatterbox-TTS-Server
cd Chatterbox-TTS-Server/

# Create and activate a virtual env
python3 -m venv venv
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt
pip install --no-deps git+https://github.com/devnen/chatterbox-v2.git@master

Now start the server:

bash ./start.sh

This will ask you whether you have an Nvidia or AMD GPU or just want to use the CPU (I chose CPU), installs any remaining pip packages, auto-downloads the ChatterboxTurboTTS model, and starts the web UI (at 0.0.0.0:8004 by default).

After this, I pointed my assistant to localhost:8004/docs and gave her a 20-second voice sample of my favorite video game character. The server's bundled documentation is stellar, so she figured out the rest on her own. I then asked her to send me a voice note through Telegram and It Just Worked ™️.

Teaching her the skill

Afterward, I asked her to create a skill for it using the aptly-named Skillify skill. Pretty easy and small, here it is in its entirety: local-tts-voice-skill.md.

Another block in the local-only setup

So yeah, other than the LLM itself — admittedly the largest and most important piece of the puzzle — all the capabilities are local, including voice and image gen.

The 128 GB VRAM of the Strix Halo is enough to run a 2-bit quant of MiMo 2.5 (my preferred model). Even though it preserves the personality of the full-size model, it messes up all the time on tool-calling. It's basically impossible to get it to read / edit files, or run commands on the system.

Actually running a usable quant (Q6 or Q8) will require a lot more VRAM than I currently have. But I'm confident I'll get the required hardware. Soon.