# Tambourine - Complete Documentation

> Open-source AI voice dictation platform. Real-time speech-to-text with AI formatting,
> multi-provider support (Groq, Deepgram, Whisper, etc.), cross-platform desktop app.

## Project Overview

## Why?

**Your voice, any app.** Tambourine gives you a universal voice-to-text interface that works everywhere: emails, messages, documents, code editors, terminals. Press a hotkey, speak, and your words are typed at your cursor. No copy-pasting, no app switching, no limitations.

**Speak at the speed of thought.** Typing averages 40-50 wpm, but speaking averages 130-160 wpm. Capture ideas before they slip away, and give your hands a break from the keyboard.

**AI that understands you.** Unlike raw transcription, Tambourine uses AI to format your speech into clean text—removing filler words, adding punctuation, and applying your personal dictionary for technical terms and proper nouns.

**Why not native dictation?** Built-in dictation is not personalized but Tambourine can be customized to your speaking and writing style, and with a personal dictionary for uncommon terms.

**Why not proprietary tools?** Unlike Wispr Flow or Superwhisper, this project gives you full control and transparency.

**Fully customizable.** This is your voice interface, built your way:
- **Choose your AI providers** — Pick your STT (Cartesia, Deepgram, AssemblyAI, Speechmatics, Azure, AWS, Google, Groq, OpenAI) and LLM (Cerebras, OpenAI, Anthropic, Gemini, Groq, OpenRouter), run fully local with Whisper and Ollama, or add more from [Pipecat's supported services](https://docs.pipecat.ai/server/services/supported-services)
- **Customize the formatting** — Modify prompts, add custom rules, build your personal dictionary
- **Extend freely** — Built on [Pipecat](https://github.com/pipecat-ai/pipecat)'s modular pipeline, fully open-source

## Platform Support

| Platform | Compatibility |
| -------- | ------------- |
| Windows  | ✅             |
| macOS    | ✅             |
| Linux    | ⚠️             |
| Android  | ❌             |
| iOS      | ❌             |

## Features

- **Dual-Mode Recording**
  - Hold-to-record: `` Ctrl+Alt+` `` - Hold to record, release to stop
  - Toggle mode: `Ctrl+Alt+Space` - Press to start, press again to stop
- **Real-time Speech-to-Text** - Fast transcription with configurable STT providers
- **LLM Text Formatting** - Removes filler words, adds punctuation using configurable LLM
- **Customizable Prompts** - Edit formatting rules, enable advanced features, add personal dictionary
- **In-App Provider Selection** - Switch STT and LLM providers without restarting
- **Automatic Typing** - Input text directly at focused position
- **Recording Overlay** - Floating visual indicator
- **Transcription History** - View and copy previous dictations
- **Paste Last Transcription** - Re-type previous dictation with `Ctrl+Alt+.`
- **Auto-Mute Audio** - Automatically mute system audio while dictating (Windows/macOS)
- **Misc.** - System tray integration, microphone selection, sound feedback, configure hotkeys

## Planned Features

- **Context-Aware Formatting** - Automatically detect which application is focused and tailor formatting accordingly. Email clients get proper salutations and sign-offs, messaging apps get casual formatting, code editors get syntax-aware output with proper casing and punctuation.
- **Voice-Driven Text Modification** - Highlight existing text and describe how to modify it. Select a paragraph and say "make this more formal" or "fix the grammar" to transform text in place.
- **Voice Shortcuts** - Create custom triggers that expand to full formatted text. Say "insert meeting link" to paste your scheduling URL, or "sign off" for your email signature.
- **Auto-Learning Dictionary** - Automatically learn new words, names, and terminology from your usage patterns rather than requiring manual dictionary entries.
- **Observability and Evaluation** - Integrate tooling from Pipecat and other voice agent frameworks to track transcription quality, latency metrics, and formatting accuracy. Use insights to continuously optimize your personal dictation workflow.
- **Hosted Service** - Optional cloud-hosted backend so you can use Tambourine without running the Python server locally.

## Architecture

```
┌─────────────────────────────────────────────────────────────┐
│                      Tauri App (app/)                       │
│  - Global hotkeys (Ctrl+Alt+Space, Ctrl+Alt+`)              │
│  - Rust backend for keyboard and audio controls             │
│  - React frontend with SmallWebRTC client                   │
│  - System tray with show/hide toggle                        │
└─────────────────────────────┬───────────────────────────────┘
                              │
                          API :8765
                              ▼
┌─────────────────────────────────────────────────────────────┐
│                  Python Server (server/)                    │
│  - Pipecat SmallWebRTC for audio streaming                  │
│  - STT providers (Cartesia, Deepgram, Groq, and more)       │
│  - LLM formatting (Cerebras, OpenAI, Anthropic, and more)   │
│  - Runtime config via WebRTC data channel (RTVI protocol)   │
│  - Returns cleaned text to app                              │
└─────────────────────────────────────────────────────────────┘
```

## Prerequisites

- Rust
- Node.js
- pnpm
- Python 3.13+
- uv (Python package manager)

### Linux Dependencies

```bash
sudo apt-get install libwebkit2gtk-4.1-dev build-essential curl wget file \
  libxdo-dev libssl-dev libayatana-appindicator3-dev librsvg2-dev libgtk-3-dev
```

## Permissions

### Microphone Access

When you first use Tambourine, your operating system will prompt you to grant microphone access. Accept this permission to enable voice dictation.

### macOS Accessibility Permissions

On macOS, Tambourine needs accessibility permissions to type text at your cursor position.

- **Running the built app**: Grant accessibility access to "Tambourine"
- **Running in development**: Grant accessibility access to the application you run the code from:
  - If running from VS Code: Add "Visual Studio Code"
  - If running from Terminal: Add "Terminal" (or your terminal app like iTerm2)

## Quick Start

> ⚠️ **Build in Progress**
> This project is under active development. Core features work well, but expect breaking changes to the code, architecture, and configuration as the project evolves.

### 1. Get API Keys

Sign up and get API keys for the providers you want to use. Some providers with generous free tiers as of this writing:
- Cartesia: https://cartesia.ai (STT)
- Cerebras: https://cloud.cerebras.ai (LLM)
- Gemini: https://aistudio.google.com (LLM)
- Groq: https://console.groq.com (STT/LLM)

**Or run fully local:**
```bash
ollama run llama3.2
```
Then in your `.env` file, set `OLLAMA_BASE_URL=http://localhost:11434` for local LLM and `WHISPER_ENABLED=true` for local STT.

### 2. Set Up the Server

```bash
cd server

# Copy environment template and add your API keys
cp .env.example .env

# Install dependencies
uv sync

# Start the server
uv run python main.py
```

### 3. Set Up the App

```bash
cd app

# Install dependencies
pnpm install

# Start development mode
pnpm dev
```

### 4. Usage

1. Start the server first (`uv run python main.py`)
2. Start the app (`pnpm dev`)
3. Use either shortcut:
   - **Toggle**: Press `Ctrl+Alt+Space` to start, press again to stop
   - **Hold**: Hold `` Ctrl+Alt+` `` while speaking, release to stop
4. Your cleaned text is typed at your cursor

## Server Commands

```bash
cd server

# Start server (default: localhost:8765)
uv run python main.py

# Start with custom host/port
uv run python main.py --host 0.0.0.0 --port 9000

# Enable verbose logging
uv run python main.py --verbose
```

## Docker Deployment

Run the server in Docker instead of installing Python dependencies locally.

```bash
cd server

# Copy environment template and add your API keys
cp .env.example .env

# Build and start the container
docker compose up --build -d

# View logs
docker compose logs -f

# Stop the container
docker compose down

# Update to latest code
docker compose down && docker compose up --build -d
```

The `.env` file is read at runtime (not baked into the image), so your API keys stay secure.

## App Commands

```bash
cd app

# Development
pnpm check         # Run all checks (lint + typecheck + knip + test + cargo)
pnpm dev           # Start Tauri app in dev mode

# Production Build
pnpm build         # Build for current platform
```

## API Reference

The server exposes HTTP endpoints on port 8765 (default). Sample endpoints:

- `GET /health` - Health check for container orchestration
- `GET /api/providers` - List available STT and LLM providers

See `server/main.py` for all endpoints. All endpoints are rate-limited.

## Configuration

### Server Configuration (.env)

Copy `.env.example` to `.env` and add API keys for at least one STT and one LLM provider. See the example file for all supported providers including Deepgram, Cartesia, OpenAI, Anthropic, Cerebras, Groq, AWS, and more. Additional [Pipecat-supported providers](https://docs.pipecat.ai/server/services/supported-services) can be added easily.

### App Configuration

The app connects to `localhost:8765` by default via WebRTC. Settings are persisted locally and include:

- **Providers** - Select active STT and LLM providers from available options
- **Audio** - Microphone selection, sound feedback, auto-mute during recording
- **Hotkeys** - Customize toggle and hold-to-record shortcuts
- **LLM Formatting Prompt** - Three customizable sections:
  - Core Formatting Rules - Filler word removal, punctuation, capitalization
  - Advanced Features - Backtrack corrections ("scratch that"), list formatting
  - Personal Dictionary - Custom words

### Data Management

Tambourine supports exporting and importing your configuration data, making it easy to backup settings, share configurations, or try community examples.

#### Export Data

Go to **Settings > Data Management** and click the export button. Select a folder and Tambourine exports 5 files:

| File                              | Description                                                |
| --------------------------------- | ---------------------------------------------------------- |
| `tambourine-settings.json`        | App settings (hotkeys, providers, audio preferences)       |
| `tambourine-history.json`         | Transcription history entries                              |
| `tambourine-prompt-main.md`       | Core formatting rules                                      |
| `tambourine-prompt-advanced.md`   | Advanced features (backtrack corrections, list formatting) |
| `tambourine-prompt-dictionary.md` | Personal dictionary for custom terminology                 |

#### Import Data

Click the import button in **Settings > Data Management** and select one or more files (`.json` or `.md`). Tambourine auto-detects file types from their content.

For history imports, you can choose a merge strategy:
- **Merge (skip duplicates)** - Add new entries, skip existing ones
- **Merge (keep all)** - Append all imported entries
- **Replace** - Delete existing history and use imported entries

#### Using Examples

The `examples/` folder contains ready-to-use prompt configurations for different use cases.

To use an example:
1. Open **Settings > Data Management**
2. Click the import button
3. Navigate to `examples/<example-name>/`
4. Select all three `.md` files
5. Click Open

Your prompts will be updated immediately. You can further customize them in **Settings > LLM Formatting Prompt**.

## Tech Stack

- **Desktop App:** Rust, Tauri
- **Frontend:** TypeScript, React, Vite
- **UI:** Mantine, Tailwind CSS
- **State Management:** Zustand, Tanstack Query, XState
- **Backend:** Python, FastAPI
- **Voice Pipeline:** Pipecat
- **Communications:** WebRTC
- **Validation:** Zod, Pydantic
- **Code Quality:** Biome, Ruff, Ty, Clippy

## Acknowledgments

Built with [Tauri](https://tauri.app/) for the cross-platform desktop app and [Pipecat](https://github.com/pipecat-ai/pipecat) for the modular voice AI pipeline.

## LLM-Friendly Documentation

For AI assistants and automated tools:
- [llms.txt](https://tambourinevoice.com/llms.txt) - Navigation index
- [llms-full.txt](https://tambourinevoice.com/llms-full.txt) - Complete documentation

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup and guidelines.

## Support

If you find Tambourine useful, here are ways to support the project:

- **Star the repo** — It helps others discover the project and motivates development
- **Report issues** — Found a bug or have a feature request? [Open an issue](https://github.com/kstonekuan/tambourine-voice/issues)
- **Join Discord** — Connect with the community for help and discussions in our [Discord server](https://discord.gg/dUyuXWVJ2a)
- **Contribute** — Check out [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines on how to contribute

## License

[AGPL-3.0](LICENSE)

---

## Contributing Guide

# Contributing to Tambourine Voice Dictation

Thanks for your interest in contributing to Tambourine! This guide will help you get started.

If you find Tambourine useful, consider giving the repo a star — it helps others discover the project and keeps contributors motivated. You can also join the [Discord server](https://discord.gg/dUyuXWVJ2a) to connect with the community.

## Development Setup

### Prerequisites

- **Rust** (use `rustup`)
- **Node.js 22+** and **pnpm**
- **Python 3.13+** and **uv**
- **Linux only**: `libwebkit2gtk-4.1-dev`, `build-essential`, `libxdo-dev`, `libssl-dev`, and other Tauri dependencies

### Server Setup

```bash
cd server
cp .env.example .env   # Add your API keys
uv sync                # Install dependencies
uv run python main.py  # Start the server (default: localhost:8765)
```

### App Setup

```bash
cd app
pnpm install   # Install dependencies
pnpm dev       # Start Tauri app in dev mode
```

## Code Quality

Pre-commit hooks automatically run linting, formatting, and type checking on commit. You can also run checks manually:

### TypeScript (app/)

```bash
pnpm lint       # Biome linting with auto-fix
pnpm typecheck  # TypeScript type checking
pnpm knip       # Detect unused code
pnpm check      # Run all checks (lint, typecheck, knip, test, cargo)
```

### Python (server/)

```bash
uv run ruff check --fix  # Linting with auto-fix
uv run ruff format       # Code formatting
uv run ty check          # Type checking
```

### Rust (app/src-tauri/)

```bash
cargo clippy --all-targets --all-features  # Linting
cargo fmt                                   # Formatting
```

Or use the pnpm wrapper from the app directory:

```bash
pnpm cargo:clippy
pnpm cargo:fmt
pnpm cargo        # Run all Rust checks
```

## Testing

```bash
# TypeScript
cd app && pnpm test

# Python
cd server && uv run pytest

# Rust
cd app && pnpm cargo:test
# or: cd app/src-tauri && cargo test
```

## Commit Conventions

Use descriptive commit messages with a type prefix:

- `Feat:` New features
- `Fix:` Bug fixes
- `Chore:` Maintenance, dependency updates
- `Docs:` Documentation changes
- `Refactor:` Code refactoring without behavior changes

Example: `Feat: add support for Azure Speech provider`

## Code Style & Philosophy

### Typing & Pattern Matching

- Prefer **explicit types** over raw dicts—make invalid states unrepresentable where practical
- Prefer **typed variants over string literals** when the set of valid values is known.
- Use **exhaustive pattern matching** (`match` in Python and Rust, `ts-pattern` in TypeScript) so the type checker can verify all cases are handled
- Structure types to enable exhaustive matching when handling variants
- Prefer **shared internal functions over factory patterns** when extracting common logic from hooks or functions—keep each export explicitly defined for better IDE navigation and readability

### Forward Compatibility

Client and server should evolve independently:

- **Unknown values**: Parse to an explicit `Unknown*` variant (never `None`), log at warn level, preserve raw data, gracefully ignore instead of raising exception

### Self-Documenting Code

- **Verbose naming**: Variable and function naming should read like documentation
- **Strategic comments**: Only for non-obvious logic or architectural decisions; avoid restating what code shows

## Pull Request Process

1. Fork the repository and create a feature branch from `main`
2. Make your changes and ensure all checks pass (`pnpm check` in app, CI will run server checks)
    * Pre-commit hooks run automatically and will auto-fix most formatting issues and check for linting / type issues
    * If a hook fails, you can review the output, stage the auto-fixed files, and commit again
3. Write clear commit messages following the conventions above
4. Submit a pull request to `main` with a description of your changes

## Community Guidelines

Be respectful and constructive in all interactions. We're building this together and value contributions of all kinds—code, documentation, bug reports, and feature suggestions.

## Adding New Providers

STT and LLM providers are defined in `server/services/provider_registry.py`:

1. Add enum value to `STTProviderId` or `LLMProviderId` in `protocol/providers.py`
2. Import the pipecat service class in `provider_registry.py`
3. Add a provider config entry to `STT_PROVIDERS` or `LLM_PROVIDERS`
4. Add the environment variable to `.env.example`

See existing providers for credential mapper patterns.

## Examples

The `examples/` directory contains pre-built prompt configurations for different domains. Community contributions are welcome.

### Creating a New Example

1. Customize your prompts in **Settings > LLM Formatting Prompt**
2. Go to **Settings > Data Management** and click Export
3. Copy the 3 `.md` files to a new directory in `examples/` (use lowercase with hyphens, e.g., `medical-transcription`)
4. Test by importing and performing dictation

### Updating LLM Documentation

When modifying README.md or CONTRIBUTING.md, regenerate llms-full.txt:

```bash
./scripts/generate-llms-full.sh
```

## Questions?

Open an issue, start a [GitHub Discussion](https://github.com/kstonekuan/tambourine-voice/discussions), or join the [Discord server](https://discord.gg/dUyuXWVJ2a).