---
sidebar_position: 1
sidebar_label: "Overview"
---

# Mobile App

The DotAgents mobile app puts AI agents in your pocket — chat by voice or text with full hands-free support, on iOS, Android, and the web.

---

## Overview

Built with Expo SDK 54 and React Native, the mobile app provides a portable interface to your DotAgents agents. It connects to your desktop instance's remote server or any OpenAI-compatible API endpoint.

### Key Capabilities

- Voice input with press-and-hold or hands-free VAD mode
- Text chat with streaming responses
- Text-to-speech for assistant replies
- Agent profile management
- Knowledge note editing
- Loop (recurring task) scheduling
- Operator dashboard for health checks, logs, tunnels, integrations, updater, and emergency actions
- Session history with search
- QR code connection setup
- Split chat view for multi-agent conversations

## Supported Platforms

| Platform | Status | Notes |
|----------|--------|-------|
| **iOS** | Full support | Requires development build (not Expo Go) |
| **Android** | Full support | Requires development build (not Expo Go) |
| **Web** | Supported | Speech recognition requires Chrome/Edge over HTTPS |

> **Important**: The app uses `expo-speech-recognition`, a native module not available in Expo Go. You must use a **development build** for native devices.

## Screens

### Chat Screen

The primary interface for conversations with your agent:

- **Text input** — Type messages and receive streaming responses
- **Voice input** — Press-and-hold the mic button for real-time transcription
- **TTS playback** — Assistant replies can be spoken aloud via `expo-speech`
- **Hands-free mode** — Toggle VAD (Voice Activity Detection) for no-hands interaction
- **Edit mode** — Release the mic while in edit state to place the transcript in the input box for review before sending

### Settings Screen

Configure your connection and preferences:

| Setting | Description |
|---------|-------------|
| **API Key** | Your API key (Bearer token) |
| **Base URL** | API endpoint URL |
| **Model** | Model identifier (e.g., `gpt-5.4-mini`) |
| **Environment** | Toggle between Local and Cloud |
| **Run API URL** | Endpoint for chat completions |
| **Manage API URL** | Endpoint for agent management |
| **Voice Preferences** | TTS voice, auto-play, language |

### Session List Screen

Browse and search your conversation history:

- Scrollable list of past sessions
- Search by content or title
- Tap to continue a previous conversation
- Delete sessions you no longer need

### Connection Settings Screen

Quick setup via QR code:

- Scan a QR code from your desktop app to auto-configure connection settings
- Manually enter connection details as an alternative
- Test connectivity with a health check

### Agent Edit Screen

Create and configure agent profiles directly on mobile:

- Set name, description, and system prompt
- Configure guidelines and properties
- Select available tools

### Knowledge Note Edit Screen

Manage durable knowledge notes from your phone:

- View existing knowledge notes
- Create new notes
- Edit note context, summary, body, tags, and references
- Delete outdated notes

### Loop Edit Screen

Schedule recurring tasks:

- Define the prompt to execute
- Set the interval (e.g., every 5 minutes, every hour)
- Choose which agent handles the loop
- Start, pause, and monitor loops

### Operations Screen

Trusted operator dashboard backed by the desktop remote server:

- View runtime status, health checks, recent errors, and audit events
- Inspect Cloudflare Tunnel, Discord, WhatsApp, push, updater, and MCP status
- Start or stop tunnel exposure
- Connect or disconnect Discord and WhatsApp integrations
- Trigger updater checks, reveal downloads, restart MCP services, restart the remote server, restart the app, or run an agent

### Split Chat Screen

Multi-agent conversation view:

- See responses from multiple agents side by side
- Compare agent outputs for the same query
- Switch between agents within a single interface

## Voice UX

### Press-and-Hold Mode

1. **Press and hold** the mic button
2. See **live transcription** as you speak
3. **Release to send** — the transcript is immediately sent to the agent
4. Or **release in edit mode** — the transcript goes into the text box for review

### Hands-Free Mode (VAD)

1. **Toggle** hands-free from the chat screen header (microphone icon)
2. The app **listens continuously** using Voice Activity Detection
3. When speech is detected, it **automatically transcribes**
4. On silence (end of speech segment), it **automatically sends**
5. Great for driving, cooking, or any hands-busy scenario

### Text-to-Speech

- Assistant replies can be read aloud via `expo-speech`
- Configure voice and auto-play preferences in Settings
- Works across all platforms (iOS, Android, Web)

## Architecture

```
┌─────────────────────────────────┐
│        Mobile App (Expo)         │
│                                 │
│  ┌───────────┐ ┌─────────────┐  │
│  │ Screens   │ │ Navigation  │  │
│  │ (React    │ │ (React      │  │
│  │  Native)  │ │  Navigation)│  │
│  └─────┬─────┘ └─────────────┘  │
│        │                        │
│  ┌─────▼─────────────────────┐  │
│  │  Store (AsyncStorage)     │  │
│  │  config, sessions,        │  │
│  │  profiles, message queue  │  │
│  └─────┬─────────────────────┘  │
│        │                        │
│  ┌─────▼─────────────────────┐  │
│  │  API Client               │  │
│  │  (OpenAI-compatible)      │  │
│  │  SSE streaming support    │  │
│  └─────┬─────────────────────┘  │
└────────┼────────────────────────┘
         │
         ▼
┌─────────────────────┐
│  Desktop Remote      │
│  Server (Fastify)    │
│  OR                  │
│  Any OpenAI-         │
│  Compatible API      │
└─────────────────────┘
```

### Key Libraries

| Library | Purpose |
|---------|---------|
| `expo-speech-recognition` | Native speech recognition |
| `expo-speech` | Text-to-speech |
| `expo-camera` | QR code scanning for connection setup |
| `@react-native-async-storage/async-storage` | Persistent config storage |
| `react-native-sse` | Server-sent events for streaming |
| `@react-navigation/native-stack` | Screen navigation |

### State Management

| Store | Purpose |
|-------|---------|
| `config.ts` | Persistent settings (API key, URLs, voice prefs) |
| `sessions.ts` | Local session history |
| `connectionManager.ts` | Connection pooling and recovery |
| `tunnelConnection.ts` | Tunnel status and reconnect state |
| `profile.ts` | Agent profile state |
| `message-queue.ts` | Queued messages for offline/slow processing |

## Connecting to Desktop

The mobile app connects to your desktop app's remote server:

1. **Start DotAgents desktop** and enable **Settings > Remote Server**
2. **On mobile**, go to **Connection Settings**
3. **Scan the QR code** displayed on your desktop, or manually enter the URL and API key
4. The mobile app now communicates with your desktop's agent engine

This gives the mobile app access to all your desktop's MCP tools, agent profiles, and conversation history.

## API Compatibility

The mobile app works with any OpenAI-compatible API endpoint:

- **OpenAI** — Direct connection to OpenAI's API
- **Groq** — Fast inference with Groq's API
- **Azure OpenAI** — Microsoft's OpenAI service
- **Ollama** — Local models at `http://localhost:11434/v1`
- **LM Studio** — Local models with OpenAI-compatible API
- **DotAgents Remote Server** — Your desktop's agent engine

The API key is sent as `Authorization: Bearer <API_KEY>`.

## Getting Started

### Prerequisites

- Node.js 20.19.4+ (Node 24.x recommended via `.nvmrc`)
- pnpm 9.x
- For native builds: Xcode (iOS) or Android Studio (Android)

### Install

```bash
git clone https://github.com/aj47/dotagents-mono.git
cd dotagents-mono
nvm use
pnpm install
```

### Run

```bash
# Start Metro bundler (choose platform in UI)
pnpm --filter @dotagents/mobile start

# Or run directly
pnpm --filter @dotagents/mobile ios      # iOS
pnpm --filter @dotagents/mobile android  # Android
pnpm --filter @dotagents/mobile web      # Web
```

### Development Build

If you see `Cannot find native module 'ExpoSpeechRecognition'`:

```bash
# Build and install the development app
pnpm --filter @dotagents/mobile android
# or
pnpm --filter @dotagents/mobile ios
```

## Troubleshooting

| Issue | Solution |
|-------|----------|
| `Cannot find native module 'ExpoSpeechRecognition'` | Use a development build, not Expo Go |
| Speech recognition not starting | Grant microphone/speech permissions |
| Web speech not working | Use Chrome or Edge over HTTPS |
| Cannot list agents | Verify Manage API URL and API key |
| No assistant response | Check Run API URL and model setting |
| Operations actions fail | Confirm the desktop remote server is reachable and the operator allowlists/API key are correct |

---

## Next Steps

- **[Voice Interface](/voice/overview)** — Advanced voice features
- **[Remote Server & Pairing](/desktop/remote-server)** — Pair mobile with your desktop agent engine
- **[AI Providers](/tools/providers)** — Configure API providers
- **[Desktop App](/desktop/overview)** — Full desktop experience
