alexandria-audiobook — AI 语音合成工具中文文档

Requirements

• Pinokio
• LLM server (one of the following):
• LM Studio (local) - recommended: Qwen3 or similar
• Ollama (local)
• OpenAI API (cloud)
• Any OpenAI-compatible API
• GPU: 8 GB VRAM minimum, 16 GB+ recommended — see compatibility table below
• Each TTS model uses ~3.4 GB; remaining VRAM determines batch size
• CPU mode available on all platforms but significantly slower
• RAM: 16 GB recommended (8 GB minimum)
• Disk: ~20 GB (8 GB venv/PyTorch, ~7 GB for model weights, working space for audio)

Installation

Option B: Google Colab (No Install Required)

No GPU or wrong OS? Run Alexandria on a free T4 GPU in your browser:

Requires a free ngrok account for the web UI tunnel. See the notebook for full instructions.

Option C: Docker (NVIDIA GPU)

For integration into automated pipelines or server deployments:

git clone https://github.com/Finrandojin/alexandria-audiobook.git
cd alexandria-audiobook
docker compose up --build

Requires Docker with the NVIDIA Container Toolkit. The web UI is available at http://localhost:4200. TTS models download on first use and are cached in a Docker volume. User data (uploads, voice configs, trained LoRA adapters, audio output) persists via bind mounts to the project directory.

Setup Tab

Configure connections to your LLM and TTS engine.

TTS Settings: - Mode - local (built-in engine) or external (connect to Gradio server) - Device - auto (recommended), cuda, cpu, or mps - Language - TTS synthesis language: English (default), Chinese, French, German, Italian, Japanese, Korean, Portuguese, Russian, Spanish, or Auto (let the model detect) - Parallel Workers - Batch size for fast batch rendering (higher = more VRAM usage) - Batch Seed - Fixed seed for reproducible batch output (leave empty for random) - Compile Codec - Enable torch.compile for 3-4x faster batch decoding (adds ~30-60s warmup on first generation) - Sub-batching - Split batches by text length to reduce wasted GPU compute on padding (enabled by default) - Min Sub-batch Size - Minimum chunks per sub-batch before allowing a split (default: 4) - Length Ratio - Maximum longest/shortest text length ratio before forcing a sub-batch split (default: 5) - Speaker Change Pause - Silence in milliseconds between different speakers during merge (default: 500) - Same Speaker Pause - Silence in milliseconds when the same speaker continues during merge (default: 250)

Prompt Settings (Advanced): - Generation Settings - Chunk size and max tokens for LLM responses - LLM Sampling Parameters - Temperature, Top P, Top K, Min P, and Presence Penalty - Banned Tokens - Comma-separated list of tokens to ban from LLM output (useful for disabling thinking mode on models like GLM4, DeepSeek-R1, etc.) - Prompt Customization - System and user prompts used for script generation. Defaults are loaded from default_prompts.txt and can be customized per-session in the UI. Click "Reset to Defaults" to reload the file-based defaults (picks up edits without restarting the app)

Dataset Builder Tab

Build LoRA training datasets interactively, one sample at a time.

• Create a project with a voice description and optional global seed
• Define samples — Set text and emotion/style per row
• Preview audio — Generate and listen to individual samples or batch-generate all at once
• Cancel batch — Stop a running batch generation without losing completed samples
• Save as dataset — Export the project as a training-ready dataset that appears in the Training tab
• Designed voices and Voice Designer descriptions drive the audio generation via Qwen3-TTS VoiceDesign model

Dataset Builder

```bash

List all dataset builder projects

curl http://127.0.0.1:4200/api/dataset_builder/list

Example: [sample.mp3](https://github.com/user-attachments/files/25276110/sample.mp3)

Quick Start

The interface is split into a 5-step core pipeline (green tabs, numbered) and advanced tools (blue tabs, unnumbered). You only need the core pipeline to produce an audiobook.

Screenshots

<img src="https://github.com/user-attachments/assets/874b5e30-56d2-4292-b754-4408fc53f5d6" width="30%"></img> <img src="https://github.com/user-attachments/assets/488cde02-6b93-47fa-874b-97a618ae482c" width="30%"></img> <img src="https://github.com/user-attachments/assets/4c0805a6-bb9d-42c1-a9ff-79bb29d0613c" width="30%"></img> <img src="https://github.com/user-attachments/assets/8e58a5bf-ed8f-4864-8545-1e3d9681b0cf" width="30%"></img> <img src="https://github.com/user-attachments/assets/531830da-8668-4189-a0dc-020e6661bfb6" width="30%"></img>

Update sample rows

curl -X POST http://127.0.0.1:4200/api/dataset_builder/update_rows \ -H "Content-Type: application/json" \ -d '{"name": "my_voice_dataset", "rows": [{"text": "Hello world.", "emotion": "cheerful"}]}'

Generate a single sample preview

curl -X POST http://127.0.0.1:4200/api/dataset_builder/generate_sample \ -H "Content-Type: application/json" \ -d '{"name": "my_voice_dataset", "description": "A warm male voice", "sample_index": 0, "samples": [{"text": "Hello.", "emotion": "cheerful"}]}'

Batch generate all samples

curl -X POST http://127.0.0.1:4200/api/dataset_builder/generate_batch \ -H "Content-Type: application/json" \ -d '{"name": "my_voice_dataset", "description": "A warm male voice", "samples": [{"text": "Hello.", "emotion": "cheerful"}]}'

Export Options

• Combined Audiobook - Single MP3 with all voices and natural pauses
• Individual Voicelines - Separate MP3 per line for DAW editing (Audacity, etc.)
• Audacity Export - One-click zip with per-speaker WAV tracks, LOF project file, and labels for automatic multi-track import into Audacity
• M4B Audiobook - Chaptered M4B (AAC) with per-chunk or auto-detected chapter markers for audiobook players (Audiobookshelf, Apple Books, VLC, etc.)

Option A: Pinokio (Recommended)

1. Install Pinokio if you haven't already 2. Open Alexandria on Pinokio: Install via Pinokio - Or manually: in Pinokio, click Download and paste https://github.com/Finrandojin/alexandria-audiobook 3. Click Install to set up dependencies 4. Click Start to launch the web interface

Advanced Tools (Optional)

These tabs are for power users who want more control over voice creation:

• Designer — Create new voices from text descriptions (e.g., "A warm elderly woman with a gentle raspy voice"). Save them to use as clone references in the Voices tab
• Dataset — Build LoRA training datasets interactively, one sample at a time with audio preview
• Training — Train LoRA adapters on voice datasets to create persistent voice identities that follow instruct directions

Recommended Settings for Batch Generation

Configuration

```bash

Get current config (empty prompts fall through to file defaults)

curl http://127.0.0.1:4200/api/config

Save config

curl -X POST http://127.0.0.1:4200/api/config \ -H "Content-Type: application/json" \ -d '{ "llm": {"base_url": "...", "api_key": "...", "model_name": "..."}, "tts": { "mode": "local", "device": "auto", "language": "English", "parallel_workers": 25, "batch_seed": 12345, "compile_codec": true, "sub_batch_enabled": true, "sub_batch_min_size": 4, "sub_batch_ratio": 5, "pause_between_speakers_ms": 500, "pause_same_speaker_ms": 250 } }' ```

Get voices and config

curl http://127.0.0.1:4200/api/voices

Save voice config

curl -X POST http://127.0.0.1:4200/api/save_voice_config \ -H "Content-Type: application/json" \ -d '{"NARRATOR": {"type": "custom", "voice": "Ryan", "character_style": "calm"}}' ```

Configure voices

voice_config = { "NARRATOR": {"type": "custom", "voice": "Ryan", "character_style": "calm narrator"}, "HERO": {"type": "custom", "voice": "Aiden", "character_style": "brave, determined"} } requests.post(f"{BASE}/api/save_voice_config", json=voice_config)

Web Interface

API Reference

Alexandria exposes a REST API for programmatic access:

... poll /api/status/audacity_export until not running ...

with open("audacity_export.zip", "wb") as f: f.write(requests.get(f"{BASE}/api/export_audacity").content) ```

AI-Powered Pipeline

• Local & Cloud LLM Support - Use any OpenAI-compatible API (LM Studio, Ollama, OpenAI, etc.)
• Automatic Script Annotation - LLM parses text into JSON with speakers, dialogue, and TTS instruct directions
• LLM Script Review - Optional second LLM pass that fixes common annotation errors: strips attribution tags from dialogue, splits misattributed narration/dialogue, merges over-split narrator entries, and validates instruct fields
• Persona Generation - LLM analyzes the script to create voice descriptions for each character, then generates reference audio via VoiceDesign and assigns clone voices automatically — one click to go from script to fully-voiced cast
• Speaker Aliases - Map multiple speaker names to the same voice (e.g. "YOUNG ELENA" → "ELENA") so variants share a single voice configuration
• Smart Chunking - Groups consecutive lines by speaker (up to 500 chars) for natural flow
• Context Preservation - Passes character roster and last 3 script entries between chunks for name and style continuity

Core Pipeline

Step 1 — Setup Configure your LLM connection and TTS engine. At minimum you need: - LLM Base URL: http://localhost:1234/v1 (LM Studio) or http://localhost:11434/v1 (Ollama) - LLM API Key: Your API key (use local for local servers) - LLM Model Name: The model to use (e.g., qwen2.5-14b) - TTS Mode: local (built-in, recommended) — loads models directly, no external server needed - Click Save Configuration when done

Step 2 — Script - Select your book file (.txt, .md, or .epub) using the file picker — it uploads automatically - Click Generate Annotated Script — this sends the book to your LLM to split it into annotated chunks with speaker labels and voice directions - (Optional) Click Review Script if the generated script has issues — this runs a second LLM pass to fix speaker misattributions or formatting problems - You can save the script for later use with the Save feature below

Step 3 — Voices Each character detected in the script gets a voice card. For each speaker: - Choose a voice type: Custom Voice (easiest), Clone Voice, LoRA Voice, or Voice Design - For Custom Voice, pick from 9 presets (Ryan, Serena, Aiden, etc.) and optionally set a character style (e.g., "Heavy Scottish accent") - Generate Personas — Click to have the LLM analyze the script, create voice descriptions for each character, generate reference audio, and assign clone voices automatically. Toggle "Advanced" for batch size control. This is the fastest way to assign unique voices to all characters - Speaker Aliases — Use the "Alias of" dropdown on any voice card to map a speaker to another character's voice (e.g., set "YOUNG ELENA" as alias of "ELENA"). Aliased speakers use the target's voice config during generation - Changes save automatically — see Voice Types for guidance on each type

Step 4 — Editor - Click Render Pending to generate audio for all chunks in batch - Listen to individual chunks or click Play Sequence to preview in order - Edit any chunk's text, speaker, or instruct inline and regenerate it individually - When satisfied, click Merge All to combine everything into the final audiobook

Step 5 — Result - Listen to the finished audiobook in the browser - Download as MP3, or click Export to Audacity for per-speaker WAV tracks

Python Integration

```python import requests

BASE = "http://127.0.0.1:4200"

JavaScript Integration

const BASE = "http://127.0.0.1:4200";

// Upload file
const formData = new FormData();
formData.append("file", fileInput.files[0]);
await fetch(`${BASE}/api/upload`, { method: "POST", body: formData });

// Generate script
await fetch(`${BASE}/api/generate_script`, { method: "POST" });

// Poll for completion
async function waitForTask(taskName) {
  while (true) {
    const res = await fetch(`${BASE}/api/status/${taskName}`);
    const data = await res.json();
    if (data.status === "completed" || data.status === "error") return data;
    await new Promise(r => setTimeout(r, 2000));
  }
}
await waitForTask("script_generation");

// Configure and generate
await fetch(`${BASE}/api/save_voice_config`, {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    NARRATOR: { type: "custom", voice: "Ryan", character_style: "calm" }
  })
});

// Fast batch render all chunks
const chunks = await (await fetch(`${BASE}/api/chunks`)).json();
const indices = chunks.map(c => c.id);
await fetch(`${BASE}/api/generate_batch_fast`, {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ indices })
});
// ... poll until all chunks done ...

// Merge into final audiobook
await fetch(`${BASE}/api/merge`, { method: "POST" });

// Export to Audacity
await fetch(`${BASE}/api/export_audacity`, { method: "POST" });
// ... poll /api/status/audacity_export until not running ...
// Download zip from GET /api/export_audacity

Troubleshooting