本地AI工具

Install dependencies

shards install

Requirements

| Component | Minimum | Recommended | | ---- | ---- | ---- | | RAM | 4 GB | 8 GB | | Storage | 3 GB free | 10 GB free | | OS | Linux | Ubuntu 22.04+ | | CPU | Any 64-bit | AVX2 support |

macOS works but is experimental. Windows support coming soon.

---

Build the binary

crystal build src/speak.cr --release -o speak

Quick Start

Usage

Chat Commands

Inside the chat, type these commands:

| Command | Action | | ---- | ---- | | exit or quit | Save and quit | | clear | Clear the screen | | history | Show conversation history | | save | Save conversation manually | | memory | Show what speak remembers about you | | clearmemory | Clear all memories | | reset | Reset working memory |

Example conversation

``` $ ./speak

Configuration

All settings are stored in ./speak/config.json. You can edit this file to customize speak.

Configuration Structure

{
  "detected": {
    "total_ram_mb": 8192,
    "available_ram_mb": 6200,
    "os_reserved_ram_mb": 512
  },
  "active": {
    "cpu_cores": 4,
    "has_avx2": true,
    "free_disk_space_mb": 51200,
    "context_size": 2048,
    "kv_cache_type": "standard",
    "model_quant": "Q4_K_M",
    "model_file": "nanbeige-3b-q4_k_m.gguf",
    "temperature": 0.7,
    "max_tokens": 512,
    "use_mmap": true
  },
  "user_overrides": {
    "os_reserved_ram_mb": null,
    "context_size": null,
    "kv_cache_type": null,
    "model_quant": null,
    "temperature": null,
    "max_tokens": null,
    "use_mmap": null
  }
}

Common Settings

| Setting | What it does | Default | ---- | ---- | ---- | |context_size | How many tokens the AI remembers| 2048| |temperature | Creativity (0.0 = strict, 1.5 = creative) | 0.7| |max_tokens | Maximum response length | 512| |model_quant | Quality vs speed (Q2_K, Q4_K_M, Q6_K) | Q4_K_M|

Make AI more creative

Edit ./speak/config.json:

"user_overrides": {
  "temperature": 1.2
}

Reduce RAM usage

"user_overrides": {
  "context_size": 1024,
  "model_quant": "Q2_K"
}

Custom System Prompt

Edit src/speak/system_prompt.txt and recompile. The prompt is embedded at build time.

---

Troubleshooting

Unable to create dir ./speak

Your binary is named speak and conflicts with the data directory. Rename the binary:

mv speak speak_app
./speak_app

401 Unauthorized during download

The model repository requires authentication. Run:

./hfd.sh Edge-Quant/Nanbeige4.1-3B-Q4_K_M-GGUF --include *.gguf --local-dir ./speak/models

Then run ./speak again.

Model loads slowly on HDD

Use the smaller Q2_K model. Edit config.json:

"user_overrides": {
  "model_quant": "Q2_K"
}

Then delete the old model file in ./speak/models/ and restart speak.

Readline not working

Install the system library:

```bash