抽象核心

AbstractCore

Unified LLM Interface > Write once, run everywhere

AbstractCore is an offline-capable, open-source-first LLM infrastructure layer for Python applications. It gives you one create_llm(...) API across local runtimes, self-hosted servers, cloud APIs, and OpenAI-compatible gateways.

Use it in-process from Python, or run it as a universal /v1 endpoint for apps that already speak the OpenAI API. The same application can run fully offline once local model assets are installed, stay private on your own inference server, or route to hosted providers when you want managed capacity.

The goal is simple: put LLM capability at your fingertips without tying your product to a vendor, network connection, or model family. AbstractCore keeps application code portable while the model underneath moves between OpenAI, Anthropic, Ollama, LM Studio, MLX, HuggingFace/GGUF, vLLM, OpenRouter, Portkey, or any OpenAI-compatible backend.

The default install is intentionally lightweight; add providers and optional subsystems via explicit install extras. For local runtimes, AbstractCore is cache-first and offline-first: it will not silently download model weights; you pull or prefetch the models you want, then run without internet when your chosen provider and tools are local.

First-class support for: - offline-capable local operation with explicit model setup (no silent downloads) - local/open-weight model backends (Ollama, LM Studio, MLX, HuggingFace/GGUF, vLLM) - cloud, hosted gateway, and generic OpenAI-compatible providers - sync + async - streaming + non-streaming - universal tool calling (native + prompted tool syntax) - structured output (Pydantic) - unified generation parameters, capability detection, and provider quirks - session memory, prompt caching, durable memory bloc cache artifacts, events, tracing, and retry-aware reliability hooks - media input (images/audio/video + documents) with explicit, policy-driven fallbacks () - optional capability plugins (core.voice/core.audio/core.vision/core.music) for deterministic TTS/STT, generative vision, and music backends (via packages such as abstractvoice, abstractvision, and abstractmusic) - glyph visual-text compression for long documents (*) - optional OpenAI-compatible /v1 gateway server (multi-provider) and single-model endpoint

() Media input is policy-driven (no silent semantic changes). If a model doesn’t support images, AbstractCore can use a configured vision model to generate short visual observations and inject them into your text-only request (vision fallback). Audio/video attachments are also policy-driven (audio_policy, video_policy) and may require capability plugins for fallbacks. See Media Handling and Centralized Config. (*) Optional visual-text compression: render long text/PDFs into images and process them with a vision model to reduce token usage. See Glyph Visual-Text Compression (install pip install "abstractcore[compression]"; for PDFs also install pip install "abstractcore[media]").

Generative vision uses abstractvision when installed. In server mode, omit model only when the server has a configured default, or use explicit provider/model ids such as diffusers/default, diffusers/<huggingface-repo>, mlx-gen/AbstractFramework/qwen-image-2512-4bit, mlx-gen/AbstractFramework/wan2.2-t2v-a14b-diffusers-8bit, sdcpp/default, or openai-compatible/<model>. Quantized MLX-Gen models are selected by their published repo id; Core does not add a separate quant override.

Docs: Getting Started · FAQ · Docs Index · https://lpalbou.github.io/AbstractCore

For AI assistants and doc-indexing tools, this repository also publishes llms.txt and llms-full.txt. MCP clients such as Context7 can query the public documentation directly as well.

Why AbstractCore

Many libraries can call an LLM. AbstractCore is for the messy middle of real applications, where you need the same product code to survive different model families, local inference servers, API dialects, offline deployments, and capability gaps.

Open-source and self-hosted models are first-class, not a demo path. AbstractCore handles the things that often break when you move beyond a single hosted API: prompted vs native tools, schema-following differences, structured-output retry, reasoning text, media support, token budget vocabulary, local server discovery, and prompt/cache behavior.

That makes it a practical foundation for privacy-sensitive assistants, local developer tools, document workflows, research machines, edge deployments, and cloud-backed production services. You can build remote-first products, fully local products, or hybrid products that move between the two as cost, privacy, latency, and hardware constraints change.

Use AbstractCore when you want a focused provider layer that stays close to your application code. Use the wider AbstractFramework stack when you also need durable runtime execution, agents, flows, gateways, agentic CLI surfaces, memory, or assistant applications such as AbstractAssistant.

AbstractFramework ecosystem

AbstractCore is part of the AbstractFramework ecosystem:

• AbstractFramework (umbrella): https://github.com/lpalbou/AbstractFramework
• AbstractCore (this package): provider-agnostic LLM I/O + reliability primitives
• AbstractRuntime: durable tool/effect execution, workflows, and state persistence (recommended host runtime) — https://github.com/lpalbou/abstractruntime
• Wider stack: agents, flows, gateway control, agentic CLI integrations, memory, semantics, coding tools, and digital assistant surfaces built on the same foundation

By default, AbstractCore is pass-through for tools (execute_tools=False): it returns structured tool calls in response.tool_calls, and your runtime decides whether/how to execute them (policy, sandboxing, retries, persistence). See Tool Calling and Architecture.

Image generation via abstractvision.

image = llm.generate("A red ceramic mug on a white table.", output="image") png_bytes = image.outputs["image"][0].data

TTS via abstractvoice.

speech = llm.generate(text="Hello from AbstractCore.", output="voice") wav_bytes = speech.outputs["voice"][0].data

Music via abstractmusic.

music = llm.generate( text="A short calm piano loop.", output={"modality": "music", "provider": "acemusic", "duration_s": 8, "format": "wav"}, ) music_wav = music.outputs["music"][0].data

when the selected AbstractVoice backend supports local or remote cloning.

clone = llm.generate(text="Optional transcript.", media="reference.wav", output="voice") voice_id = clone.resources["voice"][0].resource_id

Text-only `generate(...)` is unchanged. For advanced/provider-specific work,
the direct `llm.vision.*`, `llm.voice.*`, `llm.audio.*`, and `llm.music.*` facades remain
available. Configure `abstractvision` and `abstractvoice` backends first for
real generation; configure `abstractmusic` for music generation. With
`abstractmusic>=0.1.13`, the default music backend is the lightweight remote
ACE Music path; set `ACEMUSIC_API_KEY` before use. Local music engines remain
optional plugin extras.

Catalog helpers are available for UI/dropdown preflight:

The HTTP server exposes equivalent discovery at /v1/vision/providers/, /v1/vision/models, /v1/audio/voices, /v1/audio/speech/models, /v1/audio/transcriptions/models, and /v1/voice/clone/providers, plus /v1/audio/music/providers and /v1/audio/music/models. Generated media HTTP routes include /v1/images/generations, /v1/images/edits, /v1/videos/generations, /v1/videos/edits, and async polling routes under /v1/vision/jobs/images/* and /v1/vision/jobs/videos/*; image and video jobs include the latest backend progress event when the selected backend reports it. /v1/models remains focused on LLM and embedding provider models. Use capability_route to filter those models by precise route-keyed support: /v1/models?capability_route=input.image,output.text, /v1/models?capability_route=input.sound,output.text, /v1/models?capability_route=input.music,output.text, or /v1/models?capability_route=embedding.text. Generated-media provider readiness and download/setup status remain on the capability plugin catalog routes, not in /v1/models.