📄 工具详情 ⚙️ 安装教程 📚 使用教程
能力标签
🤖 Agent🔄 工作流🐳 Docker🧬 Embedding🖼 视觉🧠 Claude🎬 视频生成
Mooncake AI技能包
🛠
AI工具

Mooncake AI技能包

基于 C++ · 开源免费，本地部署，数据完全自主可控
英文名：Mooncake
⭐ 5.3k Stars 🍴 751 Forks 💻 C++ 📄 Apache-2.0 🏷 AI 8.2分
8.2AI 综合评分
LLM推理高性能服务分布式推理KV缓存优化RDMA网络
🌐 访问官网 📺 TG 频道
✦ AI Skill Hub 推荐

经 AI Skill Hub 精选评估，Mooncake AI技能包 获评「强烈推荐」。已获得 5.3k 颗 GitHub Star，这款AI工具在功能完整性、社区活跃度和易用性方面表现出色，AI 评分 8.2 分，适合有一定技术背景的用户使用。

📚 深度解析

Mooncake AI技能包 是一款基于 C++ 的开源工具，在 GitHub 上收获 5k+ Star，是LLM推理、高性能服务、分布式推理、KV缓存优化领域中的优质开源项目。开源工具的最大优势在于代码完全透明，你可以审计每一行代码的安全性，也可以根据自身需求进行二次开发和定制。

**为什么要使用开源工具而非商业 SaaS？**
对于个人开发者和有隐私需求的用户，本地部署的开源工具意味着数据不离本机，不受第三方服务商的数据政策约束。同时，开源工具通常没有使用次数限制和月度费用，一次安装即可长期使用，对于高频使用场景的总拥有成本（TCO）远低于订阅制商业工具。

**安装与环境准备**
Mooncake AI技能包 依赖 C++ 运行环境。建议通过 pyenv（Python）或 nvm（Node.js）管理 C++ 版本，避免全局环境污染。对于新手用户，推荐先创建虚拟环境（python -m venv venv && source venv/bin/activate），再安装依赖，这样即使出现问题也可以随时删除虚拟环境重新开始，不影响系统稳定性。

**社区与维护**
GitHub Issue 和 Discussion 是获取帮助的最快渠道。在提问前建议先检查 Closed Issues（已关闭的问题），大多数常见问题都已有解答。遇到 Bug 时，提供 pip list 的输出、完整错误堆栈和最小可复现示例，能显著提高开发者响应速度。AI Skill Hub 将持续追踪 Mooncake AI技能包 的版本更新，及时通知重要功能变化。

📋 工具概览

Kimi的开源推理服务平台，采用C++实现高性能LLM推理。支持KV缓存优化、RDMA通信和推理服务分解架构，适合部署和优化大型语言模型推理，面向开发者和企业级用户。

Mooncake AI技能包 是一款基于 C++ 开发的开源工具，专注于 LLM推理、高性能服务、分布式推理 等核心功能。作为 GitHub 开源项目，它拥有活跃的社区支持和持续的版本迭代，代码完全透明可审计，支持本地部署以保护数据隐私。无论是个人使用还是集成到企业工作流，都能提供稳定可靠的解决方案。

GitHub Stars
⭐ 5.3k
开发语言
C++
支持平台
Windows / macOS / Linux
维护状态
持续维护，定期更新
开源协议
Apache-2.0
AI 综合评分
8.2 分
工具类型
AI工具
Forks
751

📖 中文文档

以下内容由 AI Skill Hub 根据项目信息自动整理，如需查看完整原始文档请访问底部「原始来源」。

Kimi的开源推理服务平台，采用C++实现高性能LLM推理。支持KV缓存优化、RDMA通信和推理服务分解架构，适合部署和优化大型语言模型推理，面向开发者和企业级用户。

Mooncake AI技能包 是一款基于 C++ 开发的开源工具，专注于 LLM推理、高性能服务、分布式推理 等核心功能。作为 GitHub 开源项目，它拥有活跃的社区支持和持续的版本迭代，代码完全透明可审计，支持本地部署以保护数据隐私。无论是个人使用还是集成到企业工作流，都能提供稳定可靠的解决方案。

📌 核心特色
  • 开源免费，支持本地部署，数据完全自主可控
  • 活跃的 GitHub 开源社区，持续迭代更新
  • 提供详细文档和使用示例，新手友好
  • 支持自定义配置，灵活适配不同使用环境
  • 可作为基础组件集成进现有技术栈或进行二次开发
🎯 主要使用场景
  • 本地部署运行，保护数据隐私，满足合规要求
  • 自定义集成到现有系统，扩展技术栈能力
  • 作为开源基础组件进行商业化二次开发
以下安装命令基于项目开发语言和类型自动生成，实际以官方 README 为准。
安装命令
# 克隆仓库
git clone https://github.com/kvcache-ai/Mooncake
cd Mooncake

# 查看安装说明
cat README.md

# 按 README 完成环境依赖安装后即可使用
📋 安装步骤说明
  1. 访问 GitHub 仓库页面
  2. 按照 README 文档完成依赖安装
  3. 根据系统环境完成初始化配置
  4. 参考官方示例或文档开始使用
  5. 遇到问题可在 GitHub Issues 中查找解答
以下用法示例由 AI Skill Hub 整理，涵盖最常见的使用场景。
常用命令 / 代码示例
# 查看帮助
mooncake --help

# 基本运行
mooncake [options] <input>

# 详细使用说明请查阅文档
# https://github.com/kvcache-ai/Mooncake
以下配置示例基于典型使用场景生成，具体参数请参照官方文档调整。
配置示例
# mooncake 配置说明
# 查看配置选项
mooncake --config-example > config.yml

# 常见配置项
# output_dir: ./output
# log_level: info
# workers: 4

# 环境变量（覆盖配置文件）
export MOONCAKE_CONFIG="/path/to/config.yml"
📑 README 深度解析 真实文档 完整度 20/100 查看 GitHub 原文 →
以下内容由系统直接从 GitHub README 解析整理，保留代码块、表格与列表结构。

简介

A KVCache-centric Disaggregated Architecture for LLM Serving

Paper | Slides | Traces | Documentation | Blog | Slack

PyPI - Downloads GitHub commit activity license Docker <br />

PyPI CUDA <=12.9 PyPI CUDA 13.0/13.1 PyPI Non-CUDA PyPI NPU PyPI MUSA PyPI EFA PyPI ROCm </div> <br/>

Mooncake is the serving platform for <a href="https://kimi.ai/"><img src="image/kimi.png" alt="icon" style="height: 16px; vertical-align: middle;"> Kimi</a>, a leading LLM service provided by <a href="https://www.moonshot.cn/"><img src="image/moonshot.jpg" alt="icon" style="height: 16px; vertical-align: middle;"> Moonshot AI</a>. Under real workloads, Mooncake’s innovative architecture enables Kimi to handle 75% more requests while adhering to SLOs.

🔄 Updates

<details> <summary>More</summary>

  • Aug 20, 2026: Mooncake is integrated into Miles as a rollout data-transfer backend for the fragmented, heterogeneous data moving between rollout and training in disaggregated RL. Blogs: KVCache.AI, lmsys.
  • Aug 17, 2026: Mooncake is integrated into Speculators as a distributed backend for multi-node online training, efficiently moves hidden-state between vLLM inference workers and Speculators trainers through RDMA, eliminating the need for massive hidden-state storage in offline training. Benchmark on GB300 NVL72.
  • Aug 5, 2026: llm-d added a Mooncake PD deployment guide, complementing its Mooncake Store tiered offloading integration for vLLM serving on Kubernetes.
  • Aug 1, 2026: Dynamo merged an event-driven Mooncake shared-cache index for SGLang HiCache routing, using Mooncake object events to track reusable KV pages in the shared pool.
  • Jul 24, 2026: FlexKV added Mooncake Store as a distributed REMOTE-tier backend, enabling content-addressed KV cache sharing alongside its existing Mooncake Transfer Engine integration.
  • Jul 12, 2026: TransferQueue added GPUDirect RDMA support for its Mooncake Store backend, enabling GPU clients to exchange RL tensors with the host-memory store through GDR.
  • May 7, 2026: 🚀 vLLM officially features Mooncake Store — a deep dive into how Mooncake's distributed KVCache engine supercharges vLLM inference with high-throughput, memory-efficient, cross-instance KV cache sharing!
  • Apr 29, 2026: SGLang introduces RDMA-based P2P weight transfer for large-scale distributed RL using Mooncake TransferEngine, achieving 7x faster weight updates for the 1T-parameter Kimi-K2 model (53s → 7.2s) with zero-copy RDMA transfer across thousands of GPUs.
  • Apr 3, 2026: LMCache introduced a native Mooncake Store L2 adapter for multiprocess mode, connecting the LMCache MP server to the distributed KV cache pool.
  • Apr 1, 2026: The TENT technical report introduces declarative data-movement orchestration for heterogeneous interconnects, with evaluations in SGLang HiCache and checkpoint-engine workloads.
  • Mar 19, 2026: TorchSpec: Speculative Decoding Training at Scale is open sourced, using Mooncake to decouple inference and training via efficient hidden states management.
  • Mar 16, 2026: FastDeploy added Mooncake Store cache read/write support to its PD pipeline, complementing its global KV cache pooling across inference instances.
  • Mar 5, 2026: LightX2V now supports disaggregated deployment based on Mooncake, enabling encoder/transformer service decoupling with Mooncake Transfer Engine for high-performance cross-device and cross-machine data transfer. Details in blog.
  • Mar 3, 2026: verl added a Mooncake checkpoint-engine backend for peer-to-peer weight synchronization between training and rollout workers.
  • Feb 25, 2026: SGLang merged Encoder Global Cache Manager, introducing a Mooncake-powered global multimodal embedding cache that enables cross-instance sharing of ViT embeddings to avoid redundant GPU computation.
  • Feb 24, 2026: vLLM-Omni introduces disaggregated inference connectors with support for both MooncakeStoreConnector and MooncakeTransferEngineConnector for multi-node omni-modality pipelines.
  • Feb 12, 2026: Mooncake Joins PyTorch Ecosystem We are thrilled to announce that Mooncake has officially joined the PyTorch Ecosystem!
  • Jan 28, 2026: FlexKV, a distributed KV store and cache system from Tencent and NVIDIA in collaboration with the community, now supports distributed KVCache reuse with the Mooncake Transfer Engine.
  • Dec 27, 2025: Collaboration with ROLL! Check out the paper here.
  • Dec 23, 2025: SGLang introduces Encode-Prefill-Decode (EPD) Disaggregation with Mooncake as a transfer backend. This integration allows decoupling compute-intensive multimodal encoders (e.g., Vision Transformers) from language model nodes, utilizing Mooncake's RDMA engine for zero-copy transfer of large multimodal embeddings.
  • Dec 19, 2025: Mooncake Transfer Engine has been integrated into TensorRT LLM for KVCache transfer in PD-disaggregated inference.
  • Dec 19, 2025: Mooncake Transfer Engine has been directly integrated into vLLM v1 as a KV Connector in PD-disaggregated setups.
  • Nov 07, 2025: RBG + SGLang HiCache + Mooncake, a role-based out-of-the-box solution for cloud native deployment, which is elastic, scalable, and high-performance.
  • Oct 15, 2025: SGLang introduced Mooncake EP and a collective communication backend as the foundation for fault-tolerant Elastic Expert Parallel inference.
  • Sept 18, 2025: Mooncake Store empowers vLLM Ascend by serving as the distributed KV cache pool backend.
  • Sept 10, 2025: SGLang officially supports Mooncake Store as a hierarchical KV caching storage backend. The integration extends RadixAttention with multi-tier KV cache storage across device, host, and remote storage layers.
  • Sept 10, 2025: The official & high-performance version of Mooncake P2P Store is open-sourced as checkpoint-engine. It has been successfully applied in K1.5 and K2 production training, updating Kimi-K2 model (1T parameters) across thousands of GPUs in ~20s.
  • Aug 23, 2025: xLLM high-performance inference engine builds hybrid KV cache management based on Mooncake, supporting global KV cache management with intelligent offloading and prefetching.
  • Aug 18, 2025: vLLM-Ascend integrates Mooncake Transfer Engine for KV cache register and disaggregate prefill, enabling efficient distributed inference on Ascend NPUs.
  • Jul 20, 2025: Mooncake powers the deployment of Kimi K2 on 128 H200 GPUs with PD disaggregation and large-scale expert parallelism, achieving 224k tokens/sec prefill throughput and 288k tokens/sec decode throughput.
  • Jun 20, 2025: Mooncake becomes a PD disaggregation backend for LMDeploy.
  • May 9, 2025: NIXL officially supports Mooncake Transfer Engine as a backend plugin.
  • May 8, 2025: Mooncake x LMCache unite to pioneer KVCache-centric LLM serving system.
  • May 5, 2025: Supported by Mooncake Team, SGLang release <a href="https://lmsys.org/blog/2025-05-05-large-scale-ep/" target="_blank">guidance</a> to deploy DeepSeek with PD Disaggregation on 96 H100 GPUs.
  • Apr 22, 2025: LMCache officially supports Mooncake Store as a <a href="https://blog.lmcache.ai/2025-04-22-tencent/" target="_blank">remote connector</a>.
  • Apr 10, 2025: SGLang officially supports Mooncake Transfer Engine for disaggregated prefilling and KV cache transfer.
  • Mar 7, 2025: We open-sourced the Mooncake Store, a distributed KVCache based on Transfer Engine. vLLM's xPyD disaggregated prefilling & decoding based on Mooncake Store will be released soon.
  • Feb 25, 2025: Mooncake receives the Best Paper Award at FAST 2025!
  • Feb 21, 2025: The updated <a href="FAST25-release/traces" target="_blank">traces</a> used in our FAST'25 paper have been released.
  • Dec 16, 2024: vLLM officially supports Mooncake Transfer Engine for disaggregated prefilling and KV cache transfer.
  • Nov 28, 2024: We open-sourced the Transfer Engine, the central component of Mooncake. We also provide two demonstrations of Transfer Engine: a P2P Store and vLLM integration.
  • July 9, 2024: We open-sourced the trace as a <a href="https://github.com/kvcache-ai/Mooncake/blob/main/FAST25-release/arxiv-trace/mooncake_trace.jsonl" target="_blank">JSONL file</a>.
  • June 27, 2024: We present a series of Chinese blogs with more discussions on <a href="https://zhuanlan.zhihu.com/p/705754254">zhihu 1</a>, <a href="https://zhuanlan.zhihu.com/p/705910725">2</a>, <a href="https://zhuanlan.zhihu.com/p/706204757">3</a>, <a href="https://zhuanlan.zhihu.com/p/707997501">4</a>, <a href="https://zhuanlan.zhihu.com/p/9461861451">5</a>, <a href="https://zhuanlan.zhihu.com/p/1939988652114580803">6</a>, <a href="https://zhuanlan.zhihu.com/p/1959366095443064318">7</a>.
  • June 26, 2024: Initial technical report release.

</details>

🎉 Overview

Mooncake is an infrastructure project for large-scale LLM inference and training. It features a KV cache-centric disaggregated architecture that separates prefill and decode clusters, while leveraging otherwise underutilized CPU, DRAM, and SSD resources in GPU clusters to build a disaggregated KV cache pool.

Mooncake includes a high-performance Transfer Engine for low-latency data movement across heterogeneous networks and accelerators; Mooncake Store for distributed KV cache and model-weight management; and Mooncake EP & PG for elastic MoE serving. Deeply integrated with ecosystems such as SGLang and vLLM, Mooncake helps LLM systems improve cache reuse, reduce serving latency, and scale efficiently across multi-node clusters.

🔥 Show Cases

SGLang Integration ([Guide](https://kvcache-ai.github.io/Mooncake/deployment/integrations/sglang/index.html))

Mooncake is deeply integrated into SGLang as a high-performance communication and storage backend. These integrations enable efficient KV cache transfer in PD-disaggregated serving, scalable multi-level KV caching through HiCache, fault-tolerant expert-parallel inference, high-performance multimodal pipeline data movement, and fast RDMA-based weight synchronization for large-scale RL training. Together, Mooncake and SGLang provide a production-oriented foundation for building elastic, high-throughput, and resource-efficient LLM and multimodal serving systems.

<details> <summary>Details</summary>

  • PD Disaggregated Serving: SGLang officially supports Mooncake Transfer Engine as a backend for disaggregated serving and KV cache transfer, enabling prefill and decode workers to exchange KV cache data efficiently across devices and machines.
  • Hierarchical KV Caching: Mooncake Store serves as an external storage backend in SGLang's HiCache system, extending RadixAttention with multi-level KV cache storage across device, host, and remote storage layers.
  • Elastic Expert Parallel: Mooncake's collective communication backend and expert parallel kernels are integrated into SGLang to enable fault-tolerant expert parallel inference (Elastic EP).
  • Cloud-Native SGLang HiCache Deployment with RBG: The RBG + SGLang HiCache + Mooncake integration provides a role-based, out-of-the-box cloud-native deployment solution that is elastic, scalable, and optimized for high-performance inference workloads.
  • Encode-Prefill-Decode Disaggregation for Multimodal Serving: SGLang introduces Encode-Prefill-Decode disaggregation with Mooncake as a transfer backend. This enables compute-intensive multimodal encoders, such as Vision Transformers, to be decoupled from language model workers while transferring large embeddings efficiently through Mooncake’s RDMA-based engine.
  • SGLang-Omni Multi-Stage Pipeline Data Transfer: SGLang-Omni integrates Mooncake as a relay backend for efficient cross-stage tensor and blob transfer in multimodal serving pipelines. This enables high-performance data movement between heterogeneous components such as thinker, talker, codec, and vocoder stages.
  • RDMA-Based P2P Weight Transfer for Distributed RL: SGLang adopts Mooncake TransferEngine for RDMA-based peer-to-peer weight transfer in large-scale distributed reinforcement learning. This enables zero-copy weight updates across thousands of GPUs and significantly accelerates synchronization for trillion-parameter models.

</details>

vLLM Integration ([Guide](https://kvcache-ai.github.io/Mooncake/deployment/integrations/vllm/index.html))

Mooncake integrates with vLLM to accelerate large language model serving through high-performance KV cache transfer and distributed KV cache storage. The integration supports both disaggregated prefill-decode serving and cross-instance KV cache sharing, helping vLLM deployments reduce TTFT, improve cache reuse, and scale more efficiently across multi-node inference clusters.

<details> <summary>Details</summary>

  • Disaggregated prefill-decode serving: Mooncake enables vLLM to split prefill and decode workloads across different nodes. Through MooncakeConnector, vLLM transfers KV cache blocks from prefill workers to decode workers using Mooncake’s high-performance transfer engine, allowing prefill and decode resources to scale independently while keeping cross-node KV transfer overhead low.
  • Distributed KV cache pooling and sharing: Mooncake Store extends vLLM from isolated per-instance KV caches to a shared, cluster-level KV cache pool. Through MooncakeStoreConnector, multiple vLLM instances can store, retrieve, and reuse KV cache blocks based on hash-based prefix caching, reducing redundant prefill computation and improving cache efficiency for workloads with repeated prefixes, especially agentic and multi-turn serving scenarios.
  • vLLM-Omni stage communication: Mooncake also integrates with vLLM-Omni through MooncakeTransferEngineConnector and MooncakeStoreConnector, enabling efficient cross-node data exchange between vLLM-Omni stages.

</details>

🌐 Ecosystem Integrations

Mooncake connects inference engines, cache middleware, RL systems, and deployment tools. The directory below links to upstream guides and implementations for each integration, checked on Oct 15, 2026.

<details> <summary>Integration directory</summary>

TE = Transfer Engine; Store = Mooncake Store; EP/PG = expert-parallel and process-group backends. Capabilities depend on the upstream version and configuration; consult the linked guide or implementation when deploying.

ProjectMooncake componentsIntegration use casesGuide or implementation
SGLangTE, Store, EP/PGPD/EPD transfer, HiCache, multimodal embedding caching, weight synchronization, and Elastic EP[Integration guide](https://kvcache-ai.github.io/Mooncake/deployment/integrations/sglang/index.html), [Elastic EP](https://github.com/sgl-project/sglang/pull/10423)
vLLMTE, StorePD transfer, shared KV pooling, encoder-cache transfer, and cross-encoder embedding reuse[PD connector](https://docs.vllm.ai/en/latest/features/mooncake_connector_usage/), [Store connector](https://docs.vllm.ai/en/latest/features/mooncake_store_connector_usage/), [encoder connector](https://docs.vllm.ai/en/latest/features/mooncake_ec_connector_usage/)
TokenSpeedTE, StorePD/EPD transfer and Store-backed L3 beneath the host KV cache[Server configuration](https://github.com/lightseekorg/tokenspeed/blob/main/docs/configuration/server.md)
TensorRT-LLMTE, StorePD transfer and shared KV pooling through the KVCMv2 Store connector merged into main[TE implementation](https://github.com/NVIDIA/TensorRT-LLM/tree/main/cpp/tensorrt_llm/executor/cache_transmission/mooncake_utils), [Store connector](https://github.com/NVIDIA/TensorRT-LLM/pull/19171)
LMDeployTE, StorePD transfer and cross-instance KV prefix reuse[TE guide](https://kvcache-ai.github.io/Mooncake/deployment/integrations/lmdeploy.html), [Store connector](https://github.com/InternLM/lmdeploy/pull/4903)
vLLM-AscendTE, StoreNPU PD transfer and KV pooling through AscendStoreConnector[PD deployment](https://github.com/vllm-project/vllm-ascend/blob/main/docs/source/tutorials/features/pd_disaggregation_mooncake_multi_node.md), [KV pool](https://github.com/vllm-project/vllm-ascend/blob/main/docs/source/user_guide/feature_guide/kv_cache_pool_ascend_store.md)
FastDeployStoreGlobal KV pooling and cache reuse in PD serving[Global cache pooling](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/features/global_cache_pooling.md)
xLLMTE, StoreKV transfer and distributed cache management[KV transfer backends](https://github.com/jd-opensource/xllm/tree/main/xllm/core/kv_cache/transfer)
ChituTEMulti-prefill/multi-decode KV transfer over RDMA[PD disaggregation](https://github.com/thu-pacman/chitu/blob/public-main/chitu/distributed/pd_disaggregation/README.md)
LMCacheTE, StoreRemote KV storage, a native MP L2 adapter, and P2P cache sharing[Store guide](https://docs.lmcache.ai/kv_cache/storage_backends/mooncake.html), [MP adapter](https://docs.lmcache.ai/mp/l2_storage.html#mooncake-store-mooncake-store-native-connector), [P2P backend](https://github.com/LMCache/LMCache/pull/4182)
FlexKVTE, StoreDistributed cache reuse and a content-addressed REMOTE tier[Distributed reuse](https://github.com/taco-project/FlexKV/blob/main/docs/dist_reuse/README_en.md), [Store backend](https://github.com/taco-project/FlexKV/pull/231)
NIXLTEMooncake transport plugin for NIXL data movement[Plugin guide](https://github.com/ai-dynamo/nixl/blob/main/src/plugins/mooncake/README.md)
DynamoStore via SGLang HiCacheShared-pool awareness and event-driven KV cache routing[HiCache integration](https://github.com/ai-dynamo/dynamo/blob/main/docs/fern/pages/developer-guide/knowledge-base/modular-components/backends/sglang/hicache.md)
llm-dTE, Store via vLLMKubernetes PD serving and tiered KV cache offloading[PD integration](https://github.com/llm-d/llm-d/pull/1511), [Tiered prefix cache](https://github.com/llm-d/llm-d/tree/main/guides/tiered-prefix-cache)
RBGTE, Store via inference enginesRole-based orchestration of Store, PD transfer, and cache reuse[Deployment examples](https://github.com/sgl-project/rbg/tree/main/examples/inference/ecosystem/mooncake)
srt-slurmTE, Store via inference enginesSlurm deployment and Mooncake service configuration[Mooncake services](https://github.com/NVIDIA/srt-slurm/blob/main/docs/mooncake-kv-store.md)
NeMo RLStore via TransferQueueRollout-to-training data exchange with optional GDR client transfers[Data plane](https://github.com/NVIDIA-NeMo/RL/blob/main/nemo_rl/data_plane/README.md), [GDR support](https://github.com/NVIDIA-NeMo/RL/pull/3501)
TransferQueueStoreRL tensor/object storage, GDR transfers, and SSD offloading[Store client](https://github.com/Ascend/TransferQueue/blob/main/transfer_queue/storage/clients/mooncake_client.py), [GDR guide](https://github.com/Ascend/TransferQueue/blob/main/docs/storage_backends/mooncake_gdr.md)
verlTECheckpoint-engine backend for trainer-to-rollout weight synchronization[Mooncake backend](https://github.com/verl-project/verl/blob/main/verl/checkpoint_engine/mooncake_checkpoint_engine.py)
checkpoint-engineTEPeer-to-peer model-weight distribution and updates[P2P backend](https://github.com/MoonshotAI/checkpoint-engine)
MilesStoreRollout-to-training data transfer[Rollout transfer guide](https://github.com/radixark/miles/blob/main/docs/advanced/mooncake-rollout-transfer.md)
TorchSpecStoreHidden-state exchange between inference and draft-model training[Training architecture](https://github.com/torchspec-project/TorchSpec)
SpeculatorsStoreHidden-state exchange for distributed online draft-model training[Hidden-state connectors](https://github.com/vllm-project/speculators/tree/main/hs_connectors)
vLLM-OmniTE, StoreData exchange between multimodal pipeline stages[Omni connectors](https://github.com/vllm-project/vllm-omni/tree/main/vllm_omni/distributed/omni_connectors/connectors)
LightX2VTEEncoder/transformer disaggregation for video generation[Transfer backend](https://github.com/ModelTC/LightX2V/blob/main/lightx2v/disagg/mooncake.py)
slimeTE via SGLangPD-disaggregated rollout serving[RL serving architecture](https://github.com/THUDM/slime/blob/main/docs/en/advanced/rl-systems.md)

The Dynamo entry describes SGLang HiCache shared-cache routing. Its separate KVBM G4 Store adapter proposal is still unmerged. NeMo RL's GDR path changes client-side tensor transfers; queued objects remain in Mooncake-managed host memory.

</details>

🖥️ Supported Hardware

Mooncake supports hardware backends across accelerator vendors, cloud fabrics, and standard datacenter interconnects, as listed below. See the supported protocols and Transfer Engine design docs for details.

<img src="image/partners/nvidia_logo.png" width="120" alt="NVIDIA"/><img src="image/partners/huawei_logo.png" width="120" alt="Huawei"/><img src="image/partners/amd_logo.png" width="120" alt="AMD"/><img src="image/hardwares/cambricon_logo.png" width="120" alt="Cambricon"/><img src="image/partners/moore_thread_logo.jpg" width="120" alt="Moore Threads"/><img src="image/partners/aws-logo.png" width="120" alt="AWS"/>
<img src="image/hardwares/MetaX_logo.png" width="120" alt="MetaX"/><img src="image/hardwares/T-Head_logo.png" width="120" alt="T-Head"/><img src="image/partners/aliyun_logo.png" width="120" alt="Alibaba Cloud"/><img src="image/partners/sunrise_logo.png" width="120" alt="Sunrise"/><img src="image/partners/hygon_logo.png" width="120" alt="Hygon"/><img src="image/hardwares/biren_logo.png" width="120" alt="Biren Technology"/>

🚀 Getting Started

Install Mooncake using pip. The mooncake-transfer-engine package includes Mooncake Transfer Engine, Mooncake Store, Mooncake EP and PG:

- CUDA < 13.0 

pip install mooncake-transfer-engine
- CUDA >= 13.0 
pip install mooncake-transfer-engine-cuda13

In addition to CUDA, Mooncake also supports other accelerator backends, along with flexible installation and deployment options. See the guides below for details:

🎯 aiskill88 AI 点评 A 级 2026-05-22

高性能LLM推理平台，采用业界先进优化技术。代码质量优秀、更新活跃，是探索推理加速的优选方案。

📚 实用指南（长尾问题）
适合谁
  • 构建企业知识库 / RAG 检索应用的团队
最佳实践
  • 生产部署优先使用 Docker Compose 隔离依赖，并挂载 volume 持久化数据
常见错误
  • API key 直接提交到 git 仓库（请用 .env 并加入 .gitignore）
  • 容器内无法访问宿主机 localhost — 使用 host.docker.internal
部署方案
  • Docker：Mooncake 提供官方镜像，docker compose up 一键启动
  • 云端托管：可放在 Vercel / Railway / Fly.io 等 PaaS 平台
相关搜索
Mooncake 中文教程Mooncake 安装报错怎么办Mooncake Docker 部署Mooncake 与同类工具对比Mooncake 最佳实践Mooncake 适合谁用

⚡ 核心功能

👥 适合谁
  • 构建企业知识库 / RAG 检索应用的团队
⭐ 最佳实践
  • 生产部署优先使用 Docker Compose 隔离依赖，并挂载 volume 持久化数据
⚠️ 常见错误
  • API key 直接提交到 git 仓库（请用 .env 并加入 .gitignore）
  • 容器内无法访问宿主机 localhost — 使用 host.docker.internal

👥 适合人群

AI 技术爱好者研究人员和学生开发者和工程师技术创业者

🎯 使用场景

  • 本地部署运行，保护数据隐私，满足合规要求
  • 自定义集成到现有系统，扩展技术栈能力
  • 作为开源基础组件进行商业化二次开发

⚖️ 优点与不足

✅ 优点
  • +GitHub 5.3k Star，社区高度认可
  • +Apache-2.0 协议，可免费商用
  • +完全开源免费，无授权费用
  • +本地部署，数据完全自主可控
  • +开发者社区支持，遇问题可查可问
⚠️ 不足
  • −安装和初始配置可能需要一定技术基础
  • −功能完整性通常不如成熟商业产品
  • −技术支持主要依赖开源社区，响应速度不稳定
⚠️ 使用须知

AI Skill Hub 为第三方内容聚合平台，本页面信息基于公开数据整理，不对工具功能和质量作任何法律背书。

建议在沙箱或测试环境中充分验证后，再部署至生产环境，并做好必要的安全评估。

📄 License 说明

✅ Apache 2.0 — 宽松开源协议，可商用，需保留版权声明和 NOTICE 文件，含专利授权条款。

🔗 相关工具推荐

vllm AI技能包
高性能开源LLM推理和服务引擎，提供高吞吐量和低内存占用。支持多种硬件平台（CUDA/AMD），适合需要快速部署大语言模
llama-cpp AI技能包
高效的大语言模型C/C++推理框架，支持在本地CPU/GPU上运行量化LLM模型，具有内存占用小、推理速度快的特点。适合
TensorRT-LLM
NVIDIA官方开源项目，为大语言模型推理优化的Python框架。提供高效的LLM编译、部署和服务能力，支持多种硬件加速
transformers AI技能包
Hugging Face开源的深度学习框架，提供预训练语言模型、视觉模型和多模态模型。集成BERT、GPT、Llama等
📰 相关 AI 新闻
前沿 AI 研究论文速览
AI 资讯 · 知识关联
Meta LLaMA 开源模型动态
AI 资讯 · 知识关联
AI Agent 自主化能力最新进展
AI 资讯 · 知识关联
AI 前沿资讯：Feel like I'm becoming the glu…
AI 资讯 · 知识关联
🗺️ 相关解决方案
workflow
ai-workflow-templates
docker
docker-deploy
embedding
rag-knowledge-base
vision
computer-vision
🧩 你可能还需要
基于当前 Skill 的能力图谱，自动补全的工具组合
gptme Agent工作流
MCP · Agent · 工作流
LEANN AI技能包
MCP · Agent · 工作流
Augustus
LLM安全测试框架，检测prompt注入、越狱等
AI公司多智能体操作系统
为Claude Code设计的开源MCP工具集，包含108个MCP工具和40+智能体模板。支持多智能体协作编排，提供完整的自主代理框架。适合
multi-agent-shogun Agent工作流
基于Claude的开源多智能体编排系统，采用武士风格架构设计，支持并行任务协调。适合需要构建复杂AI工作流、自动化多步骤任务的开发者和企业用
Redmine MCP服务器
为Redmine项目管理系统设计的生产级MCP服务器，提供安全认证、分页查询和企业级功能。支持AI Agent与Redmine深度集成，适合

❓ 常见问题 FAQ

Mooncake是Kimi的官方推理平台，专为大规模LLM优化，支持KV缓存和RDMA等高级特性。
💡 AI Skill Hub 点评

AI Skill Hub 点评：Mooncake AI技能包 的核心功能完整，质量优秀。对于AI 技术爱好者来说，这是一个值得纳入个人工具库的选择。建议先在非生产环境试用，再逐步推广。

📚 深入学习 Mooncake AI技能包
查看分步骤安装教程和完整使用指南，快速上手这款工具
⚙️ 安装教程 📚 使用教程
🌐 原始信息
原始名称 Mooncake
原始描述 开源AI工具：Mooncake is the serving platform for Kimi, a leading LLM service provided by Moo。⭐5.3k · C++
Topics LLM推理高性能服务分布式推理KV缓存优化RDMA网络
GitHub https://github.com/kvcache-ai/Mooncake
License Apache-2.0
语言 C++
🔗 原始来源
🐙 GitHub 仓库  https://github.com/kvcache-ai/Mooncake 🌐 官方网站  https://kvcache-ai.github.io/Mooncake/

收录时间：2026-05-16 · 更新时间：2026-05-19 · License：Apache-2.0 · AI Skill Hub 不对第三方内容的准确性作法律背书。

📺 订阅 AI Skill Hub Daily Telegram 频道
每天 8 条精选 AI Skill、MCP、Agent 与自动化工具推送
加入频道 →