mlx-serve
Copyright (c) 2026 David Dalcu

mlx-serve as a whole is distributed under the MIT License (see LICENSE). It
incorporates third-party software that remains under its own license, listed
below. Those licenses continue to apply to those portions. A copy of the Apache
License, Version 2.0 ships with this distribution as LICENSE-APACHE-2.0.

The source comments at each site name their upstream too.


================================================================================
Apache License, Version 2.0
================================================================================

--------------------------------------------------------------------------------
MTPLX
https://github.com/youssofal/mtplx
Copyright 2026 Youssof Altoukhi
Licensed under the Apache License, Version 2.0.

Reproduced from MTPLX's NOTICE file:

    MTPLX
    Copyright 2026 Youssof Altoukhi

    MTPLX is a native MTP speculative decoding project for Apple Silicon.

    Preferred attribution for public projects, products, benchmarks, articles,
    and research that use or build on MTPLX:

      Powered by MTPLX by Youssof Altoukhi
      https://github.com/youssofal/mtplx

    If MTPLX informs academic or technical writing, please cite the repository
    using the included CITATION.cff metadata.

    This product includes Metal kernel code adapted from dflash-mlx
    (https://github.com/bstnxbt/dflash-mlx), Copyright dflash-mlx contributors,
    licensed under the Apache License 2.0. See mtplx/nax_verify.py for details.

Used in mlx-serve (`src/transformer.zig`):

  * The verify-shaped quantized matmul kernel family (`verifyQmm`), ported from
    `mtplx/verify_kernels.py`. Their design: one threadgroup per column tile,
    split-K reduction across simdgroups, sequential per-column dequant and FMA
    chains, named-scalar codegen with literal accumulator indices.
  * The NAX m16 tensor-ops tile for M5-class GPUs, ported from
    `mtplx/nax_verify.py`, together with its host contract: the M-padding
    dispatch, the environment gating and the availability probes.

Modifications by mlx-serve: the kernels were reimplemented in Zig against
mlx-c, the Metal sources are generated by comptime codegen rather than Python
f-strings, and the NAX tile's dequantization loop was extended from 4-bit only
to 4-, 5- and 6-bit affine weights.

MTPLX's own NOTICE also credits vllm-metal and PipeNetwork for parts of MTPLX
that mlx-serve does not use. Those notices are omitted here as permitted by
Apache-2.0 section 4(d).

--------------------------------------------------------------------------------
dflash-mlx
https://github.com/bstnxbt/dflash-mlx
Copyright dflash-mlx contributors
Licensed under the Apache License, Version 2.0.

Reached through MTPLX's `nax_verify.py`, which is a port of dflash-mlx's
`dflash_mlx/verify_qmm.py`, itself based on DFlash (arXiv:2602.06036). The
matmul2d tensor extents and strides in our NAX kernel are dflash's working
convention.

--------------------------------------------------------------------------------
oMLX, by jundot
Copyright oMLX contributors
Licensed under the Apache License, Version 2.0.

Used in mlx-serve (`src/transformer.zig`):

  * The GatedDeltaNet blocked-sequence prefill kernel, ported from
    `custom_kernels/qwen35_prefill/gdn.py` (`gated_delta_blocked_seq`).
  * The per-dispatch key-chunk budget in the head-dim-256 fused causal
    attention path, whose design comes from `qwen35_fa256_attention.py`.
  * The direct-index Qwen4 QSA prefill design (attend to each query's
    selected blocks by gathered index, never through a dense [S, kv] mask;
    one threadgroup per query token), from the exact QSA prefill of PR #3244
    (`qsa_fast.py`, `qwen4_qsa_sparse_gqa`). The `msv_attn_qsa256` kernel
    body itself derives from our own `msv_attn_p256`, not from their source.
    thank you Jonathan Spangler.
    
Used in mlx-serve (`lib/ane/`, `src/ane.zig`):

  * The Neural Engine instance-affinity option keys and the requirement that
    they be passed together, from `custom_kernels/qwen35_prefill/csrc/
    qwen35_ane.mm`; the per-procedure request construction and the practice
    of reading a procedure's symbol indices from the model rather than
    assuming them come from the same file.
  * The procedure-bank program layout (many linears as procedures inside one
    compiled program) as a response to the runtime's resident-handle limit.

Modifications by mlx-serve: reimplemented in Zig against mlx-c, and the fa256
chunking carries exact online-softmax state between dispatches so a chunked run
is bit-identical to a single dispatch. The ANE MIL emitter is our own
(`constexpr_affine_dequantize` with an int8 zero point, our K-chunked down
conv and power-of-two accumulator wrap); only the bank structure and the
affinity handle are theirs.

--------------------------------------------------------------------------------
jinja.cpp
https://github.com/wangzhaode/jinja.cpp
Copyright 2025 Wang Zhaode
Licensed under the Apache License, Version 2.0.

Vendored under `lib/jinja_cpp/`, compiled to `libjinja.a` and linked into the
mlx-serve binary. It renders the chat templates that ship inside model
directories.

Modifications by mlx-serve: `value.cpp` and `value.h` were changed to add
`tojson(sort_keys=..., separators=...)`, which some chat templates require.
`jinja_wrapper.*`, `caps.*`, `jinja_string.*` and `log.h` are mlx-serve
additions providing the C interface the Zig side calls.


================================================================================
MIT License
================================================================================

--------------------------------------------------------------------------------
h3.c-ane (Apple Neural Engine port of h3.c)
https://github.com/maderix/h3.c-ane
https://github.com/maderix/ANE
Copyright (c) 2026 Manjeet Singh (maderix)
Licensed under the MIT License.

Vendored into `lib/ane/ane_bridge.{h,m}` (from `h3_ane_bridge.{h,m}`): the
objc-runtime bridge to the private AppleNeuralEngine framework — in-memory
MIL model compile/load/evaluate, IOSurface I/O binding, and the
content-addressed compile cache. The ANE port is maderix's addition to a
fork of antirez's h3.c (whose own code is Metal/GPU only and none of which
is vendored here), building on the private-API groundwork in
https://github.com/maderix/ANE. mlx-serve changes: the persistent cache
entries move to `~/.mlx-serve/ane-cache` (staging stays in $TMPDIR — the
compile runs inside aned, which cannot read a home-directory path) and the
cache env var is renamed `MLX_SERVE_ANE_CACHE`. `lib/ane/ane_mlp.m` (the
per-layer SwiGLU-MLP MIL program builder used by `--ane-prefill`) is an
mlx-serve addition modeled on the same project's `h3_ane_linear.m` blob
layout and int8 constexpr-dequantize idioms.

--------------------------------------------------------------------------------
SwaTex
https://github.com/PhraseHQ/SwaTex
Copyright (c) 2026 Phrase (https://phrase.so) and SwaTex contributors
Licensed under the MIT License.

Used by the MLX Core app to parse, lay out, and natively render LaTeX in
assistant chat responses. SwaTex includes work derived from:

  * RaTeX (https://github.com/erweixin/RaTeX), Copyright (c) erweixin,
    licensed under the MIT License. SwaTex's engine architecture, layout
    mathematics, generated tables, and golden corpus derive from RaTeX.
  * KaTeX (https://katex.org), Copyright (c) 2013-2020 Khan Academy and other
    contributors, licensed under the MIT License. Font metrics, symbols,
    stretchy-SVG geometry, macros, and layout rules derive from KaTeX.
  * mhchem for KaTeX, Copyright (c) Martin Hensel, licensed under the MIT
    License. The chemistry command state machine reaches SwaTex through
    RaTeX's Rust port.

The KaTeX_*.ttf fonts bundled in SwaTexRender use the SIL Open Font License
1.1. SwaTex_SwaTexRender.bundle includes the upstream OFL.txt and
FONT_NOTICE.txt alongside those fonts, and the entire bundle is copied into the
app's Resources directory by every packaging path.

--------------------------------------------------------------------------------
MLX
https://github.com/ml-explore/mlx
Copyright (c) 2023 Apple Inc.

Pinned submodule at `lib/mlx-src`, built by `scripts/build-mlx.sh` and linked
into the mlx-serve binary. Unmodified as a dependency.

Separately, Metal source from MLX was ported into `src/transformer.zig`: the
fused head-dim-256 prefill attention kernel (`msv_attn_p256`) began as a
self-contained port of MLX's steel attention (FA-2 online softmax, float32
accumulation, exp2 softmax) specialized to head_dim 256, and its fragment
layout mirrors MLX's `BaseMMAFrag<float,8,8>`. Modifications by mlx-serve: the
v2 tiling (BQ=64, BK=32, 8 simdgroups, register-resident Q, uint4 staging) was
picked by our own microbenchmark sweep, and a sliding-window band arm and
key-axis chunked dispatch were added.

Several other kernels of ours deliberately reproduce the exact arithmetic of
MLX ops (reduction order, accumulator dtype, `fast::exp`, the `precise::rsqrt`
variant) so a fused kernel stays bit-identical to the op chain it replaces.
Those are our own implementations written against MLX's observable behaviour,
not copies of its source.

`src/kernels/qsa_nax_header.metal` adapts the cooperative NAX fragment layout
and MMA helpers from MLX's `steel/attn/nax.h` (Copyright (c) 2025 Apple
Inc., MIT License, the same license as this project's LICENSE), and
`src/kernels/qsa_nax.metal` adapts the split-head-dimension attention
algorithm to per-query sparse block selection with a two-term bf16 PV.
Contributed by Nikolai V.

--------------------------------------------------------------------------------
mlx-lm
https://github.com/ml-explore/mlx-lm
Copyright (c) 2023-2026 Apple Inc.

Used in mlx-serve:

  * The GatedDeltaNet fused recurrence kernel in `src/transformer.zig`
    (`GDN_KERNEL_SOURCE`) is a port of `mlx_lm/models/gated_delta.py`'s
    `_make_gated_delta_kernel(has_mask=False, vectorized=False)`.
  * The global-sort MoE gather (`_gather_sort`: argsort the flattened
    (token, expert) pairs, gather rows, run with sorted indices, then inverse
    permute) is theirs, used by our multi-position MoE paths in
    `src/transformer.zig` and `src/deepseek_v4.zig`.

Modifications by mlx-serve: reimplemented in Zig against mlx-c, and the GDN
kernel is extended by a sequence variant (`GDN_KERNEL_SEQ_SOURCE`) that records
the recurrence state after every intermediate timestep so speculative decoding
can roll back without re-running the prefix.

--------------------------------------------------------------------------------
mlx-c
https://github.com/ml-explore/mlx-c
Copyright (c) 2023 ml-explore

Pinned submodule at `lib/mlxc-src`, built by `scripts/build-mlx.sh` and linked
into the mlx-serve binary. Pinned to an upstream commit chosen for
compatibility with the pinned MLX version. Unmodified.

--------------------------------------------------------------------------------
llama.cpp
https://github.com/ggml-org/llama.cpp
Copyright (c) 2023-2026 The ggml authors

Prebuilt release staged into `lib/llama` by `scripts/fetch-llama.sh` and linked
into the mlx-serve binary as the generic GGUF engine. Unmodified.

--------------------------------------------------------------------------------
ds4.c
https://github.com/antirez/ds4
Copyright (c) 2026 The ds4.c authors
Copyright (c) 2023-2026 The ggml authors

Pinned submodule at `lib/ds4`, compiled into the mlx-serve binary as the
DeepSeek-V4-Flash GGUF engine. Unmodified; mlx-serve supplies its own Metal
kernel staging and FFI bridge in `src/arch/ds4.zig`.

--------------------------------------------------------------------------------
nlohmann/json
https://github.com/nlohmann/json
SPDX-FileCopyrightText: 2013-2023 Niels Lohmann

Vendored under `lib/jinja_cpp/nlohmann/` as a dependency of jinja.cpp.
Unmodified.

--------------------------------------------------------------------------------
mlxfast-challenge
https://github.com/Layr-Labs/mlxfast-challenge
Copyright (c) 2026 Layr Labs, Inc.

Four decode fast paths in `src/transformer.zig` were derived from their tree
over two porting rounds in July 2026:

  * The certified lm_head prune, a port of `LagunaLmHeadPrune.swift`: an MXFP8
    coarse GEMV producing a certified per-row bound, a mask, and an exact pass
    over the surviving candidates.
  * Decode-only dense-attention requant (`--decode-attn-quant`): int8 side
    copies of dense attention weights served at decode widths, with real
    nvfp4-g16 copies on the tail layers at their measured split point.
  * The fused decode QK-norm + RoPE kernel, from their
    `laguna_sliding_qk_norm_rope_*`.
  * The fused MoE down + router-weighting + K-reduction tail, from their fused
    routed down-reduce, including its measured 4-rows-per-simdgroup choice.

Modifications by mlx-serve: all four were written in Zig against mlx-c rather
than Swift. The down-reduce kernel's per-row arithmetic is our own `gatherQmv`
body with their fusion structure around it; the QK-norm+RoPE kernel extracts
cos/sin by a probe row through stock `mlx_fast_rope` instead of re-deriving the
angles; the lm_head prune is gated on argmax-only requests and ships opt-in
behind `MLX_SERVE_LMHEAD_PRUNE=1` after our own A/B measured it slower in the
live graph than in a microbenchmark.

--------------------------------------------------------------------------------
qwen-3.8-mtp-challenge
https://github.com/Layr-Labs/qwen-3.8-mtp-challenge
Copyright (c) 2026 Layr Labs, Inc. and the challenge submitters

Ideas and code derived from validated challenge submissions at commit b6ce964,
ported in August 2026 (each item names the submission's co-author handle):

  * The hd-256 causal sdpa query split for 6-9-row verify blocks
    (`splitCausalSdpa` in `src/transformer.zig`), from
    `Vendor/mlx-swift-lm/.../AttentionUtils.swift` at b6ce964 (a-github-name):
    split the queries at row 5 so both halves ride MLX's sdpa vector path,
    with bottom-right causal alignment keeping each half's key window exact.
  * Requantizing a dense bf16 MTP draft head at load time, from the 4-bit/g64
    head declared in `mtp-head.manifest.json` at deb63ad (noskillcoding) — an
    idea rather than code: the draft head only proposes tokens, verification
    corrects them, so its weights can be served narrow.
  * The crossrow multi-row QMV kernel (`verifyQmmCrossrowSource` in
    `src/transformer.zig`), from `qmv_fast_crossrow_affine4_g64` in their
    vendored `quantized.h` as widened to M 2..9 at 08897af (hadakang): one
    packed-weight read serves two input rows, with /16-power prescaled
    activations against masked (unshifted) nibbles. Ships opt-in
    (`MLX_SERVE_VERIFY_QMM_CROSSROW=1`) — measured slower than stock in our
    stack, retained as an A/B lever.
  * The draft-rerank top-32 shortlist kernels (`msv_mtp_top32_partial` /
    `msv_mtp_top32_finalize` in `src/mtp.zig`), from
    `qwen35DraftTop32PartialKernel` / `qwen35DraftTop32FinalizeKernel` in
    their `Qwen35.swift` at 474c750 (submission 942e5ab2): an exact top-32
    of a full-vocab row in two dispatches (per-thread slot selection under a
    taken-bitmask, simd_max rounds, ordinal-mapped floats), replacing a
    full GPU argsort. The surrounding scheme (coarse 2-bit proposal readout
    re-scored by the exact head's own 32 rows) is theirs as well; our
    re-score uses the trunk lm_head instead of a compact draft head.

Modifications by mlx-serve: reimplemented in Zig against mlx-c; the query
split is gated per shape and kill-switched (`MLX_SERVE_SDPA_SPLIT`), the
head requantization uses our existing per-weight quantization path, and the
crossrow kernel is emitted by comptime codegen with our own host dispatch
(paired input groups instead of their idle-odd-threadgroup reuse).


================================================================================
Other licenses
================================================================================

--------------------------------------------------------------------------------
libwebp
https://chromium.googlesource.com/webm/libwebp
Copyright (c) 2010, Google Inc. All rights reserved.
Licensed under the BSD 3-Clause License.

Linked into the mlx-serve binary for WebP image decoding. Unmodified.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

  * Redistributions of source code must retain the above copyright notice, this
    list of conditions and the following disclaimer.
  * Redistributions in binary form must reproduce the above copyright notice,
    this list of conditions and the following disclaimer in the documentation
    and/or other materials provided with the distribution.
  * Neither the name of Google nor the names of its contributors may be used to
    endorse or promote products derived from this software without specific
    prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

--------------------------------------------------------------------------------
stb_image / stb_image_write
https://github.com/nothings/stb
Copyright (c) 2017 Sean Barrett
Public domain (Unlicense), or MIT at your option.

Vendored under `lib/` and compiled into the mlx-serve binary. Unmodified.


================================================================================
Reference implementations
================================================================================

No code from the following was copied into mlx-serve. They were read as
specifications while writing our own implementations of the same architectures,
and they are listed because they made that work possible.

  * poolside `modeling_laguna.py` (Apache-2.0), the reference for the `laguna`
    architecture.
  * The `inkling_mlx/` package bundled with Thinking Machines' Inkling Small
    checkpoints (Apache-2.0), the reference and parity oracle for
    `inkling_mm_model`.
  * hexgrad/misaki (Apache-2.0), whose pronunciation dictionaries are fetched
    verbatim into the Kokoro conversion we publish at
    `ddalcu/Kokoro-82M-MLX-Serve`, where they are redistributed under their own
    Apache-2.0 terms.
  * DeepSeek's `inference/model.py` and `encoding_dsv4.py`, shipped with the
    DeepSeek-V4-Flash release, the reference for the `deepseek_v4` architecture
    and its chat template.
  * The diffusers `minimax-music3-integration` branch (Apache-2.0, The MiniMax
    Team + The HuggingFace Team), the reference for the `minimax_music3`
    architecture: its prompt-normalization contract, the hierarchical AR loop,
    condition encoder, flow-matching DiT, chunked denoise and Flow-VAE vocoder.
    Its module code is transcribed into plain torch by
    `tests/dump_music3_fixtures.py` as the parity oracle; the fixtures it
    produces are golden values, not source.
  * ComfyUI's `comfy/ldm/minimax/*` and `comfy_extras/nodes_minimax_h3.py`
    (GPL-3.0), the reference for the `minimax_h3` architecture, its packed
    layout and its dual sigma schedule. Its layout/schedule output is executed
    directly as a parity oracle by `tests/dump_minimax_h3_layout.py`; the
    fixtures it produces are golden values, not source.
  * ComfyUI's `comfy/latent_formats.py` and `latent_preview.py` (GPL-3.0), the
    reference for the per-step denoise preview: the published latent-to-RGB
    fits `LTXAV` (128 channels) and `MiniMaxH3Video` (24), and the
    `Latent2RGBPreviewer` + `preview_to_image` arithmetic they are applied
    with. `tests/dump_latent_rgb_factors.py` EXECUTES those classes to emit
    `src/latent_rgb.zig` and `src/fixtures/latent_rgb.json`; the coefficients
    and reference outputs it produces are golden values, not source, and no
    ComfyUI code is transcribed or shipped.
  * Larryvrh/ComfyUI-MiniMax-H3-Turbo (Apache-2.0), whose `_turbo_sampler`
    is the reference for stepping H3's audio stream by the exact mapped
    delta-sigma at low step counts (`minimax_h3.audioStepFactor`). The Turbo
    LoRA weights it loads (larryvrh/MiniMax-H3-Turbo-Lora, Apache-2.0) are
    downloaded at runtime and not redistributed here.
  * filipstrand/mflux (MIT), the reference for stacking several LoRA adapters
    at once — summing per-adapter deltas at forward time rather than merging
    them into the base weight, the `lora_paths`/`lora_scales` request shape,
    and splitting a fused QKV up-projection across q/k/v targets.

Model weights downloaded at runtime carry their own licenses from their
publishers. mlx-serve does not redistribute them.
