mlx-serve
Copyright (c) 2026 David Dalcu

mlx-serve as a whole is distributed under the MIT License (see LICENSE). It
incorporates third-party software that remains under its own license, listed
below. Those licenses continue to apply to those portions. A copy of the Apache
License, Version 2.0 ships with this distribution as LICENSE-APACHE-2.0.

The source comments at each site name their upstream too.


================================================================================
Apache License, Version 2.0
================================================================================

--------------------------------------------------------------------------------
NVIDIA Nemotron / RADIO and Model Optimizer LiLiCorr
https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16
https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16
https://github.com/NVIDIA/Model-Optimizer
Copyright (c) 2026 NVIDIA CORPORATION. All rights reserved.
Licensed under the Apache License, Version 2.0.

The RADIO tower/processor in src/radio_vision.zig and the LiLiCorr head in
src/lilicorr.zig follow NVIDIA's reference implementations. The fixture
generators in tests/dump_radio_* and tests/dump_lilicorr_fixtures.py execute
those references to validate the native engine.

--------------------------------------------------------------------------------
Cloudflare Clef and mlx-community Clef MLX
https://huggingface.co/Cloudflare/clef
https://huggingface.co/mlx-community/clef-flash-4bit
Licensed under the Apache License, Version 2.0.

The joint schema head, input encoding and SystemOne response mapping in
src/clef.zig are adapted from joint_schema_model.py and clef_mlx.py.

--------------------------------------------------------------------------------
FastVideo
https://github.com/hao-ai-lab/FastVideo
Copyright FastVideo contributors
Licensed under the Apache License, Version 2.0.

The VSA-H3 sparse attention in src/h3_vsa.zig (tile geometry, routing, the
gate branch and the SIMD-group block-sparse Metal kernel) is ported from
`fastvideo/mlx_runtime/minimax_h3_vsa.py` and `minimax_h3_vsa_simd.py`, and
the PDD sampler in src/minimax_h3.zig follows `fastvideo/layers/pdd.py`.
tests/dump_h3_vsa_fixtures.py executes the reference to validate the port.

--------------------------------------------------------------------------------
SageAttention
https://github.com/thu-ml/SageAttention
Licensed under the Apache License, Version 2.0.

The int8 attention recipe in src/int8_attn.zig (Q int8 per token, K smoothed and
int8 per key block, P int8 against each block's row max, V int8 per channel)
follows the SageAttention papers; the Metal kernels are our own.

--------------------------------------------------------------------------------
MTPLX
https://github.com/youssofal/mtplx
Copyright 2026 Youssof Altoukhi
Licensed under the Apache License, Version 2.0.

Reproduced from MTPLX's NOTICE file:

    MTPLX
    Copyright 2026 Youssof Altoukhi

    MTPLX is a native MTP speculative decoding project for Apple Silicon.

    Preferred attribution for public projects, products, benchmarks, articles,
    and research that use or build on MTPLX:

      Powered by MTPLX by Youssof Altoukhi
      https://github.com/youssofal/mtplx

    If MTPLX informs academic or technical writing, please cite the repository
    using the included CITATION.cff metadata.

    This product includes Metal kernel code adapted from dflash-mlx
    (https://github.com/bstnxbt/dflash-mlx), Copyright dflash-mlx contributors,
    licensed under the Apache License 2.0. See mtplx/nax_verify.py for details.

Used in mlx-serve (`src/transformer.zig`):

  * The verify-shaped quantized matmul kernel family (`verifyQmm`), ported from
    `mtplx/verify_kernels.py`. Their design: one threadgroup per column tile,
    split-K reduction across simdgroups, sequential per-column dequant and FMA
    chains, named-scalar codegen with literal accumulator indices.
  * The NAX m16 tensor-ops tile for M5-class GPUs, ported from
    `mtplx/nax_verify.py`, together with its host contract: the M-padding
    dispatch, the environment gating and the availability probes.
  * The physical-M4 paired routed gate/up q4 producer, adapted from
    `mtplx/kernels/qwen4_m4_routed_glu.py`. Its row-paired arithmetic and
    one-threadgroup-per-eight-output-rows geometry are retained.

Modifications by mlx-serve: the kernels were reimplemented in Zig against
mlx-c, the Metal sources are generated by comptime codegen rather than Python
f-strings, and the NAX tile's dequantization loop was extended from 4-bit only
to 4-, 5- and 6-bit affine weights. The paired gate/up kernel uses the target
checkpoint's separate q4/group-64 gate and up banks, explicit sorted row and
expert indices, and a BF16 sigmoid table to preserve mlx-serve's fused SwiGLU
rounding. It is installed only after pack validation and per-layer self-check.

MTPLX's own NOTICE also credits vllm-metal and PipeNetwork for parts of MTPLX
that mlx-serve does not use. Those notices are omitted here as permitted by
Apache-2.0 section 4(d).

--------------------------------------------------------------------------------
Qwen-Image
https://github.com/QwenLM/Qwen-Image
Copyright Alibaba Qwen team
Licensed under the Apache License, Version 2.0.

The image Enhance rules for text in a picture (`textLayoutShape` in
app/Sources/MLXServe/Services/PromptRewriter.swift) are condensed from the
text-image instructions of `polish_prompt_en` in
src/examples/tools/prompt_utils_2512.py.

--------------------------------------------------------------------------------
MOSS-Image
https://github.com/OpenMOSS/MOSS-Image
Copyright OpenMOSS
Licensed under the Apache License, Version 2.0.

src/moss_image.zig ports the MOSS-Image diffusers pipeline
(pipeline_moss_image.py: the transformer, scheduler and sampling loop) to MLX.
The image Enhance shape for MOSS-Image (`mossShape` in
app/Sources/MLXServe/Services/PromptRewriter.swift) is condensed from
skills/moss-image-prompt-enhancer/SKILL.md, and
app/Sources/MLXServe/Services/MossLayout.swift ports its
scripts/compile_prompt.py.

--------------------------------------------------------------------------------
laya-mlx
https://github.com/mizorewww/laya-mlx
Copyright laya-mlx contributors
Licensed under the Apache License, Version 2.0.

src/laya.zig is an op-for-op port of the laya_mlx runtime: prompt and sequence
construction, the ModernBERT encoder, decision head, marker scorer, action head
and calibration, so /v1/decisions answers match the reference.

--------------------------------------------------------------------------------
Inco Splash
https://github.com/incoai/splash
Licensed under the Apache License, Version 2.0.

The matmul2d packed-KV attention kernel (msv_qkv_mpp in src/transformer.zig)
follows the structure of Splash's q8 attention tile
(runtime/metal/kernels/common/q8_attention_tile.h): per-page QK and PV through
the MetalPerformancePrimitives tensor op with an online softmax and split-KV
partials. Re-implemented over MLX's affine group-quantized cache.

--------------------------------------------------------------------------------
dflash-mlx
https://github.com/bstnxbt/dflash-mlx
Copyright dflash-mlx contributors
Licensed under the Apache License, Version 2.0.

Reached through MTPLX's `nax_verify.py`, which is a port of dflash-mlx's
`dflash_mlx/verify_qmm.py`, itself based on DFlash (arXiv:2602.06036). The
matmul2d tensor extents and strides in our NAX kernel are dflash's working
convention.

--------------------------------------------------------------------------------
FlashInfer (Local Inference Lab fork)
https://github.com/local-inference-lab/flashinfer
Copyright FlashInfer contributors
Licensed under the Apache License, Version 2.0.

Used in mlx-serve (`src/modelopt_pack.zig`, `decodeCsf32`): the `csf_32x2_v1`
NVFP4 scale-block layout, ported from
`flashinfer/experimental/b12x/loader/_csf_blocks.py`.

--------------------------------------------------------------------------------
oMLX, by jundot
Copyright oMLX contributors
Licensed under the Apache License, Version 2.0.

Used in mlx-serve (`src/transformer.zig`):

  * The GatedDeltaNet blocked-sequence prefill kernel, ported from
    `custom_kernels/qwen35_prefill/gdn.py` (`gated_delta_blocked_seq`).
  * The per-dispatch key-chunk budget in the head-dim-256 fused causal
    attention path, whose design comes from `qwen35_fa256_attention.py`.
  * The direct-index Qwen4 QSA prefill design (attend to each query's
    selected blocks by gathered index, never through a dense [S, kv] mask;
    one threadgroup per query token), from the exact QSA prefill of PR #3244
    (`qsa_fast.py`, `qwen4_qsa_sparse_gqa`). The `msv_attn_qsa256` kernel
    body itself derives from our own `msv_attn_p256`, not from their source.
    thank you Jonathan Spangler.
  * The segmented sorted NAX gather_qmm tile scan and affine matmul schedules,
    with row-mapped inputs and paired gate/up SwiGLU epilogue, ported from
    `omlx/patches/m5_gather_qmm_nax.py` at d6b2b92.
  * The tensor-unit (NAX) prefill attention at q/k 192, v 128 with key-range
    passes and the head-dim split (`src/nax_attention.zig`,
    `src/kernels/nax_attention.metal`), and the NAX tile primitives
    (`src/kernels/nax_tiles.metal`), ported from `omlx/utils/nax_attention.py`
    and `omlx/custom_kernels/nax_tiles.py` at 5dcfe243. Both are MLX's own
    steel NAX attention and tile code (Copyright (c) 2024-25 Apple Inc., MIT);
    the sliding-window bound is ours.
  * The block-sparse VSA attention (`src/kernels/h3_vsa_nax.metal`): that
    attention kernel's loop walking each query tile's selected key tiles.

Used in mlx-serve for GLM-5-Next (`src/glm5_next.zig`, `src/dsa_nax.zig`,
`src/kda_recurrence.zig`, `src/kernels/kda_*.metal`, `src/kernels/glm5_*.metal`),
from oMLX 0.7.0:

  * The sparse latent attention on the tensor units, from
    `omlx/patches/glm_moe_dsa/sparse_mla_nax.py` (`src/dsa_nax.zig` adds the
    roped part of DeepSeek-V3.2's MLA), and the indexer scores on the tensor
    units, from `omlx/patches/glm_moe_dsa/indexer_nax.py`.
  * The per-core KDA prefill recurrence (`kda_percore.metal`), generated by
    `omlx/patches/glm53_kda_recurrence.py`.
  * From `omlx/patches/mlx_vlm_glm5_next_compat/decode_kernels.py`: the
    one-token KDA layer step (`kda_decode.metal`), the one-token MoE gate/up
    and down + combine kernels with their qmv header (`glm5_moe_*.metal`,
    `glm5_qmv_header.metal`), the selected-pool expansion, and the design of
    the one-dispatch hyper-connection collapse with the expand deferred into
    the next sublayer (`hc_pre_fused`; the cross-threadgroup reduction is
    ours).

Used in mlx-serve (`lib/ane/`, `src/ane.zig`):

  * The Neural Engine instance-affinity option keys and the requirement that
    they be passed together, from `custom_kernels/qwen35_prefill/csrc/
    qwen35_ane.mm`; the per-procedure request construction and the practice
    of reading a procedure's symbol indices from the model rather than
    assuming them come from the same file.
  * The procedure-bank program layout (many linears as procedures inside one
    compiled program) as a response to the runtime's resident-handle limit.

Modifications by mlx-serve: reimplemented in Zig against mlx-c, and the fa256
chunking carries exact online-softmax state between dispatches so a chunked run
is bit-identical to a single dispatch. The ANE MIL emitter is our own
(`constexpr_affine_dequantize` with an int8 zero point, our K-chunked down
conv and power-of-two accumulator wrap); only the bank structure and the
affinity handle are theirs.

--------------------------------------------------------------------------------
jinja.cpp
https://github.com/wangzhaode/jinja.cpp
Copyright 2025 Wang Zhaode
Licensed under the Apache License, Version 2.0.

Vendored under `lib/jinja_cpp/`, compiled to `libjinja.a` and linked into the
mlx-serve binary. It renders the chat templates that ship inside model
directories.

Modifications by mlx-serve: `value.cpp` and `value.h` were changed to add
`tojson(sort_keys=..., separators=...)`, which some chat templates require.
`jinja_wrapper.*`, `caps.*`, `jinja_string.*` and `log.h` are mlx-serve
additions providing the C interface the Zig side calls.


--------------------------------------------------------------------------------
NAF (valeoai)
https://github.com/valeoai/NAF
Licensed under the Apache License, Version 2.0.

`src/naf.zig` reimplements the released NAF feature upsampler operation for
operation; Pixal3D conditions its voxels on DINOv3 features upsampled by it.
The Pixal3D pack we publish on Hugging Face carries the released NAF weights
with NAF's license beside them.

================================================================================
MIT License
================================================================================

--------------------------------------------------------------------------------
Svelte 5.57.2
https://github.com/sveltejs/svelte/tree/svelte@5.57.2
Copyright (c) 2016-2025 Svelte Contributors
Licensed under the MIT License (the permission and warranty terms in LICENSE).

The Svelte runtime is compiled into `src/html/index.html`, the web console
built from `app-web/`.

--------------------------------------------------------------------------------
MLX Serve Studio (the web console)
https://github.com/ddalcu/mlx-serve/pull/755
Contributed under the MIT License of this repository by Alin Selea (Studio, #755),
Samuel Reed (the Monitoring panel, #701) and Pan Patryk (mount-prefix resolution,
#698), with earlier console work by other contributors.

`app-web/` re-implements Studio in Svelte and TypeScript: its client logic,
presets, prompts, translations and layout are ported from Studio's modules.

--------------------------------------------------------------------------------
Sushi n-gram gather calibration, MTP depth bounds and the e4m3 GEMV
https://github.com/beamivalice/sushi
Copyright (c) 2026 Theinruj Toranavikrai
Copyright (c) 2026 David Dalcu
Licensed under the MIT License (the permission and warranty terms in LICENSE).

The load-time read-path calibration in `src/qwen4_exp.zig` is adapted from
Sushi commit feb9ed7d92149e7ebc46a34595e19d440a929a0a.
The CLI depth-range helpers in `src/mtp_depth_bounds.zig`, planner clamping
and associated regression tests in `src/generate.zig` and `src/scheduler.zig`
are adapted from Sushi commit 09e6d4792e6178b864b809299d53dbd05cd054cd.
The e4m3 GEMV in `src/sushi_pack.zig` is adapted from `src/fp8_block.zig` of
Sushi v1.2.3 (commit 2c602b9e).

--------------------------------------------------------------------------------
mlx-stream
https://github.com/davidtai/mlx-stream
Copyright (c) 2026 David Tai
Licensed under the MIT License (the permission and warranty terms in LICENSE).

`lib/mlx-stream`, built into the macOS server: DeepSeek-V4.1's EXL3 streaming
repack (its arch, Metal kernels, EXL3 decoder and SSD expert streamer) and
MiMo-V2.6-Flash's prepared MXFP4 pack (its arch, kernels and MTP draft lane). Its own
NOTICE lists the material it incorporates (MLX's steel headers, ExLlamaV3's
EXL3 format).

--------------------------------------------------------------------------------
h3.c-ane (Apple Neural Engine port of h3.c)
https://github.com/maderix/h3.c-ane
https://github.com/maderix/ANE
Copyright (c) 2026 Manjeet Singh (maderix)
Licensed under the MIT License.

Vendored into `lib/ane/ane_bridge.{h,m}` (from `h3_ane_bridge.{h,m}`): the
objc-runtime bridge to the private AppleNeuralEngine framework — in-memory
MIL model compile/load/evaluate, IOSurface I/O binding, and the
content-addressed compile cache. The ANE port is maderix's addition to a
fork of antirez's h3.c (whose own code is Metal/GPU only and none of which
is vendored here), building on the private-API groundwork in
https://github.com/maderix/ANE. mlx-serve changes: the persistent cache
entries move to `~/.mlx-serve/ane-cache` (staging stays in $TMPDIR — the
compile runs inside aned, which cannot read a home-directory path) and the
cache env var is renamed `MLX_SERVE_ANE_CACHE`. `lib/ane/ane_mlp.m` (the
per-layer SwiGLU-MLP MIL program builder used by `--ane-prefill`) is an
mlx-serve addition modeled on the same project's `h3_ane_linear.m` blob
layout and int8 constexpr-dequantize idioms.

--------------------------------------------------------------------------------
ANEMLL Forge
https://github.com/Anemll/anemll-forge
Copyright (c) 2026 ANEMLL contributors
Licensed under the MIT License.

Ported (commit 6c572a8), not vendored: `src/arch/ane_qwen35.zig` and `src/arch/ane_dflash.zig` emit, as
MIL text from MLX packs, the Neural Engine graphs of Forge's builders (`coreai/qwen38_coreai_build.py`,
`scripts/qwen38_ane_chunk.py`, `scripts/dflash2_gdn_lazy.py`, `coreai/dflash2_coreai_build.py`): the
lazy-commit Gated DeltaNet and its UT-transform form, the attention over host-owned KV, the fp16 numerics
(scale-free RMSNorm, tanh SiLU, overflow-safe softplus, q / v range scales, the drafter's residual scale)
and the DFlash2 drafter's draft / context entries. `src/arch/ane_lm.zig` is an mlx-serve port of Forge's
host runtime (`scripts/qwen38_coreai_model.py`, `scripts/qwen38_kv_cache.py`): the per-call positions and
masks, the DeltaNet state ping-pong, the KV commits, the speculative cycle and the selector walk.

--------------------------------------------------------------------------------
SwaTex
https://github.com/PhraseHQ/SwaTex
Copyright (c) 2026 Phrase (https://phrase.so) and SwaTex contributors
Licensed under the MIT License.

Used by the MLX Core app to parse, lay out, and natively render LaTeX in
assistant chat responses. SwaTex includes work derived from:

  * RaTeX (https://github.com/erweixin/RaTeX), Copyright (c) erweixin,
    licensed under the MIT License. SwaTex's engine architecture, layout
    mathematics, generated tables, and golden corpus derive from RaTeX.
  * KaTeX (https://katex.org), Copyright (c) 2013-2020 Khan Academy and other
    contributors, licensed under the MIT License. Font metrics, symbols,
    stretchy-SVG geometry, macros, and layout rules derive from KaTeX.
  * mhchem for KaTeX, Copyright (c) Martin Hensel, licensed under the MIT
    License. The chemistry command state machine reaches SwaTex through
    RaTeX's Rust port.

The KaTeX_*.ttf fonts bundled in SwaTexRender use the SIL Open Font License
1.1. SwaTex_SwaTexRender.bundle includes the upstream OFL.txt and
FONT_NOTICE.txt alongside those fonts, and the entire bundle is copied into the
app's Resources directory by every packaging path.

--------------------------------------------------------------------------------
MLX
https://github.com/ml-explore/mlx
Copyright (c) 2023 Apple Inc.

Pinned submodule at `lib/mlx-src`, built by `scripts/build-mlx.sh` and linked
into the mlx-serve binary. Unmodified as a dependency.

Separately, Metal source from MLX was ported into `src/transformer.zig`: the
fused head-dim-256 prefill attention kernel (`msv_attn_p256`) began as a
self-contained port of MLX's steel attention (FA-2 online softmax, float32
accumulation, exp2 softmax) specialized to head_dim 256, and its fragment
layout mirrors MLX's `BaseMMAFrag<float,8,8>`. Modifications by mlx-serve: the
v2 tiling (BQ=64, BK=32, 8 simdgroups, register-resident Q, uint4 staging) was
picked by our own microbenchmark sweep, and a sliding-window band arm and
key-axis chunked dispatch were added.

The verifier expert-pair kernel (`VERIFY_EXPERT_REUSE_SOURCE`) adapts the
4-bit affine `qmv_impl`, `load_vector` and `qdot` arithmetic from MLX's
`backend/metal/kernels/quantized.h`. Modifications: sorted expert-run pairing,
shared packed-weight reads for two inputs, and verification-only shape guards.
The fused expert reduction preserves the bf16 partial-sum order of MLX
`reduction/reduce_col.h:col_reduce_small` while gathering, weighting and
reducing in one dispatch. The batched coarse-head pair kernel
(`MTP_COARSE_PAIRS_SOURCE`) adapts the 3-bit affine qmv-fast reduction from
quantized.h, sharing each packed read across two input rows.

Several other kernels of ours deliberately reproduce the exact arithmetic of
MLX ops (reduction order, accumulator dtype, `fast::exp`, the `precise::rsqrt`
variant) so a fused kernel stays bit-identical to the op chain it replaces.
Those are our own implementations written against MLX's observable behaviour,
not copies of its source.

The original QSA NAX kernel adapted MLX's `steel/attn/nax.h` (Copyright
(c) 2025 Apple Inc., MIT License). The current `src/kernels/qsa_nax.metal`
and `src/kernels/qsa_nax_header.metal` port oMLX's occupancy-tuned QSA
kernel from `omlx/patches/mlx_vlm_qwen4_exp_compat/vendor/mlx_vlm/models/
qwen4_exp/qsa_nax.py` at d6b2b92 (Apache-2.0).
https://github.com/jundot/omlx/pull/4020
Original mlx-serve QSA NAX implementation contributed by Nikolai V.

`src/kernels/gather_qmm_nax_header.metal` adapts the NAX matmul fragment and
MMA helpers from MLX's `steel/gemm/nax.h` (Copyright (c) 2025 Apple Inc., MIT
License).

`src/lane_qmm.zig`'s `PREFILL` kernel ports MLX's affine `qmm_t_nax`
(`quantized_nax.h`) to a tiled weight layout and inlines `steel/gemm/nax.h`
with the headers it includes, embedded from the pinned submodule (Copyright
(c) 2023-2025 Apple Inc., MIT License).

--------------------------------------------------------------------------------
mlx-lm
https://github.com/ml-explore/mlx-lm
Copyright (c) 2023-2026 Apple Inc.

Used in mlx-serve:

  * The GatedDeltaNet fused recurrence kernel in `src/transformer.zig`
    (`GDN_KERNEL_SOURCE`) is a port of `mlx_lm/models/gated_delta.py`'s
    `_make_gated_delta_kernel(has_mask=False, vectorized=False)`.
  * The global-sort MoE gather (`_gather_sort`: argsort the flattened
    (token, expert) pairs, gather rows, run with sorted indices, then inverse
    permute) is theirs, used by our multi-position MoE paths in
    `src/transformer.zig` and `src/deepseek_v4.zig`.

Modifications by mlx-serve: reimplemented in Zig against mlx-c, and the GDN
kernel is extended by a sequence variant (`GDN_KERNEL_SEQ_SOURCE`) that records
the recurrence state after every intermediate timestep so speculative decoding
can roll back without re-running the prefix.

--------------------------------------------------------------------------------
mlx-c
https://github.com/ml-explore/mlx-c
Copyright (c) 2023 ml-explore

Pinned submodule at `lib/mlxc-src`, built by `scripts/build-mlx.sh` and linked
into the mlx-serve binary. Pinned to an upstream commit chosen for
compatibility with the pinned MLX version. Unmodified.

--------------------------------------------------------------------------------
llama.cpp
https://github.com/ggml-org/llama.cpp
Copyright (c) 2023-2026 The ggml authors

Prebuilt release staged into `lib/llama` by `scripts/fetch-llama.sh` and linked
into the mlx-serve binary as the generic GGUF engine. Unmodified.

--------------------------------------------------------------------------------
ds4.c
https://github.com/antirez/ds4
Copyright (c) 2026 The ds4.c authors
Copyright (c) 2023-2026 The ggml authors

Pinned submodule at `lib/ds4`, compiled into the mlx-serve binary as the
DeepSeek-V4-Flash GGUF engine. Unmodified; mlx-serve supplies its own Metal
kernel staging and FFI bridge in `src/arch/ds4.zig`.

--------------------------------------------------------------------------------
nlohmann/json
https://github.com/nlohmann/json
SPDX-FileCopyrightText: 2013-2023 Niels Lohmann

Vendored under `lib/jinja_cpp/nlohmann/` as a dependency of jinja.cpp.
Unmodified.

--------------------------------------------------------------------------------
mlxfast-challenge
https://github.com/Layr-Labs/mlxfast-challenge
Copyright (c) 2026 Layr Labs, Inc.

Three decode fast paths in `src/transformer.zig` were derived from their tree
over two porting rounds in July 2026:

  * Decode-only dense-attention requant (`--decode-attn-quant`): int8 side
    copies of dense attention weights served at decode widths, with real
    nvfp4-g16 copies on the tail layers at their measured split point.
  * The fused decode QK-norm + RoPE kernel, from their
    `laguna_sliding_qk_norm_rope_*`.
  * The fused MoE down + router-weighting + K-reduction tail, from their fused
    routed down-reduce, including its measured 4-rows-per-simdgroup choice.

Modifications by mlx-serve: all three were written in Zig against mlx-c rather
than Swift. The down-reduce kernel's per-row arithmetic is our own `gatherQmv`
body with their fusion structure around it; the QK-norm+RoPE kernel extracts
cos/sin by a probe row through stock `mlx_fast_rope` instead of re-deriving the
angles.

--------------------------------------------------------------------------------
qwen-3.8-mtp-challenge
https://github.com/Layr-Labs/qwen-3.8-mtp-challenge
Copyright (c) 2026 Layr Labs, Inc. and the challenge submitters

Ideas and code derived from validated challenge submissions at commit b6ce964,
ported in August 2026 (each item names the submission's co-author handle):

  * The hd-256 causal sdpa query split for 6-9-row verify blocks
    (`splitCausalSdpa` in `src/transformer.zig`), from
    `Vendor/mlx-swift-lm/.../AttentionUtils.swift` at b6ce964 (a-github-name):
    split the queries at row 5 so both halves ride MLX's sdpa vector path,
    with bottom-right causal alignment keeping each half's key window exact.
  * Requantizing a dense bf16 MTP draft head at load time, from the 4-bit/g64
    head declared in `mtp-head.manifest.json` at deb63ad (noskillcoding) — an
    idea rather than code: the draft head only proposes tokens, verification
    corrects them, so its weights can be served narrow.
  * The crossrow multi-row QMV kernel (`verifyQmmCrossrowSource` in
    `src/transformer.zig`), from `qmv_fast_crossrow_affine4_g64` in their
    vendored `quantized.h` as widened to M 2..9 at 08897af (hadakang): one
    packed-weight read serves two input rows, with /16-power prescaled
    activations against masked (unshifted) nibbles. Ships opt-in
    (`MLX_SERVE_VERIFY_QMM_CROSSROW=1`) — measured slower than stock in our
    stack, retained as an A/B lever.
  * The draft-rerank top-32 shortlist kernels (`msv_mtp_top32_partial` /
    `msv_mtp_top32_finalize` in `src/mtp.zig`), from
    `qwen35DraftTop32PartialKernel` / `qwen35DraftTop32FinalizeKernel` in
    their `Qwen35.swift` at 474c750 (submission 942e5ab2): an exact top-32
    of a full-vocab row in two dispatches (per-thread slot selection under a
    taken-bitmask, simd_max rounds, ordinal-mapped floats), replacing a
    full GPU argsort. The surrounding scheme (coarse 2-bit proposal readout
    re-scored by the exact head's own 32 rows) is theirs as well; our
    re-score uses the trunk lm_head instead of a compact draft head.

Modifications by mlx-serve: reimplemented in Zig against mlx-c; the query
split is gated per shape and kill-switched (`MLX_SERVE_SDPA_SPLIT`), the
head requantization uses our existing per-weight quantization path, and the
crossrow kernel is emitted by comptime codegen with our own host dispatch
(paired input groups instead of their idle-odd-threadgroup reuse).

--------------------------------------------------------------------------------
TensorFold
https://github.com/ashhart/TensorFold
Copyright (c) 2026 TensorFold contributors
Licensed under the MIT License.

Ported into mlx-serve:
  - `src/rowqmv.zig`: the row-exact 4-bit matvec, routed-expert and router
    kernels of `rows.py` (one simdgroup per input row, MLX's qmv_fast loop).
  - `src/simd_qmm.zig`: the `simd_qmm` fixed-order matmul kernels.
  - `src/row_attn.zig`: the fixed-split row attention of `row_attention`.
  - `src/lane_qmm.zig`: the row-exact tensor-unit lane matmul of `lane_qmm.py`
    and the 5/6/8-bit group widening of `lane_widen.py`.
  - `src/gdn_decode.zig`: the draft-tree GDN launch layout of `stream_gdn.py`.
  - `src/generate.zig`: the backed context-copy rounds of the DFlash proposer.
  - `src/lane_attn.zig`: the lane/stream attention of `lane_attention.py` and
    `stream_attention.py`.
  - `src/keyed_sample.zig`: the keyed Gumbel-max GPU sampler of
    `gpu_sampling.py` (Metal source reproduced unchanged).
  - `src/dflash.zig`: the best-first draft-tree parameters and the
    lattice/tree construction scheme.

Modifications by mlx-serve: reimplemented in Zig against mlx-c with our own
host dispatch; `simd_qmm` gains a 2-3 row `rows` kind; draft trees verify
through our GDN fold kernel and path attention.


--------------------------------------------------------------------------------
TRELLIS.2 and TRELLIS (Microsoft)
https://github.com/microsoft/TRELLIS.2
https://huggingface.co/microsoft/TRELLIS.2-4B
https://huggingface.co/microsoft/TRELLIS-image-large
Copyright (c) Microsoft Corporation.
Licensed under the MIT License.

`src/trellis2*.zig` port the TRELLIS.2 image-to-3D pipeline: the sparse-structure,
shape and texture flow transformers with their sampler recipes, the sparse
SC-VAE decoders, the dual-grid mesh extraction and the PBR attribute layout;
the TRELLIS v1 sparse-structure decoder is ported alongside. Written in Zig
against mlx-c, not shipped as their source. The 8-bit packs we publish on
Hugging Face repackage the released weights.

--------------------------------------------------------------------------------
Pixal3D (TencentARC)
https://github.com/TencentARC/Pixal3D
https://huggingface.co/TencentARC/Pixal3D
Copyright (c) 2026 Tencent.
Licensed under the MIT License.

`src/pixal3d.zig` ports the pixel-aligned conditioning (each voxel projected
through the camera into the DINOv3 features), the multi-view averaging and the
per-block feature projection of the Pixal3D flows. The 8-bit pack we publish on
Hugging Face repackages the released weights.

================================================================================
Other licenses
================================================================================

--------------------------------------------------------------------------------
libwebp
https://chromium.googlesource.com/webm/libwebp
Copyright (c) 2010, Google Inc. All rights reserved.
Licensed under the BSD 3-Clause License.

Linked into the mlx-serve binary for WebP image decoding. Unmodified.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

  * Redistributions of source code must retain the above copyright notice, this
    list of conditions and the following disclaimer.
  * Redistributions in binary form must reproduce the above copyright notice,
    this list of conditions and the following disclaimer in the documentation
    and/or other materials provided with the distribution.
  * Neither the name of Google nor the names of its contributors may be used to
    endorse or promote products derived from this software without specific
    prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

--------------------------------------------------------------------------------
DINOv3 (Meta)
https://github.com/facebookresearch/dinov3
https://huggingface.co/facebook/dinov3-vitl16-pretrain-lvd1689m
Copyright (c) Meta Platforms, Inc. and affiliates.
Licensed under the DINOv3 License (Meta's own agreement, neither MIT nor Apache).

`src/dinov3.zig` implements the ViT-L/16 image encoder the TRELLIS.2 and Pixal3D
pipelines condition on. The packs we publish on Hugging Face include its weights
in float16 under that agreement, with its text as `dinov3/LICENSE.md`; mlx-serve
itself does not redistribute them. The agreement must travel with any copy of
the weights and restricts some uses (trade-control and military end uses).

--------------------------------------------------------------------------------
Fast-Quadric-Mesh-Simplification
https://github.com/sp4cerat/Fast-Quadric-Mesh-Simplification
Copyright (c) 2015-2019 Spacerat and contributors
Licensed under the MIT License (see lib/fqms/LICENSE).

`Simplify.h` at 65df07dc, vendored unmodified under `lib/fqms/` with a C shim
and compiled into the mlx-serve binary: the quadric decimation the Hunyuan3D
texture stage runs before its UV unwrap.

--------------------------------------------------------------------------------
xatlas
https://github.com/jpcy/xatlas
Copyright (c) 2018-2020 Jonathan Young
Licensed under the MIT License (see lib/xatlas/LICENSE).

Vendored under `lib/xatlas/` with a C shim and compiled into the mlx-serve
binary: the UV unwrap of the Hunyuan3D paint stage and of the TRELLIS.2 and
Pixal3D texture bake.

--------------------------------------------------------------------------------
Stable Audio 3 (MLX reference implementation)
https://github.com/Stability-AI/stable-audio-3
Copyright (c) 2026 Stability AI
Licensed under the MIT License.

`src/stable_audio.zig` ports the T5Gemma encoder, DiT, ping-pong sampler and
SAME-S decoder from `optimized/mlx/models/defs`. The model WEIGHTS are not
part of mlx-serve: they are the Stability AI Community License
(https://stability.ai/license), T5Gemma the Gemma Terms of Use.

--------------------------------------------------------------------------------
YuE2 (m-a-p) and ahmadw/YuE2-3B-MLX
https://huggingface.co/m-a-p/YuE2-3B
https://huggingface.co/ahmadw/YuE2-3B-MLX
Oobleck decoder: Copyright (c) 2023 Stability AI (stable-audio-tools), MIT.
SnakeBeta: Copyright (c) 2022 NVIDIA CORPORATION (BigVGAN), MIT.

`src/yue2.zig` follows the YuE2 inference protocol and the pack layout of
ahmadw's MLX port, including the decoder those packs fold in. The model WEIGHTS
are not part of mlx-serve: they are CC BY-NC 4.0 (non-commercial); credit YuE2,
its model name and https://huggingface.co/m-a-p/YuE2-3B when you publish output.

--------------------------------------------------------------------------------
stb_image / stb_image_write
https://github.com/nothings/stb
Copyright (c) 2017 Sean Barrett
Public domain (Unlicense), or MIT at your option.

Vendored under `lib/` and compiled into the mlx-serve binary. Unmodified.


================================================================================
Reference implementations
================================================================================

No code from the following was copied into mlx-serve. They were read as
specifications while writing our own implementations of the same architectures,
and they are listed because they made that work possible.

  * poolside `modeling_laguna.py` (Apache-2.0), the reference for the `laguna`
    architecture.
  * The `inkling_mlx/` package bundled with Thinking Machines' Inkling Small
    checkpoints (Apache-2.0), the reference and parity oracle for
    `inkling_mm_model`.
  * hexgrad/misaki (Apache-2.0), whose pronunciation dictionaries are fetched
    verbatim into the Kokoro conversion we publish at
    `ddalcu/Kokoro-82M-MLX-Serve`, where they are redistributed under their own
    Apache-2.0 terms.
  * DeepSeek's `inference/model.py` and `encoding_dsv4.py`, shipped with the
    DeepSeek-V4-Flash release, the reference for the `deepseek_v4` architecture
    and its chat template.
  * DeepSeek's `inference/model.py`, `inference/engram.py` and
    `encoding/encoding.py` (MIT), shipped with the DeepSeek-V4.1-Flash release,
    the reference for the `deepseek_v41` architecture, its Engram hash, its
    Engram token map (`src/fixtures/dsv41_engram_token_map.bin`, derived from
    the tokenizer by `build_compressed_token_map`) and its chat template. PipeNetwork's `deepseek-v41-mlx` kernel stub runs that
    reference on CPU as the parity oracle (`tests/dump_dsv41_fixtures.py`).
    The Engram hash multipliers reproduce NumPy's `default_rng` (BSD-3-Clause:
    SeedSequence, PCG64, bounded integers) in `src/dsv41_engram.zig`.
  * The diffusers `minimax-music3-integration` branch (Apache-2.0, The MiniMax
    Team + The HuggingFace Team), the reference for the `minimax_music3`
    architecture: its prompt-normalization contract, the hierarchical AR loop,
    condition encoder, flow-matching DiT, chunked denoise and Flow-VAE vocoder.
    Its module code is transcribed into plain torch by
    `tests/dump_music3_fixtures.py` as the parity oracle; the fixtures it
    produces are golden values, not source.
  * ComfyUI's `comfy/ldm/minimax/*` and `comfy_extras/nodes_minimax_h3.py`
    (GPL-3.0), the reference for the `minimax_h3` architecture, its packed
    layout and its dual sigma schedule. Its layout/schedule output is executed
    directly as a parity oracle by `tests/dump_minimax_h3_layout.py`; the
    fixtures it produces are golden values, not source.
  * ComfyUI's `comfy/latent_formats.py` and `latent_preview.py` (GPL-3.0), the
    reference for the per-step denoise preview: the published latent-to-RGB
    fits `LTXAV` (128 channels) and `MiniMaxH3Video` (24), and the
    `Latent2RGBPreviewer` + `preview_to_image` arithmetic they are applied
    with. `tests/dump_latent_rgb_factors.py` EXECUTES those classes to emit
    `src/latent_rgb.zig` and `src/fixtures/latent_rgb.json`; the coefficients
    and reference outputs it produces are golden values, not source, and no
    ComfyUI code is transcribed or shipped.
  * Larryvrh/ComfyUI-MiniMax-H3-Turbo (Apache-2.0), whose `_turbo_sampler`
    is the reference for stepping H3's audio stream by the exact mapped
    delta-sigma at low step counts (`minimax_h3.audioStepFactor`). The Turbo
    LoRA weights it loads (larryvrh/MiniMax-H3-Turbo-Lora, Apache-2.0) are
    downloaded at runtime and not redistributed here.
  * Layr-Labs/mlxfast-bonsai2-27b-engine (MIT), the reference for the few-row
    2-bit tensor path in `src/qmv_nax2.zig`: its `qmm_m16_block` fragment
    coordinate mapping, the 256-k weight line loaded across a fragment quad and
    exchanged with `simd_shuffle`, the dequantize-into-the-right-operand
    arrangement, and the two-slice threadgroup reduction. Transcribed into a
    JIT `metal_kernel` here, not shipped as their source.
  * Layr-Labs/mlxfast-bonsai2-27b-engine (MIT), the reference for the 2-bit
    simdgroup-MMA verify kernel in `src/qmv2.zig` (`msv_qmv2_mma`): its
    `FewRowPackedMatmul` fragment coordinate mapping, two codes per lane
    decoded straight into the weight fragment, and per-group scaling of the
    partial products. Transcribed into a JIT `metal_kernel` here, not shipped
    as their source.
  * Layr-Labs/mlxfast-bonsai2-27b-engine (MIT), the reference for the opt-in
    int8 prefill in `src/qmm_int8.zig`: its register-staged prompt matmul (2-bit
    codes expanded to int8 straight into the tensor op's operand, signed
    per-group activation codes quantized with the rotation) and the factored
    ternary epilogue. Written as a JIT `metal_kernel` here, not shipped as their
    source.
  * filipstrand/mflux (MIT), the reference for stacking several LoRA adapters
    at once — summing per-adapter deltas at forward time rather than merging
    them into the base weight, the `lora_paths`/`lora_scales` request shape,
    and splitting a fused QKV up-projection across q/k/v targets.
  * LiquidAI/d1-3B (LFM Open License v1.0, https://huggingface.co/LiquidAI/d1-3B),
    the reference for the D1 decision prompt, option codes and readout in
    `src/d1.zig`: `prompt.py`'s question and state rendering, its single-token
    alias assignment and readout groups, and `api.py`'s answer shapes. Written
    in Zig here, not shipped as their source; the checkpoint is downloaded at
    runtime and not redistributed.

Model weights downloaded at runtime carry their own licenses from their
publishers. mlx-serve does not redistribute them.
