# Inference Optimization

> KV cache, quantisation (AWQ/GPTQ/EXL2), speculative decoding, and high-throughput serving with vLLM and TensorRT-LLM.

![Status: developing](https://img.shields.io/badge/status-developing-yellow)

Open [`30-inference-optimization.ipynb`](./30-inference-optimization.ipynb) for learning objectives, prerequisites, and recommended reading order.

## Topics Covered

- PagedAttention and KV cache management
- Quantisation: AWQ, GPTQ, EXL2, GGUF, and FP8
- Serving runtimes: vLLM, TensorRT-LLM, SGLang, and TGI
- Speculative decoding and continuous batching

## Prerequisites

[14-local-llms/README.md](../14-local-llms/README.md) · [04-token/README.md](../04-token/README.md)

## What Comes Next

[09-mlops/README.md](../09-mlops/README.md) or [29-ai-hardware-llm-validation/README.md](../29-ai-hardware-llm-validation/README.md)
