# Advanced Mathematics for Machine Learning

Research-level mathematical topics and learning theory for understanding modern ML research.

**Prerequisites:** Complete [foundational/](../foundational/) and [mml-book/](../mml-book/) sections first.

This folder is for deepening theory after the core curriculum starts making concrete sense. It is not a first-pass requirement, and it should be approached selectively based on your actual interests.

## Notebooks

### Part I: Learning Theory

| # | Notebook | Topics |
|---|---------|--------|
| 01 | [Introduction to Learning Theory](01_introduction_learning_theory.ipynb) | Generalization, bias-variance tradeoff |
| 02 | [Concentration Inequalities](02_concentration_inequalities.ipynb) | Hoeffding, Bernstein, McDiarmid's inequality |
| 03 | [Rademacher Complexity](03_rademacher_complexity.ipynb) | Uniform convergence, capacity measures |
| 04 | [PAC-Bayes Theory](04_pac_bayes_theory.ipynb) | PAC learning framework, Bayesian perspective |
| 05 | [Neural Tangent Kernel](05_neural_tangent_kernel.ipynb) | Infinite-width neural networks, kernel methods |

### Part II: Advanced Optimization & Inference

| # | Notebook | Topics |
|---|---------|--------|
| 06 | [Variational Inference](06_variational_inference.ipynb) | Mean-field approximation, ELBO |
| 07 | [Bayesian Nonparametrics](07_bayesian_nonparametrics.ipynb) | Dirichlet Process, Chinese Restaurant Process |
| 08 | [Expectation Maximization](08_expectation_maximization.ipynb) | EM algorithm, convergence proofs, GMM |
| 09 | [Gradient Descent Convergence](09_gradient_descent_convergence.ipynb) | Implicit bias, convergence analysis |

### Part III: Advanced Models & Theory

| # | Notebook | Topics |
|---|---------|--------|
| 10 | [State Space Models](10_state_space_models.ipynb) | Kalman Filters, Hidden Markov Models |
| 11 | [Copula Theory](11_copula_theory.ipynb) | Dependency modeling, multivariate distributions |
| 12 | [Determinantal Point Processes](12_determinantal_point_processes.ipynb) | Diversity modeling, sampling |
| 13 | [Johnson-Lindenstrauss](13_johnson_lindenstrauss.ipynb) | Random projections, dimensionality reduction |
| 14 | [Duality Theory](14_duality_theory.ipynb) | Lagrangian duality, KKT conditions |
| 15 | [Conjugate Gradients](15_conjugate_gradients.ipynb) | Efficient second-order optimization |
| 16 | [Matrix Concentration](16_matrix_concentration_inequalities.ipynb) | Matrix-valued concentration inequalities |

## Learning Paths

**Theoretical ML Researcher:** 01 -> 02 -> 03 -> 04 -> 05

**Probabilistic ML:** 06 -> 07 -> 08 -> 10

**Optimization:** 09 -> 14 -> 15

## How To Use This Folder Well

- Pick one path based on your interest instead of trying to complete all sixteen notebooks at once.
- Use this folder to support research reading or a specific technical curiosity.
- Return to practical phases after each deep dive so the theory stays grounded.

## Prerequisites

- Solid understanding of linear algebra, multivariable calculus, probability, and basic ML
- Python 3.8+, NumPy, SciPy, Matplotlib

## Related

- [foundational/](../foundational/) - Core prerequisites
- [mml-book/](../mml-book/) - Intermediate theory
- [mlpp-book/](../mlpp-book/) - Probabilistic ML depth

## What Comes Next

- Continue to [../../24-advanced-deep-learning/README.md](../../24-advanced-deep-learning/README.md) if you want theory connected to modern architectures.
- Continue to [../../28-practical-data-science/README.md](../../28-practical-data-science/README.md) if you want to reconnect theory to applied work.
- Return to [../README.md](../README.md) to choose another math path only if you have a clear reason.
