01The 8-Layer AI Chip Stack
Every AI chip is this stack. Each layer builds on the one below it.
02Background All, Master One
Go deep in one layer — that becomes your job.
| Master Layer | Phase 4 Track | Phase 5 | Job Titles |
|---|---|---|---|
| L1 Application | Track B + C Part 2 | Edge AI / HPC | ML Inference Eng, Edge AI Eng, GenAI Eng, ML Eng, MLOps Eng |
| L2 Compiler | Track C | HPC / AI Chip | AI Compiler Eng, Graph Opt Eng, Kernel Optimization Eng |
| L3 Runtime | Track A §5 / B §8 | HPC | GPU Runtime Eng, Linux Kernel Eng, Infra Eng |
| L4 Firmware | Track B (FSP, L4T) | Autonomous Veh. | Firmware Eng, Embedded SW Eng, Embedded Linux Eng, IoT Eng |
| L5 Architecture | Track A + C | AI Chip Design | AI Accelerator Architect, SoC Architect |
| L6 RTL | Track A (full) | AI Chip Design | RTL Design Eng, FPGA Eng, DV Eng |
035-Phase Curriculum
2.5–5 years part-time. Each phase builds on the previous.
Phase 1: Digital Foundations (6–12 months) — L5/L6, L3/L4, L1
Digital design & HDL, computer architecture, operating systems, C++/CUDA/HIP/SYCL. Five sub-tracks from SIMD to GPU kernels.
Phase 2: Embedded Systems (6–12 months) — L4, L3
ARM Cortex-M, FreeRTOS, Yocto, embedded Linux, SPI/I2C/CAN. The firmware and BSP foundation.
Phase 3: Artificial Intelligence (6–12 months) — L1
Neural networks, edge AI, computer vision, sensor fusion. PyTorch, tinygrad, micrograd. The workloads your hardware must run.
Phase 4: Hardware Deployment & Compilation (6–12 months each)
Track A — Xilinx FPGA: Vivado, Zynq, HLS, advanced design, runtime & drivers, projects.
Track B — NVIDIA Jetson: JetPack, carrier board, L4T, FSP firmware, security, manufacturing, runtime.
Track C — ML Compiler: LLVM, MLIR, TVM, tinygrad, graph optimization, DL inference optimization.
Phase 5: Specialization (ongoing)
A: GPU Infrastructure | B: HPC (CUDA-X) | C: Edge AI | D: Robotics | E: Autonomous Vehicles | F: AI Chip Design
04AI Chip Market
The AI chip market is projected to grow from $71B (2025) to $227B (2030) at 26% CAGR. Inference chips are the fastest segment at 30% CAGR.
05Job Postings by Sub-Layer
Monthly US postings total ~34K–42K/month across 23 sub-layers. ML Engineering dominates volume (~9K). Agentic AI is growing fastest (+120% YoY). Hardware layers have fewer postings but highest scarcity.
06Senior Compensation
Compiler engineers (L2b) and accelerator architects (L5a) command $400K–$550K+. Embedded software starts lower but has the most entry points and highest job count.
07Year-over-Year Growth
Agentic AI (+120%), accelerator architecture (+70%), and compiler roles (+55–60%) are surging as the AI chip wave creates unprecedented demand for specialized talent.
08Volume vs Compensation
The fundamental trade-off: high-volume roles (L4a, L1e) are easier to find but pay less. Low-volume roles (L5a, L2b) command premium compensation. Bubble size shows remote work availability.
09Remote Work Availability
AI hardware engineering is predominantly onsite (65–94%). Agentic AI (25%) and ML Engineering (20%) offer the most flexibility. Physical design and silicon validation are almost entirely onsite.
10Key Takeaways
- The AI chip stack has 8 layers and 23 sub-layers — each is a career.
- Background all layers, master one. Phases 1–3 build the foundation, Phase 4 picks your track.
- Compiler (L2) and Architecture (L5) are the highest-paid, scarcest roles in the stack.
- ML Engineering (L1e) and Embedded Software (L4a) have the most job openings.
- Agentic AI (L1d) is growing at +120% YoY — the fastest category.
- AI hardware engineering is 65–94% onsite. Only L1 application roles offer significant remote.
- The AI chip market is $71B (2025) → $227B (2030) at 26% CAGR.
- A chip startup needs ~10 key hires across L1–L8, costing $3M–5M in year one.
- This roadmap covers L1–L6 practically and L7–L8 in theory.
- The goal: design a custom AI inference chip. Every phase gets you closer.