从零开始的LLM 是 AI Skill Hub 本期精选AI工具之一。综合评分 7.5 分，整体质量较高。我们推荐使用将其纳入你的 AI 工具库，帮助提升工作效率。
从零开始的LLM 是一款基于 Jupyter Notebook 开发的开源工具，专注于 artificial-intelligence、library-development、jupyter notebook 等核心功能。作为 GitHub 开源项目，它拥有活跃的社区支持和持续的版本迭代，代码完全透明可审计，支持本地部署以保护数据隐私。无论是个人使用还是集成到企业工作流，都能提供稳定可靠的解决方案。
从零开始的LLM 是一款基于 Jupyter Notebook 开发的开源工具，专注于 artificial-intelligence、library-development、jupyter notebook 等核心功能。作为 GitHub 开源项目，它拥有活跃的社区支持和持续的版本迭代，代码完全透明可审计，支持本地部署以保护数据隐私。无论是个人使用还是集成到企业工作流，都能提供稳定可靠的解决方案。
# 克隆仓库 git clone https://github.com/ashworks1706/llm-from-scratch cd llm-from-scratch # 查看安装说明 cat README.md # 按 README 完成环境依赖安装后即可使用
# 查看帮助 llm-from-scratch --help # 基本运行 llm-from-scratch [options] <input> # 详细使用说明请查阅文档 # https://github.com/ashworks1706/llm-from-scratch
# llm-from-scratch 配置说明 # 查看配置选项 llm-from-scratch --config-example > config.yml # 常见配置项 # output_dir: ./output # log_level: info # workers: 4 # 环境变量（覆盖配置文件） export LLM_FROM_SCRATCH_CONFIG="/path/to/config.yml"
this repository is where i build machine learning models from scratch to deeply understand how they work. everything is implemented from the ground up with detailed notes explaining the math, intuition and design choices behind each component.
the goal is complete ml engineering knowledge covering architectures, training pipelines, optimization techniques and deployment.
pretraining trains models on massive text corpora to predict next tokens. this is where models learn language patterns, facts and reasoning.
sft does supervised finetuning on instruction response pairs to teach models to follow instructions and have conversations. only computes loss on responses not instructions.
rl implements direct preference optimization to align models with human preferences using chosen vs rejected response pairs. simpler and more stable.
rl also includes ppo and grpo training paths for comparing classic policy optimization, reward driven updates and group relative preference learning.
distillation compresses a large teacher model into a smaller student by training on soft probability distributions rather than hard labels.
data contains dataset and packing utilities for turning raw examples into efficient training batches. packing matters because wasted padding directly becomes wasted compute.
该项目提供了一个理论和实践的深入探讨大语言模型及其应用的好例子，值得关注。
AI Skill Hub 为第三方内容聚合平台，本页面信息基于公开数据整理，不对工具功能和质量作任何法律背书。
建议在沙箱或测试环境中充分验证后，再部署至生产环境，并做好必要的安全评估。
✅ Apache 2.0 — 宽松开源协议，可商用，需保留版权声明和 NOTICE 文件，含专利授权条款。
经综合评估，从零开始的LLM 在AI工具赛道中表现稳健，质量良好。如果你已有明确的使用需求，可以直接上手体验；如果还在评估阶段，建议对比同类工具后再做决策。
| 原始名称 | llm-from-scratch |
| 原始描述 | 开源AI工具：A theoretical and practical deep dive into Large Language Models and their appli。⭐11 · Jupyter Notebook |
| Topics | artificial-intelligencelibrary-developmentjupyter notebook |
| GitHub | https://github.com/ashworks1706/llm-from-scratch |
| License | Apache-2.0 |
| 语言 | Jupyter Notebook |
收录时间：2026-05-22 · 更新时间：2026-05-22 · License：Apache-2.0 · AI Skill Hub 不对第三方内容的准确性作法律背书。