I'm Anran. I work on LLM inference and AI infrastructure — and I learn it the only way that sticks: by writing the thing from scratch, running it, and measuring it.
Most of what I publish is the trail of that process. An inference engine in Rust + CUDA with validation reports. Thirteen handbooks that walk from a tensor to a serving system, where every code example is executed in CI and its output compared line by line. Hundreds of exercises you can solve in the browser.
Read the paper. Write the kernel. Measure it. Then explain it to someone else.
LLM Inference · CUDA Kernels · PyTorch · Distributed Training · Diffusion · Rust Runtimes
AI Infra Handbooks · Learning in public · Now building · Contributions · Project index
ai-infra-handbooks · read it → · English edition →
Thirteen interlinked handbooks that go from "what is a tensor" to "here is a working inference engine", in Chinese and English.
| Handbook | What it covers | |
|---|---|---|
| 🧠 | LLM Internals | Attention, RoPE, KV Cache, quantization, and the arithmetic behind capacity estimates. |
| 🔥 | PyTorch in a Hurry | Tensors, shapes, autograd, nn.Module, training loops — enough to read everything else. |
| ⚡ | Advanced CUDA | Reductions, GEMM, FlashAttention, Tensor Cores, Triton, Nsight, torch.compile. |
| 🌱 | Train a Small Model | Corpus → BPE tokenizer → a small GPT → SFT, LoRA, DPO, distillation, on your own machine. |
| 🧩 | Distributed Training | DDP, ZeRO/FSDP, tensor / pipeline / expert parallelism, RL training. |
| 🚀 | Inference Systems | Continuous batching, paged KV, prefill/decode disaggregation, speculative decoding, serving ops. |
| 🛠 | mini-sglang from Scratch | Build a complete inference engine chapter by chapter — scheduler, radix cache, CUDA graphs, TP. |
| 🎨 | Image & Video Generation | Diffusion and flow matching, samplers, and how generative media is actually served. |
| 📜 | SGLang Design Evolution | Reading a real engine's design through its commit history. |
Plus four foundations that the rest leans on: Python · C++ · CS Fundamentals · Math.
What makes it different from notes:
- Every example really runs. Python, C++ and CUDA samples are executed by CI on each push and their output is compared with the page, line by line — if a number drifts, the build goes red.
- 288 exercises, graded in your browser (Python on WebAssembly, plus a CUDA simulator) or from the command line on a real GPU.
- A learning roadmap and 2060 spaced-repetition flashcards generated from the chapters themselves.
- Bilingual, with the English edition built from the same sources so the code can never diverge.
| Layer | What I'm learning | Where it lives |
|---|---|---|
| Model internals | Attention variants, positional encodings, normalization, MoE routing | LLM Internals · tiny-llm |
| Framework | Autograd, dispatcher, memory allocator, torch.compile |
PyTorch in a Hurry |
| Kernels | Reductions, tiled GEMM, online softmax, FlashAttention, Triton | Advanced CUDA |
| Training | Pretraining a small GPT, SFT, LoRA, DPO, distillation; ZeRO and 3D parallelism | Train a Small Model · Distributed Training |
| Inference | Scheduling, paged KV cache, radix prefix reuse, speculative decoding, PD disaggregation | Inference Systems · nano-vllm-rs |
| Engines | Writing one end to end, then reading how a production engine got there | mini-sglang · SGLang Evolution |
| Generative media | Diffusion, flow matching, video pipelines and their serving cost | Image & Video Generation |
Everything above is verified on real hardware where it needs to be: the repository ships a setup-gpu.sh and a 14-item gpu_check.py self-check that runs the GPU-only examples on an actual card.
| Track | Projects & direction |
|---|---|
| AI learning platform | ai-infra-handbooks — 13 bilingual handbooks, 291 chapters, 288 browser-graded exercises, every sample verified in CI. |
| Inference engineering | nano-vllm-rs — single-GPU Qwen3 inference in Rust + CUDA, with attention validation and reproducible benchmarks. No Python, no PyTorch, no libtorch in the inference path. |
| Inference education | sglang-interactive-guide — 13 Chinese chapters with browser experiments, source links, and exercises. Learn offline, without a GPU. |
| Runtimes & delivery | wasm-split-tool — split large WebAssembly modules and load code on demand. |
| Everyday tools | DesktopPulse · inline-vocab-translator — desktop widgets and tools for reading across languages. |
| Game-engine tooling | godot_for_minigame · tiktok-minigame-unity-demo — exports, platform SDK integration, and working API examples. |
nano-vllm-rs connects implementation to evidence:
- Attention validation
- Stress tests and reproducible results
- Build, run, and inspect the implementation
Validation currently targets Qwen3-0.6B. Known chunked-output differences are documented in the validation report.
mini-sglang (inside the handbooks) goes the other way: 25 chapters that build a working engine — scheduler, radix prefix cache, CUDA graphs, tensor parallelism, an OpenAI-compatible server — with 63 tests that compare its output against Hugging Face token by token.
Selected merged contributions to deepcoldy/botmux, a bridge between IM platforms and AI coding CLIs:
| Contribution | Merged work |
|---|---|
| Codex integration | PR #157 — support goal passthrough. |
| Resource efficiency | PR #153 — reduce memory leaks, redundant disk writes, and idle screenshot overhead. |
| Remote configuration | PR #132 — add /botconfig with owner-only configuration cards. |
AI & inference
Languages
Runtimes, apps & tooling
| Project | Role | Description |
|---|---|---|
| ai-infra-handbooks | Learning platform | 13 bilingual handbooks, 291 chapters, 288 graded exercises, CI-verified examples. |
| nano-vllm-rs | Implementation | Rust + CUDA inference engine and validation reports. |
| sglang-interactive-guide | Learning guide | 13 interactive chapters on inference serving. |
| leetcode | Practice | Python and Rust algorithm solutions. |
| Project | Stack | Description |
|---|---|---|
| godot_for_minigame | Godot · JavaScript | Mini Game export tooling and platform integration. |
| tiktok-minigame-unity-demo | Unity · C# | Reference examples across 14 SDK API categories, with bilingual documentation. |
| wasm-split-tool | Rust · WebAssembly | On-demand loading for split WebAssembly modules. |
| Project | Stack | Description |
|---|---|---|
| DesktopPulse | Swift · WidgetKit | System, weather, and AI-tool usage widgets for macOS. |
| inline-vocab-translator | JavaScript | Inline translation and vocabulary learning. |
| wallpaper | TypeScript | macOS wallpaper project. |
📚 Learning forks & earlier experiments
These repositories are forks or practice projects, listed separately from the work above.
Inference & ML learning forks
Runtime & tooling forks
- botmux · claude-code
- rust · deno · node · cocos-engine
Personal configuration & practice
📖 Documentation & releases
🏆 GitHub statistics & contribution activity
Learning something in the same direction? The handbooks are open — corrections, better explanations and new exercises are all welcome, and every chapter links to the source it came from.
Have a question, an edge case, or an idea for a tool? Open an issue in the relevant project — reproducible examples are always useful.
Learning through implementation, sharing through working software.