Reading Notes: “Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning”

Reading Notes: “GPipe: Easy Scaling with Micro-Batch Pipeline”

Distributed Training Basics

Reading Note: Megatron-LM v1

Quantization for NN Inference

Reading Note: TVM

Reading Note: Triton

© Lifan Sun 2023 - 2025