MLSys | Tags | Lifan Sun’s Blog

Reading Notes: “DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving”

Reading Notes: “DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving”

Reading Notes: “Preble: Efficient Distributed Prompt Scheduling for LLM Serving”

Reading Notes: “Preble: Efficient Distributed Prompt Scheduling for LLM Serving”

Reading Note: “ORCA: A Distributed Serving System for Transformer-Based Generative Models”

Reading Note: “ORCA: A Distributed Serving System for Transformer-Based Generative Models”

Reading Notes: “FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness”

Reading Notes: “FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness”

Reading Notes: “Efficient Memory Management for Large Language Model Serving with PagedAttention”

Reading Notes: “Efficient Memory Management for Large Language Model Serving with PagedAttention”

Reading Notes: “Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning”

Reading Notes: “Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning”

Reading Notes: “GPipe: Easy Scaling with Micro-Batch Pipeline”

Reading Notes: “GPipe: Easy Scaling with Micro-Batch Pipeline”

Distributed Training Basics

Distributed Training Basics

Reading Note: Megatron-LM v1

Reading Note: Megatron-LM v1

Quantization for NN Inference

Quantization for NN Inference

Reading Note: TVM

Reading Note: TVM

Reading Note: Triton

Reading Note: Triton

1 2

Lifan Sun

Software Dev Engineer

Latest posts

Some Ideas on Research

Class Loading in Java

LLVM IR Tutorial: Phis, GEPs, and other things, oh my! - LLVM Developers Conference 2019

The Elements of Search-Based Software Testing Techniques

Reading Note: “An Industrial Evaluation of Unit Test Generation: Finding Real Faults in a Financial Application”, ICSE-SEIP '17

Reading Note: “Whole Test Suite Generation”, TSE 2012