Explore projects

[MLSys'25] QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving; [MLSys'25] LServe: Efficient Long-sequence LLM Serving with Unified Sparse Attention

KernelBench: Can LLMs Write GPU Kernels? - Benchmark with Torch -> CUDA problems

llmc is an efficient LLM compression tool with various advanced compression methods, supporting multiple inference backends.

AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:

A high-throughput and memory-efficient inference and serving engine for LLMs

PyTorch extensions for high performance and large scale training.