No video available

Please refer to the materials section for this topic.

Multi-GPU Serving

Learning Objectives

•Tensor Parallelism
•Pipeline Parallelism
•Communication Overhead

Weekly Outcome

Deploy optimized models and benchmark their performance.

Resources

Benchmarking LLM Inference Backends vLLM vs TensorRT-LLM: RTX 4090 Inference Benchmark – TildAlice Quick Start Guide — TensorRT LLM