Serve Your Own LLM with vLLM & SGLang | Live Bootcamp on Maven
Vizuara · 2:00 · Watch on YouTube
Dr. Sridhar Padmanabhan argues that an AI product’s profit depends on how cheaply and quickly each answer is served, and that outside a few leading labs that scheduling job is almost always handled by the open-source...