동일한 GPU로 처리량 극대화하기: Kakao AI Platform의 LLM 추론 최적화 | Model & Agent
kakao tech · 17:03 · Watch on YouTube
Kakao's inference platform team shows how to serve more LLM requests on the same GPUs by cutting interference, reuse, and wasted generation with four techniques — PD separation, prefix routing, KV-cache offloading, an...