#serving
1 paper
-
inspiration
Latency and Throughput Are Not the Same Goal
Two systems serving the same model can feel completely different because they optimize opposite things.
1 paper
Two systems serving the same model can feel completely different because they optimize opposite things.