The noisy neighbor problem: serving LLMs

3 points

by _josh_meyer_

2 days ago