Optimizing LLM Inference Serving

You have used LLMs. You may have deployed one. But do you know what happens inside the inference engine between the moment a request arrives and the moment the first token streams back?
This meetup goes under the hood of production LLM inference serving — the systems layer that sits between your API endpoint and the GPU cluster. We will cover the real bottlenecks, the algorithms that solve them, and the architectural trade-offs that every ML practitioner working with large models should understand.

Codes for past meetups posted: Github
Recordings posted at: YanAITalk
Feel free to reach out if you want to present at upcoming meetups!

Note: You must have a Zoom account to login (free account is sufficient). Zoom Link will be posted to the event page one day before the meetup.