Inference latency and throughput
Prepare an explanation of how you would measure and improve inference latency and throughput for a large model serving system.
The role covers systems that determine inference latency, throughput and stability.
Source posting