ML systems under latency constraints
Prepare an example of optimizing a machine learning system for low latency or high reliability, and describe the tradeoffs you made.
The prediction pipelines must meet strict latency targets, with every prediction returned in under 300 milliseconds.
Source posting