In this tutorial, we will practice selected techniques for evaluating machine learning systems, and then monitoring them in production. It is one of a 3-part series:
- Offline evaluation of ML systems
- Online evaluation of ML systems (this part!)
- Evaluation of ML systems by closing the feedback loop
In this particular section, we will practice evaluation and monitoring in the online stage - when a system is serving some or all real users.
Follow along at Online evaluation of ML systems.
This tutorial uses: one m1.medium VM at KVM@TACC, and one floating IP.
This material is based upon work supported by the National Science Foundation under Grant No. 2230079.