Data Telling & Model Monitoring

In this webinar, we will be discussing Machine Learning pipelines in production and the challenges that are faced in terms of data. Once a model has been deployed, the next step is to observe its behavior by calculating performance metrics.

Once a degradation is detected, what happens next? Normally, there is an investigation step to understand the motivation and fix it. What if just by adding some simple tasks to the pipeline one could monitor the data and be alerted even before the degradation of metrics occurs? In this webinar, we will be addressing some techniques that can help detect the decay of a model and the motive.

Typically an ML model performance degrades over time and this can come due to many factors such as a change in the KPI, an issue in the data integration, and a shift in the distribution of the data used for prediction with respect to the data used for training the model, etc. Once a decay in the performance of the model is observed there is the need to find the cause and retrain the model with the updated data. One analysis that can be performed is the study of the distribution of data.

The main idea is that if the data that is being used for prediction purposes is very different from the data that was used for training the model, then there is the need to retrain the model with the new data. This is also called Drift Analysis.