Using Bayesian Stats to Evaluate Your Model During a Pilot

Jun 21, 2024

In this week’s newsletter, we highlight three recent papers that have caught the attention of our data scientists. We also discuss upcoming features for NannyML Cloud, which have been in high demand. Additionally, we share two recent additions to our Post Deployment Data Science Blog. Finally, we provide details about our upcoming webinar. Let’s kick things off!

Papers that caught our eyes 👀

A Short Survey on Importance Weighting for Machine Learning

Paper: https://arxiv.org/abs/2403.10175v2

Concept Drift Detection using Ensemble of Integrally Private Models

Paper: https://arxiv.org/abs/2406.04903

Improving the Validity and Practical Usefulness of AI/ML Evaluations Using an Estimands Framework

Paper: https://arxiv.org/abs/2406.10366

NannyML News

New feature coming soon

One of the most requested features is coming to NannyML Cloud: Segmentation! 🎉 This feature lets you split your data into groups to analyze them separately.

Now you can know if your model’s issues are coming from a single segment or all!

A Comprehensive Guide to Univariate Drift Detection Methods

Article: https://www.nannyml.com/blog/comprehensive-guide-univariate-methods

Feeling overwhelmed by all the univariate drift methods? Kavita wrote an experimental comparison between univariate drift methods to understand when to use one or the other.

Don’t Drift Away with Your Data

Article: https://www.nannyml.com/blog/monitoring-data-drift

Data is going to drift, but it is up to you to prevent it from affecting your model’s performance. Check out Taliya’s blog if you are looking for a gentle introduction to data drift.