Post-deployment hype. Advances on Performance Estimation and the White House.
In this week issue we share educational material to get into post-deployment data science, new findings on performance estimation by our research team and the key points of Biden’s executive order on generative AI.
But first, lets start with some proper hype! 🔥
Reaching +3500 monthly active instances
We couldn’t be more grateful to know that more than 3500 processes out there are running nannyML every month to keep ML models tidy.
It took us several months to reach 1000 monthly instances but only one month to triple that amount! 💪📈 If that doesn't show perseverance, I don't know what will 😂.
If you are one of those users running nannyML to monitor your ML model, I would love to speak with you and learn what can be done to make your monitoring life easier! Send us a DM on LinkedIn.
Mastering post-deployment data science
Last month, we launched the first-ever course on post-deployment data science! It is a (FREE) 5-hour long course that covers the basics of model monitoring, including concepts like the optimal monitoring flow and data drift detection.
More than 600 Data Scientists have already started learning about Post-Deployment Data Science! Be sure to join the movement
You can join the course for free using this link.
Best part about building educational content?, sharing it with the community!
Better performance estimation algorithms
During the last couple of weeks our research team (shout out to Jakub, Nikos and Wojtek) have been working on a improved performance algorithm that shows 10% more accurate results than the previous SoTA method, Confidence-based Performance Estimation (CBPE).
This new method, called MCBPE, is a multi-calibrated version of good old CBPE. In a nutshell it calibrates the model taking into account the covariate shift present in the chunk and then runs CBPE. In contrast, CBPE calibrates the model only looking at the reference input distribution as a whole.
This algorithm, and others like Concept-shift detection are available for all nannyML cloud customers.
NannyML + Marvelous MLOps blog post colab 🤝
Together with
we wrote a blog post about monitoring and retraining strategies for demand forecasting models.In the blog we cover some challenges of monitoring these types of models and how to overcome them. Specifically, we explain strategies on how to intelligently retrain your models to decrease time to deploy and reduce cloud costs.
Biden’s executive order to reduce AI risk
This week, US President Joe Biden signed an executive order providing rules to reduce AI risk. This initial document sets the preview of what we might expect in a longer regulation act.
Hakim posted a summary of the document on LinkedIn. Join the conversation here.
Until the next one! ✌️