Maestro: Data/ML Workflow Orchestrator at NetflixBy Jun He, Natallia Dzenisenka, Praneeth Yenugutala, Yingyi Zhang, and Anjali NorwoodTL;DRWe are thrilled to announce that the Maestro source code is now open to the public! Please visit the Maestro GitHub repository to get started. If you find it useful, please give us a star.What is MaestroMaestro is...

Applying Quality of Service techniques at the application levelAnirudh Mendiratta, Kevin Wang, Joey Lynch, Javier Fernandez-Ivern, Benjamin FedorkaIntroductionIn November 2020, we introduced the concept of prioritized load shedding at the API gateway level in our blog post, Keeping Netflix Reliable Using Prioritized Load Shedding. Today, we’re excited...

A summary of sessions at the first Data Engineering Open Forum at Netflix on April 18th, 2024The Data Engineering Open Forum at Netflix on April 18th, 2024.At Netflix, we aspire to entertain the world, and our data engineering teams play a crucial role in this mission by enabling data-driven decision-making at scale. Netflix is not the only place where...

Video annotator: a framework for efficiently building video classifiers using vision-language models and active learningAmir Ziai, Aneesh Vartakavi, Kelli Griggs, Eugene Lok, Yvonne Jukes, Alex Alonso, Vi Iyengar, Anna Pulidohttps://medium.com/media/02a5bbf97c619182adba24b45e42edcb/hrefIntroductionProblemHigh-quality and consistent annotations are fundamental...

At Netflix, we want to ensure that every current and future member finds content that thrills them today and excites them to come back for more. Causal inference is an essential part of the value that Data Science and Engineering adds towards this mission. We rely heavily on both experimentation and quasi-experimentation to help our teams make the best...

Liwei Guo, Vinicius Carvalho, Anush Moorthy, Aditya Mavlankar, Lishan ZhuThis is the second post in a multi-part series from Netflix. See here for Part 1 which provides an overview of our efforts in rebuilding the Netflix video processing pipeline with microservices. This blog dives into the details of building our Video Encoding Service (VES), and...