Big Data Efficiency Software Engineering 
Dynatrace
MARCH 17, 2025
Until recently, improvements in data center power efficiency compensated almost entirely for the increasing demand for computing resources. The rise of big data, cryptocurrencies, and AI means the IT sector contributes significantly to global greenhouse gas emissions. However, this trend is now reversing.
The Netflix TechBlog
OCTOBER 28, 2021
Interview with Pallavi Phadnis This post is part of our “ Data Engineers of Netflix ” series, where our very own data engineers talk about their journeys to Data Engineering @ Netflix. Pallavi Phadnis is a Senior Software Engineer at Netflix. What is your favorite project?
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
The Netflix TechBlog
MARCH 25, 2019
Building and Scaling Data Lineage at Netflix to Improve Data Infrastructure Reliability, and Efficiency By: Di Lin , Girish Lingappa , Jitender Aswani Imagine yourself in the role of a data-inspired decision maker staring at a metric on a dashboard about to make a critical business decision but pausing to ask a question?—?“Can
The Netflix TechBlog
JUNE 20, 2024
To handle errors efficiently, Netflix developed a rule-based classifier for error classification called “Pensive.” To address this, we propose developing an intelligent agent that can automatically discover, map, and query all data within an enterprise.
Dynatrace
APRIL 20, 2023
With more automated approaches to log monitoring and log analysis, however, organizations can gain visibility into their applications and infrastructure efficiently and with greater precision—even as cloud environments grow. “The weakness of a data lake is they fail when you need to access them fast,” Pawlowski said.
The Netflix TechBlog
NOVEMBER 20, 2023
It also improves the engineering productivity by simplifying the existing pipelines and unlocking the new patterns. We will show how we are building a clean and efficient incremental processing solution (IPS) by using Netflix Maestro and Apache Iceberg. Backfill: Backfilling datasets is a common operation in big data processing.
The Netflix TechBlog
OCTOBER 18, 2022
by Jun He , Akash Dwivedi , Natallia Dzenisenka , Snehal Chennuru , Praneeth Yenugutala , Pawan Dixit At Netflix, Data and Machine Learning (ML) pipelines are widely used and have become central for the business, representing diverse use cases that go beyond recommendations, predictions and data transformations.
Expert insights. Personalized for you.
147 
Let's personalize your content