The Netflix TechBlog

JULY 15, 2021

Data Engineers of Netflix?—?Interview Interview with Kevin Wylie This post is part of our “Data Engineers of Netflix” series, where our very own data engineers talk about their journeys to Data Engineering @ Netflix. Kevin, what drew you to data engineering?

Data Engineering 197

Data Engineering Engineering Entertainment Big Data 197

The Netflix TechBlog

JUNE 1, 2021

Data Engineers of Netflix?—?Interview Interview with Samuel Setegne Samuel Setegne This post is part of our “Data Engineers of Netflix” interview series, where our very own data engineers talk about their journeys to Data Engineering @ Netflix. What drew you to Netflix?

Data Engineering 130

Data Engineering Engineering Big Data Healthcare 130

The Netflix TechBlog

SEPTEMBER 10, 2019

This enables us to optimize their experience at speed. Instead of relying on engineers to productionize scientific contributions, we’ve made a strategic bet to build an architecture that enables data scientists to easily contribute. Our data scientists faced numerous challenges in our previous infrastructure.

Metrics 221

Metrics Architecture Infrastructure Innovation 221

The Netflix TechBlog

NOVEMBER 20, 2023

For example, a job would reprocess aggregates for the past 3 days because it assumes that there would be late arriving data, but data prior to 3 days isn’t worth the cost of reprocessing. Backfill: Backfilling datasets is a common operation in big data processing. data arrives too late to be useful).

Processing 219

Processing Big Data Efficiency Engineering 219

The Netflix TechBlog

MAY 26, 2020

These characteristics allow for an on-call response time that is relaxed and more in line with traditional big data analytical pipelines. Spark could look up and retrieve the data in the s3 files that the Mouthful represented. And excellent logging is needed for debugging purposes and supportability.

Network 154

Network Tuning AWS Traffic 154

All Things Distributed

OCTOBER 7, 2015

Today, I am excited to share with you a brand new service called Amazon QuickSight that aims to simplify the process of deriving insights from a wide variety of data sources in a fast and affordable manner. Big data challenges. We believe this is one of the critical parts of our big data offerings.

Cloud 114

Cloud Big Data AWS Analytics 114

All Things Distributed

NOVEMBER 15, 2016

They require teams of data engineers to spend months building complex data models and synthesizing the data before they can generate their first report. The cost and complexity to implement, scale, and use BI makes it difficult for most companies to make data analysis ubiquitous across their organizations.

Analytics 126

Analytics Availability Media Social Media 126