Remove Data Engineering Remove Metrics Remove Software Engineering
article thumbnail

Bringing Software Engineering Rigor to Data

DZone

In software engineering, we've learned that building robust and stable applications has a direct correlation with overall organization performance. The data community is striving to incorporate the core concepts of engineering rigor found in software communities but still has further to go. Posted with permission.

article thumbnail

Building and Scaling Data Lineage at Netflix to Improve Data Infrastructure Reliability, and…

The Netflix TechBlog

Building and Scaling Data Lineage at Netflix to Improve Data Infrastructure Reliability, and Efficiency By: Di Lin , Girish Lingappa , Jitender Aswani Imagine yourself in the role of a data-inspired decision maker staring at a metric on a dashboard about to make a critical business decision but pausing to ask a question?—?“Can

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Netflix at AWS re:Invent 2019

The Netflix TechBlog

In this session, we discuss the technologies used to run a global streaming company, growing at scale, billions of metrics, benefits of chaos in production, and how culture affects your velocity and uptime. Wednesday?—?December This session looks at what it takes to accept, produce, encode, and stream your favorite content.

AWS 38
article thumbnail

Incremental Processing using Netflix Maestro and Apache Iceberg

The Netflix TechBlog

The need for backfilling could be due to a variety of factors, e.g. (1) upstream data sets got repopulated due to changes in business logic of its data pipeline, (2) business logic was changed in a data pipeline, (3) anew metric was created that needs to be populated for historical time ranges, (4) historical data was found missing, etc.

article thumbnail

Netflix at AWS re:Invent 2019

The Netflix TechBlog

In this session, we discuss the technologies used to run a global streaming company, growing at scale, billions of metrics, benefits of chaos in production, and how culture affects your velocity and uptime. Wednesday?—?December This session looks at what it takes to accept, produce, encode, and stream your favorite content.

AWS 100
article thumbnail

Netflix at AWS re:Invent 2019

The Netflix TechBlog

In this session, we discuss the technologies used to run a global streaming company, growing at scale, billions of metrics, benefits of chaos in production, and how culture affects your velocity and uptime. Wednesday?—?December This session looks at what it takes to accept, produce, encode, and stream your favorite content.

AWS 100
article thumbnail

Organise your engineering teams around the work by reteaming

Abhishek Tiwari

For instance, if you are fast-growing VC funded e-commerce startup and your number one business priority is multiplying current growth and performing exceptionally well on key financial metrics charted out by your investors. Is it possible to draw inspiration from outside of software engineering? You want to move fast.