The Netflix TechBlog

OCTOBER 28, 2021

Interview with Pallavi Phadnis This post is part of our “ Data Engineers of Netflix ” series, where our very own data engineers talk about their journeys to Data Engineering @ Netflix. Pallavi Phadnis is a Senior Software Engineer at Netflix. Pallavi, what’s your journey to data engineering at Netflix?

Data Engineering 177

Data Engineering Engineering Big Data Software Engineering 177

Dynatrace

FEBRUARY 24, 2021

SRE is the transformation of traditional operations practices by using software engineering and DevOps principles to improve the availability, performance, and scalability of releases by building resiliency into apps and infrastructure. Designating and managing Service Level Objectives (SLOs) as availability targets for a service.

DevOps 189

DevOps Software Engineering Speed Google 189

DZone

MARCH 8, 2024

Finite state machines (FSMs) offer a solution by modeling system behavior as states and transitions, a useful tool that can help software engineers understand software behavior and design effective test cases. Ensuring application reliability is a never-ending quest.

Software 162

Software Software Testing Software Engineering 162

DZone

FEBRUARY 27, 2021

As a Software Engineer, the mind is trained to seek optimizations in every aspect of development and ooze out every bit of available CPU Resource to deliver a performing application. This begins not only in designing the algorithm or coming out with efficient and robust architecture but right onto the choice of programming language.

Java 214

Java C++ Benchmarking Programming 214

Dynatrace

NOVEMBER 3, 2023

Platform engineering is a practice that outlines how development teams build internal platforms to create self-service capabilities for software engineering teams. The result is a cloud-native approach to software delivery.

Engineering 213

Engineering DevOps Software Engineering Scalability 213

J. Paul Reed

MARCH 14, 2019

The 737Max and Why Software Engineers Might Want to Pay Attention As someone with a bit of a reputation for talking about aviation and software development and operations , I’ve been asked about the 737Max repeatedly over the past week. a selling point ?—?and

Software Engineering 75

Software Engineering Engineering Software Software 75

The Netflix TechBlog

NOVEMBER 9, 2021

To achieve this goal, the Encoding Technologies team made the following design decisions about AV1 encoding recipes: We always encode at the highest available source resolution and frame rate. The Encoding Technologies team currently has a number of openings, and we can’t wait to have more stunning engineers joining us.

Media 246

Media Open Source Software Engineering Efficiency 246

Dynatrace

MAY 23, 2024

Stream processing One approach to such a challenging scenario is stream processing, a computing paradigm and software architectural style for data-intensive software systems that emerged to cope with requirements for near real-time processing of massive amounts of data. We designed experimental scenarios inspired by chaos engineering.

Engineering 182

Engineering Tuning Latency Open Source 182

The Netflix TechBlog

JUNE 20, 2024

SQLMesh, an open-source project born from our collective experience at companies like Airbnb, Apple, Google, and Netflix, is designed to handle the complexities of evolving data pipelines at an internet scale. In this talk, Iaroslav Zeigerman discusses challenges faced by data practitioners today and how core SQLMesh concepts solve them.

Data Engineering 203

Data Engineering Engineering Entertainment Software Engineering 203

The Netflix TechBlog

NOVEMBER 29, 2023

Bitrate versus quality comparison HDR-VMAF is designed to be format-agnostic — it measures the perceptual quality of HDR video signal regardless of its container format, for example, Dolby Vision or HDR10. The graphic below (Fig. 1) depicts the migration of traffic from fixed bitrates to DO encodes. Krasula, A. Choudhury, S. Malfait, A.

Open Source 256

Open Source Software Engineering Internet Internet 256

The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 202 A day in the life of a Netflix Engineer Dave Hahn , SRE Engineering Manager Abstract : Netflix is a large, ever-changing ecosystem serving millions of customers across the globe through cloud-based systems and a globally distributed CDN. Wednesday?—?December

AWS 38

AWS Entertainment Open Source Benchmarking 38

The Netflix TechBlog

MARCH 25, 2019

Now, imagine yourself in the role of a software engineer responsible for a micro-service which publishes data consumed by few critical customer facing services (e.g. Design a flexible data model ? —?Represent Therefore, the ingestion approach for data lineage is designed to work with many disparate data sources.

Infrastructure 222

Infrastructure Big Data Transportation Architecture 222

Dynatrace

NOVEMBER 22, 2021

EC2 is Amazon’s Infrastructure-as-a-service (IaaS) compute platform designed to handle any workload at scale. AWS Lambda makes it easy to design, run, and maintain application systems without having to provision or manage infrastructure. Here are a few of the most popular. Amazon EC2. Amazon Fargate.

Best Practices 213

Best Practices AWS Monitoring Serverless 213

The Netflix TechBlog

DECEMBER 3, 2019

mainly because of mundane reasons related to software engineering. By design, Metaflow is a deceptively simple Python library: Data scientists can structure their workflow as a Directed Acyclic Graph of steps, as depicted above. It leverages elasticity of the cloud by design?—?both Metaflow is a cloud-native framework.

Open Source 235

Open Source AWS Infrastructure Energy 235

Dynatrace

NOVEMBER 7, 2023

Platform engineering is on the rise. According to leading analyst firm Gartner, “80% of software engineering organizations will establish platform teams as internal providers of reusable services, components, and tools for application delivery…” by 2026.

Engineering 275

Engineering DevOps Best Practices Infrastructure 275

High Scalability

OCTOBER 5, 2018

Because the device in question is a high-speed unit designed to process a high volume of ballots for an entire county, hacking just one of these machines could enable anattacker to flip the Electoral College and determine the outcome of a presidential election?. I used to comment sparingly.

Internet 156

Internet Internet Scalability Software Engineering 156

Dynatrace

JUNE 4, 2024

Build an umbrella for Development and Operations In modern software engineering, the discipline of platform engineering delivers DevSecOps practices to developers to bridge the gaps between development, security, and operations and enhance the developer experience.

Metrics 233

Metrics Engineering Code Tuning 233

The Netflix TechBlog

FEBRUARY 1, 2022

The interview panel consists of two or three engineers, a hiring manager and a recruiter. The engineers assess your technical skills by asking you to solve various design and coding problems. The interview panel comprises an engineering director, a partner engineer or manager, and another engineering leader.

Engineering 204

Engineering Games Entertainment Innovation 204

O'Reilly

OCTOBER 19, 2021

This is both frustrating for companies that would prefer making ML an ordinary, fuss-free value-generating function like software engineering, as well as exciting for vendors who see the opportunity to create buzz around a new category of enterprise software. Software Architecture. The new category is often called MLOps.

DevOps 139

DevOps Software Engineering Open Source Infrastructure 139

DZone

OCTOBER 11, 2021

This article is more focused on overall system design and architecture than any other written by me till today — so consider yourself warned. Additionally, I want to show you laws and their mathematical equations that can help you calculate the impact of these 3 mechanics on your application.

Software 246

Software Software Architecture Design 246

The Netflix TechBlog

JUNE 1, 2021

Dhevi joined Netflix in July 2020 and is one of many Data Engineers who have onboarded remotely during the pandemic. In this post, Dhevi talks about her passion for data engineering and taking on a new role during the pandemic. One great thing about working with data is the impact you can create as an engineer.

Data Engineering 138

Data Engineering Engineering Software Engineering Big Data 138

The Netflix TechBlog

APRIL 26, 2022

We will share how its design has evolved over the years and the lessons learned while building it. To understand Axion’s design, we need to know the various components that interact with it. The motivation has not changed since then; the design has. Design evolution Axion fact store has four components?—?fact

Storage 191

Storage Design Scalability Latency 191

Dynatrace

JUNE 15, 2021

And the last sentence of the email was what made me want to share this story publicly, as it’s a testimonial to how modern software engineering and operations should make you feel. This meant there was no negative impact on our end-users, Service Level Objectives (SLOs), or Service Level Agreements (SLAs).

AWS 191

AWS Traffic Architecture Azure 191

Dynatrace

MARCH 17, 2021

Application security is a software engineering term that refers to several different types of security practices designed to ensure applications do not contain vulnerabilities that could allow illicit access to sensitive data, unauthorized code modification, or resource hijacking. Dynatrace news. So, why is all this important?

Open Source 167

Open Source Cloud Games Java 167

Tasktop

MAY 31, 2019

We’re typically heads-down, and it seems any interaction we have is just business: problem solving, design discussions, code review comments, story planning. The post Team building in software engineering appeared first on Tasktop Blog. When this was brought up in a retrospective meeting, we agreed to try a weekly team lunch.

Software Engineering 40

Software Engineering Engineering Software Software 40

Dynatrace

JUNE 2, 2022

Site reliability engineering (SRE) continues to gain popularity as organizations embrace hybrid cloud strategies and IT automation at scale. By applying software engineering principles to operations and infrastructure practices, SRE enables organizations to streamline and automate IT processes. Dynatrace news.

DevOps 147

DevOps Innovation Engineering Benchmarking 147

Dynatrace

NOVEMBER 14, 2023

Customer empathy is key to a fully optimized site reliability engineering practice Software engineering can often be an impersonal discipline. Breaking out of the mindset that DevOps, SRE, and platform engineering are opposing forces is a vital step toward mitigating silos and ensuring satisfied SLO criteria.

Best Practices 193

Best Practices Engineering DevOps Software Engineering 193

The Netflix TechBlog

OCTOBER 27, 2021

Requirements for a Metaflow GUI Metaflow is a human-centered system by design. Based on our experiences with other GUIs operating at Netflix-scale, this is not a trivial requirement: scalability needs to be baked into the design from the very beginning. The GUI should integrate well with other GUIs.

Open Source 226

Open Source Monitoring Scalability Code 226

The Morning Paper

FEBRUARY 16, 2020

Migrating a privacy-safe information extraction system to a software 2.0 design , Sheng, CIDR’20. This is a comparatively short (7 pages) but very interesting paper detailing the migration of a software system to a ‘Software 2.0’ ’ design. " Switching to a Software 2.0

Systems 84

Systems Design Software Software 84

High Scalability

AUGUST 4, 2020

has hours of system design content. They also do live system design discussions every week. T riplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Who's Hiring? InterviewCamp.io Try out their platform. Generous free tier.

Education 115

Education AWS Scalability Java 115

High Scalability

APRIL 16, 2019

Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Join Etleap , an Amazon Redshift ETL tool to learn the latest trends in designing a modern analytics infrastructure. Make your job search O (1), not O ( n ). Apply here.

Analytics 26

Analytics Java Metrics Software Engineering 26

High Scalability

APRIL 10, 2019

Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Join Etleap , an Amazon Redshift ETL tool to learn the latest trends in designing a modern analytics infrastructure. Make your job search O (1), not O ( n ). Apply here.

Analytics 26

Analytics Java Metrics Software Engineering 26

High Scalability

APRIL 14, 2020

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! Learn to balance architecture trade-offs and design scalable enterprise-level software.

Education 105

Education Software Engineering Scalability Engineering 105

O'Reilly

JANUARY 21, 2025

When we talk about conversational AI, were referring to systems designed to have a conversation, orchestrate workflows, and make decisions in real time. By predefined, tested workflows, we mean creating workflows during the design phase, using AI to assist with ideas and patterns. What Does Structured Automation Look Like in Practice?

Software Engineering 82

Software Engineering Efficiency Engineering Systems 82

High Scalability

MAY 12, 2020

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! Learn to balance architecture trade-offs and design scalable enterprise-level software.

Education 102

Education Software Engineering Scalability Engineering 102

High Scalability

APRIL 28, 2020

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! Learn to balance architecture trade-offs and design scalable enterprise-level software.

Education 102

Education Software Engineering Scalability Engineering 102

High Scalability

JANUARY 8, 2019

Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Companies like Apple, Dropbox, Mixpanel, and Instacart now let Triplebyte-recommended engineers skip their own screening steps. Who's Hiring? Make your job search O (1), not O ( n ).

Java 137

Java Metrics Software Engineering DevOps 137