The Netflix TechBlog

JUNE 20, 2024

A summary of sessions at the first Data Engineering Open Forum at Netflix on April 18th, 2024 The Data Engineering Open Forum at Netflix on April 18th, 2024. At Netflix, we aspire to entertain the world, and our data engineering teams play a crucial role in this mission by enabling data-driven decision-making at scale.

Data Engineering 206

Data Engineering Engineering Entertainment Software Engineering 206

The Netflix TechBlog

OCTOBER 28, 2021

Data Engineers of Netflix?—?Interview Interview with Pallavi Phadnis This post is part of our “ Data Engineers of Netflix ” series, where our very own data engineers talk about their journeys to Data Engineering @ Netflix. Pallavi Phadnis is a Senior Software Engineer at Netflix.

Data Engineering 181

Data Engineering Engineering Big Data Software Engineering 181

Dynatrace

APRIL 20, 2023

Modern organizations ingest petabytes of data daily, but legacy approaches to log analysis and management cannot accommodate this volume of data. At Dynatrace Perform 2023 , Maciej Pawlowski, senior director of product management for infrastructure monitoring at Dynatrace, and a senior software engineer at a U.K.-based

Analytics 246

Analytics Infrastructure Storage Architecture 246

Dynatrace

JUNE 15, 2021

Software reliability and resiliency don’t just happen by simply moving your software to a modern stack, or by moving your workloads to the cloud. And the last sentence of the email was what made me want to share this story publicly, as it’s a testimonial to how modern software engineering and operations should make you feel.

AWS 246

AWS Traffic Architecture Azure 246

The Netflix TechBlog

MARCH 25, 2019

Building and Scaling Data Lineage at Netflix to Improve Data Infrastructure Reliability, and Efficiency By: Di Lin , Girish Lingappa , Jitender Aswani Imagine yourself in the role of a data-inspired decision maker staring at a metric on a dashboard about to make a critical business decision but pausing to ask a question?—?“Can

Infrastructure 225

Infrastructure Big Data Transportation Architecture 225

The Netflix TechBlog

DECEMBER 3, 2019

Netflix applies data science to hundreds of use cases across the company, including optimizing content delivery and video encoding. Data scientists at Netflix relish our culture that empowers them to work autonomously and use their judgment to solve problems independently. How could we improve the quality of life for data scientists?

Open Source 238

Open Source AWS Infrastructure Energy 238

Dynatrace

JANUARY 23, 2024

AI data analysis can help development teams release software faster and at higher quality. So how can organizations ensure data quality, reliability, and freshness for AI-driven answers and insights? And how can they take advantage of AI without incurring skyrocketing costs to store, manage, and query data?

Performance 246

Performance DevOps Innovation Artificial Intelligence 246

The Morning Paper

JULY 7, 2019

Software engineering for machine learning: a case study Amershi et al., More specifically, we’ll be looking at the results of an internal study with over 500 participants designed to figure out how product development and software engineering is changing at Microsoft with the rise of AI and ML. ICSE’19.

Software Engineering 92

Software Engineering Engineering Software Software 92

The Netflix TechBlog

JANUARY 11, 2022

Here we describe the role of Experimentation and A/B testing within the larger Data Science and Engineering organization at Netflix, including how our platform investments support running tests at scale while enabling innovation. Curious to learn more about other Data Science and Engineering functions at Netflix?

Innovation 245

Innovation Metrics Engineering Testing 245

O'Reilly

JANUARY 21, 2025

This approach delivers substantial benefits: consistent execution, lower costs, better security, and systems that can be maintained like traditional software. 90% accuracy for software will often be a deal-breaker, but the promise of agents rests on the ability to chain them together: even five in a row will fail over 40% of the time!

Software Engineering 117

Software Engineering Efficiency Engineering Systems 117

The Netflix TechBlog

MARCH 4, 2025

Building services that adhere to software best practices, such as Object-Oriented Programming (OOP), the SOLID principles, and modularization, is crucial to have success at this stage. Store the data in an optimized, highly distributed datastore. Additionally, some collectors will instead poll our kafka queue for impressions data.

Traffic 180

Traffic Strategy Entertainment Innovation 180

Dynatrace

MAY 6, 2024

Problem remediation is too time-consuming According to the DevOps Automation Pulse Survey 2023 , on average, a software engineer takes nine hours to remediate a problem within a production application. Challenges organizations face in using observability and security data to drive automation. In-context topology identification.

DevOps 306

DevOps Software Engineering Games Java 306

The Netflix TechBlog

AUGUST 11, 2020

Jeffrey Wong , Colin McFarland Every Netflix data scientist, whether their background is from biology, psychology, physics, economics, math, statistics, or biostatistics, has made meaningful contributions to the way Netflix analyzes causal effects. We also optimize for memory and data alignment.

Software Engineering 238

Software Engineering Scalability Engineering Strategy 238

O'Reilly

OCTOBER 19, 2021

As with many burgeoning fields and disciplines, we don’t yet have a shared canonical infrastructure stack or best practices for developing and deploying data-intensive applications. Why: Data Makes It Different. All ML projects are software projects. The new category is often called MLOps.

DevOps 145

DevOps Software Engineering Open Source Infrastructure 145

Scalegrid

OCTOBER 18, 2024

Structured Query Language (SQL) is a simple declarative programming language utilized by various technology and business professionals to extract and transform data. Offering comprehensive access to files, software features, and the operating system in a more user-friendly manner to ensure control.

Open Source 147

Open Source Database Cloud Operating System 147

DZone

MARCH 14, 2023

Site Reliability Engineering (SRE) is a systematic and data-driven approach to improving the reliability, scalability, and efficiency of systems. It combines principles of software engineering, operations, and quality assurance to ensure that systems meet performance goals and business objectives.

Engineering 246

Engineering Software Engineering Scalability Efficiency 246

Dynatrace

APRIL 29, 2021

The website was born as a collaboration between the Innovation Lab and R&D Employer Branding team, with a double aim of showcasing our engineering excellence and of attracting even greater talent to our company. The post Showcasing engineering excellence at Dynatrace appeared first on Dynatrace blog.

Engineering 264

Engineering Open Source Website Innovation 264

DZone

JULY 14, 2021

Fei Xu (Software Engineer at PingCAP). However, when large amounts of data are involved, the CPU becomes the bottleneck for processing queries that include JOIN statements and/or aggregation functions. Authors: Ruoxi Sun (Tech Lead of Analytical Computing Team at PingCAP).

Analytics 223

Analytics Software Engineering Performance Database 223

The Netflix TechBlog

OCTOBER 18, 2022

by Jun He , Akash Dwivedi , Natallia Dzenisenka , Snehal Chennuru , Praneeth Yenugutala , Pawan Dixit At Netflix, Data and Machine Learning (ML) pipelines are widely used and have become central for the business, representing diverse use cases that go beyond recommendations, predictions and data transformations.

Java 211

Java Scalability Traffic Architecture 211

Dynatrace

JUNE 4, 2024

Build an umbrella for Development and Operations In modern software engineering, the discipline of platform engineering delivers DevSecOps practices to developers to bridge the gaps between development, security, and operations and enhance the developer experience. However, other data formats, like logs, can also be employed.

Metrics 297

Metrics Engineering Code Tuning 297

Dynatrace

MARCH 3, 2020

For software engineering teams, this demand means not only delivering new features faster but ensuring quality, performance, and scalability too. One way to apply improvements is transforming the way application performance engineering and testing is done. Here is a shortlist to get you started.

Performance 314

Performance Education Innovation Software Architecture 314

The Netflix TechBlog

JUNE 1, 2021

Data Engineers of Netflix?—?Interview Interview with Dhevi Rajendran Dhevi Rajendran This post is part of our “Data Engineers of Netflix” interview series, where our very own data engineers talk about their journeys to Data Engineering @ Netflix.

Data Engineering 139

Data Engineering Engineering Software Engineering Big Data 139

Dynatrace

AUGUST 3, 2022

In today’s complex, data-driven world, many security vulnerabilities and attacks can jeopardize an organization’s data. To ensure the safety of their customers, employees, and business data, organizations must have a strategy to protect against zero-day vulnerabilities. Application logs are a good data source for this method.

Java 261

Java Traffic Benchmarking Strategy 261

Dynatrace

NOVEMBER 7, 2023

Platform engineering is on the rise. According to leading analyst firm Gartner, “80% of software engineering organizations will establish platform teams as internal providers of reusable services, components, and tools for application delivery…” by 2026. Open source logs and metrics take precedence in the monitoring process.

Engineering 306

Engineering DevOps Best Practices Infrastructure 306

Dynatrace

NOVEMBER 6, 2023

Cloud complexity and data proliferation are two of the most significant challenges that IT teams are facing today. Computing environments are scaling to new heights, resulting in more data that makes pinpointing root causes and vulnerabilities even more challenging. Why is developer observability important for engineers?

Development 290

Development DevOps Programming Cloud 290

Dynatrace

JUNE 28, 2022

After investigating, the software engineering team discovered that it wasn’t leveraging application performance monitoring (APM) tooling data to its full potential. So, the team decided to leverage mobile analytics data to deliver a seamless user experience. Analyze the data and develop an action plan.

Mobile 246

Mobile Analytics Best Practices Software Engineering 246

Dynatrace

DECEMBER 5, 2022

By helping teams release new software more frequently, DevOps practices are an essential component of digital transformation. DevOps is a widely practiced set of procedures and tools for streamlining the development, release, and updating of software. DevOps orchestration in practice. Get started with DevOps orchestration.

DevOps 264

DevOps Virtualization Best Practices Innovation 264

High Scalability

JANUARY 11, 2022

from a client it performs two parallel operations: i) persisting the action in the data store ii) publish the action in a streaming data store for a pub-sub model. User Feed Service, Media Counter Service) read the actions from the streaming data store and performs their specific tasks. Data Models. Graph Data Models.

Design 334

Design Media Storage Logistics 334

Dynatrace

NOVEMBER 17, 2020

We sat together with Armin Ruech and Daniel Dyla, software engineers at Dynatrace and leaders within the OpenTelemetry community, to hear about their involvement with the second most active CNCF project. My name is Armin Ruech, I’m a Software Engineer at Dynatrace and I started as a software developer around 3.5

Open Source 207

Open Source Software Engineering Government Java 207

Dynatrace

MARCH 6, 2024

Platform engineering creates and manages a shared infrastructure and set of tools, such as internal developer platforms (IDPs) , to enable software developers to build, deploy, and operate applications more efficiently. The Dynatrace Operator automatically ingests all observability data from OpenTelemetry and Prometheus.

Engineering 264

Engineering Development DevOps Infrastructure 264

The Netflix TechBlog

NOVEMBER 9, 2021

The Android launch leveraged the open-source software decoder dav1d built by the VideoLAN, VLC, and FFmpeg communities and sponsored by AOMedia. While software decoders enable AV1 playback for more powerful devices, a majority of Netflix members enjoy their favorite shows on TVs. Some titles (e.g.,

Media 246

Media Open Source Software Engineering Efficiency 246

DZone

JUNE 27, 2022

Before DevOps took the software engineering world by storm, developers were left in the dark once their applications were up and running. Instead of being the first to know when outages occurred, engineers would only find out when customers or stakeholders complained of “laggy websites” or one too many 503 pages.

Software Engineering 130

Software Engineering DevOps Website Engineering 130

Dynatrace

NOVEMBER 14, 2023

Effective site reliability engineering requires enterprise-wide transformation Without a unified understanding of SRE practices, organizational silos can quickly form between departments. Lack of collaboration leads to siloed observability data and leaves teams with little information to work from when attempting to deliver value.

Best Practices 246

Best Practices Engineering DevOps Software Engineering 246

The Netflix TechBlog

NOVEMBER 29, 2023

A vital aspect of such development is subjective testing with HDR encodes in order to generate training data. Fixed-ladder HDR encodes have been fully replaced by optimized ones, reducing storage footprint and Internet data usage — and most importantly, improving the video quality for our members. Krasula, A. Choudhury, S. Malfait, A.

Open Source 260

Open Source Software Engineering Internet Internet 260

Dynatrace

FEBRUARY 12, 2024

The various presenters in this session aligned platform engineering use cases with the software development lifecycle. Check out the following use cases to learn how to drive innovation from development to production efficiently and securely with platform engineering observability. Real-time detection for fast remediation.

Engineering 246

Engineering Development Infrastructure Cloud 246

The Netflix TechBlog

JUNE 1, 2021

Data Engineers of Netflix?—?Interview Interview with Samuel Setegne Samuel Setegne This post is part of our “Data Engineers of Netflix” interview series, where our very own data engineers talk about their journeys to Data Engineering @ Netflix. For example?—?clinical

Data Engineering 130

Data Engineering Engineering Big Data Healthcare 130

High Scalability

FEBRUARY 19, 2019

Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Shape the future of software in your industry. Receive occasional invitations to chat with for 30 minutes about your area of expertise and software usage. Who's Hiring?

Software 125

Software Software Analytics Infrastructure 125