The Netflix TechBlog

JUNE 7, 2021

Flow Exporter The Flow Exporter is a sidecar that uses eBPF tracepoints to capture TCP flows at near real time on instances that power the Netflix microservices architecture. After several iterations of the architecture and some tuning, the solution has proven to be able to scale. What is BPF?

Network 327

Network Transportation AWS Cloud 327

The Netflix TechBlog

MAY 4, 2023

Migrating Critical Traffic At Scale with No Downtime — Part 1 Shyam Gala , Javier Fernandez-Ivern , Anup Rokkam Pratap , Devang Shah Hundreds of millions of customers tune into Netflix every day, expecting an uninterrupted and immersive streaming experience. This technique facilitates validation on multiple fronts.

Traffic 347

Traffic Latency Tuning Systems 347

Dynatrace

APRIL 17, 2024

Mainframe is a strong choice for hybrid cloud, but it brings observability challenges IBM Z is a mainframe computing platform chosen by many organizations with a hybrid cloud strategy because of its security, resiliency, performance, scalability, and sustainability. You can now install OneAgent on Linux with s390 architecture.

Operating System 246

Operating System Energy Infrastructure Architecture 246

Dynatrace

MAY 23, 2024

Stream processing One approach to such a challenging scenario is stream processing, a computing paradigm and software architectural style for data-intensive software systems that emerged to cope with requirements for near real-time processing of massive amounts of data. Recovery time of the latency p90. However, we noticed that GPT 3.5

Engineering 245

Engineering Tuning Latency Open Source 245

The Netflix TechBlog

FEBRUARY 18, 2022

To this end, we developed a Rapid Event Notification System (RENO) to support use cases that require server initiated communication with devices in a scalable and extensible manner. This separation allows us to tune system configuration and scaling policies independently for different event priorities and traffic patterns.

Systems 334

Systems Traffic Architecture Mobile 334

Dynatrace

OCTOBER 2, 2023

Transforming an application from monolith to microservices-based architecture can be daunting, and knowing where to start can be difficult. Unsurprisingly, organizations are breaking away from monolithic architectures and moving toward event-driven microservices. Migration is time-consuming and involved. create a microservice; 2.

Architecture 278

Architecture Artificial Intelligence Cloud Open Source 278

DZone

SEPTEMBER 6, 2019

Application performance Review (also known as Application Performance Walkthrough or Application Performance Assessment) is the process of review of an existing application (in production) to evaluate its performance and scalability attributes. The performance characteristics of the application are determined by its architecture and design.

Processing 147

Processing Performance Scalability Best Practices 147

Dynatrace

APRIL 20, 2023

Logs highlight observability challenges Ingesting, storing, and processing the unprecedented explosion of data from sources such as software as a service, multicloud environments, containers, and serverless architectures can be overwhelming for today’s organizations. Seamless integration.

Analytics 246

Analytics Infrastructure Storage Architecture 246

The Netflix TechBlog

FEBRUARY 9, 2021

In particular, we’ll define plans and offers, review the legacy architecture and some of its shortcomings, and dig into our new architecture and some of its advantages. Let’s take a deeper look at the architecture, protocols, and systems involved. A plan is essentially a set of features with a price.

Engineering 145

Engineering Scalability Architecture Innovation 145

The Netflix TechBlog

MARCH 14, 2023

Specifically, we will dive into the architecture that powers search capabilities for studio applications at Netflix. In addition, we were able to perform a handful of A/B tests to validate or negate our hypotheses for tuning the search experience. Media Search Platform (MSP) is the initiative to address these requirements.

Media 299

Media Innovation Energy Architecture 299

The Netflix TechBlog

MARCH 25, 2019

In the rest of this blog, we will a) touch on the complexity of Netflix cloud landscape, b) discuss lineage design goals, ingestion architecture and the corresponding data model, c) share the challenges we faced and the learnings we picked up along the way, and d) close it out with “what’s next” on this journey.

Infrastructure 225

Infrastructure Big Data Transportation Architecture 225

Dynatrace

JUNE 9, 2021

Especially as software development continually evolves using microservices, containerized architecture, distributed multicloud platforms, and open-source code. Positive filters are highly effective at blocking attacks but require constant tuning. And open-source software is rife with vulnerabilities.

Open Source 261

Open Source Entertainment Tuning Internet 261

The Netflix TechBlog

OCTOBER 18, 2022

As Big data and ML became more prevalent and impactful, the scalability, reliability, and usability of the orchestrating ecosystem have increasingly become more important for our data scientists and the company. Motivation Scalability and usability are essential to enable large-scale workflows and support a wide range of use cases.

Java 211

Java Scalability Traffic Architecture 211

The Netflix TechBlog

OCTOBER 8, 2024

In previous blog posts, we introduced the Key-Value Data Abstraction Layer and the Data Gateway Platform , both of which are integral to Netflix’s data architecture. Scalability Our users may operate with limited information at the time of provisioning their namespaces, resulting in best-effort provisioning estimates.

Latency 239

Latency Storage Traffic Tuning 239

The Netflix TechBlog

NOVEMBER 12, 2024

Stay tuned for Part 3 of Composite Abstractions at Netflix, where we’ll introduce our Graph Abstraction , a new service being built on top of the Key-Value Abstraction and the TimeSeries Abstraction to handle high-throughput, low-latency graphs. Along the way, we make various trade-offs to meet the diverse counting requirements at Netflix.

Latency 251

Latency Cache Infrastructure Strategy 251

Dynatrace

NOVEMBER 17, 2020

The rapidly evolving digital landscape is one important factor in the acceleration of such transformations – microservices architectures, service mesh, Kubernetes, Functions as a Service (FaaS), and other technologies now enable teams to innovate much faster. Technical scalability without limits. So please stay tuned for updates.

Analytics 236

Analytics Technology Technology Cloud 236

The Netflix TechBlog

SEPTEMBER 18, 2024

Central to this infrastructure is our use of multiple online distributed databases such as Apache Cassandra , a NoSQL database known for its high availability and scalability. Data Model At its core, the KV abstraction is built around a two-level map architecture.

Latency 260

Latency Storage Cache Efficiency 260

Dynatrace

FEBRUARY 9, 2021

The devil is in the detail, though because of the sheer number, breadth, and volatility of technologies used in modern architectures and the immense volume, velocity, and variety of data they produce. Our goal is to make this process simple, scalable, and enjoyable. So please stay tuned for updates. .

Software 221

Software Software Technology Technology 221

Dynatrace

JUNE 27, 2023

The increasing complexity of cloud service architectures requires a rock-solid understanding of the activity, health status, and security of cloud services. If so, stay tuned for more news about direct AWS Kinesis Data Firehose configuration in AWS console. Or explore the recently introduced support for AWS Lambda logs.

AWS 246

AWS Cloud Lambda Analytics 246

Dynatrace

JULY 18, 2023

This is especially crucial in microservice architectures, where the number of components can be overwhelming. Dedicated configuration files are used to create teams and maintain relevant information, such as responsibilities and contact details, in a scalable and automated way.

Best Practices 246

Best Practices Code Infrastructure Latency 246

The Netflix TechBlog

JANUARY 10, 2024

This architecture shift greatly reduced the processing latency and increased system resiliency. Reloaded was well-architected, providing good stability, scalability, and a reasonable level of flexibility. Several promising ideas were abandoned owing to the outsized work needed to overcome architectural limitations.

Processing 238

Processing Media Latency Innovation 238

The Netflix TechBlog

JUNE 20, 2024

This “Enterprise Data Model/Architect Agent” employs generative AI techniques for autonomous enterprise data modeling and architecture. This talk will delve into the creative solutions Netflix deploys to manage this high-volume, real-time data requirement while balancing scalability and cost. Until next time!

Data Engineering 206

Data Engineering Engineering Entertainment Software Engineering 206

The Netflix TechBlog

OCTOBER 27, 2020

At Netflix, we also heavily embrace a microservice architecture that emphasizes separation of concerns. As the paved path for moving data to key-value stores, Bulldozer provides a scalable and efficient no-code solution. Figure 1 shows how we use Bulldozer to move data at Netflix. Moving data with Bulldozer at Netflix.

Latency 252

Latency Storage Big Data Tuning 252

Dynatrace

AUGUST 16, 2021

Automatically collect and evaluate business, service, and architectural indicator metrics to promote or roll back deployments. Providing standardized self-service pipeline templates, best practices, and scalable automation for monitoring, testing, and SLO validation. SLO validation – ?Automatically Topics in this blog series.

Software 246

Software Software Open Source Azure 246

Scalegrid

JANUARY 10, 2025

Heading into 2024, SQL databases will remain essential in data management, increasingly using distributed systems to meet growing needs for scalability and reliability. The main advantages of distributed SQL databases are scalability and continuous operation.

Database 130

Database Scalability Best Practices Blockchain 130

Dynatrace

JANUARY 19, 2021

Serverless architectures help developers innovate more efficiently and effectively by removing the burden of managing underlying infrastructure. Dynatrace is happy to announce its enhanced AWS Lambda extension, expanding its support for Amazon Web Services (AWS) Lambda and serverless architectures. stay tuned?for functionality?in

Lambda 264

Lambda Serverless AWS Mobile 264

Dynatrace

JULY 16, 2020

The Fan-Out/Fan-In pattern is nowadays found when building serverless functions for high scalability. I’ve gone through some very technical aspects of the Dynatrace APIs and details of my architecture, which allows me to manage a large Dynatrace Managed Setup. I found a good read here.

Cache 289

Cache Serverless Efficiency Tuning 289

Dynatrace

DECEMBER 8, 2020

It inherits the automation, AI, scalability, and enterprise-grade robustness of the Dynatrace platform. With new RASP capabilities of the Dynatrace OneAgent, the same trusted approach extends the Dynatrace platform to application security: automatic, intelligent, highly scalable. Stay tuned – this is only the start.

Cloud 306

Cloud Open Source Internet Internet 306

All Things Distributed

NOVEMBER 28, 2012

Werner Vogels weblog on building scalable and robust distributed systems. s architecture and underlying platform are also optimized to deliver high performance for data warehousing workloads. s MPP architecture makes it easy to resize your cluster to keep pace with your storage and performance requirements. Comments ().

Cloud 152

Cloud Storage Architecture Scalability 152

The Netflix TechBlog

NOVEMBER 2, 2021

As VMAF evolves and is integrated with more encoding and streaming workflows within Netflix, we need scalable ways of fostering video quality innovations. The Reloaded system is a well-matured and scalable system, but its monolithic architecture can slow down rapid innovation. via bug fixes). We call this system Cosmos.

Media 178

Media Innovation Metrics Latency 178

Dynatrace

OCTOBER 22, 2019

This is why threads are often the source of scalability as well as performance issues. Use case #1: Identify scalability issues. A scalable architecture needs to distribute work across many threads in order to facilitate all the CPUs of a physical or virtual machine. Dynatrace news.

Java 145

Java Scalability Hardware Code 145

The Netflix TechBlog

MARCH 1, 2021

The third generation, called Reloaded , has been online for about seven years and has proven to be stable and massively scalable. While we were at it, we also made improvements to scalability, reliability, security, and other system qualities. Stay tuned to learn more details of how Cosmos works and how we use it.

Serverless 235

Serverless Media Latency Social Media 235

Dynatrace

NOVEMBER 4, 2019

Kubernetes was architected to allow for additional technologies and services to assist in speed, scalability and reducing the overall complexity which can arise from a Microservices environment. Yet as a platform, it is in no way considered a standalone environment, containing all the functionality needed for Cloud Native development.

Monitoring 223

Monitoring Cloud Metrics Code 223

Percona

MARCH 23, 2023

Conclusion PostgreSQL is a top choice for production-ready databases due to its scalability, reliability, flexibility, security, and community support. Percona provides managed PostgreSQL services that include database monitoring, performance tuning, and backup and recovery services.

Database 139

Database Open Source Traffic Small Business 139

The Netflix TechBlog

MAY 14, 2021

Managing and operating asynchronous workflows can be difficult without the proper tools and architecture that puts observability, debugging, and tracing at the forefront. We wanted a scalable service that was near real-time, 2. Written by Colby Callahan , Megha Manohara , and Mike Azar. Conductor and one of the new offerings Cosmos.

Traffic 166

Traffic Java Latency Google 166

The Netflix TechBlog

AUGUST 17, 2023

Watching every moment of content to find the best frames and select them manually takes a lot of time, and this approach is often not scalable. Each algorithm needed a process of evaluation and tuning to get great results in AVA Discovery View. For many teams and titles, Stills are essential to Netflix’s promotional asset strategy.

Media 177

Media Scalability Architecture Strategy 177

InfoQ

JANUARY 26, 2024

The company evolved the guild component, which is responsible for fanning out billions of message notifications, in a series of performance and scalability improvements supported by system observability and performance tuning. By Rafal Gancarz

Servers 107

Servers Tuning Scalability Performance 107