The Netflix TechBlog

MARCH 4, 2025

Accurately Reflecting Production Behavior A key part of our solution is insights into production behavior, which necessitates our requests to the endpoint result in traffic to the real service functions that mimics the same pathways the traffic would take if it came from the usualcallers. Time Travel to validate ahead oftime.

Traffic 180

Traffic Strategy Entertainment Innovation 180

The Netflix TechBlog

JULY 26, 2021

This happens at an unprecedented scale and introduces many interesting challenges; one of the challenges is how to provide visibility of Studio data across multiple phases and systems to facilitate operational excellence and empower decision making. With the latest Data Mesh Platform, data movement in Netflix Studio reaches a new stage.

Big Data 257

Big Data Government Processing Analytics 257

The Netflix TechBlog

OCTOBER 8, 2024

Rajiv Shringi Vinay Chella Kaidan Fullerton Oleksii Tkachuk Joey Lynch Introduction As Netflix continues to expand and diversify into various sectors like Video on Demand and Gaming , the ability to ingest and store vast amounts of temporal data — often reaching petabytes — with millisecond access latency has become increasingly vital.

Latency 239

Latency Storage Traffic Tuning 239

The Netflix TechBlog

MARCH 10, 2023

This platform has evolved from supporting studio applications to data science applications, machine-learning applications to discover the assets metadata, and build various data facts. Hence we built the data pipeline that can be used to extract the existing assets metadata and process it specifically to each new use case.

Traffic 243

Traffic Media Processing Design 243

The Netflix TechBlog

DECEMBER 17, 2024

Metadata and assets must be correctly configured, data must flow seamlessly, microservices must process titles without error, and algorithms must function as intended. This allows us to focus on data analysis and problem-solving rather than managing complex systemchanges. This could lead to an exponential increase in logged data.

Traffic 172

Traffic Scalability Strategy Monitoring 172

Dynatrace

JUNE 25, 2020

With Dynatrace actively managing business-critical applications, some of our globally distributed enterprise customers require Dynatrace Managed to continue operating even when an entire data center goes down. Near-zero RPO and RTO—monitoring continues seamlessly and without data loss in failover scenarios.

Availability 264

Availability Hardware Latency Traffic 264

Dynatrace

SEPTEMBER 7, 2022

Dynatrace and the Dynatrace Intelligent Observability Platform have added support for the newly introduced Amazon VPC Flow Logs to Amazon Kinesis Data Firehose. This support enables customers to define specific endpoint delivery of real-time streaming data to platforms such as Dynatrace. What is VPC Flow Logs? Why Dynatrace?

Traffic 246

Traffic AWS Network Cloud 246

Dynatrace

AUGUST 29, 2024

In today’s data-driven world, businesses across various industry verticals increasingly leverage the Internet of Things (IoT) to drive efficiency and innovation. IoT is transforming how industries operate and make decisions, from agriculture to mining, energy utilities, and traffic management.

IoT 264

IoT Analytics Transportation Metrics 264

Dynatrace

JUNE 15, 2021

Through the RUM data, Dynatrace’s AI engine, Davis, detected seven users were impacted by the outage when they tried to access the Web Interface. This number was so low because the automatic traffic redirect was so fast it kept the impact so low. In our case that includes the login to our SaaS tenants and exploring captured data.

AWS 246

AWS Traffic Architecture Azure 246

Dynatrace

MAY 3, 2024

Log data—the most verbose form of observability data, complementing other standardized signals like metrics and traces—is especially critical. As cloud complexity grows, it brings more volume, velocity, and variety of log data. When trying to address this challenge, your cloud architects will likely choose Amazon Data Firehose.

Cloud 279

Cloud Lambda AWS Analytics 279

The Netflix TechBlog

JUNE 1, 2023

To do this, we devised a novel way to simulate the projected traffic weeks ahead of launch by building upon the traffic migration framework described here. New content or national events may drive brief spikes, but, by and large, traffic is usually smoothly increasing or decreasing.

Traffic 349

Traffic Best Practices Systems Testing 349

Scalegrid

FEBRUARY 24, 2025

Scaling RabbitMQ ensures your system can handle growing traffic and maintain high performance. Youll also learn strategies for maintaining data safety and managing node failures so your RabbitMQ setup is always up to the task. This setup prioritizes data safety, with most replicas online at any given time.

Best Practices 147

Best Practices Traffic Strategy Scalability 147

The Netflix TechBlog

SEPTEMBER 18, 2024

Second, developers had to constantly re-learn new data modeling practices and common yet critical data access patterns. To overcome these challenges, we developed a holistic approach that builds upon our Data Gateway Platform. Data Model At its core, the KV abstraction is built around a two-level map architecture.

Latency 260

Latency Storage Cache Servers 260

Dynatrace

FEBRUARY 28, 2024

In a digital-first world, site reliability engineers and IT data analysts face numerous challenges with data quality and reliability in their quest for cloud control. Increasingly, organizations seek to address these problems using AI techniques as part of their exploratory data analytics practices.

Analytics 246

Analytics Metrics Media Monitoring 246

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. No locks on tables are ever acquired, which prevent impacting write traffic on the source database. Writing events to any output.

Database 221

Database Traffic Transportation Open Source 221

The Netflix TechBlog

JUNE 14, 2023

Testing Strategies: A Summary Two key factors determined our testing strategies: Functional vs. non-functional requirements Idempotency If we were testing functional requirements like data accuracy, and if the request was idempotent , we relied on Replay Testing. In such cases, we were not testing for response data but overall behavior.

Traffic 357

Traffic Latency Metrics Cache 357

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. No locks on tables are ever acquired, which prevent impacting write traffic on the source database. Writing events to any output.

Database 206

Database Traffic Transportation Open Source 206

The Netflix TechBlog

OCTOBER 18, 2022

by Jun He , Akash Dwivedi , Natallia Dzenisenka , Snehal Chennuru , Praneeth Yenugutala , Pawan Dixit At Netflix, Data and Machine Learning (ML) pipelines are widely used and have become central for the business, representing diverse use cases that go beyond recommendations, predictions and data transformations.

Java 211

Java Scalability Traffic Architecture 211

The Netflix TechBlog

FEBRUARY 14, 2025

Every image you hover over isnt just a visual placeholder; its a critical data point that fuels our sophisticated personalization engine. This nuanced integration of data and technology empowers us to offer bespoke content recommendations. This queue ensures we are consistently capturing raw events from our global userbase.

Tuning 166

Tuning Latency Efficiency Storage 166

Dynatrace

JUNE 20, 2023

Modern IT organizations are generating more data from more tools and technologies than ever. Data is proliferating in separate silos from containers and Kubernetes to open source APIs and software to serverless compute services, such as AWS and Azure. However, they had numerous custom applications with separate APIs.

Open Source 246

Open Source Google Serverless Azure 246

Scalegrid

AUGUST 30, 2019

Redis , short for Remote Dictionary Server, is a BSD-licensed, open-source in-memory key-value data structure store written in C language by Salvatore Sanfillipo and was first released on May 10, 2009. Instead, Redis stores data in data structures which makes it very flexible to use. Data Structures in Redis.

Cache 232

Cache Ecommerce Social Media Database 232

Dynatrace

FEBRUARY 16, 2023

How do you get more value from petabytes of exponentially exploding, increasingly heterogeneous data? The short answer: The three pillars of observability—logs, metrics, and traces—converging on a data lakehouse. To solve this problem, Dynatrace launched Grail, its causational data lakehouse , in 2022.

Analytics 246

Analytics Innovation Metrics Database 246

Dynatrace

AUGUST 16, 2022

The data locked in your log files can be a goldmine for your application developers, operations teams, and your enterprise as a whole. However, it can be complicated , expensive , or even impossible to set up robust observability that makes use of this data. Log format inconsistency makes it a challenge to access critical data.

Processing 246

Processing Metrics Monitoring Java 246

Dynatrace

APRIL 13, 2023

The massive volumes of log data associated with a breach have made cybersecurity forensics a complicated, costly problem to solve. As organizations adopt more cloud-native technologies, observability data—telemetry from applications and infrastructure, including logs, metrics, and traces—and security data are converging.

Java 246

Java Analytics Infrastructure Cloud 246

Dynatrace

MARCH 16, 2022

Over the last year, Dynatrace extended its AI-powered log monitoring capabilities by providing support for all log data sources. We added monitoring and analytics for log streams from Kubernetes and multicloud platforms like AWS, GCP, and Azure, as well as the most widely used open-source log data frameworks.

Metrics 246

Metrics Lambda Infrastructure Monitoring 246

Dynatrace

MARCH 25, 2025

We recently announced Dynatrace Live Debugger , which gives developers unprecedented access to real-time data and runtime behavior insights. How can you tell if an algorithm or data source changed or a new feature flag worked? Test data collection Accurate test data can mean life or death.

Benchmarking 147

Benchmarking Code Open Source Engineering 147

DZone

MARCH 2, 2023

In the past 15+ years, online video traffic has experienced a dramatic boom utterly unmatched by any other form of content. It must be said that this video traffic phenomenon primarily owes itself to modernizations in the scalability of streaming infrastructure, which simply weren’t present fifteen years ago.

Social Media 281

Social Media Java Traffic Internet 281

Dynatrace

APRIL 28, 2020

Dynatrace automatically detects and connects monitored services and web applications, considers synthetic data in AI-powered answers by Davis, and even lets you drill down from a synthetic transaction to code-level if necessary. Easily integrate Amazon CloudWatch Synthetics data into Dynatrace. This blog post explains how to do this.

AWS 189

AWS Monitoring Traffic Website 189

Dynatrace

JULY 2, 2020

Large enterprise environments are often distributed across multiple data centers around the world. Unnecessary traffic between such data centers can result in wasted resources, unpredictable downtimes, and lost business. optimizing traffic routing. preventing unrelated traffic between data centers and regions.

Network 240

Network Traffic Infrastructure Tuning 240

Dynatrace

MAY 17, 2023

OpenTelemetry , the open source observability tool, has become the go-to standard for instrumenting custom applications to collect observability telemetry data. For this third and final part of our series, we saved the best for last: How you can enhance telemetry data even more and with less effort on your end with Dynatrace OneAgent.

Metrics 243

Metrics Database Monitoring Network 243

Dynatrace

MAY 6, 2020

Over the last two month s, w e’ve monito red key sites and applications across industries that have been receiving surges in traffic , including government, health insurance, retail, banking, and media. Readers who share our privacy concerns, please note, all the data we monitor is publicly available. . Monitoring with ?the

Website 278

Website Monitoring Retail Media 278

DZone

APRIL 4, 2023

Service meshes are becoming increasingly popular in cloud-native applications as they provide a way to manage network traffic between microservices. Istio, one of the most popular service meshes, uses Envoy as its data plane. This is where Aperture comes in.

Traffic 274

Traffic Network Architecture Monitoring 274

DZone

JULY 8, 2023

Envoy proxy, the data plane of Istio service mesh, is used for handling east-west traffic ( service-to-service communication within a data center). However, to make Istio manage a network of multicloud applications, Envoy was configured as a sidecar proxy for handling north-south traffic (traffic in and out of data centers).

Traffic 257

Traffic Network Development 257

Dynatrace

APRIL 22, 2020

While most government agencies and commercial enterprises have digital services in place, the current volume of usage — including traffic to critical employment, health and retail/eCommerce services — has reached levels that many organizations have never seen before or tested against.

Traffic 285

Traffic Ecommerce Retail Government 285

The Netflix TechBlog

NOVEMBER 9, 2022

A quick canary test was free of errors and showed lower latency, which is expected given that our standard canary setup routes an equal amount of traffic to both the baseline running on 4xl and the canary on 12xl. a contiguous chunk of data (typically 64 bytes on x86 systems) transferred to and from the cache.

Hardware 363

Hardware Cache Performance Latency 363

Dynatrace

JULY 1, 2020

In my last blog , I’ve provided an example of this happening, whereby the traffic spiked and quadrupled the usual incoming traffic. These are all interesting metrics from marketing point of view, and also highly interesting to you as they allow you to engage with the teams that are driving the traffic against your IT-system.

Traffic 264

Traffic Analytics Metrics Servers 264

Dynatrace

MAY 21, 2021

This becomes even more challenging when the application receives heavy traffic, because a single microservice might become overwhelmed if it receives too many requests too quickly. It controls the delivery of service requests to other services, performs load balancing, encrypts data, and discovers other services.

Traffic 279

Traffic DevOps Infrastructure Network 279