The Netflix TechBlog

FEBRUARY 20, 2019

by Jason Koch , with Martin Spier , Brendan Gregg , Ed Hunter Improving the tools available to our engineers to help them diagnose, triage, and work through software performance challenges in the cloud is a key goal for the cloud performance engineering team at Netflix. to the broader community.

Performance 217

Performance Latency Open Source Metrics 217

The Netflix TechBlog

MAY 4, 2023

When undertaking system migrations, one of the main challenges is establishing confidence and seamlessly transitioning the traffic to the upgraded architecture without adversely impacting the customer experience. It provides a good read on the availability and latency ranges under different production conditions.

Traffic 349

Traffic Latency Tuning Systems 349

The Netflix TechBlog

JANUARY 25, 2023

The service should be able to serve real-time, aka UI, applications so CRUD and search operations should be achieved with low latency. Our service will be used by a lot of internal UI applications hence the latency for CRUD and search operations must be low. Teams should be able to define their data model for annotation.

Scalability 293

Scalability Latency Media Architecture 293

The Netflix TechBlog

SEPTEMBER 18, 2024

Firstly, developers struggled to reason about consistency, durability and performance in this complex global deployment across multiple stores. These include challenges with tail latency and idempotency, managing “wide” partitions with many rows, handling single large “fat” columns, and slow response pagination.

Latency 254

Latency Storage Cache Efficiency 254

Dynatrace

OCTOBER 1, 2021

As dynamic systems architectures increase in complexity and scale, IT teams face mounting pressure to track and respond to conditions and issues across their multi-cloud environments. In a monitoring scenario, you typically preconfigure dashboards that are meant to alert you to performance issues you expect to see later.

Metrics 363

Metrics Open Source Monitoring Cloud 363

Dynatrace

DECEMBER 27, 2022

As more organizations embrace microservices-based architecture to deliver goods and services digitally, maintaining customer satisfaction has become exponentially more challenging. When organizations implement SLOs, they can improve software development processes and application performance. SLOs improve software quality. Reliability.

Software 276

Software Software Benchmarking Latency 276

The Netflix TechBlog

SEPTEMBER 24, 2021

The ProRes codec family provides great editing performance and image quality. Table 1: Movie and File Size Examples Initial Architecture A simplified view of our initial cloud video processing pipeline is illustrated in the following diagram. Uploading and downloading data always come with a penalty, namely latency.

Cloud 242

Cloud Media Storage Cache 242

Dynatrace

NOVEMBER 28, 2022

The new Amazon capability enables customers to improve the startup latency of their functions from several seconds to as low as sub-second (up to 10 times faster) at P99 (the 99th latency percentile). This can cause latency outliers and may lead to a poor end-user experience for latency-sensitive applications.

Lambda 235

Lambda AWS Serverless Latency 235

The Netflix TechBlog

MARCH 5, 2024

Reduced tail latencies In both our GRPC and DGS Framework services, GC pauses are a significant source of tail latencies. In fact, we’ve found for our services and architecture that there is no such trade off. Reference processing is also only performed in major collections with ZGC. There is no best garbage collector.

Latency 240

Latency Java Tuning Efficiency 240

Dynatrace

NOVEMBER 10, 2022

As companies accelerate digital transformation, cloud services such as AWS Lambda help companies to modernize their application architectures to quickly adapt to the needs of their customers while offloading the operational complexity to their cloud vendor.

Lambda 240

Lambda Serverless AWS Latency 240

The Netflix TechBlog

OCTOBER 8, 2024

Rajiv Shringi Vinay Chella Kaidan Fullerton Oleksii Tkachuk Joey Lynch Introduction As Netflix continues to expand and diversify into various sectors like Video on Demand and Gaming , the ability to ingest and store vast amounts of temporal data — often reaching petabytes — with millisecond access latency has become increasingly vital.

Latency 242

Latency Storage Traffic Tuning 242

The Netflix TechBlog

NOVEMBER 12, 2024

By: Rajiv Shringi , Oleksii Tkachuk , Kartik Sathyanarayanan Introduction In our previous blog post, we introduced Netflix’s TimeSeries Abstraction , a distributed service designed to store and query large volumes of temporal event data with low millisecond latencies. Today, we’re excited to present the Distributed Counter Abstraction.

Latency 253

Latency Cache Infrastructure Strategy 253

InfoQ

APRIL 26, 2024

Allegro experimented with different performance optimization options to improve Apache Kafka producer tail latency and eventually switched all its clusters to the XFS filesystem. The company used Kafka protocol sniffing, JVM profiling, and eBPF, which proved instrumental in identifying and eliminating performance bottlenecks.

Latency 114

Latency Performance Tuning Design 114

The Netflix TechBlog

JANUARY 10, 2024

This architecture shift greatly reduced the processing latency and increased system resiliency. We expanded pipeline support to serve our studio/content-development use cases, which had different latency and resiliency requirements as compared to the traditional streaming use case. divide the input video into small chunks 2.

Processing 240

Processing Media Latency Innovation 240

The Netflix TechBlog

FEBRUARY 18, 2022

By: Ankush Gulati , David Gevorkyan Additional credits: Michael Clark , Gokhan Ozer Intro Netflix has more than 220 million active members who perform a variety of actions throughout each session, ranging from renaming a profile to watching a title. This approach enables the computing power to catch up quickly when the queues grow.

Systems 334

Systems Traffic Architecture Mobile 334

Dynatrace

MAY 19, 2022

Example 1: Architecture boundaries. This multinational information technology service and consulting company was asked to help a global automotive manufacturer with the management goal of measuring service flow performance. First, they took a big step back and looked at their end-to-end architecture (Figure 2).

Automotive 279

Automotive Latency Architecture Azure 279

Dynatrace

MAY 13, 2022

This extension provides fully app-centric Cassandra performance monitoring for Azure Managed Instance for Apache Cassandra. Because of its scalability and distributed architecture, thousands of companies trust it to run their cloud and hybrid-based workloads at high availability without compromising performance.

Azure 242

Azure Latency Metrics Infrastructure 242

Dynatrace

JANUARY 26, 2021

Within this paradigm, it is possible to run entire architectures without touching a traditional virtual server, either locally or in the cloud. In a serverless architecture, applications are distributed to meet demand and scale requirements efficiently. When an application is triggered, it can cause latency as the application starts.

Serverless 246

Serverless Efficiency Lambda Azure 246

Dynatrace

JANUARY 14, 2022

That is because Kubernetes provides several benefits from a performance perspective. However, setting the right parameters for Kubernetes clusters to ensure application availability, performance, and resilience while avoiding overspending isn’t a walk in the park. Dynatrace news. below 500ms) and error rates (e.g. lower than 2%.).

Latency 217

Latency Tuning Efficiency AWS 217

The Netflix TechBlog

JUNE 4, 2019

Because microprocessors are so fast, computer architecture design has evolved towards adding various levels of caching between compute units and the main memory, in order to hide the latency of bringing the bits to the brains. As it turns out, for the large majority of Netflix use cases, its performance is far from optimal.

Cache 260

Cache Latency Airlines Logistics 260

High Scalability

SEPTEMBER 8, 2018

RISELabs , those wonderfully innovative folks over at Berkeley, have uplifted their Anna datatabase —a shared-nothing, thread-per-core architecture to achieve lightning-fast speeds by avoiding all coordination mechanisms—to become cloud-aware. While database neogenesis has slowed down considerably, it has not gone necrotic.

Storage 199

Storage Performance AWS Efficiency 199

Dynatrace

FEBRUARY 4, 2021

As a discipline, SRE focuses on improving software system reliability across key categories including availability, performance, latency, efficiency, capacity, and incident response. SRE applies DevOps principles to developing systems and software that help increase site reliability and performance. Solving for SR.

Engineering 260

Engineering DevOps Government Latency 260

The Netflix TechBlog

NOVEMBER 17, 2022

While conventional video codecs remain prevalent, NN-based video encoding tools are flourishing and closing the performance gap in terms of compression efficiency. Architecture of the deep downscaler model, consisting of a preprocessing block followed by a resizing block. On a CPU, we leveraged oneDnn to further reduce latency.

Network 302

Network Media Innovation Efficiency 302

The Netflix TechBlog

NOVEMBER 3, 2022

The original assumptions and architectural choices were no longer viable. Overview The figure below depicts a simplified high-level architecture of a single Titus cluster (a.k.a We started seeing increased response latencies and leader servers running at dangerously high utilization.

Cache 235

Cache Latency Traffic Systems 235

High Scalability

JANUARY 11, 2022

Architecture. from a client it performs two parallel operations: i) persisting the action in the data store ii) publish the action in a streaming data store for a pub-sub model. User Feed Service, Media Counter Service) read the actions from the streaming data store and performs their specific tasks. High Level Design.

Design 334

Design Media Storage Logistics 334

The Netflix TechBlog

MARCH 1, 2021

It supports both high throughput services that consume hundreds of thousands of CPUs at a time, and latency-sensitive workloads where humans are waiting for the results of a computation. The subsystems all communicate with each other asynchronously via Timestone, a high-scale, low-latency priority queuing system.

Serverless 236

Serverless Media Latency Social Media 236

Scalegrid

OCTOBER 17, 2019

Moving to a multithreaded architecture will require extensive rewrites. But that causes a problem with PostgreSQL’s architecture – forking a process becomes expensive when transactions are very short, as the common wisdom dictates they should be. The PostgreSQL Architecture | Source. The Connection Pool Architecture.

Architecture 200

Architecture Database Latency Servers 200

Dynatrace

MAY 23, 2024

Stream processing One approach to such a challenging scenario is stream processing, a computing paradigm and software architectural style for data-intensive software systems that emerged to cope with requirements for near real-time processing of massive amounts of data. This significantly increases event latency.

Engineering 192

Engineering Tuning Latency Open Source 192

Dynatrace

APRIL 7, 2023

Dynatrace is a launch partner in support of AWS Lambda Response Streaming , a new capability enabling customers to improve the efficiency and performance of their Lambda functions. Customers can use AWS Lambda Response Streaming to improve performance for latency-sensitive applications and return larger payload sizes.

Lambda 227

Lambda AWS Serverless Latency 227

Alex Russell

FEBRUARY 28, 2022

It's being reposted here for completeness, but if you care about web performance, make sure to check out the whole series and get subscribed to the RSS feed to avoid missing any of next year's posts. The predominant answer: a unified theory of web performance. What, in particular, is "web performance"? How do we do it?

Performance 113

Performance Latency Architecture Network 113

The Morning Paper

OCTOBER 4, 2020

Understanding operational 5G: a first measurement study on its coverage, performance and energy consumption , Xu et al., What is the end-to-end throughput and latency, and where are the bottlenecks? Future 5G Standalone Architecture (SA) deployments with a native 5G control plane will not have this problem. SIGCOMM’20.

Energy 130

Energy Latency Performance Network 130

Dynatrace

SEPTEMBER 18, 2020

Sure, cloud infrastructure requires comprehensive performance visibility, as Dynatrace provides , but the services that leverage cloud infrastructures also require close attention. Cloud-based application architectures commonly leverage microservices. High latency or lack of responses. Soaring number of active connections.

Infrastructure 194

Infrastructure Latency Metrics Analytics 194

The Netflix TechBlog

SEPTEMBER 8, 2020

We tried a few iterations of what this new service should look like, and eventually settled on a modern architecture that aimed to give more control of the API experience to the client teams. For us, it means that we now need to have ~15 MDN tabs open when writing routes :) Let’s briefly discuss the architecture of this microservice.

Latency 243

Latency Cache Java Traffic 243

Dynatrace

MAY 31, 2023

With so many of their transactions occurring online, customers are becoming more demanding, expecting websites and applications to always perform perfectly. Microservices-based architectures and software containers enable organizations to deploy and modify applications with unprecedented speed.

Best Practices 233

Best Practices DevOps Latency Metrics 233

Dynatrace

MARCH 29, 2024

During a breakout session at Dynatrace Perform 2024 , Alois Mayr, principal product manager at Dynatrace, and Stefano Doni, CTO at Akamas, broke down how Dynatrace and Akamas can help organizations enhance Kubernetes cluster management. Consider a sudden drop in performance. “Additionally, we are full-stack and goal-oriented.

Engineering 238

Engineering DevOps Operating System Open Source 238

Dynatrace

JANUARY 31, 2024

This blog post explores how AI observability enables organizations to predict and control costs, performance, and data reliability. Figure 1: Sample RAG architecture While this approach significantly improves the response quality of GenAI applications, it also introduces new challenges.

Cache 215

Cache Azure Infrastructure Monitoring 215

The Netflix TechBlog

APRIL 27, 2023

High Level Architecture The idea, at a high level, was to avoid the need to query the Atlas database almost entirely and transition most alert queries to streaming evaluation. First and foremost, we have successfully alleviated our initial scalability problem with the polling based architecture. OK, Results?

Storage 300

Storage Cache Metrics Database 300