The Netflix TechBlog

MARCH 1, 2021

It supports both high throughput services that consume hundreds of thousands of CPUs at a time, and latency-sensitive workloads where humans are waiting for the results of a computation. The subsystems all communicate with each other asynchronously via Timestone, a high-scale, low-latency priority queuing system. Warm capacity.

Serverless 235

Serverless Media Latency Social Media 235

High Scalability

DECEMBER 21, 2018

Tim Bray : How to talk about [Serverless Latency] · To start with, don’t just say “I need 120ms.” Whether it’s database or message queues it’s a really weird combo of licenses and features for hostage. Because nobody knows how to make money. Support contracts aren’t enough.

Internet 137

Internet Internet Scalability Serverless 137

The Netflix TechBlog

JANUARY 10, 2024

This architecture shift greatly reduced the processing latency and increased system resiliency. We expanded pipeline support to serve our studio/content-development use cases, which had different latency and resiliency requirements as compared to the traditional streaming use case. divide the input video into small chunks 2.

Processing 238

Processing Media Latency Innovation 238

High Scalability

JULY 20, 2018

A typical example of modern "microservices-inspired" Java application would function along these lines: Netflix : We observed during experimentation that RAM random read latencies were rarely higher than 1 microsecond whereas typical SSD random read speeds are between 100–500 microseconds.

Internet 121

Internet Internet Scalability Automotive 121

Brendan Gregg

OCTOBER 8, 2018

Screenshot: tracing read latency for PID 181: # bpftrace -e 'kprobe:vfs_read /pid == 30153/ { @start[tid] = nsecs; } kretprobe:vfs_read /@start[tid]/ { @ns = hist(nsecs - @start[tid]); delete(@start[tid]); }'. Here's key differences as of August 2018: Type DTrace bpftrace. It's shaping up to be a DTrace version 2.0: I wrote seeksize.d

C++ 105

C++ Virtualization Programming Latency 105

High Scalability

JANUARY 24, 2019

For the first time in history, a VC-backed startup in 2018 was more likely to sell to a PE buyer than a fellow VC-backed company. TServerless : We sat with a solution architect, apparently they are aware of the latency issue and suggested to ditch api gw and build our own solution. 10% : Netflix captured screen time in US; 8.3

Internet 29

Internet Internet Scalability Games 29

Brendan Gregg

JULY 4, 2021

## References I've reproduced the talk references below, so you can click on links: - [Gregg 08] Brendan Gregg, “ZFS L2ARC,” [link] Jul 2008 - [Gregg 10] Brendan Gregg, “Visualizations for Performance Analysis (and More),” [link] 2010 - [Greenberg 11] Marc Greenberg, “DDR4: Double the speed, double the latency? Ford, et al., “TCP

Performance 121

Performance Latency Hardware Storage 121

High Scalability

AUGUST 17, 2018

12 million requests / hour with sub-second latency, ~300GB of throughput / day. @coryodaniel : Rewrote an #AWS APIGateway & #lambda service that was costing us about $16000 / month in #elixir. Its running in 3 nodes that cost us about $150 / month. myelixirstatus !#Serverless.No Serverless.No

Internet 105

Internet Internet Scalability Games 105

KeyCDN

MARCH 10, 2021

The image below shows a significant drop in latency once we've launched the new point of presence in Israel. In fact, latency has been reduced by almost 50%! With a total of 5 POPs in Oceania, this continent benefits from lower latency with every POP added. With a population of 2.5

Latency 110

Latency Internet Internet Speed 110

All Things Distributed

NOVEMBER 13, 2018

ENEL is using AWS to transform its entire business, closing all of their data centers by 2018, migrating workloads from over 6,000 on-premises servers onto AWS in nine months, and using AWS IoT services to better manage and understand energy consumption. ENEL is one of the leading energy operators in the world. million unique visits.

AWS 164

AWS Energy Automotive Traffic 164

Brendan Gregg

FEBRUARY 28, 2023

My personal opinion is that I don't see a widespread need for more capacity given horizontal scaling and servers that can already exceed 1 Tbyte of DRAM; bandwidth is also helpful, but I'd be concerned about the increased latency for adding a hop to more memory. Ford, et al., “TCP

Performance 103

Performance Latency Cache Google 103

The Netflix TechBlog

MARCH 13, 2020

As mentioned in our earlier blog post , Intel and Netflix have been collaborating on the SVT-AV1 encoder and decoder framework since August 2018. The teams have been working closely on SVT-AV1 development, discussing architectural decisions, implementing new tools, and improving compression efficiency.

Open Source 159

Open Source Efficiency C++ Speed 159

Abhishek Tiwari

DECEMBER 27, 2017

There are several emerging data trends that will define the future of ETL in 2018. In 2018, we anticipate that ETL will either lose relevance or the ETL process will disintegrate and be consumed by new data architectures. With the arrival of new cloud-native tools and platform, ETL is becoming obsolete. More details on this approach.

Big Data 49

Big Data Artificial Intelligence Storage Hardware 49

All Things Distributed

SEPTEMBER 25, 2017

This Region will consist of three Availability Zones at launch, and it will provide even lower latency to users across the Middle East. Plus another AWS GovCloud (US) Region in the United States is coming online by the end of 2018. This news marks the 22nd AWS Region we have announced globally.

Cloud 117

Cloud Education Energy Government 117

All Things Distributed

DECEMBER 12, 2018

They can run applications in Sweden, serve end users across the Nordics with lower latency, and leverage advanced technologies such as containers, serverless computing, and more. Starting today, developers, startups, and enterprises—as well as government, education, and non-profit organizations—can use the new AWS Europe (Stockholm) Region.

AWS 119

AWS Cloud Games Serverless 119

The Netflix TechBlog

DECEMBER 17, 2019

Passive instances across regions are also possible, though it is recommended to operate in the same region as the database host in order to keep the change capture latencies low. Delta is used in production since 2018 for datastore synchronization and event processing use cases in Netflix studio applications.

Database 221

Database Traffic Transportation Open Source 221

The Netflix TechBlog

DECEMBER 17, 2019

Passive instances across regions are also possible, though it is recommended to operate in the same region as the database host in order to keep the change capture latencies low. Delta is used in production since 2018 for datastore synchronization and event processing use cases in Netflix studio applications.

Database 206

Database Traffic Transportation Open Source 206

ACM Sigarch

MARCH 18, 2019

How many buffers are needed to track pending requests as a function of needed bandwidth and expected latency? Can one both minimize latency and maximize throughput for unscheduled work? These models are useful for insight regarding the basic computer system performance metrics of latency and throughput (bandwidth). Little’s Law.

Systems 60

Systems Latency Performance Analytics 60

Brendan Gregg

OCTOBER 7, 2018

Screenshot: tracing read latency for PID 181: # bpftrace -e 'kprobe:vfs_read /pid == 30153/ { @start[tid] = nsecs; } kretprobe:vfs_read /@start[tid]/ { @ns = hist(nsecs - @start[tid]); delete(@start[tid]); }'. Here's key differences as of August 2018: Type DTrace bpftrace. It's shaping up to be a DTrace version 2.0: I wrote seeksize.d

C++ 40

C++ Virtualization Programming Latency 40

Brendan Gregg

JULY 4, 2021

## References I've reproduced the talk references below, so you can click on links: - [Gregg 08] Brendan Gregg, “ZFS L2ARC,” [link] Jul 2008 - [Gregg 10] Brendan Gregg, “Visualizations for Performance Analysis (and More),” [link] 2010 - [Greenberg 11] Marc Greenberg, “DDR4: Double the speed, double the latency? Ford, et al., “TCP

Performance 52

Performance Latency Hardware Storage 52

Brendan Gregg

AUGUST 18, 2019

For example, iostat(1), or a monitoring agent, may tell you your average disk latency, but not the distribution of this latency. For smaller environments, it can be of more use helping eliminate latency outliers. Block I/O latency as a histogram. 6 * 7 * Copyright 2018 Netflix, Inc. 1 /* 2 * biolatency.bt

Latency 57

Latency C++ Cache Programming 57

John McCalpin

JANUARY 9, 2019

This week I updated three of those posts — two really old ones (primarily of interest to computer architecture historians), and one from 2018: July 2012: Local and Remote Memory Latency on AMD Processors in 2-socket and 4-socket servers December 2013: Notes on Memory Bandwidth on the Xeon Phi (Knights Corner) Coprocessor January 2018: A Peculiar (..)

Latency 40

Latency Architecture Servers Performance 40

John McCalpin

JANUARY 9, 2019

This week I updated three of those posts — two really old ones (primarily of interest to computer architecture historians), and one from 2018: July 2012: Local and Remote Memory Latency on AMD Processors in 2-socket and 4-socket servers. December 2013: Notes on Memory Bandwidth on the Xeon Phi (Knights Corner) Coprocessor.

Latency 40

Latency Architecture Servers Performance 40

ACM Sigarch

MARCH 25, 2019

How many buffers are needed to track pending requests as a function of needed bandwidth and expected latency? Can one both minimize latency and maximize throughput for unscheduled work? The M/M/1 queue will show us a required trade-off among (a) allowing unscheduled task arrivals, (b) minimizing latency, and (c) maximizing throughput.

Systems 53

Systems Latency Performance C++ 53

Alex Russell

FEBRUARY 3, 2023

They understood that most websites lack tight latency budgeting, dedicated performance teams, hawkish management reviews, ship gates to prevent regressions, and end-to-end measurements of critical user journeys. This raises a follow-on question: what's an interaction?

Social Media 130

Social Media Website Technology Technology 130

Brendan Gregg

AUGUST 29, 2021

1072-aws (xxx) 12/18/2018 _x86_64_ (16 CPU) avg-cpu: %user %nice %system %iowait %steal %idle 5.03 biolatency From [bcc], this eBPF tool shows a latency histogram of disk I/O. 1072-aws (xxx) 12/19/2018 _x86_64_ (16 CPU) avg-cpu: %user %nice %system %iowait %steal %idle 12.25 avg-cpu: %user %nice %system %iowait %steal %idle 14.81

Cache 78

Cache C++ AWS Systems 78

Brendan Gregg

FEBRUARY 28, 2023

My personal opinion is that I don't see a widespread need for more capacity given horizontal scaling and servers that can already exceed 1 Tbyte of DRAM; bandwidth is also helpful, but I'd be concerned about the increased latency for adding a hop to more memory. Ford, et al., “TCP

Performance 52

Performance Latency Cache Google 52

Smashing Magazine

OCTOBER 31, 2023

The truth is that the two tools were fairly distinct until PSI was updated in 2018 to use Lighthouse reporting. INP is a measure of the latency for all interactions on a given page, where the highest latency — or close to it — informs the final score. It’s right there in the name!

Speed 117

Speed Google Website Metrics 117

CSS - Tricks

APRIL 1, 2021

Google’s industry benchmarks from 2018 also provide a striking breakdown of how each second of loading affects bounce rates. Source: Google /SOASTA Research, 2018. I’m going to update my referenced URL to the new site to help decrease latency that adds drag to the initial page load. Improvement #2: The Critical Render Path.

Cache 94

Cache Social Media Media Website 94

KeyCDN

OCTOBER 24, 2018

Although both countries are relatively close to one another, they are separated by a distance of approximately 500km, which adds up in terms of latency. As of August 2018, Helsinki alone had a population of almost 650,000 people. As for Finland, it covers an area of over 338,000 square kilometers and has a population of over 5.5

Internet 48

Internet Internet Speed Network 48

Alex Russell

JANUARY 30, 2024

If you or your company are able to generate a credible worldwide latency estimate in the higher percentiles for next year's update, please get in touch. Since 2010, volumes have been on a slow downward glide path , shrinking from ~350MM per year in a decade ago to ~260MM in 2018. but which blessedly sports 8GB of RAM.

Performance 98

Performance Network Mobile Speed 98

SQL Performance

FEBRUARY 26, 2020

Azure SQL Database Managed Instance became generally available in late 2018. The General Purpose tier is designed for applications with typical performance and I/O latency requirements and provides built-in HA. The Business Critical tier is designed for applications that require low I/O latency and higher HA requirements.

Azure 63

Azure Performance Storage Latency 63

Abhishek Tiwari

DECEMBER 27, 2017

In 2018, a widespread adaptation of Kubernetes for big data processing is anitcipated. direct access to raw block storage [18] without the abstraction of a filesystem for workloads that require consistent I/O performance and low latency. Kubernetes has emerged as go to container orchestration platform for data engineering teams.

Big Data 63

Big Data Storage Benchmarking Hardware 63

Alex Russell

SEPTEMBER 11, 2018

I can count on one hand the number of teams I’ve worked with who have goals that allow them to block launches for latency regressions, including Google products. — Alex Russell (@slightlylate) August 27, 2018. Few teams I’ve encountered have actionable metrics associated with the real experiences of their users.

Development 73

Development Government Mobile Network 73

Speed Curve

FEBRUARY 5, 2024

Charlie Vazac introduced server timing in a Performance Calendar post circa 2018. Latency – How much time does it take to deliver a packet from A to B. In practice, it looks something like this: server-timing: processing_time; dur=123.4; desc="Time to process request at origin" NOTE: This is not a new API.

Servers 64

Servers Cache Benchmarking Retail 64

Alex Russell

MARCH 6, 2021

India became a 4G-centric market sometime in 2018. Sadly, data on latency is harder to get, even from Google's perch, so progress there is somewhat more difficult to judge. If there's a bright spot in our construction of a 2021 baseline for performance, this is it. 5G looks set to continue a bumpy rollout for the next half-decade.

Performance 127

Performance Network Mobile Metrics 127

Percona

AUGUST 10, 2018

sec < 2018-08-06 07:22:35.199 EDT > LOG: automatic analyze of table "vactest.scott.employee" system usage: CPU 0.00s/0.02u sec elapsed 0.15 sec < 2018-08-06 07:22:35.199 EDT > LOG: automatic analyze of table "vactest.scott.employee" system usage: CPU 0.00s/0.02u sec elapsed 0.15 1 second = 1000 milliseconds).

Tuning 69

Tuning Cache Database Storage 69