The Netflix TechBlog

NOVEMBER 9, 2021

We were very pleased to see that AV1 streaming improved members’ viewing experience, particularly under challenging network conditions. AV1 playback on TV platforms relies on hardware solutions, which generally take longer to be deployed. Throughout 2020 the industry made impressive progress on AV1 hardware solutions.

Media 246

Media Open Source Software Engineering Efficiency 246

Dotcom-Montior

OCTOBER 6, 2021

A site reliability engineer, or SRE, is a role that that encompasses aspects of both software engineering and operations/infrastructure. The term site reliability engineering first came into existence at Google in 2003 when a site reliability team was created. At that time, the team was made up of software engineers.

Engineering 67

Engineering DevOps Monitoring Google 67

Adrian Cockcroft

JANUARY 17, 2022

There were five trends and topics for 2021, Serverless First, Chaos Engineering, Wardley Mapping, Huge Hardware, Sustainability. The need for systems to be resilient is still increasing, and chaos engineering tools and techniques are developing as a key way to validate that resilience is working as designed.

Serverless 98

Serverless Hardware AWS Architecture 98

Dynatrace

NOVEMBER 1, 2022

These metrics help to keep a network system up and running?, Mean time to recovery (MTTR) measures the entire amount of time it takes to get a downed network or system back up and running. MTTF measures the reliability of a network and durability of its hardware. a critical task that’s easier said than done.

DevOps 199

DevOps Artificial Intelligence Metrics Network 199

The Morning Paper

MARCH 8, 2020

Building an elastic query engine on disaggregated storage , Vuppalapati, NSDI’20. This paper presents Snowflake design and implementation along with a discussion on how recent changes in cloud infrastructure (emerging hardware, fine-grained billing, etc.) From shared-nothing to disaggregation.

Storage 112

Storage Engineering Cache Serverless 112

The Netflix TechBlog

FEBRUARY 25, 2021

In all these cases, prior to being delivered through our content delivery network Open Connect , our award-winning TV shows, movies and documentaries like The Crown need to be packaged to enable crucial features for our members. Decryption modules need to be initialized with the appropriate scheme and initialization vector. We’re hiring!

Technology 190

Technology Technology Open Source Media 190

DZone

JANUARY 16, 2021

CPU consumption in Unix/Linux operating systems is broken down into 8 different metrics: User CPU time , System CPU time , nice CPU time , Idle CPU time , Waiting CPU time , Hardware Interrupt CPU time , Software Interrupt CPU time , and Stolen CPU time. In this article, let us study ‘waiting CPU time’. What Is ‘Waiting’ CPU Time?

Operating System 124

Operating System Hardware Network Metrics 124

ACM Sigarch

APRIL 26, 2018

HPU: Holographic Processing Unit (HPU) is the specific hardware of Microsoft’s Hololens. They use the graph as the basic representation for many AI-related algorithms, including neural network, Bayesian network, Markov Field, and some other emerging methods. HPU1 with TSMC 28nm process was announced in HOTCHIPS’17.

Hardware 99

Hardware Artificial Intelligence Architecture Google 99

Dynatrace

APRIL 5, 2021

You will likely need to write code to integrate systems and handle complex tasks or incoming network requests. As a bonus, operations staff never needs to update operating systems or hardware, because AWS manages servers with no stoppage of application functionality. Customizing and connecting these services requires code.

Lambda 174

Lambda AWS Serverless Hardware 174

VoltDB

MAY 14, 2024

Low usage or over-engineered legacy systems Until the arrival of cloud hosting, nobody knew how to size anything properly, or they did know but never got a chance to. Prototypes, experiments, and tests Development and testing historically involved end-of-life or ‘spare’ hardware. When is the cloud a bad idea?

Latency 52

Latency Availability Cloud Hardware 52

Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on. I also wrote about these topics in detail for my recent [Systems Performance 2nd Edition] book.

Performance 140

Performance Latency Hardware Storage 140

CSS - Tricks

JULY 25, 2019

Lighthouse uses Chrome’s Remote Debugging Protocol to read network request information, measure JavaScript performance, observe accessibility standards and measure user-focused timing metrics like First Contentful Paint , Time to Interactive or Speed Index. Lighthouse is an open source project run by a dedicated team from Google Chrome.

Google 72

Google Engineering Speed Mobile 72

The Netflix TechBlog

JUNE 4, 2019

The idea CFS operates by very frequently (every few microseconds) applying a set of heuristics which encapsulate a general concept of best practices around CPU hardware use. We could then feed this information directly into the optimization engine to move towards a more supervised learning approach.

Cache 258

Cache Latency Airlines Logistics 258

Dynatrace

SEPTEMBER 30, 2021

It requires purchasing, powering, and configuring physical hardware, training and retaining the staff capable of servicing and securing the machines, operating a data center, and so on. They need enough hardware to serve their anticipated volume and keep things running smoothly without buying too much or too little. Reduced cost.

Cloud 159

Cloud Traffic Best Practices Strategy 159

Scalegrid

JANUARY 5, 2024

Components of DBMS The primary component of a DBMS is the storage engine, which operates alongside software components such as the query language, query processor, optimization engine, metadata catalog, and log manager. Types of DBMS DBMS can be classified into hierarchical, network, relational, and object-oriented types.

Efficiency 147

Efficiency Storage Database Scalability 147

Smashing Magazine

AUGUST 3, 2021

Lots can go wrong: a network request fails, a third-party library breaks, a JavaScript feature is unsupported (assuming JavaScript is even available), a CDN goes down, a user behaves unexpectedly (they double-click a submit button), the list goes on. The more enriched sentence (right) is an enhancement for when the network request succeeds.

Network 145

Network Engineering Availability Monitoring 145

CSS Wizardry

OCTOBER 27, 2018

Things always always feel fast when we’re developing because, more often than not, we’re working on high-spec machines on dedicated networks, and also serving from localhost which removes the bulk of the latency and bandwidth issues that a real user would suffer. Who: Engineers. Who: Engineers, Product Owners.

Performance Testing 175

Performance Testing Testing Performance Strategy 175

Sutter's Mill

NOVEMBER 11, 2023

Instead, you want a library that is tuned for your target hardware architecture and ready for par_unseq vectorized algorithms, for blazing speed. SG4 (Networking) continued working on updating the networking proposal for std::execution senders and receivers. This is that library.

C++ 119

C++ Lambda Network Programming 119

HammerDB

APRIL 19, 2023

As database performance is heavily influenced by the performance of storage, network, memory, and processors, we must understand the upper limit of these key components. For the network, we can use Iperf to assess the network bandwidth between the client and the database server to ensure it will be enough to meet our peak requirement.

Performance 54

Performance Benchmarking Cache Storage 54

Adrian Cockcroft

JUNE 2, 2023

One example is that Netflix launched it’s streaming service in 2007, just at the point when the cost of streaming a movie over the network (which was dropping fast) became less than the cost of shipping a DVD. Russell Kaplan of Scale AI talked about prompt engineering. Tristan Zajonc, CEO of Continual.ai

Social Media 98

Social Media Innovation Open Source Entertainment 98

Percona

MAY 2, 2023

At the same time that I see database engineers relying on the tool, sites such as StackOverflow are banning ChatGPT. However, it’s important to note that the optimal value may vary depending on your specific workload and hardware configuration. 16) and monitoring the server’s performance. Let’s make it harder now.

Social Media 122

Social Media Database Servers Cache 122

Percona

JUNE 22, 2023

Another related variable, innodb_buffer_pool_instances, determines the number of buffer pool instances for the InnoDB storage engine, which can improve the performance of multi-core systems by reducing contention on the buffer pool latch.

Performance 105

Performance Monitoring Traffic Database 105

Adrian Cockcroft

JUNE 21, 2022

So in 1988 I joined the Sun sales force and became a Systems Engineer (nowadays we would say Solutions Architect). I became the Sun UK local specialist in performance and hardware, and as Sun transitioned from a desktop workstation company to sell high end multiprocessor servers I was helping customers find and fix scalability problems.

Tuning 52

Tuning Benchmarking Engineering C++ 52

The Morning Paper

JUNE 13, 2019

As we saw with the SOAP paper last time out, even with a fixed model variant and hardware there are a lot of different ways to map a training workload over the available hardware. Different hardware architectures (CPUs, GPUs, TPUs, FPGAs, ASICs, …) offer different performance and cost trade-offs.

Hardware 58

Hardware Latency Serverless Energy 58

Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on. I also wrote about these topics in detail for my recent [Systems Performance 2nd Edition] book.

Performance 52

Performance Latency Hardware Storage 52

The Morning Paper

JANUARY 30, 2020

physics engine that simulates 3D cubes falling from the air. These use their regression models to estimate processing time (which will depend on the hardware available, current load, etc.). The kind of edge server envisaged here might, for example, be integrated with your WiFi access point. The opencv app has the largest state (4.6

Mobile 104

Mobile Cloud Latency Games 104

All Things Distributed

APRIL 5, 2012

From tax preparation to safe social networks, Amazon RDS brings new and innovative applications to the cloud. Recently I had great conversations with Troy Otillio, Senior Development Manager at Intuit and Jack Murgia, Senior DevOps Engineer at Edmodo. Jack and his engineers have created a safe social app for teachers and students.

Innovation 138

Innovation AWS Education Network 138

Brendan Gregg

FEBRUARY 28, 2023

Make sure your system can handle next-generation DRAM,” [link] Nov 2011 - [Hruska 12] Joel Hruska, “The future of CPU scaling: Exploring options on the cutting edge,” [link] Feb 2012 - [Gregg 13] Brendan Gregg, “Blazing Performance with Flame Graphs,” [link] 2013 - [Shimpi 13] Anand Lal Shimpi, “Seagate to Ship 5TB HDD in 2014 using Shingled Magnetic (..)

Performance 109

Performance Latency Cache Virtualization 109

Dotcom-Montior

DECEMBER 8, 2021

In the world of a site reliability engineer (SRE) , failure is not only an option, but also expected. In a different article, we talked about chaos engineering and how SRE teams proactively seek out and test for failures to prevent the worst from happening. Read : Top 13 Site Reliability Engineer (SRE) Tools. Asset Management.

Social Media 90

Social Media Monitoring Latency DevOps 90

Abhishek Tiwari

JULY 12, 2018

These data pipelines can process data at petabytes scale and to some extent, their success can be attributed to an army of engineers devoted to build and maintain internal data pipelines. Not everyone is operating at Netflix or Spotify scale data engineering function. A data pipeline is a software which runs on hardware.

Latency 63

Latency Analytics Scalability Engineering 63

The Netflix TechBlog

FEBRUARY 9, 2021

Growth Engineering at Netflix?—?Automated In the Growth Engineering team, we refer to this as the top of the signup funnel. For more background on the signup funnel and Growth Engineering’s role in the signup funnel, please read our initial post on the topic: Growth Engineering at Netflix? Accelerating Innovation.

Engineering 172

Engineering Storage Latency Entertainment 172

Brendan Gregg

AUGUST 18, 2019

bpftrace uses BPF (Berkeley Packet Filter), an in-kernel execution engine that processes a virtual instruction set. BPF has been extended (aka eBPF) in recent years for providing a safe way to extend kernel functionality, and has become a hot topic in systems engineering, with at least 24 talks on BPF at the last Linux Plumber's conference.

Latency 68

Latency C++ Cache Programming 68

Abhishek Tiwari

DECEMBER 27, 2017

Kubernetes has emerged as go to container orchestration platform for data engineering teams. optimised container networking and security. Using default scheduler's node affinity feature you can ensure that certain pods only schedule on nodes with specialized hardware like GPU, memory-optimised, I/O optimised etc.

Big Data 63

Big Data Storage Benchmarking Hardware 63

Abhishek Tiwari

DECEMBER 13, 2017

More importantly, if this works out well, this could lead to a radical improvement in performance by leveraging hardware trends such as GPUs and TPUs. Their result is particularly promising for specialized index structures termed as learned indexes which has potential to replace B-Trees without major re-engineering. Learned indexes.

Artificial Intelligence 40

Artificial Intelligence Hardware Google Benchmarking 40

Testsigma

JANUARY 30, 2021

Each smartphone comes with various screen sizes and resolutions, operates on different network speeds, and has different hardware capabilities. Test how user-friendly an application is: Google search engine gives high priority to websites in comparison to desktop apps. Signup Now. Best Practices For Mobile Website Testing.

Mobile 52

Mobile Testing Website Internet 52

Dynatrace

JUNE 25, 2020

The network latency between cluster nodes should be around 10 ms or less. Our Premium High Availability comes with the following features: Active-active deployment model for optimum hardware utilization. Minimized cross-data center network traffic. – A Dynatrace customer, Head of Performance Engineering.

Availability 238

Availability Hardware Latency Traffic 238

All Things Distributed

SEPTEMBER 6, 2013

Shell''s scientists, especially the geophysicists and drilling engineers, frequently use cloud computing to run models. In my many conversations with customers, and with the media, I encountered surprise and excitement about the extent that European enterprises have already been using the Amazon Web Services for some time.

Cloud 123

Cloud Energy AWS Healthcare 123