The Netflix TechBlog

OCTOBER 19, 2020

which is difficult when troubleshooting distributed systems. Troubleshooting a session in Edgar When we started building Edgar four years ago, there were very few open-source distributed tracing systems that satisfied our needs. Investigating a video streaming failure consists of inspecting all aspects of a member account.

Infrastructure 238

Infrastructure Transportation Storage Open Source 238

Brendan Gregg

DECEMBER 31, 2017

My last talk for 2017 was at AWS re:Invent, on "How Netflix Tunes EC2 Instances for Performance," an updated version of my [2014] talk. WARNING: These tunables were developed in late 2017, for Ubuntu Xenial instances on EC2.** File System. echo tsc > /sys/devices/system/clocksource/clocksource0/current_clocksource.

Tuning 53

Tuning AWS Best Practices Network 53

Brendan Gregg

APRIL 14, 2022

PMC analysis (2017). I hope my last two books, [Systems Performance 2nd Ed] and [BPF Performance Tools] serve Netflix well in my absence and everyone else who reads them. Netflix has been the best job of my career so far, and I'll miss my colleagues and the culture. offer letter logo (2014). flame graphs (2014). eBPF tools (2014-2019).

Java 142

Java Cloud Cache AWS 142

Brendan Gregg

DECEMBER 30, 2017

My last talk for 2017 was at AWS re:Invent, on "How Netflix Tunes EC2 Instances for Performance," an updated version of my [2014] talk. WARNING: These tunables were developed in late 2017, for Ubuntu Xenial instances on EC2.** File System. echo tsc > /sys/devices/system/clocksource/clocksource0/current_clocksource.

Tuning 52

Tuning AWS Best Practices Network 52

Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on. I also wrote about these topics in detail for my recent [Systems Performance 2nd Edition] book.

Performance 121

Performance Latency Hardware Storage 121

Brendan Gregg

JULY 15, 2019

BPF (eBPF) tracing is a superpower that can analyze everything, and I'll show you how in my upcoming book BPF Performance Tools: Linux System and Application Observability , coming soon from Addison Wesley. I'm also grateful that my fiancée, Deirdré Straughan, could find the time outside of her AWS job to do technical copy-edit.

Systems 111

Systems Performance Education Technology 111

O'Reilly

FEBRUARY 18, 2020

Up until 2017, the ML+AI topic had been amongst the fastest growing topics on the platform. After several years of steady climbing—and after outstripping Java in 2017—Python-related interactions now comprise almost 10% of all usage. Not necessarily: Java-related searches increased by 5% between 2017 and 2018. Coincidence?

Software Architecture 145

Software Architecture DevOps Data Engineering Architecture 145

Scalegrid

APRIL 2, 2019

PostgreSQL is an open source relational database system that has soared in popularity over the past 30 years from its active, loyal, and growing community. In a predictable discovery, we found that Amazon Web Services (AWS) claimed the majority at 55% of use for all PostgreSQL hosting activities in a public cloud environment.

Database 185

Database Cloud Open Source Azure 185

All Things Distributed

MARCH 30, 2017

This weekend I am travelling to Australia for the first AWS Summit of 2017. One topic that always gets me excited is how to take computer science research and implement it in production systems. real systems do not fail by stopping in a nice and clean way). At times, it makes research unsuitable for production (e.g.

Systems 77

Systems AWS Design Technology 77

Dynatrace

JUNE 15, 2023

If your app runs in a public cloud, such as Amazon Web Services (AWS), Microsoft Azure, or Google Cloud Platform (GCP), the provider secures the infrastructure, while you’re responsible for security measures within applications and configurations. What are some key characteristics of securing cloud applications?

Cloud 245

Cloud Open Source Architecture Monitoring 245

All Things Distributed

SEPTEMBER 29, 2016

Voice makes access to digital services far more inclusive than traditional screen-based interaction, for example, an aging population may be much more comfortable interacting with voice-based systems than through tablets or keyboards. However, it is still Day One, and a lot of innovation is underway in this world.

Artificial Intelligence 111

Artificial Intelligence Healthcare AWS Government 111

All Things Distributed

SEPTEMBER 25, 2017

Today, I am excited to announce plans for Amazon Web Services (AWS) to bring an infrastructure Region to the Middle East! Based in Bahrain, this will be the first Region for AWS in the Middle East. This news marks the 22nd AWS Region we have announced globally. 2017 continues a busy year for AWS in the Middle East.

Cloud 119

Cloud Education Energy Government 119

Brendan Gregg

NOVEMBER 28, 2017

At Netflix, we've been using these technologies as they've been made available for instance types in the AWS EC2 cloud. The latest AWS hypervisor, Nitro, uses everything to provide a new hardware-assisted hypervisor that is easy to use and has near bare-metal performance. AWS called this [enhanced networking]. The first was c3.

Virtualization 40

Virtualization AWS Hardware Storage 40

Brendan Gregg

SEPTEMBER 5, 2017

What follows are topics that may be of interest to anyone looking to migrate their systems and skillset: scan these to find topics that interest you. ## ZFS ZFS is available for Linux via the [zfsonlinux] and [OpenZFS] projects, and more recently was included in Canonical's Ubuntu Linux distribution: Ubuntu Xenial 16.04 LTS (April 2016).

Virtualization 75

Virtualization AWS Engineering Hardware 75

Dynatrace

OCTOBER 17, 2019

Everything as Code can be described as a methodology or practice which consists of extending the idea of how applications are treated as code and applying these concepts to all other IT components like operating systems, network configurations, and pipelines. We went from managing five EC2 instances in 2011 to around 1,000 in 2017.

Code 167

Code Infrastructure DevOps Serverless 167

All Things Distributed

OCTOBER 2, 2017

With these requirements in mind, and a willingness to question the status quo, a small group of distributed systems experts came together and designed a horizontally scalable distributed database that would scale out for both reads and writes to meet the long-term needs of our business. This was the genesis of the Amazon Dynamo database.

Internet 111

Internet Internet AWS Performance 111

All Things Distributed

MAY 19, 2017

Soon after the launch of Amazon Relation Dabase Service (RDS) AWS customers were giving us feedback that they would love to migrate to RDS but what they would love even more was if we could also unshackle them from the high-cost, punitive licensing schemes that came with the proprietary databases.

Database 89

Database Design Cloud Storage 89

Brendan Gregg

FEBRUARY 28, 2023

This talk originated from my updates to [Systems Performance 2nd Edition], and this was the first time I've given this talk in person! CXL in a way allows a custom memory controller to be added to a system, to increase memory capacity, bandwidth, and overall performance. Ford, et al., “TCP

Performance 103

Performance Latency Cache Virtualization 103

Brendan Gregg

SEPTEMBER 25, 2021

CLI tools The Cassandra systems were EC2 virtual machine (Xen) instances. Microbenchmark os::javaTimeMillis() on both systems. Running this on the two systems saw similar results. Checking those available: $ cat /sys/devices/system/clocksource/clocksource0/available_clocksource. Try changing the kernel clocksource.

Speed 103

Speed Java AWS Virtualization 103

All Things Distributed

JUNE 30, 2017

Once connected, the systems that power call centers generally don't do a good job of using and sharing available information. Today, AWS customers are using the cloud to better serve their customers in many different ways. This means that you're going to wait on hold regardless of whether your issue is simple or complex.

Artificial Intelligence 115

Artificial Intelligence Lambda AWS Virtualization 115

The Symphonia

DECEMBER 13, 2017

that of the ability to overwhelm downstream systems with unfettered load?—?got got a lot smoother at re:Invent 2017. For other teams it has meant Lambda is simply not a suitable technology for certain types of system. Enter Reserved Concurrency At re:Invent 2017 the Lambda team announced a new feature?—?

Lambda 40

Lambda AWS Architecture Database 40

Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on. I also wrote about these topics in detail for my recent [Systems Performance 2nd Edition] book.

Performance 52

Performance Latency Hardware Storage 52

Abhishek Tiwari

DECEMBER 1, 2017

At re:Invent 2017, Amazon Web Services (AWS) announced Amazon Time Sync Service which provides a highly accurate and reliable global time reference. This service is immediately available in all AWS regions. With the launch of Amazon Time Sync Service, AWS regions are equipped with similar GPS and atomic clocks.

AWS 55

AWS Google Database Virtualization 55

The Symphonia

MARCH 14, 2018

AWS announced a new service?—?the at re:Invent 2017. The general goal of SAR is to make it easier to distribute, and consume, applications that have been developed using AWS Serverless products, like Lambda. SAM itself is built on top of AWS CloudFormation ?—?Amazon’s the Serverless Application Repository (SAR)?—?at

Serverless 40

Serverless AWS Lambda Open Source 40

The Morning Paper

APRIL 7, 2019

You might also want to look at git-secrets which does this for AWS keys, and is extensible with additional patterns. In one case, we found what we believe to be AWS credentials for a major website relied upon by millions of college applications in the United States, possibly leaked by a contractor. For example: ( Enlarge ).

Google 76

Google AWS Website Government 76

Brendan Gregg

JULY 14, 2019

BPF (eBPF) tracing is a superpower that can analyze everything, and I'll show you how in my upcoming book BPF Performance Tools: Linux System and Application Observability , coming soon from Addison Wesley. I'm also grateful that my fiancée, Deirdré Straughan, could find the time outside of her AWS job to do technical copy-edit.

Systems 40

Systems Performance Education Technology 40

Brendan Gregg

FEBRUARY 28, 2023

This talk originated from my updates to Systems Performance 2nd Edition , and this was the first time I've given this talk in person! CXL in a way allows a custom memory controller to be added to a system, to increase memory capacity, bandwidth, and overall performance. Ford, et al., “TCP

Performance 52

Performance Latency Cache Virtualization 52

Percona

APRIL 12, 2023

Instead of relational (SQL) databases defined primarily through a hierarchy of related sets via tables and columns, their non-relational structure used a system of collections and documents. 2017: MongoDB goes public, trading as MDB. They came up with a horizontally scalable NoSQL database. Is MongoDB free for commercial use?

Open Source 98

Open Source Programming Database Servers 98

Brendan Gregg

APRIL 14, 2022

offer letter logo (2014) flame graphs (2014) eBPF tools (2014-2019) PMC analysis (2017) my pandemic-abandoned desk (2020); office wall I joined Netflix in 2014, a company at the forefront of cloud computing with an attractive work culture. Netflix has been the best job of my career so far, and I'll miss my colleagues and the culture.

Java 52

Java Cloud Cache AWS 52

O'Reilly

MARCH 23, 2023

It can pretend to be an operating system. All of these models are based on a technology called Transformers , which was invented by Google Research and Google Brain in 2017. And some of these things are mind blowing. It can explain code that you don’t understand, including code that has been intentionally obfuscated. It’s much more.

Google 138

Google Open Source Programming Code 138

Brendan Gregg

SEPTEMBER 25, 2021

CLI tools The Cassandra systems were EC2 virtual machine (Xen) instances. Microbenchmark os::javaTimeMillis() on both systems. Running this on the two systems saw similar results. Aftermath I provided details to AWS and Canonical, and then moved onto the other performance issues as part of the migration.

Speed 40

Speed Java AWS Virtualization 40

Tasktop

AUGUST 23, 2021

Try logging into your typical government-built system and you can immediately tell it was built differently. The UI is awful and the system crashes under heavy use. Even more common, we’re building data processing capabilities for a space or airborne system that won’t be deployed until a future date. . About the Author.

Software 52

Software Software Development Social Media 52

Enprowess

JANUARY 12, 2022

Twitter Lite PWA : It was integrated as the standard User Interface in 2017. IoT tracking systems. Use in traffic systems: Using IoT technology, delivery, traffic or transportation tracking, asset tracking, inventory control, customer management, and individual order tracking can be more cost-effective with the right tracking system.

Blockchain 59

Blockchain IoT Development Serverless 59

Smashing Magazine

JANUARY 7, 2019

You could go as far as evaluating a framework (or any JavaScript library) on Sacha Greif’s 12-point scale scoring system by exploring features, accessibility, stability, performance, package ecosystem, community, learning curve, documentation, tooling, track record, team, compatibility, security for example. Are web fonts optimized?

Performance 111

Performance Cache Network Metrics 111