Brendan Gregg

JULY 15, 2019

BPF (eBPF) tracing is a superpower that can analyze everything, and I'll show you how in my upcoming book BPF Performance Tools: Linux System and Application Observability , coming soon from Addison Wesley. A time where you can pose arbitrary questions of the system, and it can answer them.

Systems 111

Systems Performance Education Technology 111

Brendan Gregg

APRIL 14, 2022

PMC analysis (2017). I hope my last two books, [Systems Performance 2nd Ed] and [BPF Performance Tools] serve Netflix well in my absence and everyone else who reads them. Netflix has been the best job of my career so far, and I'll miss my colleagues and the culture. offer letter logo (2014). flame graphs (2014). eBPF tools (2014-2019).

Java 142

Java Cloud Cache AWS 142

Dynatrace

MAY 6, 2021

Operating systems support. Future Dynatrace Managed operating systems support changes. The following operating systems will no longer be supported starting 01 September 2021. Linux: Amazon Linux AMI 2017.x. The following operating systems will no longer be supported starting 01 October 2021. x86 (64bit-only).

Operating System 246

Operating System AWS Metrics Storage 246

Dynatrace

MARCH 2, 2023

Focusing on the fundamentals Grabner recalled a 2017 tweet from Hightower that outlined how his technology career gathered steam once he started to “focus on the fundamentals.” The whole goal is, we want to reduce the amount that a human needs to give to the system in order to make it do what you want it to do,” he said.

Best Practices 246

Best Practices Internet Internet Cloud 246

O'Reilly

FEBRUARY 18, 2020

Up until 2017, the ML+AI topic had been amongst the fastest growing topics on the platform. After several years of steady climbing—and after outstripping Java in 2017—Python-related interactions now comprise almost 10% of all usage. Not necessarily: Java-related searches increased by 5% between 2017 and 2018. Coincidence?

Software Architecture 145

Software Architecture DevOps Data Engineering Architecture 145

Brendan Gregg

JULY 4, 2021

I also wrote about these topics in detail for my recent [Systems Performance 2nd Edition] book. TCP Extensions for Multipath Operation with Multiple Addresses,” [link] Mar 2020 - [Gregg 20] Brendan Gregg, “Systems Performance: Enterprise and the Cloud, Second Edition,” Addison-Wesley, 2020 - [Hruska 20] Joel Hruska, “Intel Demos PCIe 5.0

Performance 121

Performance Latency Hardware Storage 121

Brendan Gregg

SEPTEMBER 5, 2017

What follows are topics that may be of interest to anyone looking to migrate their systems and skillset: scan these to find topics that interest you. ## ZFS ZFS is available for Linux via the [zfsonlinux] and [OpenZFS] projects, and more recently was included in Canonical's Ubuntu Linux distribution: Ubuntu Xenial 16.04 LTS (April 2016).

Virtualization 75

Virtualization AWS Engineering Hardware 75

Brendan Gregg

APRIL 27, 2023

Keeping overhead low can require tradeoffs in other areas: tcpdump(8), for example, will drop packets if the system is overloaded, resulting in incomplete visibility. Long ago I encountered systems which met strict security auditing requirements with the following behavior: If the kernel could not log an event, it would immediately **halt**!

Open Source 138

Open Source Monitoring Engineering Design 138

The Netflix TechBlog

OCTOBER 19, 2020

which is difficult when troubleshooting distributed systems. Troubleshooting a session in Edgar When we started building Edgar four years ago, there were very few open-source distributed tracing systems that satisfied our needs. Investigating a video streaming failure consists of inspecting all aspects of a member account.

Infrastructure 238

Infrastructure Transportation Storage Open Source 238

All Things Distributed

MARCH 30, 2017

This weekend I am travelling to Australia for the first AWS Summit of 2017. One topic that always gets me excited is how to take computer science research and implement it in production systems. real systems do not fail by stopping in a nice and clean way). At times, it makes research unsuitable for production (e.g.

Systems 77

Systems AWS Design Technology 77

Brendan Gregg

DECEMBER 31, 2017

My last talk for 2017 was at AWS re:Invent, on "How Netflix Tunes EC2 Instances for Performance," an updated version of my [2014] talk. A video of the talk is on youtube : The slides are on slideshare : I love this talk as I get to share more about what the Performance and Operating Systems team at Netflix does, rather than just my work.

Tuning 53

Tuning AWS Best Practices Network 53

The Netflix TechBlog

OCTOBER 28, 2021

During earlier years of my career, I primarily worked as a backend software engineer, designing and building the backend systems that enable big data analytics. I also built online serving systems and microservices powering Walmart’s e-commerce. At this conference, I also got to know my current team: Consolidated Logging (CL).

Data Engineering 181

Data Engineering Engineering Big Data Software Engineering 181

Brendan Gregg

DECEMBER 30, 2017

My last talk for 2017 was at AWS re:Invent, on "How Netflix Tunes EC2 Instances for Performance," an updated version of my [2014] talk. A video of the talk is on youtube : The slides are on slideshare : I love this talk as I get to share more about what the Performance and Operating Systems team at Netflix does, rather than just my work.

Tuning 52

Tuning AWS Best Practices Network 52

Dynatrace

FEBRUARY 25, 2021

since 2017, and many customers have used it with great success while we collected requirements for the next iteration of our Lambda extension. Lambda functions often serve as a gluing tier or a gateway from on-premises systems to AWS services; they’re almost never standalone. Dynatrace has offered a Lambda code module for Node.js

Lambda 278

Lambda AWS Monitoring Serverless 278

The Netflix TechBlog

DECEMBER 10, 2019

By Adam Wang , Andy Swan , Raja Senapati , Shilpa Jois , Anjali Chablani , Deepa Krishnan , Vidya Sundaram , and Casey Wilms You can also check out highlights from our past events: May 2019 , November 2018 , March 2018 , August 2017 , January 2017 , May 2016 , November 2015 , March 2015 , February 2014 & August 2014.

Innovation 130

Innovation Mobile Technology Technology 130

The Netflix TechBlog

DECEMBER 10, 2019

By Adam Wang , Andy Swan , Raja Senapati , Shilpa Jois , Anjali Chablani , Deepa Krishnan , Vidya Sundaram , and Casey Wilms You can also check out highlights from our past events: May 2019 , November 2018 , March 2018 , August 2017 , January 2017 , May 2016 , November 2015 , March 2015 , February 2014 & August 2014.

Innovation 130

Innovation Mobile Technology Technology 130

Brendan Gregg

FEBRUARY 28, 2023

This talk originated from my updates to [Systems Performance 2nd Edition], and this was the first time I've given this talk in person! CXL in a way allows a custom memory controller to be added to a system, to increase memory capacity, bandwidth, and overall performance. Ford, et al., “TCP

Performance 103

Performance Latency Cache Virtualization 103

Dynatrace

JUNE 15, 2023

If your application runs on servers you manage, either on-premises or on a private cloud, you’re responsible for securing the application as well as the operating system, network infrastructure, and physical hardware. What are some key characteristics of securing cloud applications?

Cloud 245

Cloud Open Source Architecture Monitoring 245

SQL Server According to Bob

OCTOBER 26, 2017

Please check out these new resources for SQL Server 2017 and SQL Server on Linux and Docker: Brk2045 upgrade sql server 2017 (on prem, iaa-s and paas) from Bob Ward. To find out what’s different and new about SQL Server 2017, go to this documentation page: What’s new in SQL Server 2017. SQL Server on Docker.

Servers 40

Servers Azure Open Source Database 40

SQL Server According to Bob

SEPTEMBER 25, 2017

Scott Guthrie and Rohan Kumar officially made the announcement today at Ignite that SQL Server 2017 will be generally available on October 2nd, 2017. Andrew Brust also wrote a very nice article detailing out all the great new features that come with SQL Server 2017. Principal Architect, Microsoft Database Systems Group.

Servers 40

Servers Availability Tuning Database 40

The Netflix TechBlog

MAY 1, 2019

In late 2017, we received some critical feedback from the brothers on the Stranger Things 2 audio mix: in some scenes, there was a reduced sense of where sounds are located in the 5.1-channel We want your experience to be brilliant even if you aren’t listening with a state-of-the-art home theater system. We began streaming 5.1

Engineering 274

Engineering Network Media Entertainment 274

All Things Distributed

SEPTEMBER 29, 2016

The new region in France will be ready for customers to use in 2017. Within the thousands of businesses using AWS in France, we count enterprises such as Schneider Electric, Lafarge and Dassault Systemes as customers as well as CAC40, multinational bank, Societe Generale Group. Ces dix dernières années, AWS s’est fortement développé.

AWS 158

AWS IoT Internet Internet 158

SQL Server According to Bob

MARCH 18, 2018

The changes made in SQL Server 2017 CU4 improve resource usage on smaller machines, such as a VM or Container used for testing and development. Studying the code I found that the ‘default max worker threads’ calculation did not take into account the RAM on the system. What Is A Small System? CLR Garbage Collection.

Servers 40

Servers Best Practices Software Engineering Operating System 40

All Things Distributed

SEPTEMBER 29, 2016

Voice makes access to digital services far more inclusive than traditional screen-based interaction, for example, an aging population may be much more comfortable interacting with voice-based systems than through tablets or keyboards. However, it is still Day One, and a lot of innovation is underway in this world.

Artificial Intelligence 111

Artificial Intelligence Healthcare AWS Government 111

Scalegrid

APRIL 2, 2019

PostgreSQL is an open source relational database system that has soared in popularity over the past 30 years from its active, loyal, and growing community. different database systems, including PostgreSQL itself. So what makes PostgreSQL so special, and how is it being used today? are using public clouds.

Database 185

Database Cloud Open Source Azure 185

Brendan Gregg

NOVEMBER 28, 2017

In this configuration, the AMI and boot is paravirt (PV), the kernel is making hypercalls instead of privileged instructions, and the system is using paravirt network and storage drivers. Xen AWS 2017 In 2015, AWS launched c4, which used hardware virtualization for EBS volumes. 0% performance overhead.

Virtualization 40

Virtualization AWS Hardware Storage 40

Brendan Gregg

SEPTEMBER 5, 2021

They claimed that the ZFS file system was consuming 30% of CPU capacity. I summarized this case study at [Kernel Recipes] in 2017 and have shared the full story here. ## 1. These included a breakdown of CPU time into percentages for "usr" (user: applications) and "sys" (system: the kernel). Ahh, it makes sense now.

Cache 141

Cache Cloud Code Java 141

Dynatrace

OCTOBER 17, 2019

Everything as Code can be described as a methodology or practice which consists of extending the idea of how applications are treated as code and applying these concepts to all other IT components like operating systems, network configurations, and pipelines. We went from managing five EC2 instances in 2011 to around 1,000 in 2017.

Code 167

Code Infrastructure DevOps Cloud 167

Brendan Gregg

OCTOBER 28, 2017

For my first trip to Paris I gave the closing keynote at [EuroBSDcon 2017] on performance methodologies, using FreeBSD 11.1 In the past I've shared similar methodologies applied to other operating systems, and finished porting them to BSD for this talk. EuroBSDcon 2017]: [link] [thread state analysis]: [link] [tstates.d]: [link]

Systems 47

Systems Performance C++ Operating System 47

The Netflix TechBlog

JULY 1, 2019

Surface Cache Surface cache is a reserved pool in main memory (or separate graphics memory on a minority of systems) that the Netflix app uses for storing textures (decoded images and cached resources). those running the 2017 SDK?—?also The SNG files have a relatively large download size at ~1.5MB each.

Cache 167

Cache Design Testing Development 167

The Netflix TechBlog

JANUARY 22, 2021

channel surround sound in 2010, Dolby Atmos in 2017 , and adaptive bitrate audio in 2019. By using metadata and decode-side gain to normalize loudness, Netflix leverages xHE-AAC to minimize the total number of gain stages in the end-to-end system, maximizing audio quality. We began streaming 5.1

Programming 275

Programming Mobile Testing Network 275

All Things Distributed

NOVEMBER 13, 2018

Since then, AWS has added another PoP in Palermo in 2017. This includes system integration (SI) and independent software vendor (ISV) partners who have built cloud practices and innovative technology solutions using AWS. In 2012, Amazon opened its first Italian office and its first Italian point of presence (PoP) based in Milan.

AWS 164

AWS Energy Automotive Traffic 164

All Things Distributed

MAY 19, 2017

We decided to use our inventive powers to design and build a new database engine that would give database systems such as MySQL and PostgreSQL reliability and performance at scale, at a level that could serve even the most demanding OLTP applications.

Database 89

Database Design Cloud Storage 89

The Netflix TechBlog

SEPTEMBER 13, 2021

Written by Jose Fernandez , Arthur Gonigberg , Julia Knecht , and Patrick Thomas In 2017, Netflix Studios was hitting an inflection point from a period of merely rapid growth to the sort of explosive growth that throws “how do we scale?” into every conversation. Examples include legacy mTLS and various webhook schemes?.

Internet 227

Internet Internet Cloud Traffic 227

All Things Distributed

OCTOBER 25, 2018

In 2017, we brought the Amazon Global Network to Africa, through AWS Direct Connect. With the AWS based system, Exatype can do this at a fraction of the cost. That's where we built many pioneering networking technologies, our next-generation software for customer support, and the technology behind our compute service, Amazon EC2.

AWS 167

AWS Cloud Education Programming 167

The Morning Paper

NOVEMBER 8, 2020

Back in 2017 the engineering team at Facebook had a problem. While ultimately this new system should be able to take advantage of the latest advances in consensus for improved performance, that’s not realistic given a 6-9 month in-production target. replacing Paxos with Raft), or they could be shims over external storage systems.

Virtualization 126

Virtualization Latency Storage Systems 126

SQL Performance

JANUARY 23, 2020

The queues component of our methodology comes from Performance Monitor counters, which provide a view of system performance from a resource standpoint.". However, some seem to have missed Davidson's point regarding the importance of resources and rely almost entirely on waits to present a picture of query performance and system health.

Tuning 115

Tuning Servers Monitoring Hardware 115