High Scalability

JANUARY 24, 2019

For the first time in history, a VC-backed startup in 2018 was more likely to sell to a PE buyer than a fellow VC-backed company. The Smalltalk group is astonishingly insular, almost childlike, but is just now opening up, looking at other systems with open-eyed curiosity and fascination. 10% : Netflix captured screen time in US; 8.3

Internet 29

Internet Internet Scalability Games 29

High Scalability

MARCH 22, 2019

µs of replication latency on lossy Ethernet, which is faster than or comparable to specialized replication systems that use programmable switches, FPGAs, or RDMA.". BrentToderian : What city went from 14% of all trips by bike in 2001, to 22% by 2012, then leaped to 30% in 3 years by 2015, & 35% by 2018? We achieve 5.5

Internet 134

Internet Internet Scalability Wireless 134

Brendan Gregg

JULY 4, 2021

I also wrote about these topics in detail for my recent [Systems Performance 2nd Edition] book. TCP Extensions for Multipath Operation with Multiple Addresses,” [link] Mar 2020 - [Gregg 20] Brendan Gregg, “Systems Performance: Enterprise and the Cloud, Second Edition,” Addison-Wesley, 2020 - [Hruska 20] Joel Hruska, “Intel Demos PCIe 5.0

Performance 121

Performance Latency Hardware Storage 121

All Things Distributed

NOVEMBER 13, 2018

ENEL is using AWS to transform its entire business, closing all of their data centers by 2018, migrating workloads from over 6,000 on-premises servers onto AWS in nine months, and using AWS IoT services to better manage and understand energy consumption. ENEL is one of the leading energy operators in the world. million unique visits.

AWS 164

AWS Energy Automotive Traffic 164

Brendan Gregg

FEBRUARY 28, 2023

This talk originated from my updates to [Systems Performance 2nd Edition], and this was the first time I've given this talk in person! CXL in a way allows a custom memory controller to be added to a system, to increase memory capacity, bandwidth, and overall performance. Ford, et al., “TCP

Performance 103

Performance Latency Cache Virtualization 103

All Things Distributed

JUNE 20, 2017

The new AWS Asia Pacific (Hong Kong) Region will have three Availability Zones and be ready for customers for use in 2018. This enables customers to serve content to their end users with low latency, giving them the best application experience. Over the past decade, we have seen tremendous growth at AWS.

AWS 126

AWS Logistics Cloud Social Media 126

ACM Sigarch

MARCH 18, 2019

Computer systems, from the Internet-of-Things devices to datacenters, are complex and optimizing them can enhance capability and save money. Developing simulators, however, is time-consuming and requires a great deal of infrastructure development regarding a prospective system. Answered in Part 2.). Bottleneck Analysis.

Systems 61

Systems Latency Performance Analytics 61

All Things Distributed

APRIL 4, 2017

The new AWS EU (Stockholm) Region will have three Availability Zones and will be ready for customers to use in 2018. This enables customers to serve content to their end users with low latency, giving them the best application experience. Over the past decade, we have seen tremendous growth at AWS.

AWS 107

AWS Airlines Latency Games 107

ACM Sigarch

MARCH 25, 2019

How many buffers are needed to track pending requests as a function of needed bandwidth and expected latency? Can one both minimize latency and maximize throughput for unscheduled work? The M/M/1 queue will show us a required trade-off among (a) allowing unscheduled task arrivals, (b) minimizing latency, and (c) maximizing throughput.

Systems 54

Systems Latency Performance C++ 54

The Netflix TechBlog

DECEMBER 17, 2019

In order to be supported, a database is required to fulfill a set of features that are commonly available in systems like MySQL, PostgreSQL, MariaDB, and others. We want to support these systems as a source so that they can provide their data for further consumption. Some of DBLog’s features are: Processes captured log events in-order.

Database 221

Database Traffic Transportation Open Source 221

The Netflix TechBlog

DECEMBER 17, 2019

In order to be supported, a database is required to fulfill a set of features that are commonly available in systems like MySQL, PostgreSQL, MariaDB, and others. We want to support these systems as a source so that they can provide their data for further consumption. Some of DBLog’s features are: Processes captured log events in-order.

Database 206

Database Traffic Transportation Open Source 206

Abhishek Tiwari

DECEMBER 27, 2017

There are several emerging data trends that will define the future of ETL in 2018. In 2018, we anticipate that ETL will either lose relevance or the ETL process will disintegrate and be consumed by new data architectures. To achieve this, organizations are utilizing distributed messaging system such as Apache Kafka.

Big Data 49

Big Data Artificial Intelligence Storage Hardware 49

All Things Distributed

SEPTEMBER 25, 2017

This Region will consist of three Availability Zones at launch, and it will provide even lower latency to users across the Middle East. Plus another AWS GovCloud (US) Region in the United States is coming online by the end of 2018. The BIPA has moved their Learning Management System to AWS, reducing costs by over 90%.

Cloud 119

Cloud Education Energy Government 119

All Things Distributed

DECEMBER 12, 2018

They can run applications in Sweden, serve end users across the Nordics with lower latency, and leverage advanced technologies such as containers, serverless computing, and more. We help Supercell to quickly develop, deploy, and scale their games to cope with varying numbers of gamers accessing the system throughout the course of the day.

AWS 119

AWS Cloud Games Serverless 119

Brendan Gregg

AUGUST 29, 2021

1072-aws (xxx) 12/18/2018 _x86_64_ (16 CPU) avg-cpu: %user %nice %system %iowait %steal %idle 5.03 avg-cpu: %user %nice %system %iowait %steal %idle 14.81 avg-cpu: %user %nice %system %iowait %steal %idle 14.86 avg-cpu: %user %nice %system %iowait %steal %idle 14.95 100.00 ^C. Tracing block device I/O. . ## 5.

Cache 78

Cache C++ AWS Java 78

Brendan Gregg

JULY 4, 2021

I also wrote about these topics in detail for my recent [Systems Performance 2nd Edition] book. TCP Extensions for Multipath Operation with Multiple Addresses,” [link] Mar 2020 - [Gregg 20] Brendan Gregg, “Systems Performance: Enterprise and the Cloud, Second Edition,” Addison-Wesley, 2020 - [Hruska 20] Joel Hruska, “Intel Demos PCIe 5.0

Performance 52

Performance Latency Hardware Storage 52

Brendan Gregg

AUGUST 18, 2019

For example, iostat(1), or a monitoring agent, may tell you your average disk latency, but not the distribution of this latency. For smaller environments, it can be of more use helping eliminate latency outliers. system(".") Block I/O latency as a histogram. 6 * 7 * Copyright 2018 Netflix, Inc.

Latency 57

Latency C++ Cache Programming 57

Brendan Gregg

OCTOBER 7, 2018

Created by Alastair Robertson , bpftrace is an open source high-level tracing front-end that lets you analyze systems in custom ways. eBPF (extended Berkeley Packet Filter) is in the Linux kernel and is the new hotness in systems engineering. Here's key differences as of August 2018: Type DTrace bpftrace. Attaching 2 probes. ^C

C++ 40

C++ Virtualization Programming Latency 40

Brendan Gregg

FEBRUARY 28, 2023

This talk originated from my updates to Systems Performance 2nd Edition , and this was the first time I've given this talk in person! CXL in a way allows a custom memory controller to be added to a system, to increase memory capacity, bandwidth, and overall performance. Ford, et al., “TCP

Performance 52

Performance Latency Cache Virtualization 52

Smashing Magazine

OCTOBER 31, 2023

The truth is that the two tools were fairly distinct until PSI was updated in 2018 to use Lighthouse reporting. If throttling is applied at the operating system level , then the metrics match what a real user with those network conditions would experience. The accuracy of observed data depends on how the test environment is set up.

Speed 117

Speed Google Website Metrics 117

Alex Russell

APRIL 29, 2021

After 20 years of neck-in-neck competition, often starting from common code lineages, there just isn't that much left to wring out of the system. Consistent improvement is the name of the game, and it can still have positive impacts, particularly as users lean on the system more heavily over time. Pointer Lock. Media Recorder.

Media 145

Media Games Education Engineering 145

KeyCDN

SEPTEMBER 12, 2019

This can be a problem in a lot of content management systems such as WordPress or Magento, because theme developers tend to scale images down in responsive themes using CSS. The main reason is because it decreases the latency to the user where they are located by serving your images from a POP physically closest to them.

Social Media 106

Social Media Media Google Website 106

SQL Performance

FEBRUARY 26, 2020

Azure SQL Database Managed Instance became generally available in late 2018. Organizations are taking advantage of having managed backups, lots of built-in security features, an uptime SLA of 99.99%, and an always up-to-date environment where they are no longer responsible for patching SQL Server or the operating system.

Azure 63

Azure Performance Storage Latency 63

Abhishek Tiwari

DECEMBER 27, 2017

In 2018, a widespread adaptation of Kubernetes for big data processing is anitcipated. A good example will be something like Pachyderm [8] [9] - a relatively young project which implements a distributed Pachyderm File System (PFS) and a data-aware scheduler Pachyderm Pipeline System (PPS) on top of Kubernetes.

Big Data 63

Big Data Storage Benchmarking Hardware 63

Speed Curve

FEBRUARY 5, 2024

Charlie Vazac introduced server timing in a Performance Calendar post circa 2018. Latency – How much time does it take to deliver a packet from A to B. In practice, it looks something like this: server-timing: processing_time; dur=123.4; desc="Time to process request at origin" NOTE: This is not a new API.

Servers 64

Servers Cache Retail Benchmarking 64

Alex Russell

JANUARY 30, 2024

If you or your company are able to generate a credible worldwide latency estimate in the higher percentiles for next year's update, please get in touch. Since 2010, volumes have been on a slow downward glide path , shrinking from ~350MM per year in a decade ago to ~260MM in 2018. was a system properly localised?

Performance 98

Performance Network Mobile Speed 98

HammerDB

OCTOBER 12, 2018

As with the previous guides as an Intel employee (#IAMINTEL) the examples are taken from a MySQL 8 on Linux on Intel system and the approach is the same for whatever system you are testing although some of the settings you see may be different. Similarly for this guide MySQL can be swapped for a mySQL based databases such as MariaDB.

Best Practices 40

Best Practices Scalability Performance C++ 40

HammerDB

MAY 5, 2019

However in the Skylake microarchitecture (you can see a list of CPUs here ) the PAUSE instruction changed and in the documentation it says “the latency of the PAUSE instruction in prior generation microarchitectures is about 10 cycles, whereas in Skylake microarchitecture it has been extended to as many as 140 cycles.”

Testing 49

Testing Tuning Latency Storage 49

Alex Russell

OCTOBER 22, 2017

Tools and CI systems help them monitor progress & prevent regressions. Some systems show median RTTs near ~100ms for 3G users. It simulates a link with a 400ms RTT and 400-600Kbps of throughput (plus latency variability and simulated packet loss). Performance budgets are set early in the life of the project.

Performance 74

Performance Network Benchmarking Mobile 74

HammerDB

OCTOBER 8, 2018

This post gives a HOWTO guide on system configuration for achieving top levels of performance with the HammerDB PostgreSQL TPC-C test. Firstly for system choice a 2 socket system is optimal for PostgreSQL OLTP performance at the time of writing. maximum transition latency: Cannot determine or is not supported.

Best Practices 40

Best Practices Scalability Performance Hardware 40

HammerDB

NOVEMBER 5, 2018

Additionally for the log disk component it is latency for an individual write that is crucial rather than the total I/O bandwidth. SQL> alter system flush buffer_cache; System altered. These parameters can be set as follows: SQL> alter system set commit_logging='BATCH' scope=spfile; System altered.

Benchmarking 40

Benchmarking Database C++ Virtualization 40

Adrian Cockcroft

NOVEMBER 18, 2019

AWS Developer Relations on how the shift from Robot Operating System (ROS) 1 to ROS 2 will change the landscape for all robot lovers. OPN304 Learnings from migrating a service from JDK 8 to JDK 11 AWS Lambda improved latency by migrating to JDK 11 with Amazon Corretto.

Open Source 40

Open Source AWS Lambda Serverless 40

Adrian Cockcroft

NOVEMBER 18, 2019

AWS Developer Relations on how the shift from Robot Operating System (ROS) 1 to ROS 2 will change the landscape for all robot lovers. OPN304 Learnings from migrating a service from JDK 8 to JDK 11 AWS Lambda improved latency by migrating to JDK 11 with Amazon Corretto.

Open Source 40

Open Source AWS Lambda Serverless 40

Brendan Gregg

AUGUST 29, 2021

1072-aws (xxx) 12/18/2018 _x86_64_ (16 CPU) avg-cpu: %user %nice %system %iowait %steal %idle 5.03 avg-cpu: %user %nice %system %iowait %steal %idle 14.81 avg-cpu: %user %nice %system %iowait %steal %idle 14.86 avg-cpu: %user %nice %system %iowait %steal %idle 14.95 41 33680 53408 0 78 100.0%

Cache 40

Cache C++ AWS Systems 40

IO River

NOVEMBER 2, 2023

However, consumers often prioritize availability in many systems. Furthermore, there are many recognized standards to measure the availability of a service or system, and the most common one is to measure it as a percentage."Five minutes of downtime per year, which means the system is almost always operational.

Availability 52

Availability Social Media Traffic Games 52

IO River

NOVEMBER 2, 2023

However, consumers often prioritize availability in many systems. Furthermore, there are many recognized standards to measure the availability of a service or system, and the most common one is to measure it as a percentage."Five minutes of downtime per year, which means the system is almost always operational.

Availability 52

Availability Social Media Traffic Games 52

Percona

JULY 1, 2023

This post was originally published in July 2018 and was updated in July 2023. It efficiently manages read and write operations, optimizes data access, and minimizes contention, resulting in high throughput and low latency to ensure that applications perform at their best. What are the differences between Aurora and RDS?

AWS 52

AWS Database Serverless Storage 52