Brendan Gregg

DECEMBER 31, 2017

My last talk for 2017 was at AWS re:Invent, on "How Netflix Tunes EC2 Instances for Performance," an updated version of my [2014] talk. WARNING: These tunables were developed in late 2017, for Ubuntu Xenial instances on EC2.** Storage I/O. We help where we can. My talk included a section on Linux kernel tunables, as follows.

Tuning 53

Tuning AWS Best Practices Network 53

Brendan Gregg

DECEMBER 30, 2017

My last talk for 2017 was at AWS re:Invent, on "How Netflix Tunes EC2 Instances for Performance," an updated version of my [2014] talk. WARNING: These tunables were developed in late 2017, for Ubuntu Xenial instances on EC2.** Storage I/O. We help where we can. My talk included a section on Linux kernel tunables, as follows.

Tuning 52

Tuning AWS Best Practices Network 52

Brendan Gregg

SEPTEMBER 5, 2017

Appliance manufacturers hire kernel engineers to develop custom features, including storage appliances. Large companies (like the one I work for) have OS teams to look after patching, releases, and performance.

Virtualization 75

Virtualization AWS Engineering Hardware 75

All Things Distributed

MAY 19, 2017

We believe the central constraint in high throughput data processing has moved from compute and storage to the network. Aurora brings a novel architecture to the relational database to address this constraint, most notably by pushing redo processing to a multi-tenant scaleout storage service, purpose-built for Aurora.

Database 88

Database Design Cloud Storage 88

Brendan Gregg

FEBRUARY 28, 2023

Titus, the Netflix container management platform, is now open source,” [link] Apr 2018 - [Cutress 19] Dr. [link] 2021 - [Trader 21] Tiffany Trader, “Cerebras Second-Gen 7nm Wafer Scale Engine Doubles AI Performance Over First-Gen Chip ,” [link] Apr 2021 - [Ghigoff 21] Yoann Ghigoff, et al., "BMC:

Performance 103

Performance Latency Cache Google 103

All Things Distributed

OCTOBER 17, 2016

In the remainder of 2016 and inq 2017, we will launch another four AWS regions in Canada, China, the United Kingdom, and France, adding another nine AZs to our global infrastructure footprint. It brings the worldwide total of AWS Availability Zones (AZs) to 38, and the number of regions globally to 14.

AWS 107

AWS Cloud Infrastructure Storage 107

Brendan Gregg

NOVEMBER 28, 2017

In this configuration, the AMI and boot is paravirt (PV), the kernel is making hypercalls instead of privileged instructions, and the system is using paravirt network and storage drivers. But not all workloads: some are network bound (proxies) and storage bound (databases). ## 5. The AMI and boot are now HVM.

Virtualization 40

Virtualization AWS Hardware Storage 40

Alex Russell

MARCH 6, 2021

TL;DR: A lot has changed since 2017 when we last estimated a global baseline resource per-page resource budget of 130-170KiB. A then-representative $200USD device had 4-8 slow (in-order, low-cache) cores, ~2GiB of RAM, and relatively slow MLC NAND flash storage. The Moto G4 , for example. Here begins our 2021 adventure. Hard Reset.

Performance 127

Performance Network Mobile Metrics 127

The Morning Paper

NOVEMBER 8, 2020

Back in 2017 the engineering team at Facebook had a problem. replacing Paxos with Raft), or they could be shims over external storage systems. A minimal Loglet needs to provide totally ordered, durable storage via the shared log API. Every little helps! The NativeLoglet.

Virtualization 126

Virtualization Latency Storage Systems 126

SQL Performance

SEPTEMBER 5, 2019

This table uses columnstore for its primary storage to produce batch mode execution later on. You can skip the WHERE clauses if you are running SQL Server 2017. OPTION ( MAXDOP 1 ) ; This time, only the COUNT_BIG aggregate achieves aggregate pushdown (SQL Server 2017 only). DROP TABLE IF EXISTS #T ; GO. c1 = @Start + N.

Performance 81

Performance C++ Storage Servers 81

All Things Distributed

OCTOBER 2, 2017

In support of Amazon Prime Day 2017, the biggest day in Amazon retail history, DynamoDB served over 12.9 million requests per second. All of this is done for you automatically and with zero downtime so that you can focus on your customers, your applications, and your business.

Internet 110

Internet Internet AWS Performance 110

Percona

JANUARY 17, 2019

now has a version which will support parallelism for SELECT queries (utilizing the read capacity of storage nodes underneath the Aurora cluster). Aurora PQ works by doing a full table scan (parallel reads are done on the storage level). sec) mysql> explain select count(*) from ontime where flightdate > '2017-01-01'G 1.

Cache 72

Cache C++ AWS Airlines 72

All Things Distributed

DECEMBER 12, 2018

In April 2017, Amazon Web Services announced that it would launch a new AWS infrastructure region Region in Sweden. We are excited to offer a comprehensive portfolio of services, from foundational technologies such as compute, storage, and networking to more advanced services such as containers and serverless computing.

AWS 119

AWS Cloud Games Serverless 119

Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on.

Performance 52

Performance Latency Hardware Storage 52

Percona

MARCH 30, 2023

Further, open source databases can be modified in infinite ways, enabling institutions to meet their specific needs for data storage, retrieval, and processing. Non-relational databases: Instead of tables, non-relational (NoSQL) databases use document-based data storage, column-oriented storage, and graph databases.

Open Source 107

Open Source Database Storage Scalability 107

Percona

AUGUST 17, 2023

StackOverflow statistics show that 26% of developers preferred it in 2017, 34% in 2019, and 40% in 2021. PostgreSQL has powerful and advanced features, including asynchronous replication, full-text searches of the database, and native support for JSON-style storage, key-value storage, and XML.

Open Source 98

Open Source Healthcare Social Media Database 98

Tim Kadlec

JANUARY 9, 2019

I recently profiled a page on a Pixel 2 (released in 2017), and an Alcatel 1x (released in 2018). Ballooning bandwidth and storage have fostered complacency that we can do without. The web’s growth is being pushed forward primarily by low-cost, underpowered Android devices that frequently struggle with today’s web.

Performance 111

Performance Energy Internet Internet 111

SQL Server According to Bob

MAY 19, 2019

For example, Azure SQL Database Edge can already use TSQL Predict Machine Learning (ML) and Column Storage. The reduction effort started when we shipped SQL Server 2017 ( [link].) SQL Server 2017. SQL Server 2017 uses function address comparisons which the enablement of COMDAT would break. SQL Server 2019. linker version.

IoT 61

IoT Servers Development Cache 61

Brendan Gregg

FEBRUARY 28, 2023

Titus, the Netflix container management platform, is now open source,” [link] , Apr 2018 [Cutress 19] Dr. [link] , 2021 [Trader 21] Tiffany Trader, “Cerebras Second-Gen 7nm Wafer Scale Engine Doubles AI Performance Over First-Gen Chip ,” [link] , Apr 2021 [Ghigoff 21] Yoann Ghigoff, et al.,

Performance 52

Performance Latency Cache Google 52

Percona

JULY 7, 2023

Backup and continuous archiving: PostgreSQL asynchronous replication relies on the write-ahead log (WAL) stream, and it’s vital to have centralized storage that all nodes can access. StackOverflow statistics show that 26% of developers preferred it in 2017, 34% in 2019, and 40% in 2021.

Availability 98

Availability Open Source Architecture Database 98

SQL Performance

AUGUST 27, 2019

TPC-H Benchmark Results with SQL Server 2017. HPE has already submitted a new official TPC-H result on SQL Server 2017 for a single-socket, HPE Proliant DL325 Gen10 1U server that was 743,750 QphH@1000GB. TPC-E Benchmark Results with SQL Server 2017. AMD Rome Second Generation EPYC Review: 2x 64-core Benchmarked.

Servers 52

Servers Benchmarking Virtualization Architecture 52

SQL Performance

FEBRUARY 3, 2020

The downside to the latter is that, in SQL Server 2017, you cannot run this online… but you will be able to in 2019. If you want to maximize compression with zero disruption , you can approach 90% storage reduction without any disruption at all, using page compression online.

Speed 113

Speed Processing Servers Storage 113

SQL Performance

MARCH 17, 2020

It comes set to SQL Server 2008 compatibility (level 100), but we will start with a more modern setting of SQL Server 2017 (level 140): ALTER DATABASE StackOverflow2013 SET COMPATIBILITY_LEVEL = 140 ; The tests were performed on my laptop using SQL Server 2019 CU 2. The serial aggregate push down result of 13ms is equally impressive.

Servers 86

Servers Tuning Database Efficiency 86

Percona

JUNE 4, 2023

This blog post was originally published in November 2017 and was updated in June 2023. MariaDB retains compatibility with MySQL, offers support for different programming languages, including Python, PHP, Java, and Perl, and works with all major open source storage engines such as MyRocks, Aria, and InnoDB. 3 GA 23 May 2017 10.2.6

Open Source 52

Open Source Servers Storage Scalability 52

Brendan Gregg

APRIL 14, 2022

offer letter logo (2014) flame graphs (2014) eBPF tools (2014-2019) PMC analysis (2017) my pandemic-abandoned desk (2020); office wall I joined Netflix in 2014, a company at the forefront of cloud computing with an attractive work culture. Netflix has been the best job of my career so far, and I'll miss my colleagues and the culture.

Java 52

Java Cloud Cache AWS 52

SQL Performance

APRIL 8, 2019

Back on December 5, 2017, Microsoft announced that they were using AMD EPYC 7551 processors in their storage-optimized Lv2-Series virtual machines. They feature low latency, local NVMe storage that can directly leverage the 128 PCIe 3.0 Microsoft handles the storage account management for you when you use Azure Managed Disks.

Azure 42

Azure Virtualization Storage Benchmarking 42

Percona

SEPTEMBER 1, 2023

This blog was originally published in October 2017 and was updated in September 2023. 3) Storage engine limitations There are a few storage engine limitations that can be a bottleneck in your use case. Database architects working with MongoDB encounter specific challenges related to database systems and system growth.

Best Practices 98

Best Practices Traffic Database Servers 98

SQL Performance

MAY 1, 2020

I was a bit perplexed by this issue, since the system was certainly no slouch — plenty of cores, 3TB of memory, and XtremIO storage. Everything was on SQL Server 2017, so why was this setting so prevalent? writes/sec average throughput: 46.96 Here is a sample of the results: Partial results from CMS query.

C++ 58

C++ Database Servers Storage 58

SQL Performance

APRIL 23, 2019

Columnstore storage. To understand this, it is necessary to first review how columnstore storage works at a high level: A compressed row group contains a column segment for each column. Value encoding can reduce the number of bits required for storage by translating raw values using a base offset and magnitude modifier.

Servers 67

Servers Testing Metrics Storage 67

Particular Software

MARCH 15, 2021

In it, we now offer a new tool for generating scripts for NServiceBus tables, enable support for SQL Server Always Encrypted, have begun to expand our testing to MariaDB and Amazon Aurora, have solved a problem with ever-expanding outbox storage, and added controls for managing transaction isolation level.

Transportation 52

Transportation Azure Database Servers 52

Alex Russell

OCTOBER 22, 2017

— Alex Russell (@slightlylate) October 4, 2017. We now have everything we need to create a ballpark perf budget for a product in 2017. Time required to read bytes from disk (it’s not zero, even on flash-based storage!). Maybe "ambush by JS"? TTI under 2 seconds for subsequent loads. First Load.

Performance 74

Performance Network Benchmarking Mobile 74

Enprowess

JANUARY 12, 2022

Other benefits: It has other benefits like a Quicker launch to the market, Easier distribution, saving device power and storage, seamless maintenance, and updating. Twitter Lite PWA : It was integrated as the standard User Interface in 2017. Blockchain technology is an encrypted database storage system. Famous PWA Use Examples.

Blockchain 59

Blockchain IoT Development Serverless 59

Percona

AUGUST 5, 2023

This blog was originally published in July 2017 and was updated in August 2023. Otherwise, the storage engine does a scatter-gather and queries ALL partitions in a UNION that is not concurrent. This method distributes data evenly across partitions to achieve balanced storage and optimal query performance.

Strategy 40

Strategy Storage Database Best Practices 40

Alex Russell

APRIL 29, 2021

Chrome has missed several APIs for 3+ years: Storage Access API. WebGL 2 launched for other platforms on Chrome and Firefox in 2017. Where Chrome Has Lagged. It's healthy for engines to have different priorities, and that can lead every browser to avoiding certain features. The underlying graphics capabilities from OpenGL ES 3.0

Media 145

Media Games Education Engineering 145

SQL Performance

AUGUST 23, 2019

The details given here apply to SQL Server 2017 — the most recently released version at the time of writing. This article is about batch mode bitmaps, which have a very different implementation. There have been major improvements since the first appearance of the batch mode execution engine in SQL Server 2012. The Query Optimizer.

Servers 75

Servers Engineering Efficiency Retail 75

The Morning Paper

JANUARY 19, 2020

STELLA: report from the SNAFU-catchers workshop on coping with complexity , Woods 2017, Coping with Complexity workshop. Today’s choice is a report from a 2017 workshop convened with that title, and recommended to me by John Allspaw – thank you John! Take a trivial example with three components A, B, and C.

C++ 50

C++ Systems Software Software 50

Tasktop

JUNE 25, 2020

When I joined Tasktop back in October of 2017, Tasktop was doing really well. Flow Fabric feeds a schemaless immutable storage of digital artifacts, across disciplines and functions. In the grand tradition of our sages, I’ve been contemplating existential questions about Tasktop and wrote a Bar Mitzvah speech of my own. .

Metrics 52

Metrics Education Innovation Software 52