The Netflix TechBlog

NOVEMBER 15, 2022

Rendering is the final step in the VFX creation process, and processing on a render farm often can take several hours to complete just a single frame of a show, even when this process runs on the latest high-end hardware. Additionally, Conductor supports render management systems?—?including

Cloud 293

Cloud Entertainment AWS Infrastructure 293

The Morning Paper

MAY 12, 2019

An open-source benchmark suite for microservices and their hardware-software implications for cloud & edge systems Gan et al., Systems built with lots of microservices have different operational characteristics to those built from a small number of monoliths, we’d like to study and better understand those differences.

Open Source 61

Open Source Hardware Benchmarking Systems 61

Scalegrid

JANUARY 25, 2024

Effective management of memory stores with policies like LRU/LFU proactive monitoring of the replication process and advanced metrics such as cache hit ratio and persistence indicators are crucial for ensuring data integrity and optimizing Redis’s performance. Cache Hit Ratio The cache hit ratio represents the efficiency of cache usage.

Metrics 130

Metrics Monitoring Latency Cache 130

Scalegrid

APRIL 16, 2020

This is why our BYOC pricing is less than our Dedicated Hosting pricing, as the costs listed for BYOC are only what you pay for ScaleGrid and don’t include your hardware costs. This becomes really important for cache solutions like Redis™. Where to host your cloud database? Expert Tip. Security Groups. More tips for you.

Cloud 242

Cloud Azure AWS Database 242

The Morning Paper

MAY 23, 2019

Compress objects, not cache lines: an object-based compressed memory hierarchy Tsai & Sanchez, ASPLOS’19. Existing cache and main memory compression techniques compress data in small fixed-size blocks, typically cache lines. Hotpads is a hardware-managed hierarchy of scratchpad-like memories called pads.

Cache 61

Cache Benchmarking Hardware Java 61

Percona

MARCH 23, 2023

Hardware Memory The amount of RAM to be provisioned for database servers can vary greatly depending on the size of the database and the specific requirements of the company. By caching hot datasets, indexes, and ongoing changes, InnoDB can provide faster response times and utilize disk IO in a much more optimal way.

Performance 130

Performance Hardware Tuning Storage 130

DZone

FEBRUARY 21, 2021

CPU consumption in Unix/Linux operating systems are studied using 8 different metrics: User CPU time , System CPU time , nice CPU time , Idle CPU time , Waiting CPU time , Hardware Interrupt CPU time , Software Interrupt CPU time , and Stolen CPU time. In this article, let's study ‘nice CPU time’. What Is ‘nice’ CPU Time?

Operating System 133

Operating System Hardware Metrics Systems 133

Brendan Gregg

MARCH 16, 2024

The problem is that this system has a default libc that has been compiled without frame pointers, so any stack walking stops at the libc layer, producing a partial stack that's missing the application frames. This is pretty common and usually goes unnoticed as the flame graph looks ok at first glance.

Java 137

Java Cache Google Hardware 137

All Things Distributed

JULY 14, 2015

In this blog post, I will explain how these three new capabilities empower you to build applications with distributed systems architecture and create responsive, reliable, and high-performance applications using DynamoDB that work at any scale. DynamoDB Streams simplifies and improves this design pattern with a distributed systems approach.

Database 159

Database Lambda AWS IoT 159

DZone

AUGUST 16, 2019

It’s not just a simple tweak you can turn on/off; it’s a long-time process that touches almost every single item in your stack, including both hardware and software sides of the system. Application scalability is the potential of an application to grow in time, being able to efficiently handle more and more requests per minute (RPM).

Scalability 145

Scalability Efficiency Hardware Software 145

The Morning Paper

MARCH 8, 2020

This paper presents Snowflake design and implementation along with a discussion on how recent changes in cloud infrastructure (emerging hardware, fine-grained billing, etc.) have altered the many assumptions that guided the design and optimization of the Snowflake system. From shared-nothing to disaggregation.

Storage 112

Storage Engineering Cache Serverless 112

The Morning Paper

NOVEMBER 5, 2019

File systems unfit as distributed storage backends: lessons from 10 years of Ceph evolution Aghayev et al., In this case, the assumption that a distributed storage backend should clearly be layered on top of a local file system. A distributed file system provides a unified view over aggregated storage from multiple physical machines.

Storage 64

Storage Systems Hardware Efficiency 64

ACM Sigarch

NOVEMBER 4, 2019

Krste Asanovic from UC Berkeley kicked off the main program sharing his experience on “ Rejuvenating Computer Architecture Research with Open-Source Hardware ”. He ended the keynote with a call to action for open hardware and tools to start the next wave of computing innovation. This year’s MICRO had three inspiring keynote talks.

Hardware 18

Hardware Architecture Programming Innovation 18

Percona

MARCH 28, 2023

Every database system has to ensure durability and reliability. ” This acts as a step to ensure durability by recovering lost data from the same journal files in case of crashes, power, and hardware failures between the checkpoints (see below) Here’s what the process looks like. wt and index-*.wt).

Hardware 94

Hardware C++ Storage Cache 94

Scalegrid

JANUARY 5, 2024

If you’re considering a database management system, understanding these benefits is crucial. Understanding Database Management Systems (DBMS) A Database Management System (DBMS) assists users in creating and managing databases. In a DBMS, data serves as the material which the system stores, structures, and controls.

Efficiency 147

Efficiency Storage Database Scalability 147

Highly Scalable

APRIL 2, 2012

In particular, we built this system on top of Oracle Coherence and designed our own data structures and indexes. The first piece of functionality is structural or hierarchical navigation through categories and products, which are the main business entities of the system. System Properties and Major Technical Requirements.

Ecommerce 100

Ecommerce Cache Storage Architecture 100

Scalegrid

DECEMBER 21, 2023

With these essential support systems in place, you can effectively monitor your databases with up-to-date data about their health and functioning status at all times. To monitor Redis® instances effectively, collect Redis metrics focusing on cache hit ratio, memory allocated, and latency threshold.

Strategy 130

Strategy Monitoring Latency DevOps 130

Highly Scalable

SEPTEMBER 18, 2012

As such, it encompasses distributed system coordination, failover, resource management and many other capabilities. These developments gradually highlight a system of relevant database building blocks with proven practical efficiency. System Coordination. Scalability is one of the main drivers of the NoSQL movement.

Database 213

Database Latency C++ Scalability 213

Percona

AUGUST 8, 2023

As such, one of the more common questions I get from my clients is whether or not their system will be able to endure an anticipated load increase. Hardware considerations The first thing we have to consider here is the resources that the underlying host provides to the database. Let’s take a look at each common resource.

Traffic 93

Traffic Cache Monitoring Database 93

ACM Sigarch

OCTOBER 12, 2020

Are caches large enough for this code? There’s some work on hardware proposals for these systems, like Zhu et al., We as a community should lead the way in developing systems (at the hardware and software levels) that will make them run faster. Is there room for accelerators? MICRO 15 , Gope et al.,

C++ 108

C++ Benchmarking Hardware Architecture 108

Percona

JUNE 22, 2023

This includes metrics such as query execution time, the number of queries executed per second, and the utilization of query cache and adaptive hash index. query cache: Disable (query_cache_size: 0, query_cache_type:OFF) innodb_adaptive_hash_index: Check adaptive hash index usage to determine its efficiency.

Performance 109

Performance Monitoring Traffic Database 109

ACM Sigarch

NOVEMBER 1, 2019

This blog post gives a glimpse of the computer systems research papers presented at the USENIX Annual Technical Conference (ATC) 2019, with an emphasis on systems that use new hardware architectures. USENIX ATC is a top-tier venue with a broad range of systems research papers from both industry and academia.

Architecture 12

Architecture Hardware Cache Storage 12

The Morning Paper

JUNE 25, 2019

This paper is all about the design of efficient data structures for far-memory, which turns out to have consequences reaching all the way down to the hardware. A far memory data structure has: far data in far memory, containing the core content of the data structure data caches at clients algorithms for operations. Refreshable vectors.

Design 80

Design Cache Hardware Scalability 80

Percona

JUNE 22, 2023

Defining high availability In general terms, high availability refers to the continuous operation of a system with little to no interruption to end users in the event of hardware or software failures, power outages, or other disruptions. Some disruption might occur, but it will be minimal. More in the following sub-section.)

Availability 114

Availability Database Open Source Hardware 114

All Things Distributed

JANUARY 18, 2012

Werner Vogels weblog on building scalable and robust distributed systems. The original Dynamo design was based on a core set of strong distributed systems principles resulting in an ultra-scalable and highly reliable database system. All Things Distributed. Amazon DynamoDB â?? By Werner Vogels on 18 January 2012 07:00 AM.

Scalability 164

Scalability Database Ecommerce Latency 164

Testsigma

JUNE 3, 2021

He may think that the software got stuck and a system restart is needed. Hardware error. We focus on software so much that we forget about the hardware failures. If the hardware gets disconnected or stops working then we cannot expect correct output from the software. Hardware issues. Caching errors.

Software 82

Software Software Testing Hardware 82

Alex Russell

APRIL 29, 2021

After 20 years of neck-in-neck competition, often starting from common code lineages, there just isn't that much left to wring out of the system. Consistent improvement is the name of the game, and it can still have positive impacts, particularly as users lean on the system more heavily over time. Media Session API. Content Indexing.

Media 145

Media Games Education Engineering 145

Brendan Gregg

FEBRUARY 28, 2023

This talk originated from my updates to [Systems Performance 2nd Edition], and this was the first time I've given this talk in person! CXL in a way allows a custom memory controller to be added to a system, to increase memory capacity, bandwidth, and overall performance. Ford, et al., “TCP

Performance 103

Performance Latency Cache Virtualization 103

Smashing Magazine

AUGUST 15, 2021

Cache Merril. Companies can use technology roadmaps to review their internal IT , DevOps, infrastructure, architecture, software, internal system, and hardware procurement policies and procedures with innovation and efficiency in mind. What gaps in systems and capabilities need to be filled to succeed? Set Priorities.

Technology 141

Technology Technology Development Strategy 141

All Things Distributed

SEPTEMBER 20, 2013

Not everybody agreed that the "N-ary Storage Model" (NSM) was the best approach for all workloads but it stayed dominant until hardware constraints, especially on caches, forced the community to revisit some of the alternatives.

Storage 72

Storage Hardware Cache C++ 72

John McCalpin

APRIL 25, 2023

For most high-end processors these values have remained in the range of 75% to 85% of the peak DRAM bandwidth of the system over the past 15-20 years — an amazing accomplishment given the increase in core count (with its associated cache coherence issues), number of DRAM channels, and ever-increasing pipelining of the DRAMs themselves.

Benchmarking 97

Benchmarking Cache Latency Tuning 97

Percona

MAY 2, 2023

ChatGPT: The InnoDB buffer pool is used by MySQL to cache frequently accessed data in memory. If we expand the cache concept more, the buffer pool could be even less if the working set (hot data) is smaller. Questions Q: I have a MySQL server with 500 GB of RAM; my data set is 100 GB. How large my InnoDB buffer pool needs to be?

Social Media 126

Social Media Database Servers Cache 126

Alex Russell

MARCH 6, 2021

A then-representative $200USD device had 4-8 slow (in-order, low-cache) cores, ~2GiB of RAM, and relatively slow MLC NAND flash storage. Hardware Past As Performance Prologue. Regardless, the overall story for hardware progress remains grim, particularly when we recall how long device replacement cycles are: Tap for a larger version.

Performance 127

Performance Network Metrics Cache 127

The Morning Paper

APRIL 14, 2019

Just as today’s systems offer memory protection, they call this time protection. The paper sets out what we can do in software given today’s hardware, and along the way also highlights areas where cooperation from hardware will be needed in the future. set out a principled basis for protecting against this class of attacks.

Hardware 44

Hardware Cache Latency Speed 44

Percona

SEPTEMBER 1, 2023

This results in expedited query execution, reduced resource utilization, and more efficient exploitation of the available hardware resources. This not only enhances performance but also enables you to make more efficient use of your hardware resources, potentially resulting in cost savings on infrastructure.

Tuning 52

Tuning Database Performance Hardware 52

Highly Scalable

AUGUST 20, 2013

In recent years, this idea got a lot of traction and a whole bunch of solutions like Twitter’s Storm, Yahoo’s S4, Cloudera’s Impala, Apache Spark, and Apache Tez appeared and joined the army of Big Data and NoSQL systems. The system should deliver performance of tens of thousands messages per second even on clusters of minimal size.

Big Data 154

Big Data Processing Lambda Database 154

HammerDB

JUNE 27, 2022

It enables the user to measure database performance and make comparative judgements about database hardware and software. HammerDB has graphical and command line interfaces for the Windows and Linux operating systems. Cached vs Scaled Workloads. HammerDB is a software application for database benchmarking. HammerDB Licensing.

Benchmarking 63

Benchmarking Open Source C++ Cache 63