The Netflix TechBlog

JANUARY 13, 2022

We have now moved to a more systematic approach where unknown errors are fed into a Machine Learning process that performs clustering to propose new regular expressions for commonly occurring errors. In the future, we are looking to automate this process. Expand Pensive with Machine Learning classifiers.

Big Data 241

Big Data Infrastructure Metrics Games 241

Dynatrace

AUGUST 8, 2019

The IBM Z platform is a range of mainframe hardware solutions that are quite frequently used in large computing shops. Typically, these shops run the z/OS operating system, but more recently, it’s not uncommon to see the Z hardware running special versions of Linux distributions. Stay tuned for more announcements on this topic.

Availability 34

Availability Hardware Java Tuning 34

Dynatrace

AUGUST 12, 2020

With Azure Functions, engineers don’t have to worry about provisioning and maintaining underlying hardware; they simply upload their code, and it’s up and running seconds later. You can find out which functions experience the highest failure rate or processing time and which of them are executed the most. So stay tuned!

Serverless 246

Serverless Azure Monitoring Lambda 246

Dynatrace

NOVEMBER 15, 2021

Logs can include data about user inputs, system processes, and hardware states. Log monitoring is a process by which developers and administrators continuously observe logs as they’re being recorded. Log analytics is the process of evaluating and interpreting log data so teams can quickly detect and resolve issues.

Analytics 264

Analytics Monitoring DevOps Artificial Intelligence 264

Dynatrace

MAY 1, 2020

This is especially the case with microservices and applications created around multiple tiers, where cheaper hardware alternatives play a significant role in the infrastructure footprint. Network measurements with per-interface and per-process resolution. Network metrics are also collected for detected processes.

Java 182

Java Hardware Metrics Tuning 182

Dynatrace

OCTOBER 2, 2023

Limits of a lift-and-shift approach A traditional lift-and-shift approach, where teams migrate a monolithic application directly onto hardware hosted in the cloud, may seem like the logical first step toward application transformation. Use SLAs, SLOs, and SLIs as performance benchmarks for newly migrated microservices.

Architecture 278

Architecture Artificial Intelligence Cloud Open Source 278

Percona

AUGUST 29, 2018

In this post, we will discuss some important kernel parameters that can affect database server performance and how these should be tuned. SHMMAX is a kernel parameter used to define the maximum size of a single shared memory segment a Linux process can allocate. A page is a chunk of RAM that is allocated to a process.

Tuning 79

Tuning Database Servers Processing 79

Dynatrace

AUGUST 12, 2020

With Azure Functions, engineers don’t have to worry about provisioning and maintaining underlying hardware; they simply upload their code, and it’s up and running seconds later. You can find out which functions experience the highest failure rate or processing time and which of them are executed the most. So stay tuned!

Serverless 217

Serverless Azure Monitoring Lambda 217

Percona

MARCH 23, 2023

Hardware Memory The amount of RAM to be provisioned for database servers can vary greatly depending on the size of the database and the specific requirements of the company. As datasets continue to grow in size, the amount of RAM required to store and process these datasets also increases. Refer to innodb_redo_log_capacity below.

Performance 144

Performance Hardware Tuning Storage 144

The Netflix TechBlog

AUGUST 30, 2021

Complementing the hardware is the software on the RAE and in the cloud, and bridging the software on both ends is a bi-directional control plane. The challenge, then, is to be able to ingest and process these events in a scalable manner, i.e., scaling with the number of devices, which will be the focus of this blog post.

Latency 220

Latency Traffic Transportation Cloud 220

All Things Distributed

MARCH 19, 2018

Bill Simmons, CTO of Dataxu, states, "We process 3 million ad requests a second - 100,000 features per request. In other words, processing the 10th gigabyte and 1000th gigabyte is conceptually the same. To seamlessly switch between CPU and GPU machines, we use Apache MXNet to interface with the underlying hardware.

Scalability 127

Scalability Hardware AWS Tuning 127

Percona

NOVEMBER 2, 2023

Upgrading to the newest release of MongoDB is the key to unlocking its full potential, but it’s not as simple as clicking a button; it requires meticulous planning, precise execution, and a deep understanding of the upgrade process. You should also review your hardware resources, how you use MongoDB, and any custom configurations.

Best Practices 111

Best Practices Hardware Tuning Scalability 111

The Morning Paper

NOVEMBER 3, 2019

Perhaps the most interesting lesson/reminder is this: it takes a lot of effort to tune a Linux kernel. Google’s data center kernel is carefully performance tuned for their workloads. On the exact same hardware, the benchmark suite is then used to test 36 Linux release versions from 3.0 Headline results.

Performance 108

Performance Benchmarking Tuning Hardware 108

Percona

SEPTEMBER 1, 2023

While there is no magic bullet for MySQL performance tuning, there are a few areas that can be focused on upfront that can dramatically improve the performance of your MySQL installation. What are the Benefits of MySQL Performance Tuning? A finely tuned database processes queries more efficiently, leading to swifter results.

Tuning 52

Tuning Database Performance Hardware 52

DZone

AUGUST 16, 2019

It’s not just a simple tweak you can turn on/off; it’s a long-time process that touches almost every single item in your stack, including both hardware and software sides of the system. Application scalability is the potential of an application to grow in time, being able to efficiently handle more and more requests per minute (RPM).

Scalability 145

Scalability Efficiency Hardware Software 145

Percona

AUGUST 27, 2018

assigning to a specific CPU) is a manageable resource, represented by the concept of “virtual CPU” as a term that includes CPU cores, hyperthreads, hardware threads, and so forth. Good examples are: ETL processes for data archiving, reporting, data consolidation and so on. Currently, CPU affinity (i.e., What is the possible usage?

Tuning 69

Tuning Virtualization Testing Servers 69

Percona

MAY 1, 2023

Out of the box, the default PostgreSQL configuration is not tuned for any particular workload. It is primarily the responsibility of the database administrator or developer to tune PostgreSQL according to their system’s workload. What is PostgreSQL performance tuning? Why is PostgreSQL performance tuning important?

Tuning 52

Tuning Database Efficiency Performance 52

Dynatrace

OCTOBER 22, 2019

Ultimately, it leads to a state where your system won’t be able to process more data even if you add more hardware. Now, let’s look at a Cassandra process group and see if we can find any CPU-hungry threads and thread groups. Such behavior not only limits speed but also your ability to increase throughput by adding resources.

Java 145

Java Scalability Hardware Code 145

O'Reilly

MARCH 11, 2025

These smaller distilled models can run on off-the-shelf hardware without expensive GPUs. And they can do useful work, particularly if fine-tuned for a specific application domain. Spending a little money on high-end hardware will bring response times down to the point where building and hosting custom models becomes a realistic option.

Artificial Intelligence 63

Artificial Intelligence Infrastructure Government Open Source 63

Percona

OCTOBER 18, 2023

It’s a complex process involving various factors and meticulous planning. Today, we’ll be taking a deep dive into the intricacies of database migration, along with specific solutions to help make the process easier. Evaluating your hardware requirements is another vital aspect of resource allocation.

Open Source 111

Open Source Database Strategy Hardware 111

O'Reilly

SEPTEMBER 19, 2023

Doubly so as hardware improved, eating away at the lower end of Hadoop-worthy work. Google goes a step further in offering compute instances with its specialized TPU hardware. You can download these models to use out of the box, or employ minimal compute resources to fine-tune them for your particular task.

Hardware 134

Hardware Storage Big Data Blockchain 134

Percona

DECEMBER 27, 2023

Most organizations are embracing DevSecOps, a term that covers processes and tooling enabling security to be integrated into the application development life cycle rather than treated as a separate process. It comprises numerous organizations from various sectors, including software, hardware, nonprofit, public, and academic.

Cloud 113

Cloud Open Source Strategy Best Practices 113

O'Reilly

NOVEMBER 28, 2023

Users and Nonusers AI adoption is in the process of becoming widespread, but it’s still not universal. Until AI reaches 100%, it’s still in the process of adoption. Automating the process of building complex prompts has become common, with patterns like retrieval-augmented generation (RAG) and tools like LangChain.

Open Source 136

Open Source Google Artificial Intelligence Infrastructure 136

Alex Podelko

OCTOBER 20, 2016

System’s configuration is not given anymore and often can’t be easily mapped to hardware. As already mentioned, performance testing is rather a performance engineering process (with tuning, optimization, troubleshooting and fixing multi-user issues) eventually bringing the system to the proper state rather than just testing.

Performance Testing 100

Performance Testing Architecture Testing Performance 100

The Morning Paper

NOVEMBER 10, 2019

Snap’s architecture is a composition of recent ideas in user-space networking, in-service upgrades, centralized resource accounting, programmable packet processing, kernel-bypass, RDMA functionality, and optimized co-design of transport, congestion control and routing. It reminds me of ZeroMQ. Snap’s control plane is orchestrated via RPCs.

Network 92

Network Transportation Latency Entertainment 92

Percona

NOVEMBER 3, 2023

About the cluster Following a step-by-step process, the objective is to create a four-node cluster consisting of: PostgreSQL version 15 Citus extension (I’ll be using version 11, but there are newer ones available.) psql pgbench <<_eof1_ qecho adding node citus3.

Database 114

Database Benchmarking Latency C++ 114

Scalegrid

JANUARY 25, 2024

Effective management of memory stores with policies like LRU/LFU proactive monitoring of the replication process and advanced metrics such as cache hit ratio and persistence indicators are crucial for ensuring data integrity and optimizing Redis’s performance. This plays a crucial role in supervising the replication process.

Metrics 130

Metrics Monitoring Latency Cache 130

O'Reilly

OCTOBER 11, 2022

There seems to be broad agreement that hyperautomation is the combination of Robotic Process Automation with AI. We’ll see it in the processing of the thousands of documents businesses handle every day. We can certainly apply the slogan to many, if not all, clerical tasks–and even to the automation process itself.

Games 141

Games Healthcare Processing Systems 141

The Morning Paper

MAY 21, 2019

Using zswap means that no new hardware solutions are required, enabling rapid deployment across clusters. ML-based auto-tuning. This model is used by a Gaussian Process (GP) Bandit machine learning model to guide the parameter search towards an optimal point with a minimal number of trials.

Software 78

Software Software Google Hardware 78

Percona

JANUARY 4, 2024

Flexible location : Data files can reside within the MySQL data directory or an independent location, enabling finer control over storage management and performance tuning. mysql> SHOW VARIABLES LIKE 'innodb_directories'; + --+ -+ | Variable_name | Value | + --+ -+ | innodb_directories | | + --+ -+ 1 row in set (0.00

Storage 114

Storage Engineering Database Open Source 114

SQL Performance

MARCH 17, 2020

Back in 2014, I wrote an article called Performance Tuning the Whole Query Plan. The Sort only has 19 rows to process, so it consumes only 1ms or so. The parallel column store aggregate push down result of 57 million rows processed in 7ms (using 40ms of CPU) is remarkable, especially considering the hardware.

Servers 86

Servers Tuning Database Efficiency 86

O'Reilly Software

APRIL 4, 2018

We live in the era of the connected experience, where our daily interactions with the world can be digitized, collected, processed, and analyzed to generate valuable insights. Accumulating all this data to process overnight is not an option anymore. Do we need to process each record individually? Process tolerance.

Architecture 80

Architecture Scalability Google Operating System 80

HammerDB

OCTOBER 18, 2019

cpupower frequency-info analyzing CPU 0: driver: intel_pstate CPUs which run at the same hardware frequency: 0 CPUs which need to have their frequency coordinated by software: 0 maximum transition latency: Cannot determine or is not supported. hardware limits: 1000 MHz - 4.00 hardware limits: 1000 MHz - 4.00

Benchmarking 63

Benchmarking Performance Hardware C++ 63

John McCalpin

APRIL 25, 2023

This metric is interesting because we don’t always have the luxury of parallelizing every application we run, and our operating systems almost always process each call (e.g., Stay tuned! This post is about a secondary performance characteristic — sustained memory bandwidth for a single thread running on a single core.

Benchmarking 97

Benchmarking Cache Latency Tuning 97

Abhishek Tiwari

SEPTEMBER 15, 2024

This allows NASA's engineers to fine-tune every aspect of the rover's behavior, optimizing for reliability rather than rapid development or ease of use. They employ rigorous development processes, including multiple layers of review and testing that would make most software companies' heads spin.

Engineering 52

Engineering Social Media Hardware C++ 52

Percona

JANUARY 19, 2023

Compression: Compression is the process of restructuring the data by changing its encoding in order to store it in fewer bytes. With hardware being more powerful and cheaper, and the technology evolving, now it is easier than ever to manage large tables in MySQL. 1 mysql mysql 592K Dec 30 02:48 tb1.ibd ibd -rw-r --. ibd -rw-r --.

Open Source 111

Open Source Storage Database Big Data 111

Brendan Gregg

SEPTEMBER 5, 2017

There's also a ZFS send/recv code path that should try to use the TASK_INTERRUPTIBLE flag (as suggested by a coworker), to avoid a kernel hang (can't kill -9 the process). On Linux, containers are a combination of namespaces (restricting what a process sees) and cgroups (similar to Solaris resource controls). amazon").

Virtualization 75

Virtualization AWS Engineering Hardware 75