Scalegrid

OCTOBER 24, 2019

AWS is the #1 cloud provider for open-source database hosting, and the go-to cloud for MySQL deployments. As organizations continue to migrate to the cloud, it’s important to get in front of performance issues, such as high latency, low throughput, and replication lag with higher distances between your users and cloud infrastructure.

AWS 183

AWS Latency Performance Performance Testing 183

Scalegrid

OCTOBER 17, 2019

In that environment, the first PostgreSQL developers decided forking a process for each connection to the database is the safest choice. It would be a shame if your database crashed, after all. Developers are often strongly discouraged from holding a database connection while other operations take place.

Architecture 200

Architecture Database Latency Servers 200

The Netflix TechBlog

OCTOBER 27, 2020

The data warehouse is not designed to serve point requests from microservices with low latency. Therefore, we must efficiently move data from the data warehouse to a global, low-latency and highly-reliable key-value store. How Bulldozer leverages Spark, Protobuf and KV DAL for moving the data.

Latency 249

Latency Storage Big Data Tuning 249

Dynatrace

JANUARY 26, 2021

REST APIs, authentication, databases, email, and video processing all have a home on serverless platforms. When an application is triggered, it can cause latency as the application starts. Security, databases, and programming languages effortlessly remain up to date and secure in the serverless model.

Serverless 246

Serverless Efficiency Lambda AWS 246

Dynatrace

MAY 17, 2023

Think about items such as general system metrics (for example, CPU utilization, free memory, number of services), the connectivity status, details of our web server, or even more granular in-application tasks like database queries. Database monitoring Once more, under Applications & Microservices, we’ll also find Databases.

Metrics 243

Metrics Database Monitoring Network 243

The Netflix TechBlog

AUGUST 13, 2020

You’re half awake and wondering, “Is there really a problem or is this just an alert that needs tuning? Telltale learns what constitutes typical health for an application, no alert tuning required. For example, a latency increase is less critical than error rate increase and some error codes are less critical than others.

Monitoring 216

Monitoring Tuning Traffic Metrics 216

Percona

NOVEMBER 3, 2023

Rather than listing the concepts, function calls, etc, available in Citus, which frankly is a bit boring, I’m going to explore scaling out a database system starting with a single host. I won’t cover all the features but show just enough that you’ll want to see more of what you can learn to accomplish for yourself.

Database 114

Database Benchmarking Latency C++ 114

InfoQ

DECEMBER 4, 2023

LinkedIn was able to dramatically improve the scalability and performance of its Espresso database by migrating it from HTTP1.1 to HTTP2, resulting in a reduction in the number of connections, latency, and garbage collection times. By Rafal Gancarz

Latency 101

Latency Scalability Database Performance 101

Scalegrid

SEPTEMBER 10, 2024

Additionally, we’ve added the Philadelphia AWS Local Zone , helping to reduce latency for customers operating in the eastern U.S. This enables ScaleGrid users in Australia and nearby regions to access lower-latency services and improved performance. Stay tuned for more exciting updates in the months to come! <p>The

Latency 130

Latency AWS Storage Tuning 130

The Netflix TechBlog

JANUARY 10, 2024

This architecture shift greatly reduced the processing latency and increased system resiliency. We expanded pipeline support to serve our studio/content-development use cases, which had different latency and resiliency requirements as compared to the traditional streaming use case. divide the input video into small chunks 2.

Processing 235

Processing Media Latency Innovation 235

The Netflix TechBlog

FEBRUARY 18, 2022

This separation allows us to tune system configuration and scaling policies independently for different event priorities and traffic patterns. Persistent Store A Cassandra database that stores all the notifications emitted by RENO for each device to allow those devices to poll for their messages at their own cadence.

Systems 334

Systems Traffic Architecture Mobile 334

The Netflix TechBlog

DECEMBER 21, 2020

These principles reduce resource usage by being more efficient and effective while lowering the end-to-end latency in data processing. Other than these principles, there are some other design considerations to support and enable: Multi-tenancy with database and table prioritization. Transparency to end-users. More processing resources.

Storage 209

Storage Latency Efficiency Data Engineering 209

The Netflix TechBlog

JULY 26, 2021

Operational Reporting is a reporting paradigm specialized in covering high-resolution, low-latency data sets, serving detailed day-to-day activities¹ and processes of a business domain. In the initial stage, data consumers set up ETL pipelines directly pulling data from databases. Please stay tuned! Endnotes ¹ Inmon, Bill.

Big Data 256

Big Data Government Processing Analytics 256

Percona

APRIL 7, 2023

If we were to select the most important MySQL setting, if we were given a freshly installed MySQL or Percona Server for MySQL and could only tune a single MySQL variable, which one would it be? That’s a heritage of the LAMP model when the same server would host both the database and the web server.

Tuning 145

Tuning Cache Servers Benchmarking 145

SQL Performance

SEPTEMBER 17, 2019

Many database administrators find themselves having to support instances of SQL Server Reporting Services (SSRS), or at least the backend databases that are required for SSRS. In each of the deployment models, the role of the database administrator is to make sure that SSRS is stable, dependable, and recoverable.

Tuning 67

Tuning Servers Database Best Practices 67

Scalegrid

JANUARY 25, 2024

Redis® is an in-memory database that provides blazingly fast performance. This makes it a compelling alternative to disk-based databases when performance is a concern. Redis returns a big list of database metrics when you run the info command on the Redis shell. This blog post lists the important database metrics to monitor.

Metrics 130

Metrics Monitoring Latency Cache 130

Percona

APRIL 13, 2023

PostgreSQL is a popular open source relational database management system that is widely used for storing and managing data. Replication lag is the delay between the time when data is written to the primary database and the time when it is replicated to the standby databases. What is replication lag?

Latency 103

Latency Tuning Open Source Network 103

The Netflix TechBlog

JUNE 13, 2023

By collecting and analyzing key performance metrics of the service over time, we can assess the impact of the new changes and determine if they meet the availability, latency, and performance requirements. They enable us to further fine-tune and configure the system, ensuring the new changes are integrated smoothly and seamlessly.

Traffic 283

Traffic Metrics Systems Strategy 283

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. In databases like MySQL and PostgreSQL, transaction logs are the source of CDC events. Designed with High Availability in mind.

Database 218

Database Traffic Transportation Open Source 218

Percona

AUGUST 10, 2018

The performance of a PostgreSQL database can be compromised by dead tuples, since they continue to occupy space and can lead to bloat. Now, though, it’s time to look at autovacuum for postgres, and the internals you to know to maintain a high performance PostgreSQL database needed by demanding applications.

Tuning 69

Tuning Cache Database Storage 69

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. In databases like MySQL and PostgreSQL, transaction logs are the source of CDC events. Designed with High Availability in mind.

Database 204

Database Traffic Transportation Open Source 204

Scalegrid

NOVEMBER 4, 2024

The results will help database administrators and decision-makers choose the right platform for their performance, scalability, and cost-efficiency needs. Introduction Purpose and Scope Cloud-hosted PostgreSQL solutions are increasingly popular among organizations seeking scalable, high-performance databases.

Benchmarking 130

Benchmarking AWS Tuning Metrics 130

Percona

MARCH 23, 2023

Hardware Memory The amount of RAM to be provisioned for database servers can vary greatly depending on the size of the database and the specific requirements of the company. Storage The type of storage and disk used for database servers can have a significant impact on performance and reliability. Benchmark before you decide.

Performance 144

Performance Hardware Tuning Storage 144

The Netflix TechBlog

OCTOBER 19, 2020

If we had an ID for each streaming session then distributed tracing could easily reconstruct session failure by providing service topology, retry and error tags, and latency measurements for all service calls. Our engineering teams tuned their services for performance after factoring in increased resource utilization due to tracing.

Infrastructure 237

Infrastructure Transportation Storage Open Source 237

The Netflix TechBlog

MAY 14, 2021

Once you finally find useful identifiers, you may begin writing SQL queries against your production database to find out what went wrong. We are expected to process 1,000 watermarks for a single distribution in a minute, with non-linear latency growth as the number of watermarks increases.

Traffic 164

Traffic Java Latency Google 164

Percona

SEPTEMBER 1, 2023

While there is no magic bullet for MySQL performance tuning, there are a few areas that can be focused on upfront that can dramatically improve the performance of your MySQL installation. What are the Benefits of MySQL Performance Tuning? A finely tuned database processes queries more efficiently, leading to swifter results.

Tuning 52

Tuning Database Performance Hardware 52

O'Reilly

FEBRUARY 18, 2025

The eval process combines: Human review Model-based evaluation A/B testing The results then inform two parallel streams: Fine-tuning with carefully curated data Prompt engineering improvements These both feed into model improvements, which starts the cycle again. Were experiencing high latency in responses.

Latency 66

Latency Tuning Metrics Testing 66

The Netflix TechBlog

AUGUST 30, 2021

By the following morning, alerts were received regarding high memory consumption and GC latencies, to the point where the service was unresponsive to HTTP requests. This setup is neither optimal nor break-proof since the Kafka message processing rate will vary depending on environmental factors, such as database latencies in our system setup.

Latency 220

Latency Traffic Transportation Cloud 220

Percona

NOVEMBER 2, 2023

MongoDB is a dynamic database system continually evolving to deliver optimized performance, robust security, and limitless scalability. Our new eBook, “ From Planning to Performance: MongoDB Upgrade Best Practices ,” guides you through the entire process to ensure your database’s long-term success. In MongoDB 6.x:

Best Practices 111

Best Practices Hardware Tuning Scalability 111

Percona

MAY 1, 2023

I’ll show you some MySQL settings to tune to get better performance, and cost savings, with AWS RDS. ” Yes, this is true, io2 volumes are expensive, and honestly, I think they should be used only where really high IO capacity at expected latencies is required, and this didn’t seem to be the case.

AWS 135

AWS Hardware Storage Tuning 135

The Netflix TechBlog

NOVEMBER 2, 2021

This enables us to use our scale to increase throughput and reduce latencies. Here, based on the video length, the throughput and latency requirements, available scale etc., Stay tuned for more details on these algorithmic innovations. VQS is called using the measureQuality endpoint. The workflow is initiated.

Media 176

Media Innovation Metrics Latency 176

SQL Performance

APRIL 27, 2020

Azure SQL Database is Microsoft's database-as-a-service offering that provides a tremendous amount of flexibility. Microsoft is continually working on improving their products and Azure SQL Database is no different. HyperScale Database. Hyperscale databases are provisioned using vCores. DTU Pricing Tier.

Azure 55

Azure Database Serverless Hardware 55

All Things Distributed

OCTOBER 2, 2017

Today marks the 10 year anniversary of Amazon's Dynamo whitepaper , a milestone that made me reflect on how much innovation has occurred in the area of databases over the last decade and a good reminder on why taking a customer obsessed approach to solving hard problems can have lasting impact beyond your original expectations.

Internet 110

Internet Internet AWS Performance 110

VoltDB

JUNE 19, 2017

Unfortunately, this means that the age-old Telco bugbears will rear their ugly heads again, including latency. 5G, as a fundamental requirement, mandates a 1 millisecond latency from the datasource to its destination. This requires 1 ms network latency. The post Latency: Will it undermine the most interesting 5G use cases?

Latency 40

Latency Retail Virtualization Hardware 40

VoltDB

JUNE 19, 2017

Unfortunately, this means that the age-old Telco bugbears will rear their ugly heads again, including latency. 5G, as a fundamental requirement, mandates a 1 millisecond latency from the datasource to its destination. This requires 1 ms network latency. The post Latency: Will it undermine the most interesting 5G use cases?

Latency 40

Latency Retail Virtualization Hardware 40

All Things Distributed

DECEMBER 12, 2013

We launched DynamoDB last year to address the need for a cloud database that provides seamless scalability, irrespective of whether you are doing ten transactions or ten million transactions, while providing rock solid durability and availability. DynamoDB stores information as database tables, which are collections of individual items.

Games 106

Games Scalability Database Retail 106

The Netflix TechBlog

FEBRUARY 9, 2021

Retrieve eligible SKUs from SKU Platform The SKU Platform consists of a rules engine, a database, and application logic. The database contains the plans, prices and offers. Stay tuned for more details on this, as well as more details on the internals of the new SKU Platform in one of our upcoming blog posts.

Engineering 144

Engineering Scalability Architecture Innovation 144