Scalegrid

JANUARY 10, 2025

Heading into 2024, SQL databases will remain essential in data management, increasingly using distributed systems to meet growing needs for scalability and reliability. According to 2023 statistics, 49% of web applications use an SQL-based database , with SQL having a 75% adoption rate in the IT industry.

Database 130

Database Scalability Best Practices Blockchain 130

The Netflix TechBlog

SEPTEMBER 18, 2024

Central to this infrastructure is our use of multiple online distributed databases such as Apache Cassandra , a NoSQL database known for its high availability and scalability. Over time as new key-value databases were introduced and service owners launched new use cases, we encountered numerous challenges with datastore misuse.

Latency 260

Latency Storage Cache Servers 260

Dynatrace

JULY 11, 2024

Strategically handle end-to-end data deletion Two key elements form the backbone of an effective deletion strategy in Dynatrace SaaS data management: retention-based and on-demand deletion. Retention-based deletion is governed by a policy outlining the duration for which data is stored in the database before it’s deleted automatically.

Storage 264

Storage Government Best Practices Media 264

Percona

DECEMBER 11, 2023

There are a wealth of options on how you can approach storage configuration in Percona Operator for PostgreSQL , and in this blog post, we review various storage strategies — from basics to more sophisticated use cases. The way it is consumed is through regular storage classes and deserves a separate blog post.

Storage 122

Storage Strategy Cloud Network 122

Scalegrid

OCTOBER 17, 2019

In that environment, the first PostgreSQL developers decided forking a process for each connection to the database is the safest choice. It would be a shame if your database crashed, after all. Developers are often strongly discouraged from holding a database connection while other operations take place.

Architecture 200

Architecture Database Latency Servers 200

Dynatrace

JULY 22, 2024

Metrics are typically aggregated and stored in time series databases for monitoring and alerting purposes. OpenTelemetry provides [extensive documentation]([link] and examples to help you fine-tune your configuration for maximum effectiveness. There are three main types of telemetry data: Metrics.

Latency 264

Latency Best Practices Metrics Open Source 264

Percona

OCTOBER 18, 2023

Migrating a proprietary database to open source is a major decision that can significantly affect your organization. Today, we’ll be taking a deep dive into the intricacies of database migration, along with specific solutions to help make the process easier.

Open Source 111

Open Source Database Hardware Strategy 111

Dynatrace

APRIL 22, 2022

Although many companies adopt solutions such as OpenTelemetry, Prometheus, and Grafana as part of their observability strategy, they often confront a common data analysis problem: data silos. Once the data is in Dynatrace, users can define dashboards and understand how SAP HANA databases are functioning relative to the overall infrastructure.

Open Source 244

Open Source Metrics Cloud Tuning 244

Percona

NOVEMBER 3, 2023

Sure, database migration is complex, particularly when you’re looking to migrate from a proprietary database to an open source one. Database migration is almost always time-consuming, tedious, and full of potential pitfalls. Database migration is complex Let’s start here. Have you tuned your environment?

Database 107

Database Open Source Benchmarking Strategy 107

The Netflix TechBlog

OCTOBER 8, 2024

Note: Contrary to what the name may suggest, this system is not built as a general-purpose time series database. Partitioning Scheme At Netflix’s scale, the continuous influx of event data can quickly overwhelm traditional databases. We do not use it for metrics, histograms, timers, or any such near-real time analytics use case.

Latency 239

Latency Storage Traffic Tuning 239

Dynatrace

OCTOBER 2, 2023

Because monolithic applications combine database, client-side interfaces, and server-side application elements in a single executable, they’re difficult to understand, even for their own administrators. This comprehensive view helps teams gain an initial understanding of a monolithic application so they can develop a migration strategy.

Architecture 278

Architecture Artificial Intelligence Cloud Open Source 278

The Netflix TechBlog

JUNE 13, 2023

They enable us to further fine-tune and configure the system, ensuring the new changes are integrated smoothly and seamlessly. Migrating Persistent Stores Stateful APIs pose unique challenges that require different strategies. This alternate migration strategy has proven effective for our systems that meet certain criteria.

Traffic 285

Traffic Metrics Systems Strategy 285

The Netflix TechBlog

JULY 26, 2021

In the initial stage, data consumers set up ETL pipelines directly pulling data from databases. With this batch style approach, several issues have surfaced like data movement is tightly coupled with database tables, database schema is not an exact mapping of business data model, and data being stale given it is not real time etc.

Big Data 257

Big Data Government Processing Analytics 257

Percona

MAY 1, 2023

Out of the box, the default PostgreSQL configuration is not tuned for any particular workload. It has default settings for all of the database parameters. It is primarily the responsibility of the database administrator or developer to tune PostgreSQL according to their system’s workload.

Tuning 52

Tuning Database Efficiency Performance 52

Dynatrace

DECEMBER 18, 2023

This challenge has given rise to the discipline of observability engineering, which concentrates on the details of telemetry data to fine-tune observability use cases. As part of our observability engineering strategy, we want that data as well and to make sure it gets sent to Dynatrace. Please stay tuned! What is Prometheus?

Metrics 245

Metrics Engineering Energy Tuning 245

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. In databases like MySQL and PostgreSQL, transaction logs are the source of CDC events. Designed with High Availability in mind.

Database 221

Database Traffic Transportation Open Source 221

Percona

SEPTEMBER 1, 2023

While there is no magic bullet for MySQL performance tuning, there are a few areas that can be focused on upfront that can dramatically improve the performance of your MySQL installation. What are the Benefits of MySQL Performance Tuning? A finely tuned database processes queries more efficiently, leading to swifter results.

Tuning 52

Tuning Database Performance Hardware 52

Dynatrace

APRIL 8, 2021

When using Lambda, you might soon end up using more serverless offerings, like databases, which makes emulating the same environment locally even harder. While AWS takes the operational overhead for operating Lambda functions off your shoulders, using Lambda functions at a larger scale requires mature deployment strategies.

Lambda 246

Lambda AWS Serverless Monitoring 246

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. In databases like MySQL and PostgreSQL, transaction logs are the source of CDC events. Designed with High Availability in mind.

Database 206

Database Traffic Transportation Open Source 206

Dynatrace

APRIL 8, 2021

When using Lambda, you might soon end up using more serverless offerings, like databases, which makes emulating the same environment locally even harder. While AWS takes the operational overhead for operating Lambda functions off your shoulders, using Lambda functions at a larger scale requires mature deployment strategies.

Lambda 246

Lambda AWS Serverless Monitoring 246

Dynatrace

DECEMBER 8, 2020

Research by the Enterprise Strategy Group in 2020 shows 60% of reported breached production applications in the past 12 months involved a known and unpatched vulnerability. Stay tuned – this is only the start. According to Gartner , 80% of vulnerabilities are introduced via transitive dependencies.

Cloud 306

Cloud Open Source Internet Internet 306

Percona

DECEMBER 27, 2023

of respondents are currently utilizing databases in Kubernetes (k8s). These indicators suggest that the adoption of databases on k8s is in its early stages and is likely to continue growing in the future. Nearly every vendor at Kubecon and every person we spoke to had some form of a multi-cloud requirement or strategy.

Cloud 113

Cloud Open Source Strategy Best Practices 113

Percona

DECEMBER 6, 2023

This blog is regarding some of the usual MySQL database conversations and responses, which can appear “wrong” or “funny,” but there’s actually more to them. Additional read Mike’s blog on How to Find and Tune a Slow SQL Query Q: What is your disaster recovery (DR) strategy?

Strategy 112

Strategy Database Best Practices Tuning 112

The Netflix TechBlog

OCTOBER 19, 2020

Our engineering teams tuned their services for performance after factoring in increased resource utilization due to tracing. In pursuit of this goal, we outlined following storage optimization strategies: Use cheaper Elastic Block Store (EBS) volumes instead of SSD instance stores in EC2.

Infrastructure 238

Infrastructure Transportation Storage Open Source 238

The Netflix TechBlog

DECEMBER 21, 2020

Other than these principles, there are some other design considerations to support and enable: Multi-tenancy with database and table prioritization. Orient: Gather tuning parameters for a particular table that changed. AutoAnalyze In short, AutoAnalyze finds the best tuning/configuration parameters for a table.

Storage 212

Storage Latency Efficiency Data Engineering 212

Scalegrid

JUNE 21, 2024

This underscores the importance of timely software upgrades and strategic planning to keep pace with advancements in database technology. Operational Risks Ensuring operational integrity is essential for the seamless functioning of any database system. Not all versions receive support, which might cause some users to feel left behind.

Database 130

Database Efficiency Strategy Tuning 130

Percona

NOVEMBER 2, 2023

MongoDB is a dynamic database system continually evolving to deliver optimized performance, robust security, and limitless scalability. Our new eBook, “ From Planning to Performance: MongoDB Upgrade Best Practices ,” guides you through the entire process to ensure your database’s long-term success. In MongoDB 6.x:

Best Practices 111

Best Practices Hardware Tuning Scalability 111

Dynatrace

JANUARY 13, 2021

Dynatrace’s OneAgent automatically captures PurePaths and analyzes transactions end-to-end across every tier of your application technology stack with no code changes, from the browser all the way down to the code and database level. Stay tuned for an upcoming Performance Clinic ! How you can get started with monitoring-as-code.

Open Source 211

Open Source Code Monitoring Azure 211

Scalegrid

JANUARY 11, 2024

This article strips away the complexities, walking you through best practices, top tools, and strategies you’ll need for a well-defended cloud infrastructure. These include alert fatigue, lack of context, and absence of strategy. Get ready for actionable insights that balance technical depth with practical advice.

Best Practices 130

Best Practices Cloud Monitoring Strategy 130

O'Reilly

FEBRUARY 18, 2025

The eval process combines: Human review Model-based evaluation A/B testing The results then inform two parallel streams: Fine-tuning with carefully curated data Prompt engineering improvements These both feed into model improvements, which starts the cycle again. Fine-tuning works best for specific jobs where you need higher accuracy.

Latency 64

Latency Tuning Metrics Testing 64

Percona

OCTOBER 4, 2023

We discussed how roles can inherit permissions from other roles, simplifying access control in your database. Access control: Tailor access permissions for child roles based on their inheritance, ensuring a granular and secure access control strategy. But what if you need to discover which roles inherit from a specific role?

C++ 122

C++ Open Source Tuning Database 122

Percona

JULY 2, 2023

In this post, we cover the methods used to achieve an enterprise-grade backup strategy for the PostgreSQL cluster. The size of the database and your database environment—if it is on colo or cloud—matters. They also ensure that backup strategies adhere to industry and legal compliance requirements.

Strategy 52

Strategy Database Storage Open Source 52

The Netflix TechBlog

NOVEMBER 2, 2021

For example, VQS relies on the Netflix Media Database (NMDB) to store and index the quality scores, while the Reloaded system uses a mix of non-queryable data models and files. Stay tuned for more details on these algorithmic innovations. Data conversions as a service To complete our design, we have to solve one last puzzle.

Media 178

Media Innovation Metrics Latency 178

Scalegrid

JANUARY 25, 2024

Redis® is an in-memory database that provides blazingly fast performance. This makes it a compelling alternative to disk-based databases when performance is a concern. Redis returns a big list of database metrics when you run the info command on the Redis shell. This blog post lists the important database metrics to monitor.

Metrics 130

Metrics Monitoring Latency Cache 130

All Things Distributed

OCTOBER 2, 2017

Today marks the 10 year anniversary of Amazon's Dynamo whitepaper , a milestone that made me reflect on how much innovation has occurred in the area of databases over the last decade and a good reminder on why taking a customer obsessed approach to solving hard problems can have lasting impact beyond your original expectations.

Internet 111

Internet Internet AWS Performance 111

Percona

FEBRUARY 24, 2023

As a result, it becomes increasingly important to protect the confidentiality of information to prevent data leaks and facilitate easy access to relevant data sources and tools for the related teams without having to sift through unrelated databases or environmental data. As a first task, Vanessa plans an access control roll-out strategy.

Metrics 135

Metrics Monitoring Database Engineering 135

Percona

JULY 30, 2018

Please join Percona’s CEO, Peter Zaitsev as he presents Using MySQL for Distributed Database Architectures on Tuesday, July 31st, 2018 at 7:00 AM PDT (UTC-7) / 10:00 AM EDT (UTC-4). Peter frequently speaks as an expert lecturer at MySQL and related conferences, and regularly posts on the Percona Database Performance Blog. Register Now.

Architecture 57

Architecture Database Open Source Internet 57