DZone

JANUARY 20, 2023

Having a distributed and scalable graph database system is highly sought after in many enterprise scenarios. Do Not Be Misled Designing and implementing a scalable graph database system has never been a trivial task.

Scalability 199

Scalability Big Data Hardware Internet 199

DZone

APRIL 11, 2023

Data processing in the cloud has become increasingly popular due to its scalability, flexibility, and cost-effectiveness. This article will explore how these technologies can be used together to create an optimized data pipeline for data processing in the cloud.

Azure 246

Azure Analytics Storage Cloud 246

Scalegrid

JANUARY 5, 2024

Enhanced data security, better data integrity, and efficient access to information. If you’re considering a database management system, understanding these benefits is crucial. Understanding Database Management Systems (DBMS) A Database Management System (DBMS) assists users in creating and managing databases.

Efficiency 147

Efficiency Storage Database Scalability 147

Scalegrid

DECEMBER 6, 2023

In the digital age, data management has transformed from locally hosted servers to cloud solutions. The choice of self-managed cloud databases vs DBaaS is a common debate among those who are looking for the best option that will cater to their particular needs.

Database 130

Database Cloud Hardware Storage 130

DZone

NOVEMBER 1, 2024

Horizontally scalable data stores like Elasticsearch , Cassandra , and CockroachDB distribute their data across multiple nodes using techniques like consistent hashing. As nodes are added or removed, the data is reshuffled to ensure that the load is spread evenly across the new set of nodes.

Monitoring 263

Monitoring Scalability Database Cloud 263

DZone

OCTOBER 9, 2024

The Scheduler service enables this and is designed to address the performance and scalability improvements on Actor reminders and the Workflow API. However, the binding approach lacked in the areas of durability and scalability, and more importantly, could not be combined with other Dapr APIs. Prior to v1.14

Scalability 317

Scalability Design Database Processing 317

Dynatrace

JUNE 9, 2023

ln a world driven by macroeconomic uncertainty, businesses increasingly turn to data-driven decision-making to stay agile. They’re unleashing the power of cloud-based analytics on large data sets to unlock the insights they and the business need to make smarter decisions. All of these factors challenge DevOps maturity.

DevOps 196

DevOps Analytics Storage Metrics 196

Percona

AUGUST 9, 2023

MySQL 8 brought a significant architectural transformation by replacing the traditional MyISAM-based system tables with the Transaction Data Dictionary (TDD), a more efficient and reliable approach. The primary goal of the Transaction Data Dictionary (TDD) is to enhance the overall performance, stability, and scalability of MySQL databases.

Database 82

Database Storage Scalability Games 82

Dynatrace

NOVEMBER 3, 2023

that offers security, scalability, and simplicity of use. already address SNMP, WMI, SQL databases, and Prometheus technologies, serving the monitoring needs of hundreds of Dynatrace customers. Python code also carries limited scalability and the burden of governing its security in production environments and lifecycle management.

Technology 255

Technology Technology Scalability Availability 255

Percona

OCTOBER 19, 2023

Welcome to the first installment of our series: Scalable Solutions with Percona Distribution for PostgreSQL. In this guide, we will demonstrate how to establish a Citus database spanning multiple nodes by using Percona Distribution for PostgreSQL. Ensure that these values align with your desired database configuration.

Scalability 91

Scalability Database Open Source Traffic 91

Dynatrace

APRIL 20, 2023

Modern organizations ingest petabytes of data daily, but legacy approaches to log analysis and management cannot accommodate this volume of data. based financial services group, discussed how the bank uses log monitoring on the Dynatrace platform with an emphasis on observability and security data.

Analytics 201

Analytics Infrastructure Storage Architecture 201

Dynatrace

MAY 1, 2023

IT operations analytics is the process of unifying, storing, and contextually analyzing operational data to understand the health of applications, infrastructure, and environments and streamline everyday operations. ITOA collects operational data to identify patterns and anomalies for faster incident management and near-real-time insights.

Analytics 201

Analytics Artificial Intelligence Big Data Open Source 201

The Netflix TechBlog

MARCH 10, 2023

This platform has evolved from supporting studio applications to data science applications, machine-learning applications to discover the assets metadata, and build various data facts. Hence we built the data pipeline that can be used to extract the existing assets metadata and process it specifically to each new use case.

Media 241

Media Traffic Processing Design 241

Percona

MAY 6, 2024

MongoDB offers powerful features and scalability, but like any database system, it has security challenges that must be addressed to protect sensitive data as well as comply with regulatory standards like GDPR, HIPAA, PCI DSS, and AM/ATF.

Best Practices 92

Best Practices Database Scalability Systems 92

DZone

JULY 3, 2023

Data engineering projects often require the setup and management of complex infrastructures that support data processing, storage, and analysis. In this article, we will explore the benefits of leveraging IaC for data engineering projects and provide detailed implementation steps to get started.

Data Engineering 150

Data Engineering Infrastructure Code Engineering 150

The Netflix TechBlog

OCTOBER 18, 2022

by Jun He , Akash Dwivedi , Natallia Dzenisenka , Snehal Chennuru , Praneeth Yenugutala , Pawan Dixit At Netflix, Data and Machine Learning (ML) pipelines are widely used and have become central for the business, representing diverse use cases that go beyond recommendations, predictions and data transformations.

Java 208

Java Scalability Traffic Architecture 208

DZone

JANUARY 2, 2024

Sharding, a database architecture pattern, involves partitioning a database into smaller, faster, more manageable parts called shards. Each shard is a distinct database, and collectively, these shards make up the entire database. What Is Sharding?

Architecture 269

Architecture Database Scalability Servers 269

Percona

JUNE 22, 2023

To make data count and to ensure cloud computing is unabated, companies and organizations must have highly available databases. A basic high availability database system provides failover (preferably automatic) from a primary database node to redundant nodes within a cluster. More in the following sub-section.)

Availability 113

Availability Database Open Source Hardware 113

Percona

MARCH 23, 2023

When it comes to enterprise-level databases, there are several options available in the market, but PostgreSQL stands out as one of the most popular and reliable choices. PostgreSQL is a free and open source object-relational database management system (ORDBMS) that has existed since the mid-1990s.

Database 121

Database Open Source Traffic Small Business 121

Percona

OCTOBER 17, 2023

This blog post discusses using the CLONE INSTANCE command with the safety option DATA DIRECTORY when you do not have enough disk space to store two datasets. However, when you clone a replica that already has a huge dataset, you may not have enough space for two datasets: one from the source server and data present on the replica.

Open Source 105

Open Source Servers Database Strategy 105

InfoQ

NOVEMBER 6, 2024

Pinterest has modernized and enhanced its Goku time-series database. The recent updates focus on optimizing storage and resource usage without compromising service quality. By Mohit Palriwal

Database 90

Database Efficiency Storage Design 90

The Netflix TechBlog

MARCH 5, 2019

While our engineering teams have and continue to build solutions to lighten this cognitive load (better guardrails, improved tooling, …), data and its derived products are critical elements to understanding, optimizing and abstracting our infrastructure. In the Reliability space, our data teams focus on two main approaches.

Infrastructure 184

Infrastructure Cloud Scalability AWS 184

Percona

OCTOBER 18, 2023

Migrating a proprietary database to open source is a major decision that can significantly affect your organization. Today, we’ll be taking a deep dive into the intricacies of database migration, along with specific solutions to help make the process easier.

Open Source 98

Open Source Database Hardware Strategy 98

DZone

NOVEMBER 21, 2023

Thanks to its expressive query language (PromQL), scalability, and configurable data format, it remains one of the most popular tools for data collection. Paired with Prometheus exporters, the tool can adapt to a variety of surroundings, which is one of its strongest points.

Best Practices 287

Best Practices Scalability Monitoring Database 287

The Netflix TechBlog

MARCH 25, 2019

Building and Scaling Data Lineage at Netflix to Improve Data Infrastructure Reliability, and Efficiency By: Di Lin , Girish Lingappa , Jitender Aswani Imagine yourself in the role of a data-inspired decision maker staring at a metric on a dashboard about to make a critical business decision but pausing to ask a question?—?“Can

Infrastructure 223

Infrastructure Big Data Transportation Architecture 223

The Netflix TechBlog

OCTOBER 27, 2020

By Tianlong Chen and Ioannis Papapanagiotou Netflix has more than 195 million subscribers that generate petabytes of data everyday. Data scientists and engineers collect this data from our subscribers and videos, and implement data analytics models to discover customer behaviour with the goal of maximizing user joy.

Latency 249

Latency Storage Big Data Tuning 249

DZone

SEPTEMBER 25, 2023

This is an article from DZone's 2023 Database Systems Trend Report. At the heart of this digital transformation lies the critical role of cloud databases — the backbone of modern data management. At the heart of this digital transformation lies the critical role of cloud databases — the backbone of modern data management.

Education 262

Education Innovation Scalability Cloud 262

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. In databases like MySQL and PostgreSQL, transaction logs are the source of CDC events.

Database 217

Database Traffic Transportation Open Source 217

Percona

NOVEMBER 3, 2023

Sure, database migration is complex, particularly when you’re looking to migrate from a proprietary database to an open source one. Database migration is almost always time-consuming, tedious, and full of potential pitfalls. Database migration is complex Let’s start here. What is the best path to migrate?

Database 89

Database Open Source Benchmarking Strategy 89

Abhishek Tiwari

SEPTEMBER 8, 2024

When a machine learning model is trained on a dataset, not all data points contribute equally to the model's performance.  Unfortunately value of data for training purposes is often nebulous and difficult to quantify. Applying data valuation to large language models (LLMs) like GPT-3, Claude 3, Llama 3.1

Scalability 52

Scalability Efficiency Database Design 52

DZone

FEBRUARY 15, 2023

In this article, I’m going to demonstrate how you can migrate a comprehensive web application from MySQL to YugabyteDB using the open-source data migration engine YugabyteDB Voyager. Nowadays, many people migrate their applications from traditional, single-server relational databases to distributed database clusters.

Open Source 298

Open Source Scalability Database Servers 298

Percona

MARCH 30, 2023

The use of open source databases has increased steadily in recent years. Past trepidation — about perceived vulnerabilities and performance issues — has faded as decision makers realize what an “open source database” really is and what it offers. What is an open source database?

Open Source 80

Open Source Database Storage Scalability 80

DZone

OCTOBER 1, 2023

As a distributed database, your data is partitioned into “shards” which are then allocated to one or more servers. Because of this sharding, a read or write request to an Elasticsearch cluster requires coordinating between multiple nodes as there is no “global view” of your data on a single server.

Open Source 251

Open Source Tuning Analytics Scalability 251

Percona

OCTOBER 20, 2023

Open source migration can be a game-changer, offering flexibility, scalability, and cost-effectiveness. Heck, typing “database migration plan” into Google and reading this blog could constitute planning. Inadequate training Transitioning to open source requires a team that understands your target database solution.

Open Source 94

Open Source Database Best Practices Strategy 94

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. In databases like MySQL and PostgreSQL, transaction logs are the source of CDC events.

Database 203

Database Traffic Transportation Open Source 203

Percona

OCTOBER 20, 2023

You can read part one here: Scalable Solutions With Percona Distribution for PostgreSQL: Set Up Three PostgreSQL Database Instances. Citus is an open-source extension for PostgreSQL that expands its capacity to manage databases of varying scales, ranging from individual nodes to extensively distributed database clusters.

Scalability 98

Scalability Database Open Source Systems 98

The Netflix TechBlog

DECEMBER 21, 2020

By Anupom Syam Background At Netflix, our current data warehouse contains hundreds of Petabytes of data stored in AWS S3 , and each day we ingest and create additional Petabytes. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage 209

Storage Latency Efficiency Data Engineering 209