DZone

OCTOBER 30, 2024

In this guide, we’ll walk through the implementation of an LSM tree in Golang , discuss features such as Write-Ahead Logging ( WAL ), block compression, and BloomFilters , and compare it with more traditional key-value storage systems and indexing strategies.

Strategy 244

Strategy Storage Efficiency Database 244

The Netflix TechBlog

NOVEMBER 12, 2024

After selecting a mode, users can interact with APIs without needing to worry about the underlying storage mechanisms and counting methods. Failures in a distributed system are a given, and having the ability to safely retry requests enhances the reliability of the service.

Latency 251

Latency Cache Infrastructure Strategy 251

The Netflix TechBlog

DECEMBER 21, 2020

At this scale, we can gain a significant amount of performance and cost benefits by optimizing the storage layout (records, objects, partitions) as the data lands into our warehouse. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage 212

Storage Latency Efficiency Data Engineering 212

Dynatrace

JULY 29, 2021

One main advantage of using a product in SaaS mode is the automatic scaling of resources based on system load. Using existing storage resources optimally is key to being able to capture the right data over time. Increased storage space availability. Compression of data that’s older than three days utilizes one virtual CPU.

Storage 246

Storage Virtualization Infrastructure Availability 246

Dynatrace

AUGUST 21, 2024

These releases often assumed ideal conditions such as zero latency, infinite bandwidth, and no network loss, as highlighted in Peter Deutsch’s eight fallacies of distributed systems. With Dynatrace, teams can seamlessly monitor the entire system, including network switches, database storage, and third-party dependencies.

Engineering 262

Engineering Systems Latency Metrics 262

Scalegrid

FEBRUARY 8, 2024

A distributed storage system is foundational in today’s data-driven landscape, ensuring data spread over multiple servers is reliable, accessible, and manageable. This guide delves into how these systems work, the challenges they solve, and their essential role in businesses and technology.

Storage 130

Storage Systems Big Data Azure 130

Dynatrace

SEPTEMBER 8, 2022

Log management is an organization’s rules and policies for managing and enabling the creation, transmission, analysis, storage, and other tasks related to IT systems’ and applications’ log data. Distributed cloud systems are complex, dynamic, and difficult to manage without the proper tools. Dynatrace news.

Cloud 260

Cloud Systems Analytics DevOps 260

Dynatrace

NOVEMBER 4, 2024

There’s a goldmine of business data traversing your IT systems, yet most of it remains untapped. Other data sources, including APIs and log files — are used to expand access, often to external or proprietary systems. In fact, it’s likely that some of your critical business systems already write business data to log files.

Analytics 241

Analytics Airlines Metrics Monitoring 241

Dynatrace

JANUARY 12, 2022

Messaging systems can significantly improve the reliability, performance, and scalability of the communication processes between applications and services. In serverless and microservices architectures, messaging systems are often used to build asynchronous service-to-service communication. Dynatrace news. This is great!

Analytics 246

Analytics Systems DevOps Healthcare 246

Dynatrace

DECEMBER 12, 2024

The Federal Reserve Regulation HH in the United States focuses on operational resilience requirements for systemically important financial market utilities. Proactive systems like Dynatrace’s Davis AI can automate responses to threats, swiftly implementing remediation while keeping executives informed of actions taken and their impact.

Analytics 289

Analytics Government Efficiency Innovation 289

Dynatrace

NOVEMBER 18, 2024

The Grail™ data lakehouse provides fast, auto-indexed, schema-on-read storage with massively parallel processing (MPP) to deliver immediate, contextualized answers from all data at scale. Through Azure Native Dynatrace Service, customers can seamlessly adopt these technologies to modernize and enhance their cloud operations.

Cloud 263

Cloud Azure Artificial Intelligence Innovation 263

DZone

JUNE 21, 2024

This article analyzes the correlation between block sizes and their impact on storage performance. This paper deals with definitions and understanding of structured data vs unstructured data, how various storage segments react to block size changes, and differences between I/O-driven and throughput-driven workloads.

Storage 162

Storage Performance Benchmarking Processing 162

Scalegrid

FEBRUARY 6, 2025

Introduction to Message Brokers Message brokers enable applications, services, and systems to communicate by acting as intermediaries between senders and receivers. This decoupling simplifies system architecture and supports scalability in distributed environments.

Latency 147

Latency Analytics Architecture Storage 147

DZone

AUGUST 6, 2021

On a small scale, this isn’t problematic but when dealing with large-scale deployments, organizations end up using lots of computing, storage, and human resources just to manage their indexes. When companies are handling terabytes of data every day, the database-backed log management system becomes untenable.

Storage 189

Storage Database Systems Efficiency 189

Dynatrace

MARCH 5, 2025

Integration with existing systems and processes : Integration with existing IT infrastructure, observability solutions, and workflows often requires significant investment and customization. Storage calculations assume that one terabyte consumes 1.2 Cloud storage is replicated twice, which doubles the energy consumption per terabyte.

Energy 223

Energy Analytics Traffic Cloud 223

The Netflix TechBlog

MARCH 4, 2025

Part 3: System Strategies and Architecture By: VarunKhaitan With special thanks to my stunning colleagues: Mallika Rao , Esmir Mesic , HugoMarques This blog post is a continuation of Part 2 , where we cleared the ambiguity around title launch observability at Netflix. The request schema for the observability endpoint.

Traffic 180

Traffic Strategy Entertainment Innovation 180

The Netflix TechBlog

FEBRUARY 14, 2025

It requires a state-of-the-art system that can track and process these impressions while maintaining a detailed history of each profiles exposure. In this multi-part blog series, we take you behind the scenes of our system that processes billions of impressions daily.

Tuning 166

Tuning Latency Efficiency Storage 166

The Netflix TechBlog

MARCH 31, 2025

Besides the need for robust cloud storage for their media, artists need access to powerful workstations and real-time playback. Local storage and compute services are connected through the Netflix Open Connect network (Netflix Content Delivery Network) to the infrastructure of Amazon Web Services (AWS).

Media 203

Media Logistics Innovation Cloud 203

Dynatrace

APRIL 16, 2025

Use cases Identifying misconfigurations: Continuously scanning cloud environments to detect misconfigurations (such as open network ports, missing security patches, and exposed storage buckets) to help maintain a secure, stable infrastructure.

Serverless 219

Serverless Azure Cloud Infrastructure 219

DZone

SEPTEMBER 22, 2022

Users can allocate different storage tiers as the resources for Alluxio workers, including MEM/SSD/HDD, which are further composed of directories. Alluxio workers are responsible for managing local resources, and they store data as blocks.

Code 246

Code Virtualization Systems Storage 246

The Netflix TechBlog

DECEMBER 17, 2024

To achieve this, we are committed to building robust systems that deliver comprehensive observability, enabling us to take full accountability for every title on ourservice. Each title represents countless hours of effort and creativity, and our systems need to honor that uniqueness. Yet, these pages couldnt be more different.

Traffic 172

Traffic Scalability Strategy Monitoring 172

DZone

FEBRUARY 27, 2025

Multimodal data processing is the evolving need of the latest data platforms powering applications like recommendation systems, autonomous vehicles, and medical diagnostics. Handling multimodal data spanning text, images, videos, and sensor inputs requires resilient architecture to manage the diversity of formats and scale.

Efficiency 130

Efficiency Processing Latency Storage 130

Scalegrid

MAY 11, 2024

MongoDB offers several storage engines that cater to various use cases. The default storage engine in earlier versions was MMAPv1, which utilized memory-mapped files and document-level locking. The newer, pluggable storage engine, WiredTiger, addresses this by using prefix compression, collection-level locking, and row-based storage.

Storage 130

Storage Engineering Cache Database 130

Scalegrid

MAY 13, 2020

High performance, query optimization, open source and polymorphic data storage are the major Greenplum advantages. The MPP system leverages a shared-nothing architecture to handle multiple operations in parallel. Typically an MPP system has one leader node and one or many compute nodes. Polymorphic Data Storage.

Big Data 321

Big Data Database Artificial Intelligence Open Source 321

Dynatrace

MARCH 11, 2025

The Dynatrace platform now enables comprehensive data exploration and interactive analytics across data sets (trace, logs, events, and metrics)empowering you to solve complex use cases, handle any observability scenario, and gain unprecedented visibility into your systems. But why stop there?

Games 165

Games Analytics Innovation Metrics 165

Dynatrace

APRIL 4, 2025

Journald provides unified structured logging for systems, services, and applications, eliminating the need for custom parsing for severity or details. Dynatrace Grail lets you focus on extracting insights rather than managing complex schemas or index and storage concepts.

Analytics 147

Analytics Operating System Scalability Infrastructure 147

Percona

DECEMBER 11, 2023

There are a wealth of options on how you can approach storage configuration in Percona Operator for PostgreSQL , and in this blog post, we review various storage strategies — from basics to more sophisticated use cases. For example, you can choose the public cloud storage type – gp3, io2, etc, or set file system.

Storage 122

Storage Strategy Cloud Network 122

Dynatrace

APRIL 22, 2025

You might have state-of-the-art surveillance systems and guards at the main entrance, but if a side door is left unlocked, all the security becomes meaningless. Insecure storage of secrets such as API keys and passwords, as well as workload configurations. Every permission granted should be scrutinized and justified. Real-world impact.

130

130

InfoQ

JANUARY 25, 2023

In this article, Rogerio Robetti discusses the challenges in auto-scaling stateful storage systems and proposes an opinionated design solution to automatically scale up (vertical) and scale out (horizontal) from a single node up to several nodes in a cluster with minimum configuration and interference of the operator.

Design 144

Design Systems Storage Data Engineering 144

High Scalability

JANUARY 11, 2022

The streaming data store makes the system extensible to support other use-cases (e.g. System Components. The system will comprise of several micro-services each performing a separate task. After that, the post gets added to the feed of all the followers in the columnar data storage. Fetching User Feed. Optimization.

Design 334

Design Media Storage Logistics 334

Scalegrid

FEBRUARY 27, 2025

Performance Optimizations PostgreSQL 17 significantly improves performance, query handling, and database management, making it more efficient for high-demand systems. Unlike full backups that duplicate everything, incremental backups store only changes since the last save, reducing storage needs and speeding up recovery.

Speed 130

Speed Database Storage Efficiency 130

The Netflix TechBlog

FEBRUARY 13, 2023

Media Feature Storage: Amber Storage Media feature computation tends to be expensive and time-consuming. This feature store is equipped with a data replication system that enables copying data to different storage solutions depending on the required access patterns.

Media 297

Media Storage Infrastructure Systems 297

DZone

JUNE 24, 2024

Objectives Modern AI innovations require proper infrastructure, especially concerning data throughput and storage capabilities. While GPUs drive faster results, legacy storage solutions often lag behind, causing inefficient resource utilization and extended times in completing the project.

Systems 130

Systems Software Software Storage 130

Dynatrace

OCTOBER 7, 2021

This means you no longer have to provision, scale, and maintain servers to run your applications, databases, and storage systems. Speed is next; serverless solutions are quick to spin up or down as needed, and there are no delays due to limited storage or resource access. AWS offers four serverless offerings for storage.

Serverless 325

Serverless AWS Lambda Storage 325

The Netflix TechBlog

SEPTEMBER 18, 2024

Our goal was to build a versatile and efficient data storage solution that could handle a wide variety of use cases, ranging from the simplest hashmaps to more complex data structures, all while ensuring high availability, tunable consistency, and low latency. Developers just provide their data problem rather than a database solution!

Latency 260

Latency Storage Cache Servers 260

The Netflix TechBlog

APRIL 27, 2023

Engineers want their alerting system to be realtime, reliable, and actionable. A few years ago, we were paged by our SRE team due to our Metrics Alerting System falling behind — critical application health alerts reached engineers 45 minutes late! In other words, false positives are bad but false negatives are the absolute worst!

Storage 297

Storage Cache Metrics Database 297

DZone

JUNE 28, 2024

At first, data tiering was a tactic used by storage systems to reduce data storage costs. This involved grouping data that was not accessed as often into more affordable, if less effective, storage array choices. Even though they are quite costly, SSDs and flash can be categorized as high-performance storage classes.

Azure 223

Azure Storage Performance Cloud 223