Scalegrid

MAY 11, 2024

MongoDB offers several storage engines that cater to various use cases. The default storage engine in earlier versions was MMAPv1, which utilized memory-mapped files and document-level locking. The newer, pluggable storage engine, WiredTiger, addresses this by using prefix compression, collection-level locking, and row-based storage.

Storage 130

Storage Engineering Cache Database 130

DZone

AUGUST 16, 2023

Caching is the process of storing frequently accessed data or resources in a temporary storage location, such as memory or disk, to improve retrieval speed and reduce the need for repetitive processing.

Cache 246

Cache Scalability Performance Latency 246

DZone

APRIL 11, 2023

Data processing in the cloud has become increasingly popular due to its scalability, flexibility, and cost-effectiveness. This article will explore how these technologies can be used together to create an optimized data pipeline for data processing in the cloud.

Azure 246

Azure Analytics Storage Cloud 246

DZone

JANUARY 20, 2023

Having a distributed and scalable graph database system is highly sought after in many enterprise scenarios. Do Not Be Misled Designing and implementing a scalable graph database system has never been a trivial task.

Scalability 199

Scalability Big Data Hardware Internet 199

DZone

JULY 13, 2023

In today's data-driven world, organizations need efficient and scalable data pipelines to process and analyze large volumes of data. Medallion Architecture provides a framework for organizing data processing workflows into different zones, enabling optimized batch and stream processing.

Azure 246

Azure Architecture Efficiency Processing 246

Percona

JANUARY 4, 2024

Managing storage and performance efficiently in your MySQL database is crucial, and general tablespaces offer flexibility in achieving this. In contrast to the single system tablespace that holds system tables by default, general tablespaces are user-defined storage containers for multiple InnoDB tables.

Storage 89

Storage Engineering Database Hardware 89

Scalegrid

JANUARY 5, 2024

Enhanced data security, better data integrity, and efficient access to information. This article cuts through the complexity to showcase the tangible benefits of DBMS, equipping you with the knowledge to make informed decisions about your data management strategies. What are the key advantages of DBMS?

Efficiency 147

Efficiency Storage Database Scalability 147

Dynatrace

JANUARY 31, 2024

Organizations choose data-driven approaches to maximize the value of their data, achieve better business outcomes, and realize cost savings by improving their products, services, and processes. However, there are many obstacles and limitations along the way to becoming a data-driven organization. Understanding the context.

Analytics 202

Analytics Processing Transportation Storage 202

The Netflix TechBlog

JUNE 20, 2024

A summary of sessions at the first Data Engineering Open Forum at Netflix on April 18th, 2024 The Data Engineering Open Forum at Netflix on April 18th, 2024. At Netflix, we aspire to entertain the world, and our data engineering teams play a crucial role in this mission by enabling data-driven decision-making at scale.

Data Engineering 204

Data Engineering Engineering Entertainment Software Engineering 204

Dynatrace

JUNE 9, 2023

ln a world driven by macroeconomic uncertainty, businesses increasingly turn to data-driven decision-making to stay agile. They’re unleashing the power of cloud-based analytics on large data sets to unlock the insights they and the business need to make smarter decisions. All of these factors challenge DevOps maturity.

DevOps 194

DevOps Analytics Storage Metrics 194

Dynatrace

OCTOBER 4, 2022

Some time ago, at a restaurant near Boston, three Dynatrace colleagues dined and discussed the growing data challenge for enterprises. At its core, this challenge involves a rapid increase in the amount—and complexity—of data collected within a company. Work with different and independent data types. Thus, Grail was born.

Artificial Intelligence 276

Artificial Intelligence Analytics Storage Architecture 276

InfoQ

MAY 15, 2023

A horizontally scalable exabyte-scale blob storage system which operates out of multiple regions, Magic Pocket is used to store all of Dropbox’s data. Adopting SMR technology and erasure codes, the system has extremely high durability guarantees but is cheaper than operating in the cloud. By Facundo Agriel

Storage 114

Storage Systems Scalability Cloud 114

Scalegrid

MAY 13, 2020

It can scale towards a multi-petabyte level data workload without a single issue, and it allows access to a cluster of powerful servers that will work together within a single SQL interface where you can view all of the data. This feature-packed database provides powerful and rapid analytics on data that scales up to petabyte volumes.

Big Data 321

Big Data Database Artificial Intelligence Open Source 321

Dynatrace

OCTOBER 6, 2023

Grail: Enterprise-ready data lakehouse Grail, the Dynatrace causational data lakehouse, was explicitly designed for observability and security data, with artificial intelligence integrated into its foundation. Tables are a physical data model, essentially the type of observability data that you can store.

Artificial Intelligence 269

Artificial Intelligence Metrics Analytics Storage 269

Dynatrace

APRIL 20, 2023

Modern organizations ingest petabytes of data daily, but legacy approaches to log analysis and management cannot accommodate this volume of data. based financial services group, discussed how the bank uses log monitoring on the Dynatrace platform with an emphasis on observability and security data.

Analytics 198

Analytics Infrastructure Storage Architecture 198

DZone

JULY 3, 2023

Data engineering projects often require the setup and management of complex infrastructures that support data processing, storage, and analysis. In this article, we will explore the benefits of leveraging IaC for data engineering projects and provide detailed implementation steps to get started.

Data Engineering 150

Data Engineering Infrastructure Engineering Code 150

Percona

FEBRUARY 1, 2023

PostgreSQL is a fantastic database, but if you’re storing images, video, audio files, or other large data objects, you need to “toast” them to get optimal performance. This post will look at using The Oversized-Attribute Storage Technique (TOAST) to improve performance and scalability.

Storage 92

Storage Scalability Strategy Performance 92

Uber Engineering

JULY 6, 2024

Kafka Tiered Storage, developed in collaboration with the Apache Kafka community, introduces the separation of storage and processing in brokers, significantly improving the scalability, reliability, and efficiency of Kafka clusters.

Storage 67

Storage Scalability Efficiency Processing 67

The Netflix TechBlog

OCTOBER 27, 2020

By Tianlong Chen and Ioannis Papapanagiotou Netflix has more than 195 million subscribers that generate petabytes of data everyday. Data scientists and engineers collect this data from our subscribers and videos, and implement data analytics models to discover customer behaviour with the goal of maximizing user joy.

Latency 249

Latency Storage Big Data Tuning 249

Dynatrace

NOVEMBER 18, 2024

AI transformation, modernization, managing intelligent apps, safeguarding data, and accelerating productivity are all key themes at Microsoft Ignite 2024. Adopting AI to enhance efficiency and boost productivity is critical in a time of exploding data, cloud complexities, and disparate technologies.

Cloud 208

Cloud Azure Artificial Intelligence Innovation 208

DZone

JULY 19, 2023

To address this need, the integration of cloud computing and virtualization has emerged as a groundbreaking solution as these technologies boast scalability and flexibility, entirely transforming the operational landscape. The IT infrastructure and services will reach $35.98 billion by 2025.

Virtualization 130

Virtualization Scalability Energy Cloud 130

DZone

OCTOBER 24, 2023

The ELK stack is an abbreviation for Elasticsearch, Logstash, and Kibana, which offers the following capabilities: Elasticsearch: a scalable search and analytics engine with a log analytics tool and application-formed database, perfect for data-driven applications.

Analytics 253

Analytics Storage Infrastructure Scalability 253

DZone

JUNE 3, 2019

Using local SSDs inside of the GPU node delivers fast access to data during training, but introduces challenges that impact the overall solution in terms of scalability, data access, and data protection.

Storage 100

Storage Performance Network Scalability 100

DZone

FEBRUARY 23, 2023

Introduction With big data streaming platform and event ingestion service Azure Event Hubs , millions of events can be received and processed in a single second. Any real-time analytics provider or batching/storage adaptor can transform and store data supplied to an event hub.

Azure 306

Azure Big Data Storage Analytics 306

Dynatrace

FEBRUARY 15, 2023

Teams can now use an observability platform to unify operations, business, and development data to gain automatic, precise answers about cloud environments and user experience. As a result, these leaders can ask different questions of the same data. Additionally, its modern architecture delivers cost-effective storage and compute.

Analytics 130

Analytics Storage Cloud Code 130

The Netflix TechBlog

APRIL 27, 2023

While we were able to put out the immediate fire by disabling the newly created alerts, this incident raised some critical concerns around the scalability of our alerting system. Atlas is an in-memory time-series database that ingests multiple billions of time-series per day and retains the last two weeks of data.

Storage 294

Storage Cache Metrics Database 294

Dynatrace

JANUARY 23, 2024

AI data analysis can help development teams release software faster and at higher quality. So how can organizations ensure data quality, reliability, and freshness for AI-driven answers and insights? And how can they take advantage of AI without incurring skyrocketing costs to store, manage, and query data?

Performance 196

Performance DevOps Innovation Artificial Intelligence 196

DZone

MARCH 2, 2023

According to data provided by Sandvine in their 2022 Global Internet Phenomena Report , video traffic accounted for 53.72% of the total volume of internet traffic in 2021, and the closest trailing category (social) came in at just 12.69%.

Social Media 279

Social Media Java Traffic Internet 279

High Scalability

JANUARY 11, 2022

from a client it performs two parallel operations: i) persisting the action in the data store ii) publish the action in a streaming data store for a pub-sub model. User Feed Service, Media Counter Service) read the actions from the streaming data store and performs their specific tasks. Data Models. Graph Data Models.

Design 334

Design Media Storage Logistics 334

Dynatrace

SEPTEMBER 5, 2024

Understanding that the first mile of getting data in can often be the hardest, Dynatrace continues to invest in log ingest, offering a range of out-of-the-box solutions within the Dynatrace Platform and apps. Dynatrace ActiveGate addresses these issues by enforcing configurable security settings and ensuring data uniformity.

Innovation 271

Innovation AWS Analytics Storage 271

InfoQ

DECEMBER 29, 2023

Zendesk reduced its data storage costs by over 80% by migrating from DynamoDB to a tiered storage solution using MySQL and S3. By Rafal Gancarz

Storage 137

Storage Scalability Database Technology 137

Dynatrace

MAY 7, 2024

Fluent Bit is a telemetry agent designed to receive data (logs, traces, and metrics), process or modify it, and export it to a destination. Fluent Bit can serve as a proxy before you send data to Dynatrace or similar. However, you can also use Fluent Bit as a processor because you can perform various actions on the data.

Best Practices 253

Best Practices IoT Metrics Storage 253

InfoQ

JUNE 16, 2023

Datadog created a dedicated data ingestion architecture offering exactly-once semantics for their third-generation event store, Husky. The event-driven architecture (EDA) can accommodate bursts in traffic in the multi-tenant platform with reasonable ingestion latency and acceptable operational costs. By Rafal Gancarz

Architecture 57

Architecture Scalability Latency Traffic 57

Dynatrace

OCTOBER 7, 2021

This means you no longer have to provision, scale, and maintain servers to run your applications, databases, and storage systems. Speed is next; serverless solutions are quick to spin up or down as needed, and there are no delays due to limited storage or resource access. Scalability. Finally, there’s scalability.

Serverless 291

Serverless AWS Lambda Storage 291

Dynatrace

NOVEMBER 11, 2022

More organizations are adopting a hybrid IT environment, with data center and virtualized components. However, today’s IT teams are stretched thin, with little time to firefight issues with deployment, integration, and data center management. But in an HCI framework, purchasing more storage means purchasing more compute.

Infrastructure 211

Infrastructure Storage Virtualization Network 211

DZone

NOVEMBER 27, 2023

An open-source distributed SQL query engine, Trino is widely used for data analytics on distributed data storage. Optimizing Trino to make it faster can help organizations achieve quicker insights and better user experiences, as well as cut costs and improve infrastructure efficiency and scalability.

Tuning 162

Tuning Speed Performance Open Source 162

Dynatrace

MAY 21, 2024

Data with context can improve your ability to deliver on your goals, modernize your organization, and accelerate business transformation. These outcomes are made easy through the platform’s unique ability to turn data into answers and action, in contextual, real-time, and cost-effective ways that were previously impossible.

Technology 210

Technology Technology Analytics Storage 210