The Netflix TechBlog

DECEMBER 21, 2020

At this scale, we can gain a significant amount of performance and cost benefits by optimizing the storage layout (records, objects, partitions) as the data lands into our warehouse. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage 209

Storage Latency Efficiency Data Engineering 209

InfoQ

MAY 15, 2023

A horizontally scalable exabyte-scale blob storage system which operates out of multiple regions, Magic Pocket is used to store all of Dropbox’s data. Adopting SMR technology and erasure codes, the system has extremely high durability guarantees but is cheaper than operating in the cloud. By Facundo Agriel

Storage 114

Storage Systems Scalability Cloud 114

Uber Engineering

JULY 6, 2024

Kafka Tiered Storage, developed in collaboration with the Apache Kafka community, introduces the separation of storage and processing in brokers, significantly improving the scalability, reliability, and efficiency of Kafka clusters.

Storage 67

Storage Scalability Efficiency Processing 67

Percona

FEBRUARY 1, 2023

This post will look at using The Oversized-Attribute Storage Technique (TOAST) to improve performance and scalability. Therefore, TOAST is a storage technique used in PostgreSQL to handle large data objects such as images, videos, and audio files.

Storage 96

Storage Scalability Strategy Performance 96

DZone

SEPTEMBER 19, 2024

RabbitMQ is a powerful and widely used message broker that facilitates communication between distributed applications by handling the transmission, storage, and delivery of messages. This decoupling allows applications to operate independently, promoting scalability, resilience, and fault tolerance.

Storage 264

Storage Performance Scalability Architecture 264

Percona

JANUARY 4, 2024

Managing storage and performance efficiently in your MySQL database is crucial, and general tablespaces offer flexibility in achieving this. In contrast to the single system tablespace that holds system tables by default, general tablespaces are user-defined storage containers for multiple InnoDB tables.

Storage 94

Storage Engineering Database Open Source 94

DZone

JULY 19, 2023

To address this need, the integration of cloud computing and virtualization has emerged as a groundbreaking solution as these technologies boast scalability and flexibility, entirely transforming the operational landscape. The IT infrastructure and services will reach $35.98 billion by 2025.

Virtualization 130

Virtualization Scalability Energy Cloud 130

The Netflix TechBlog

OCTOBER 8, 2024

The Key-Value Abstraction offers a flexible, scalable solution for storing and accessing structured key-value data, while the Data Gateway Platform provides essential infrastructure for protecting, configuring, and deploying the data tier.

Latency 236

Latency Storage Traffic Tuning 236

DZone

JUNE 3, 2019

Using local SSDs inside of the GPU node delivers fast access to data during training, but introduces challenges that impact the overall solution in terms of scalability, data access, and data protection. For example, one well-respected vendor's standard solution is limited to 7.5TB of internal storage, and it can only scale to 30TB.

Storage 100

Storage Performance Network Scalability 100

InfoQ

OCTOBER 14, 2023

times the storage per node than before” without a price change. Google recently announced various improvements to Cloud Spanner, its distributed, decoupled relational database service with a “50% increase in throughput and 2.5 By Steef-Jan Wiggers

Google 103

Google Storage Cloud Database 103

DZone

JULY 13, 2023

In today's data-driven world, organizations need efficient and scalable data pipelines to process and analyze large volumes of data. Each zone has a specific purpose and plays a critical role in building efficient and scalable data pipelines.

Azure 246

Azure Architecture Efficiency Processing 246

DZone

OCTOBER 24, 2023

The ELK stack is an abbreviation for Elasticsearch, Logstash, and Kibana, which offers the following capabilities: Elasticsearch: a scalable search and analytics engine with a log analytics tool and application-formed database, perfect for data-driven applications.

Analytics 253

Analytics Storage Infrastructure Scalability 253

DZone

JUNE 12, 2019

A system that has the ability to easily scale resources to meet the increasing workload without affecting the performance is known as a scalable system. The workload could refer to anything from an increase in users, storage, or a number of transactions.

Scalability 100

Scalability Software Architecture Storage Architecture 100

DZone

MARCH 2, 2023

It must be said that this video traffic phenomenon primarily owes itself to modernizations in the scalability of streaming infrastructure, which simply weren’t present fifteen years ago.

Social Media 279

Social Media Java Traffic Internet 279

The Netflix TechBlog

SEPTEMBER 18, 2024

Central to this infrastructure is our use of multiple online distributed databases such as Apache Cassandra , a NoSQL database known for its high availability and scalability. This flexibility allows our Data Platform to route different use cases to the most suitable storage system based on performance, durability, and consistency needs.

Latency 248

Latency Storage Cache Efficiency 248

DZone

FEBRUARY 23, 2023

Any real-time analytics provider or batching/storage adaptor can transform and store data supplied to an event hub. Event Hubs is a simple, dependable, and scalable real-time data intake solution. Build dynamic data pipelines that stream millions of events per second from any source to quickly address business concerns.

Azure 306

Azure Big Data Storage Analytics 306

High Scalability

FEBRUARY 1, 2019

Glacier is cheap because it isn't "always ready" storage. Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading). There are TRADEOFFS YOU HAVE TO CONSIDER. Works great.

Internet 146

Internet Internet Scalability Games 146

Scalegrid

SEPTEMBER 10, 2024

At ScaleGrid, we’re always pushing the boundaries to offer more flexibility and scalability to our customers. Customer-Requested Features We’re always listening to your feedback, so we added the ability to access additional storage without upgrading to a larger plan.

Latency 130

Latency AWS Storage Tuning 130

Dynatrace

NOVEMBER 18, 2024

The Grail™ data lakehouse provides fast, auto-indexed, schema-on-read storage with massively parallel processing (MPP) to deliver immediate, contextualized answers from all data at scale. Through Azure Native Dynatrace Service, customers can seamlessly adopt these technologies to modernize and enhance their cloud operations.

Cloud 150

Cloud Azure Artificial Intelligence Innovation 150

InfoQ

DECEMBER 29, 2023

Zendesk reduced its data storage costs by over 80% by migrating from DynamoDB to a tiered storage solution using MySQL and S3. The company considered different storage technologies and decided to combine the relational database and the object store to strike a balance between querybility and scalability while keeping the costs down.

Storage 137

Storage Scalability Database Technology 137

Dynatrace

NOVEMBER 11, 2022

Therefore, they need an environment that offers scalable computing, storage, and networking. Hyperconverged infrastructure (HCI) is an IT architecture that combines servers, storage, and networking functions into a unified, software-centric platform to streamline resource management. What is hyperconverged infrastructure?

Infrastructure 214

Infrastructure Storage Virtualization Network 214

The Netflix TechBlog

APRIL 27, 2023

While we were able to put out the immediate fire by disabling the newly created alerts, this incident raised some critical concerns around the scalability of our alerting system. It became clear to us that we needed to solve the scalability problem with a fundamentally different approach. OK, Results?

Storage 294

Storage Cache Metrics Database 294

Dynatrace

OCTOBER 7, 2021

This means you no longer have to provision, scale, and maintain servers to run your applications, databases, and storage systems. Speed is next; serverless solutions are quick to spin up or down as needed, and there are no delays due to limited storage or resource access. Scalability. Finally, there’s scalability.

Serverless 294

Serverless AWS Lambda Storage 294

DZone

APRIL 11, 2023

Data processing in the cloud has become increasingly popular due to its scalability, flexibility, and cost-effectiveness. It provides built-in connectors for various data sources such as databases, file systems, cloud storage, and more.

Azure 246

Azure Analytics Storage Cloud 246

High Scalability

DECEMBER 21, 2018

It's HighScalability time: Have a very scalable Xmas everyone! Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading). See you in the New Year. I'd really appreciate it.

Internet 137

Internet Internet Scalability Serverless 137

High Scalability

JANUARY 11, 2022

Firstly, the synchronous process which is responsible for uploading image content on file storage, persisting the media metadata in graph data-storage, returning the confirmation message to the user and triggering the process to update the user activity. Fetching User Feed. Sample Queries supported by Graph Database. Optimization.

Design 334

Design Media Storage Logistics 334

DZone

NOVEMBER 27, 2023

An open-source distributed SQL query engine, Trino is widely used for data analytics on distributed data storage. Optimizing Trino to make it faster can help organizations achieve quicker insights and better user experiences, as well as cut costs and improve infrastructure efficiency and scalability. But how do we do that?

Tuning 162

Tuning Speed Performance Open Source 162

Scalegrid

JANUARY 5, 2024

Despite initial investment costs, DBMS presents long-term savings and improved efficiency through automated processes, efficient query optimizations, and scalability, contributing to enhanced decision-making and end-user productivity. Practical Applications of DBMS DBMS finds practical applications in various fields.

Efficiency 147

Efficiency Storage Database Scalability 147

High Scalability

DECEMBER 16, 2022

Cloud computing is a model of computing that delivers computing services over the internet, including storage, data processing, and networking. It allows users to access and use shared computing resources, such as servers, storage, and applications, on demand and without the need to manage the underlying infrastructure. Can you expand?

Cloud 201

Cloud Serverless Internet Internet 201

Dynatrace

AUGUST 11, 2022

Before an organization moves to function as a service, it’s important to understand how it works, its benefits and challenges, its effect on scalability, and why cloud-native observability is essential for attaining peak performance. Infrastructure as a service (IaaS) handles compute, storage, and network resources. What is FaaS?

Development 262

Development Serverless Best Practices Lambda 262

Dynatrace

MAY 7, 2024

New processors: Introducing new processors, including Metric Selector and Content Modifier, for selective data processing and metadata adjustment, improving data relevance and storage efficiency. By default, you have a storage type memory, but you may exceed this buffer limit if you have a lot of data.

Best Practices 256

Best Practices IoT Metrics Storage 256

DZone

JULY 3, 2023

Data engineering projects often require the setup and management of complex infrastructures that support data processing, storage, and analysis. Traditionally, this process involved manual configuration, leading to potential inconsistencies, human errors, and time-consuming deployments.

Data Engineering 150

Data Engineering Infrastructure Code Engineering 150

Dynatrace

MAY 21, 2024

That’s because it does not require any pre-prepared schemas, and access to cold/hot storage is fully automatic and with zero latency. Insights are therefore dispersed in a multitude of data lakes, storage systems, and reporting platforms. Moreover, it is fast, powered by its massively parallel processing data lakehouse.

Technology 212

Technology Technology Analytics Storage 212

Dynatrace

SEPTEMBER 5, 2024

Dynatrace supports scalable data ingestion, ensuring your observability infrastructure grows with your cloud environment. The dashboard tracks a histogram chart of total storage utilized with logs daily. You can see in a table retention periods by the number of logs and storage they consumed.

Innovation 273

Innovation AWS Analytics Storage 273

Scalegrid

MARCH 28, 2024

In this comparison of Redis vs Memcached, we strip away the complexity, focusing on each in-memory data store’s performance, scalability, and unique features. can enhance Redis by handling management tasks, backups, and scalability, facilitating global reach and easy cloud integration for global businesses.

Cache 130

Cache Storage Architecture Scalability 130

Dynatrace

OCTOBER 23, 2023

Secondly, determining the correct allocation of resources (CPU, memory, storage) to each virtual machine to ensure optimal performance without over-provisioning can be difficult. Firstly, managing virtual networks can be complex as networking in a virtual environment differs significantly from traditional networking.

Efficiency 227

Efficiency Virtualization Hardware Performance 227

High Scalability

JUNE 15, 2018

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading). They'll love you even more. Domain Specific Architectures are getting 20x and 40x improvements, not just 5-10%.

Internet 113

Internet Internet Scalability Blockchain 113