The Netflix TechBlog

MARCH 4, 2025

The Insight TriadAPI To efficiently understand the health of a title and triage issues quickly, all implementations of the observability endpoint must answer: is the title eligible for this phase of promotion, if notwhy is it not eligible, and what can be done to fix any problems. The request schema for the observability endpoint.

Traffic 160

Traffic Strategy Entertainment Innovation 160

Scalegrid

MAY 13, 2020

It’s architecture was specially designed to manage large-scale data warehouses and business intelligence workloads by giving you the ability to spread your data out across a multitude of servers. Greenplum uses an MPP database design that can help you develop a scalable, high performance deployment. Greenplum Architectural Design.

Big Data 321

Big Data Database Artificial Intelligence Open Source 321

Scalegrid

FEBRUARY 24, 2025

This guide will cover how to distribute workloads across multiple nodes, set up efficient clustering, and implement robust load-balancing techniques. The architecture of RabbitMQ is meticulously designed for complex message routing, enabling dynamic and flexible interactions between producers and consumers.

Best Practices 147

Best Practices Traffic Strategy Efficiency 147

The Netflix TechBlog

DECEMBER 17, 2024

How can we design systems that recognize these nuances and empower every title to shine and bring joy to ourmembers? This led to a suite of fragmented scripts, runbooks, and ad hoc solutions scattered across teamsan approach that was neither sustainable nor efficient. Yet, these pages couldnt be more different.

Traffic 168

Traffic Scalability Strategy Monitoring 168

Uber Engineering

AUGUST 14, 2019

Maintaining Uber’s large-scale data warehouse comes with an operational cost in terms of ETL functions and storage. In our experience, optimizing for operational efficiency requires answering one key question: for which tables does the maintenance cost supersede utility?

Efficiency 110

Efficiency Engineering Design Storage 110

The Netflix TechBlog

SEPTEMBER 18, 2024

In this post, we dive deep into how Netflix’s KV abstraction works, the architectural principles guiding its design, the challenges we faced in scaling diverse use cases, and the technical innovations that have allowed us to achieve the performance and reliability required by Netflix’s global operations.

Latency 256

Latency Storage Cache Servers 256

Dynatrace

OCTOBER 7, 2021

This means you no longer have to provision, scale, and maintain servers to run your applications, databases, and storage systems. Instead of worrying about infrastructure management functions, such as capacity provisioning and hardware maintenance, teams can focus on application design, deployment, and delivery. Reliability.

Serverless 325

Serverless AWS Lambda Storage 325

Dynatrace

MAY 7, 2024

Fluent Bit is a telemetry agent designed to receive data (logs, traces, and metrics), process or modify it, and export it to a destination. Fluent Bit was designed to help you adjust your data and add the proper context, which can be helpful in the observability backend. What’s the difference between Fluent Bit and Fluentd?

Best Practices 279

Best Practices IoT Metrics Storage 279

Scalegrid

FEBRUARY 8, 2024

A distributed storage system is foundational in today’s data-driven landscape, ensuring data spread over multiple servers is reliable, accessible, and manageable. Understanding distributed storage is imperative as data volumes and the need for robust storage solutions rise.

Storage 130

Storage Systems Big Data Azure 130

Scalegrid

JANUARY 5, 2024

Enhanced data security, better data integrity, and efficient access to information. Despite initial investment costs, DBMS presents long-term savings and improved efficiency through automated processes, efficient query optimizations, and scalability, contributing to enhanced decision-making and end-user productivity.

Efficiency 147

Efficiency Storage Database Scalability 147

Dynatrace

AUGUST 8, 2019

One of our design principles is to make sure that Dynatrace is as easy to set up and use as possible. Smaller network and Dynatrace cluster storage footprint. The post Rebuilt OneAgent installer for Windows provides more efficient installation appeared first on Dynatrace blog. Dynatrace news. msi-based deployment.

Efficiency 164

Efficiency Network Storage Architecture 164

Dynatrace

FEBRUARY 24, 2025

Theyre often categorized by their function; core processes directly create customer value, support processes increase departmental efficiency, and management processes drive strategic goals and compliance. Regardless of their role, every business process is designed to improve business outcomes.

Processing 147

Processing Analytics Infrastructure Monitoring 147

Dynatrace

JANUARY 17, 2024

As organizations turn to artificial intelligence for operational efficiency and product innovation in multicloud environments, they have to balance the benefits with skyrocketing costs associated with AI. The good news is AI-augmented applications can make organizations massively more productive and efficient.

Strategy 288

Strategy Artificial Intelligence Storage Cloud 288

Dynatrace

AUGUST 1, 2022

Kubernetes was initially designed with a strong focus on stateless workloads, meaning these workloads do not need to store any persistent data. You quickly realize that it will take ages to fill up the overprovisioned database storage. Two days later, your database runs out of storage in the middle of the night. Dynatrace news.

Storage 246

Storage Database Network Metrics 246

The Netflix TechBlog

SEPTEMBER 24, 2021

Figure 1: A Simplified Video Processing Pipeline With this architecture, chunk encoding is very efficient and processed in distributed cloud computing instances. From chunk encoding to assembly and packaging, the result of each previous processing step must be uploaded to cloud storage and then downloaded by the next processing step.

Cloud 240

Cloud Media Storage Cache 240

The Netflix TechBlog

OCTOBER 8, 2024

Building on these foundational abstractions, we developed the TimeSeries Abstraction  — a versatile and scalable solution designed to efficiently store and query large volumes of temporal event data with low millisecond latencies, all in a cost-effective manner across various use cases.

Latency 235

Latency Storage Traffic Infrastructure 235

Dynatrace

SEPTEMBER 5, 2024

These developments open up new use cases, allowing Dynatrace customers to harness even more data for comprehensive AI-driven insights, faster troubleshooting, and improved operational efficiency. Customers have had a positive response to our native syslog implementation, noting its easy setup and efficiency.

Innovation 290

Innovation AWS Analytics Storage 290

DZone

JANUARY 23, 2022

Hardware - servers/storage hardware/software faults such as disk failure, disk full, other hardware failures, servers running out of allocated resources, server software behaving abnormally, intra DC network connectivity issues, etc. . – this is addressed through monitoring and redundancy. Redundancy by building additional data centers.

Hardware 243

Hardware DevOps Network Storage 243

Dynatrace

OCTOBER 4, 2022

While data lakes and data warehousing architectures are commonly used modes for storing and analyzing data, a data lakehouse is an efficient third way to store and analyze data that unifies the two architectures while preserving the benefits of both. Unlike data warehouses, however, data is not transformed before landing in storage.

Artificial Intelligence 246

Artificial Intelligence Storage Analytics Government 246

Dynatrace

OCTOBER 4, 2022

A data lakehouse addresses these limitations and introduces an entirely new architectural design. This architecture offers rich data management and analytics features (taken from the data warehouse model) on top of low-cost cloud storage systems (which are used by data lakes). Grail is built for such analytics, not storage.

Artificial Intelligence 290

Artificial Intelligence Analytics Storage Architecture 290

Scalegrid

JULY 17, 2020

JSONB supports indexing the JSON data, and is very efficient at parsing and querying the JSON data. JSONB storage has some drawbacks vs. traditional columns: PostreSQL does not store column statistics for JSONB columns. JSONB storage results in a larger storage footprint. It is a decomposed binary format to store JSON.

Storage 321

Storage Database Efficiency Availability 321

The Morning Paper

MARCH 8, 2020

Building an elastic query engine on disaggregated storage , Vuppalapati, NSDI’20. This paper describes the design decisions behind the Snowflake cloud-based data warehouse. have altered the many assumptions that guided the design and optimization of the Snowflake system. From shared-nothing to disaggregation.

Storage 112

Storage Engineering Cache Serverless 112

Dynatrace

JUNE 10, 2024

As a result, organizations are implementing security analytics to manage risk and improve DevSecOps efficiency. Security analytics solutions are designed to handle modern applications that rely on dynamic code and microservices. Dehydrated data has been compressed or otherwise altered for storage in a data warehouse.

Analytics 264

Analytics Network Open Source Hardware 264

High Scalability

SEPTEMBER 8, 2018

Anna is not only incredibly fast, it’s incredibly efficient and elastic too: an autoscaling, multi-tier, selectively-replicating cloud service. The issue is that Anna is now orders of magnitude more efficient than competing systems, in addition to being orders of magnitude faster. What's changed ?

Storage 199

Storage Performance AWS Cloud 199

Dynatrace

OCTOBER 17, 2023

AI can help automate tasks, improve efficiency, and identify potential problems before they occur. Data, AI, analytics, and automation are key enablers for efficient IT operations Data is the foundation for AI and IT automation. IT automation also helps improve operational efficiency by automating repetitive tasks.

Monitoring 278

Monitoring Artificial Intelligence Performance Analytics 278

Dynatrace

OCTOBER 4, 2022

As a result, organizations need software to work perfectly to create customer experiences, deliver innovation, and generate operational efficiency. Traditionally, though, to gain true business insight, organizations had to make tradeoffs between accessing quality, real-time data and factors such as data storage costs.

Analytics 246

Analytics Software Software Storage 246

Dynatrace

OCTOBER 17, 2023

Dynatrace has developed the purpose-built data lakehouse, Grail , eliminating the need for separate management of indexes and storage. All data is readily accessible without storage tiers, such as costly solid-state drives (SSDs). No storage tiers, no archiving or retrieval from archives, and no indexing or reindexing.

Retail 264

Retail Storage Best Practices Architecture 264

The Netflix TechBlog

FEBRUARY 13, 2023

We have been leveraging machine learning (ML) models to personalize artwork and to help our creatives create promotional content efficiently. Media Feature Storage: Amber Storage Media feature computation tends to be expensive and time-consuming. We accomplish this by paving the path to: Accessing and processing media data (e.g.

Media 293

Media Storage Infrastructure Systems 293

Dynatrace

MAY 26, 2021

Trace-based sampling can help you save storage costs. This can help you save money in storage costs in the long run. After all, it’s key to sample and store traces efficiently while not missing out on important events. OTel is designed to ensure flexibility and stability. So, the design of OTel allows for flexibility.

AWS 246

AWS Strategy Storage Metrics 246

Dynatrace

JUNE 6, 2023

Kubernetes enables efficient resource utilization by easily scaling applications and services based on demand. With the ability to monitor resource utilization metrics such as CPU and memory in real time, teams can optimize their operations, resulting in reduced cost and greater overall efficiency. Automated scaling. Self-healing.

Open Source 246

Open Source Storage Analytics Innovation 246

Dynatrace

NOVEMBER 11, 2020

Dynatrace OneAgent deployment and life-cycle management are already widely considered to be industry benchmarks for reliability and efficiency. Note that most of the changes we’ve introduced so far and those that are detailed below are all designed to be invisible to you, taking place entirely automatically in the background.

Best Practices 246

Best Practices Storage Java Benchmarking 246

Dynatrace

SEPTEMBER 13, 2023

This is a set of best practices and guidelines that help you design and operate reliable, secure, efficient, cost-effective, and sustainable systems in the cloud. The framework comprises six pillars: Operational Excellence, Security, Reliability, Performance Efficiency, Cost Optimization, and Sustainability.

AWS 305

AWS Efficiency Azure Cloud 305

The Netflix TechBlog

OCTOBER 27, 2020

The data warehouse is not designed to serve point requests from microservices with low latency. Therefore, we must efficiently move data from the data warehouse to a global, low-latency and highly-reliable key-value store. It leverages Netflix Scheduler for scheduling the Bulldozer Jobs. Moving data with Bulldozer at Netflix.

Latency 248

Latency Storage Big Data Tuning 248

Dynatrace

JANUARY 31, 2024

Designed to reach beyond petabyte (PB) scale, OpenPipeline will—at its launch— quintuple the ingest throughput from 100 TB a day per tenant to 500 TB a day per tenant. OpenPipeline high-performance filtering and preprocessing provides full ingest and storage control for the Dynatrace platform. Commitment to privacy.

Analytics 272

Analytics Processing Transportation Storage 272

The Netflix TechBlog

OCTOBER 19, 2020

Now let’s look at how we designed the tracing infrastructure that powers Edgar. Our distributed tracing infrastructure is grouped into three sections: tracer library instrumentation, stream processing, and storage. A second job taps the data feed from the first job, does tail sampling of data and writes traces to the storage system.

Infrastructure 236

Infrastructure Transportation Storage Open Source 236

Dynatrace

NOVEMBER 20, 2023

The first goal is to demonstrate how generative AI can bring key business value and efficiency for organizations. While technologies have enabled new productivity and efficiencies, customer expectations have grown exponentially, cyberthreat risks continue to mount, and the pace of business has sped up.

AWS 277

AWS Artificial Intelligence Cloud Analytics 277

The Netflix TechBlog

APRIL 26, 2022

We will share how its design has evolved over the years and the lessons learned while building it. To understand Axion’s design, we need to know the various components that interact with it. The motivation has not changed since then; the design has. Design evolution Axion fact store has four components?—?fact

Storage 191

Storage Design Scalability Latency 191