The Netflix TechBlog

DECEMBER 17, 2024

We kick off with a few topics focused on how were empowering Netflix to efficiently produce and effectively deliver high quality, actionable analytic insights across the company. Subsequent posts will detail examples of exciting analytic engineering domain applications and aspects of the technical craft.

Analytics 212

Analytics Engineering Entertainment Metrics 212

Dynatrace

SEPTEMBER 8, 2023

In part 2, we’ll show you how to retrieve business data from a database, analyze that data using dashboards and ad hoc queries, and then use a Davis analyzer to predict metric behavior and detect behavioral anomalies. Similar to the tutorial extension, we created an extension that performs queries against databases.

Database 290

Database Artificial Intelligence Metrics Monitoring 290

Scalegrid

FEBRUARY 27, 2025

Incremental Backups: Speeds up recovery and makes data management more efficient for active databases. Improved JSON Handling & Security: Improved logical replication and the new MAINTAIN privilege give database administrators more control and flexibility. Start your free trial today!

Speed 130

Speed Database Storage Efficiency 130

Dynatrace

MAY 8, 2023

This demand for rapid innovation is propelling organizations to adopt agile methodologies and DevOps principles to deliver software more efficiently and securely. And how do DevOps monitoring tools help teams achieve DevOps efficiency? Lost efficiency. 54% reported deploying updates every two hours or less.

DevOps 246

DevOps Efficiency Monitoring Infrastructure 246

Scalegrid

JUNE 22, 2020

MySQL is the all-time number one open source database in the world, and a staple in RDBMS space. MySQL on DigitalOcean is a natural fit, but what’s the best way to deploy your cloud database? ScaleGrid provides 30% more storage on average vs. DigitalOcean for MySQL at the same affordable price. At a glance – TLDR.

Database 217

Database Benchmarking Latency Performance 217

The Netflix TechBlog

MARCH 6, 2019

Mounting object storage in Netflix’s media processing platform By Barak Alon (on behalf of Netflix’s Media Cloud Engineering team) MezzFS (short for “Mezzanine File System”) is a tool we’ve developed at Netflix that mounts cloud objects as local files via FUSE. Our object storage service splits objects into many parts and stores them in S3.

Media 218

Media Storage Processing Cache 218

DZone

AUGUST 6, 2021

Most log management solutions store log data in a database and enable search by storing an index of the data. As the database grows in size, so does the index management cost. When companies are handling terabytes of data every day, the database-backed log management system becomes untenable.

Storage 189

Storage Database Systems Efficiency 189

The Netflix TechBlog

SEPTEMBER 18, 2024

Central to this infrastructure is our use of multiple online distributed databases such as Apache Cassandra , a NoSQL database known for its high availability and scalability. Over time as new key-value databases were introduced and service owners launched new use cases, we encountered numerous challenges with datastore misuse.

Latency 260

Latency Storage Cache Servers 260

DZone

JANUARY 29, 2020

To resolve the problem it was suggested to find more suitable data storage. It's still not possible to continue using light-weighted databases such as HSQL or H2 to implement tests. There are several limitations to store and fetch such data (all restrictions could be found in official documentation ).

Storage 124

Storage Efficiency Testing Database 124

Scalegrid

JANUARY 5, 2024

Enhanced data security, better data integrity, and efficient access to information. If you’re considering a database management system, understanding these benefits is crucial. Understanding Database Management Systems (DBMS) A Database Management System (DBMS) assists users in creating and managing databases.

Efficiency 147

Efficiency Storage Database Scalability 147

Dynatrace

OCTOBER 7, 2021

This means you no longer have to provision, scale, and maintain servers to run your applications, databases, and storage systems. Speed is next; serverless solutions are quick to spin up or down as needed, and there are no delays due to limited storage or resource access. AWS offers four serverless offerings for storage.

Serverless 325

Serverless AWS Lambda Storage 325

Dynatrace

AUGUST 1, 2022

Interestingly, our partner RedHat reported in 2021 that around 80% of deployed workloads are databases or data caches, storing data in persistent volume claims (PVCs). You also decide to run your database for storing user uploads – such as images or videos – directly in Kubernetes. However, you lack insights into your PVCs.

Storage 246

Storage Database Network Metrics 246

Scalegrid

JULY 17, 2020

Why should a relational database even care about unstructured data? JSON database in 9.2 It is useful to validate incoming JSON and store in the database. JSONB supports indexing the JSON data, and is very efficient at parsing and querying the JSON data. JSONB storage results in a larger storage footprint.

Storage 321

Storage Database Efficiency Availability 321

Scalegrid

FEBRUARY 8, 2024

A distributed storage system is foundational in today’s data-driven landscape, ensuring data spread over multiple servers is reliable, accessible, and manageable. Understanding distributed storage is imperative as data volumes and the need for robust storage solutions rise.

Storage 130

Storage Systems Big Data Azure 130

Scalegrid

MAY 11, 2024

MongoDB offers several storage engines that cater to various use cases. The default storage engine in earlier versions was MMAPv1, which utilized memory-mapped files and document-level locking. The newer, pluggable storage engine, WiredTiger, addresses this by using prefix compression, collection-level locking, and row-based storage.

Storage 130

Storage Engineering Cache Database 130

All Things Distributed

JANUARY 18, 2012

a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Today is a very exciting day as we release Amazon DynamoDB , a fast, highly reliable and cost-effective NoSQL database service designed for internet scale applications. Werner Vogels weblog on building scalable and robust distributed systems.

Scalability 162

Scalability Database Ecommerce Latency 162

Dynatrace

JANUARY 11, 2023

Database monitoring. This ensures the database queries are performant, while also identifying host problems. For example, uptime detection can identify database instability and help to improve mean time to restoration. Cloud storage monitoring. Website monitoring. Virtual machine (VM) monitoring.

Cloud 295

Cloud Monitoring Best Practices Infrastructure 295

Dynatrace

SEPTEMBER 5, 2024

These developments open up new use cases, allowing Dynatrace customers to harness even more data for comprehensive AI-driven insights, faster troubleshooting, and improved operational efficiency. Customers have had a positive response to our native syslog implementation, noting its easy setup and efficiency.

Innovation 290

Innovation AWS Analytics Storage 290

DZone

APRIL 11, 2023

Modern tech stacks such as Apache Spark, Azure Data Factory, Azure Databricks, and Azure Synapse Analytics offer powerful tools for building optimized data pipelines that can efficiently ingest and process data on the cloud. It provides built-in connectors for various data sources such as databases, file systems, cloud storage, and more.

Azure 246

Azure Analytics Storage Cloud 246

Dynatrace

NOVEMBER 2, 2020

IT infrastructure is the heart of your digital business and connects every area – physical and virtual servers, storage, databases, networks, cloud services. If you don’t have insight into the software and services that operate your business, you can’t efficiently run your business. Minimizes downtime and increases efficiency.

Infrastructure 246

Infrastructure Monitoring Virtualization Serverless 246

Dynatrace

JULY 29, 2022

With the right log management and observability platform, IT teams can efficiently identify the root cause of problems during these peak times and maintain three-nines of availability — or 99.98%. Traditional databases help users and machines find data with a quick search. Cold storage and rehydration. Indexing overhead.

Analytics 246

Analytics Storage Retail DevOps 246

Dynatrace

JULY 29, 2022

With the right log management and observability platform, IT teams can efficiently identify the root cause of problems during these peak times and maintain three-nines of availability — or 99.98%. Traditional databases help users and machines find data with a quick search. Cold storage and rehydration. Indexing overhead.

Analytics 245

Analytics Storage Retail DevOps 245

Dynatrace

OCTOBER 4, 2022

They can call on dozens of databases and deliver gigabytes of data across myriad devices. In most data storage models, indexing engines enable faster access to query logs. But indexing requires schema management and additional storage to be effective, which adds cost and overhead. Eliminates team silos.

Analytics 264

Analytics Storage Infrastructure Metrics 264

Dynatrace

APRIL 20, 2023

With more automated approaches to log monitoring and log analysis, however, organizations can gain visibility into their applications and infrastructure efficiently and with greater precision—even as cloud environments grow. ” A data warehouse, on the other hand, is an efficient and fast option for querying data.

Analytics 246

Analytics Infrastructure Storage Architecture 246

The Netflix TechBlog

OCTOBER 8, 2024

Building on these foundational abstractions, we developed the TimeSeries Abstraction  — a versatile and scalable solution designed to efficiently store and query large volumes of temporal event data with low millisecond latencies, all in a cost-effective manner across various use cases. Let’s dive into the various aspects of this abstraction.

Latency 239

Latency Storage Traffic Tuning 239

Highly Scalable

MARCH 1, 2012

NoSQL databases are often compared by various non-functional criteria, such as scalability, performance, and consistency. At the same time, NoSQL data modeling is not so well studied and lacks the systematic theory found in relational databases. Document databases advance the BigTable model offering two significant improvements.

Database 279

Database Ecommerce Efficiency Engineering 279

High Scalability

SEPTEMBER 8, 2018

New databases used to be announced seemingly every week. While database neogenesis has slowed down considerably, it has not gone necrotic. Anna is not only incredibly fast, it’s incredibly efficient and elastic too: an autoscaling, multi-tier, selectively-replicating cloud service. What's changed ?

Storage 199

Storage Performance AWS Media 199

InfoQ

NOVEMBER 6, 2024

Pinterest has modernized and enhanced its Goku time-series database. The recent updates focus on optimizing storage and resource usage without compromising service quality. By Mohit Palriwal

Database 93

Database Efficiency Storage Scalability 93

Dynatrace

JULY 11, 2024

Retention-based deletion is governed by a policy outlining the duration for which data is stored in the database before it’s deleted automatically. For instance, if data is mistakenly ingested into the database, it may need to be deleted to prevent inaccuracies or sensitive data from being stored. Get started New to Dynatrace?

Storage 264

Storage Government Best Practices Media 264

Scalegrid

DECEMBER 6, 2023

The choice of self-managed cloud databases vs DBaaS is a common debate among those who are looking for the best option that will cater to their particular needs. Database as a Service (DBaaS) and managed databases offer distinct advantages along with certain challenges.

Database 130

Database Cloud Hardware Storage 130

All Things Distributed

JUNE 21, 2018

A common question that I get is why do we offer so many database products? To do this, they need to be able to use multiple databases and data models within the same application. Seldom can one database fit the needs of multiple distinct use cases. Seldom can one database fit the needs of multiple distinct use cases.

Database 167

Database AWS Games Latency 167

Percona

APRIL 14, 2023

A shared characteristic in most (if not all) databases, be them traditional relational databases like Oracle, MySQL, and PostgreSQL or some kind of NoSQL-style database like MongoDB, is the use of a caching mechanism to keep (a copy of) part of the data in memory. MySQL does. Download Percona Monitoring and Management Today

Cache 107

Cache Efficiency Database Monitoring 107

Percona

JANUARY 4, 2024

Managing storage and performance efficiently in your MySQL database is crucial, and general tablespaces offer flexibility in achieving this. In contrast to the single system tablespace that holds system tables by default, general tablespaces are user-defined storage containers for multiple InnoDB tables.

Storage 114

Storage Engineering Database Open Source 114

Dynatrace

OCTOBER 4, 2022

As a result, organizations need software to work perfectly to create customer experiences, deliver innovation, and generate operational efficiency. Traditionally, though, to gain true business insight, organizations had to make tradeoffs between accessing quality, real-time data and factors such as data storage costs.

Analytics 246

Analytics Software Software Storage 246

All Things Distributed

AUGUST 20, 2015

Today, we are releasing a plugin that allows customers to use the Titan graph engine with Amazon DynamoDB as the backend storage layer. It opens up the possibility to enjoy the value that graph databases bring to relationship-centric use cases, without worrying about managing the underlying storage. Enter graph databases.

Database 121

Database Logistics Availability Social Media 121

The Netflix TechBlog

OCTOBER 27, 2020

Therefore, we must efficiently move data from the data warehouse to a global, low-latency and highly-reliable key-value store. What is Bulldozer Bulldozer is a self-serve data platform that moves data efficiently from data warehouse tables to key-value stores in batches. Figure 1 shows how we use Bulldozer to move data at Netflix.

Latency 252

Latency Storage Big Data Tuning 252

The Netflix TechBlog

OCTOBER 19, 2020

Our distributed tracing infrastructure is grouped into three sections: tracer library instrumentation, stream processing, and storage. We earned the trust of our engineers by developing empathy for their operational burden and by focusing on providing efficient tracer library integrations in runtime environments.

Infrastructure 238

Infrastructure Transportation Storage Open Source 238