The Netflix TechBlog

APRIL 27, 2023

Ruchir Jha , Brian Harrington , Yingwu Zhao TL;DR Streaming alert evaluation scales much better than the traditional approach of polling time-series databases. It allows us to overcome high dimensionality/cardinality limitations of the time-series database. It opens doors to support more exciting use-cases.

Storage 297

Storage Cache Metrics Database 297

High Scalability

SEPTEMBER 3, 2019

Depending on how it is configured, Redis can act like a database, a cache or a message broker. It’s important to note that Redis is a NoSQL database system.

Open Source 55

Open Source Cache C++ Database 55

Dynatrace

OCTOBER 7, 2021

This means you no longer have to provision, scale, and maintain servers to run your applications, databases, and storage systems. Scalability. Finally, there’s scalability. Serverless architecture shifts application hosting functions away from local servers onto those managed by providers. Data Store.

Serverless 325

Serverless AWS Lambda Storage 325

All Things Distributed

JUNE 21, 2018

A common question that I get is why do we offer so many database products? To do this, they need to be able to use multiple databases and data models within the same application. Seldom can one database fit the needs of multiple distinct use cases. Seldom can one database fit the needs of multiple distinct use cases.

Database 166

Database AWS Games Latency 166

Scalegrid

JANUARY 10, 2025

Heading into 2024, SQL databases will remain essential in data management, increasingly using distributed systems to meet growing needs for scalability and reliability. According to 2023 statistics, 49% of web applications use an SQL-based database , with SQL having a 75% adoption rate in the IT industry.

Database 130

Database Scalability Best Practices Blockchain 130

The Netflix TechBlog

SEPTEMBER 18, 2024

Central to this infrastructure is our use of multiple online distributed databases such as Apache Cassandra , a NoSQL database known for its high availability and scalability. Over time as new key-value databases were introduced and service owners launched new use cases, we encountered numerous challenges with datastore misuse.

Latency 260

Latency Storage Cache Servers 260

High Scalability

NOVEMBER 30, 2018

8 : successful Mars landings; $250,000 : proposed price for Facebook Graph API; 33 : countries where mobile internet is faster than WiFi; 1000s : Facebook cache poisoning; 8.2 The scalability of a native graph database. They'll love it and you'll be their hero forever. million : concurrent Fortnite players; 6.2 servers of Neo4j.

Internet 28

Internet Internet Scalability Blockchain 28

Dynatrace

MAY 13, 2022

Apache Cassandra is an open-source, distributed, NoSQL database. Because of its scalability and distributed architecture, thousands of companies trust it to run their cloud and hybrid-based workloads at high availability without compromising performance. Microsoft Azure offers multiple ways to manage Apache Cassandra databases.

Azure 246

Azure Latency Metrics Infrastructure 246

All Things Distributed

JULY 14, 2015

I am excited to share with you that today we are expanding DynamoDB with streams, cross-region replication, and database triggers. Streams provide you with the underlying infrastructure to create new applications, such as continuously updated free-text search indexes, caches, or other creative extensions requiring up-to-date table changes.

Database 156

Database Lambda AWS IoT 156

High Scalability

FEBRUARY 17, 2021

This article is to simply report the YCSB bench test results in detail for five NoSQL databases namely Redis, MongoDB, Couchbase, Yugabyte and BangDB and compare the result side by side. I have used latest versions for each NoSQL DB and have followed the recommendations to run all the databases in optimized conditions. Load and 2.

Benchmarking 308

Benchmarking Latency C++ Database 308

The Netflix TechBlog

MARCH 5, 2019

the order of the rows on your Netflix home page, issuing content licenses when you click play, finding the Open Connect cache closest to you with the content you requested, and many more). A majority of the Netflix product features are either partially or completely dependent on one of our many micro-services (e.g.,

Infrastructure 185

Infrastructure Cloud Scalability AWS 185

Dynatrace

NOVEMBER 3, 2021

For example, when monitoring a database, you’ll want to know about any latency when writing data to a disk or average query response time. Experienced database administrators learn to spot patterns that can lead to common problems. DevOps practitioners struggle to maintain highly available and scalable applications.

Monitoring 264

Monitoring Metrics DevOps Scalability 264

Dynatrace

DECEMBER 21, 2023

Reduce the volume of data volumes requested from databases (for example, request all, filter in memory). Implement appropriate caching layers (for example, read-only cache for static data). A few examples: Reduce roundtrips between services (for example, the N+1 query pattern). Reduce inter-process communications overhead.

Cloud 264

Cloud Energy Best Practices Cache 264

The Netflix TechBlog

MAY 19, 2023

A cleanup process to prune stale relationships from the database. The Beginning of SpiceDB Caveats The SpiceDB community had already explored integrating SpiceDB with Open Policy Agent (OPA) and concluded it strayed too far from Zanzibar’s core promise of global horizontal scalability with strong consistency.

Cache 255

Cache Google Open Source Systems 255

Percona

JUNE 22, 2023

To make data count and to ensure cloud computing is unabated, companies and organizations must have highly available databases. A basic high availability database system provides failover (preferably automatic) from a primary database node to redundant nodes within a cluster. HA is sometimes confused with “fault tolerance.”

Availability 132

Availability Database Open Source Hardware 132

All Things Distributed

SEPTEMBER 3, 2013

Today, we added two important choices for customers running high performance apps in the cloud: support for Redis in Amazon ElastiCache and a new high memory database instance (db.cr1.8xlarge) for Amazon RDS. No single database architecture or solution can meet all of Amazon.com’s or our customers’ needs.

Cache 97

Cache Cloud Performance Retail 97

Percona

APRIL 6, 2023

PostgreSQL is a popular open source relational database management system many organizations use to store and manage their data. One of the key benefits of using PostgreSQL is its reliability, scalability, and performance. However, as the size of your database grows, it can become challenging to manage and optimize its performance.

Cache 138

Cache Open Source Benchmarking Database 138

All Things Distributed

OCTOBER 12, 2016

Fast Data is an emerging industry term for information that is arriving at high volume and incredible rates, faster than traditional databases can manage. While caching continues to be a dominant use of ElastiCache for Redis, we see customers increasingly use it as an in-memory NoSQL database.

Scalability 100

Scalability Cache Analytics AWS 100

Dynatrace

OCTOBER 21, 2020

This is a great example of how valuable Dynatrace is for diagnosing performance or scalability issues, and a great testimony that we at Dynatrace use our own product and its various capabilities across our globally distributed systems. One of them being a small cache that would have brought the initial startup time down by about 95%.

Code 246

Code Cloud Engineering Java 246

Percona

NOVEMBER 1, 2023

version, like this: ANALYZE TABLE removes the table from the table definition cache, which requires a flush lock. This makes the query wait for any long-running queries to finish but also can trigger cascading waiting for other incoming requests. In short, ANALYZE could lead to nasty stalls in busy production environments. x are all safe.

Servers 117

Servers Database Cache C++ 117

Scalegrid

MARCH 28, 2024

In this comparison of Redis vs Memcached, we strip away the complexity, focusing on each in-memory data store’s performance, scalability, and unique features. Redis is better suited for complex data models, and Memcached is better suited for high-throughput, string-based caching scenarios.

Cache 130

Cache Storage Architecture Scalability 130

The Morning Paper

JUNE 16, 2019

Towards multiverse databases Marzoev et al., The central idea behind multiverse databases is to push the data access and privacy rules into the database itself. With multiverse databases, each user sees a consistent “parallel universe” database containing only the data that user is allowed to see.

Database 90

Database Cache Benchmarking Efficiency 90

InfoQ

JANUARY 29, 2024

RevenueCat extensively uses caching to improve the availability and performance of its product API while ensuring consistency. The company shared its techniques to deliver the platform, which can handle over 1.2 billion daily API requests. The team at RevenueCat created an open-source memcache client that provides several advanced features.

Cache 109

Cache Open Source Availability Performance 109

Scalegrid

MAY 11, 2024

WiredTiger excels with operational databases and transactional workloads as it offers b-tree-based storage and well-ordered data structures. It uses a filesystem cache and write-ahead log for crash recovery. A database level lock is held during the compaction operation. Compaction operation defragments data files & indexes.

Storage 130

Storage Engineering Cache Database 130

The Netflix TechBlog

OCTOBER 8, 2024

The Key-Value Abstraction offers a flexible, scalable solution for storing and accessing structured key-value data, while the Data Gateway Platform provides essential infrastructure for protecting, configuring, and deploying the data tier. We do not use it for metrics, histograms, timers, or any such near-real time analytics use case.

Latency 239

Latency Storage Traffic Tuning 239

All Things Distributed

AUGUST 22, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Today AWS has launched Amazon ElastiCache , a new service that makes it easy to add distributed in-memory caching to any application. Systems that make extensive use of caching almost all report a significant reduction in the cost of their database tier.

Cloud 103

Cloud Cache AWS Storage 103

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. In databases like MySQL and PostgreSQL, transaction logs are the source of CDC events. Designed with High Availability in mind.

Database 221

Database Traffic Transportation Open Source 221

High Scalability

JANUARY 8, 2020

It’s a very simple utility that does exactly one thing – it sits between the database and the clients and speaks the PostgreSQL protocol, emulating a PostgreSQL server. When it comes to connection pooling in the PostgreSQL world, PgBouncer is probably the most popular option.

Database 48

Database Servers Open Source Cache 48

Scalegrid

JANUARY 5, 2024

If you’re considering a database management system, understanding these benefits is crucial. DBMS enhances data security with encryption, implements various access controls, and enables improved data sharing and concurrent access, thus facilitating quick response to changes and maintaining consistent database accuracy.

Efficiency 147

Efficiency Storage Database Scalability 147

Percona

MARCH 17, 2023

These updates are designed to keep databases running at peak performance and simplify database operations. But as companies grow and see more demand for their databases, we need to ensure that PMM also remains scalable so you don’t need to worry about its performance while tending to the rest of your environment.

Monitoring 117

Monitoring Scalability Database Cache 117

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. In databases like MySQL and PostgreSQL, transaction logs are the source of CDC events. Designed with High Availability in mind.

Database 206

Database Traffic Transportation Open Source 206

Percona

FEBRUARY 1, 2023

PostgreSQL is a fantastic database, but if you’re storing images, video, audio files, or other large data objects, you need to “toast” them to get optimal performance. This post will look at using The Oversized-Attribute Storage Technique (TOAST) to improve performance and scalability. Thus, you can use less disk space.

Storage 111

Storage Scalability Strategy Performance 111

Percona

MARCH 27, 2023

On MySQL and Percona Server for MySQL , there is a schema called information_schema (I_S) which provides information about database tables, views, indexes, and more. having to open each table.frm (and in which my test runs, I have purposely read a very high number of tables compared to “Table-open-cache” variable).

Cache 124

Cache Hardware Servers Benchmarking 124

Smashing Magazine

SEPTEMBER 28, 2022

Active Memory Caching. When you want to get data that you already had quickly, you need to do caching — caching stores data that a user recently retrieved. Caching partially stores your data and is not used as permanent storage. Caching partially stores your data and is not used as permanent storage.

Cache 141

Cache Performance Servers Architecture 141

Scalegrid

JANUARY 21, 2025

Redis Monitoring Essentials Ensuring the performance, reliability, and safety of a Redis database requires active monitoring. With these essential support systems in place, you can effectively monitor your databases with up-to-date data about their health and functioning status at all times.

Strategy 130

Strategy Monitoring Latency DevOps 130

Percona

FEBRUARY 15, 2023

Bloom filters are an essential component of an LSM-based database engine like MyRocks. For good performance, the filter blocks are cached in the RocksDB block cache and normally stay there since they are accessed frequently. A good understanding of how bloom filters work can only lead to improved database performance.

Storage 138

Storage Tuning Cache Engineering 138

High Scalability

OCTOBER 3, 2019

ScaleGrid, a rapidly growing leader in the Database-as-a-Service (DBaaS) space, has just launched their new fully managed Redis on Azure service. Redis, the #1 key-value store and top 10 database in the world, has grown by over 300% in popularity over that past 5 years, per the DB-Engines knowledge base.

Azure 23

Azure Cloud Open Source Cache 23