The Netflix TechBlog

DECEMBER 3, 2019

by David Berg , Ravi Kiran Chirravuri , Romain Cledat , Savin Goyal , Ferras Hamad , Ville Tuulos tl;dr Metaflow is now open-source! both for compute and storage. For the open-source release, we partnered with AWS to provide a seamless integration between Metaflow and various AWS services.

Open Source 135

Open Source AWS Infrastructure Energy 135

Percona

MARCH 30, 2023

The use of open source databases has increased steadily in recent years. Past trepidation — about perceived vulnerabilities and performance issues — has faded as decision makers realize what an “open source database” really is and what it offers. What is an open source database?

Open Source 107

Open Source Database Storage Scalability 107

Dynatrace

MARCH 11, 2025

Say hello to advanced trace an alytics and new data storage and capture options. These game-changing features elevate your data interactions, opening up vast possibilities for advanced queries and efficient data management tailored to your needs. This precision reduces storage costs while ensuring you retain the data that matters most.

Games 165

Games Analytics Innovation Metrics 165

Dynatrace

JANUARY 16, 2023

Open-source software drives a vibrant Kubernetes ecosystem. Open source software drives a vibrant Kubernetes ecosystem. Across all categories in the Kubernetes survey, open source projects rank among the most frequently used solutions. Dynatrace’s investment in open source technologies keeps growing.

Open Source 358

Open Source Java Operating System Programming 358

Scalegrid

FEBRUARY 6, 2025

Message brokers handle validation, routing, storage, and delivery, ensuring efficient and reliable communication. RabbitMQ is an open-source message broker that supports multiple messaging protocols , including AMQP, STOMP, MQTT, and RabbitMQ Streams. What is RabbitMQ? What is Apache Kafka?

Latency 147

Latency Analytics Architecture Storage 147

Percona

OCTOBER 18, 2023

Migrating a proprietary database to open source is a major decision that can significantly affect your organization. Advantages of migrating to open source For many reasons mentioned earlier, organizations are increasingly shifting towards open source databases for their data management needs.

Open Source 111

Open Source Database Hardware Strategy 111

Dynatrace

JUNE 6, 2023

Kubernetes has become the leading container orchestration platform for organizations adopting open source solutions to manage, scale, and automate application deployment. Kubernetes is an open source container orchestration platform for managing, automating, and scaling containerized applications. What is Kubernetes?

Open Source 246

Open Source Storage Analytics Innovation 246

DZone

SEPTEMBER 14, 2023

Apache Spark , a robust open-source data processing framework, has emerged as a game-changer in this domain. Spark takes full advantage of this storage property by exclusively reading the columns that are involved in subsequent computations.

Big Data 279

Big Data Processing Open Source Games 279

Abhishek Tiwari

JANUARY 6, 2018

Here we present a list of 10 open-source Kubernetes tools to make your SRE and Ops teams more effective to achieve their service level objectives. The backup files are stored in an object storage service (e.g. Ark server performs the actual backup, validates it and loads backup files in cloud object storage. Telepresence.

Open Source 111

Open Source DevOps Storage Engineering 111

Scalegrid

JULY 13, 2020

The unstoppable rise of open source databases. One database in particular is causing a huge dent in Oracle’s market share – open source PostgreSQL. See how open source PostgreSQL Community version costs compare to Oracle Standard Edition and Oracle Enterprise Edition. What’s causing this massive shift?

Open Source 244

Open Source Tuning C++ Database 244

The Netflix TechBlog

MARCH 4, 2025

Data storage and distribution through HollowFeeds Netflix Hollow is an Open Source java library and toolset for disseminating in-memory datasets from a single producer to many consumers for high performance read-only access.

Traffic 180

Traffic Entertainment Strategy Innovation 180

DZone

MAY 5, 2023

This blog post explores the benefits of combining both open-source frameworks, shows unique differentiators of Flink versus Kafka, and discusses when to use a Kafka-native streaming engine like Kafka Streams instead of Flink.

Open Source 246

Open Source Storage Innovation Engineering 246

Scalegrid

SEPTEMBER 16, 2019

PostgreSQL graphical user interface (GUI) tools help these open source database users to manage, manipulate, and visualize their data. It supports all PostgreSQL operations and features while being free and open-source. pgAdmin Cost: Free (open source). Let’s start with the first and most popular one.

Open Source 262

Open Source Database Azure Servers 262

The Netflix TechBlog

MARCH 8, 2021

Machine Configuration: Spinnaker Starting at the left of the chart, Spinnaker is an open-source platform that controls the creation of workstation pools. They could need a GPU when doing graphics-intensive work or extra large storage to handle file management. Spinnaker uses “pipelines” as instructions for creating the pools.

Entertainment 307

Entertainment Storage Open Source Hardware 307

The Netflix TechBlog

OCTOBER 19, 2020

Troubleshooting a session in Edgar When we started building Edgar four years ago, there were very few open-source distributed tracing systems that satisfied our needs. Our tactical approach was to use Netflix-specific libraries for collecting traces from Java-based streaming services until open source tracer libraries matured.

Infrastructure 238

Infrastructure Transportation Storage Open Source 238

Dynatrace

OCTOBER 4, 2022

This architecture offers rich data management and analytics features (taken from the data warehouse model) on top of low-cost cloud storage systems (which are used by data lakes). This decoupling ensures the openness of data and storage formats, while also preserving data in context. Start using Grail now.

Artificial Intelligence 290

Artificial Intelligence Analytics Storage Architecture 290

Dynatrace

JUNE 7, 2023

Kubernetes is an open source container orchestration platform that enables organizations to automatically scale, manage, and deploy containerized applications in distributed environments. Like Kubernetes, OpenShift is an open source Kubernetes-based container platform. What is Kubernetes? What is OpenShift? Ease of use.

Open Source 246

Open Source Social Media Infrastructure Operating System 246

Scalegrid

JUNE 9, 2020

MySQL and PostgreSQL are the top two open source relational databases in the world, and Redis is the top key-value database. The open source model is not only popular with the developer market, but also enterprise companies looking to modernize their infrastructure and reduce spend.

Availability 227

Availability Open Source Benchmarking Database 227

Dynatrace

JUNE 10, 2024

This includes everything from multicloud deployments to microservices to Kubernetes instances and the use of open source software. While it’s easy for analytics solutions to pinpoint symptoms, determining the communal source is much more difficult. The net result is a growing challenge in getting to the root cause.

Analytics 264

Analytics Network Open Source Hardware 264

DZone

OCTOBER 11, 2022

Why haven’t cash-strapped American schools embraced open source? Email Reveals Google App Engine Search API About Ready For Preview Release, Charges Planned For Storage, Operations ( TechCrunch). Pandora launches new HTML5 site for TVs and gaming consoles, available now on PS3 and Xbox 360 ( The Next Web). Hacker News).

Java 244

Java Best Practices Google Open Source 244

Dynatrace

JULY 29, 2021

The complexity of such deployments has accelerated with the adoption of emerging, open-source technologies that generate telemetry data, which is exploding in terms of volume, speed, and cardinality. Extend your topology mapping to include open-source observability data.

Analytics 246

Analytics Open Source Serverless Architecture 246

DZone

JANUARY 8, 2020

Prometheus is an open-source system monitoring and alerting toolkit. Data related to monitoring is stored in RAM and LevelDB nevertheless data can be stored to other storage systems such as ElasticSearch, InfluxDb, and others, [link]. Watch out for your self-destructing apps!

Monitoring 191

Monitoring Open Source Storage Systems 191

Scalegrid

APRIL 28, 2020

Since database hosting is more dependent on memory (RAM) than storage, we are going to compare various instance sizes ranging from just 1GB of RAM up to 64GB of RAM so you can see how costs vary across different application workloads. MongoDB is the #3 open source and #1 NoSQL database in the world. EC2 instances. VM instances.

Azure 344

Azure AWS Database Latency 344

Scalegrid

JULY 17, 2020

JSONB storage has some drawbacks vs. traditional columns: PostreSQL does not store column statistics for JSONB columns. JSONB storage results in a larger storage footprint. JSONB storage does not deduplicate the key names in the JSON. If that doesn’t work, the data is moved to out-of-line storage.

Storage 321

Storage Database Efficiency Availability 321

Dynatrace

SEPTEMBER 29, 2021

Prometheus is an open-source monitoring and alerting toolkit for services and applications that run in containers. Prometheus components include client libraries for application code instrumentation, special-purpose exporters for popular services, and the optional Prometheus server for orchestrating service discovery and data storage.

Technology 246

Technology Technology Monitoring AWS 246

Dynatrace

NOVEMBER 17, 2020

With these release candidate APIs available, instrumentation for web frameworks, storage clients, and much more can be built. We at Dynatrace understand the importance of contributing our expertise in enterprise-grade intelligent observability to the open source community. Dynatrace fully embraces OpenTelemetry.

Open Source 207

Open Source Software Engineering Government Java 207

DZone

JANUARY 16, 2023

BindPlane OP is a powerful open-source tool that makes it easy to build and manage telemetry pipelines to ship data from IT environments of any kind and size to any analysis tool or storage destination.

Open Source 211

Open Source Storage Processing 211

Percona

JANUARY 4, 2024

Managing storage and performance efficiently in your MySQL database is crucial, and general tablespaces offer flexibility in achieving this. In contrast to the single system tablespace that holds system tables by default, general tablespaces are user-defined storage containers for multiple InnoDB tables.

Storage 114

Storage Engineering Database Open Source 114

Dynatrace

MARCH 24, 2023

Problems include provisioning and deployment; load balancing; securing interactions between containers; configuration and allocation of resources such as networking and storage; and deprovisioning containers that are no longer needed. Originally created by Google, Kubernetes was donated to the CNCF as an open source project.

Infrastructure 246

Infrastructure Open Source Operating System Cloud 246

Dynatrace

OCTOBER 4, 2022

Traditionally, though, to gain true business insight, organizations had to make tradeoffs between accessing quality, real-time data and factors such as data storage costs. Additionally, it provides index-free storage and direct analytics access to source data without requiring data rehydration. Don’t reinvent the wheel.

Analytics 246

Analytics Software Software Storage 246

Scalegrid

NOVEMBER 12, 2019

There are many things I love about it – PL/ PG SQL, smart defaults, replication (that actually works out of the box), and an active and vibrant open source community. Hopefully we can generate enough interest in the open source community to prioritize these features.

Database 184

Database Open Source Servers Storage 184

High Scalability

SEPTEMBER 8, 2018

They've posted about Anna's new superpowers in Going Fast and Cheap: How We Made Anna Autoscale : Using Anna v0 as an in-memory storage engine, we set out to address the cloud storage problems described above. Each storage server collects statistics about the requests it serves, the data it stores, etc. Related Articles.

Storage 199

Storage Performance AWS Open Source 199

The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 209 File system as a service at Netflix Kishore Kasi , Senior Software Engineer Abstract : As Netflix grows in original content creation, its need for storage is also increasing at a rapid pace. Technology advancements in content creation and consumption have also increased its data footprint.

AWS 38

AWS Entertainment Open Source Benchmarking 38

The Netflix TechBlog

OCTOBER 28, 2021

I developed many batch and real-time data pipelines using open source technologies for AOL Advertising and eBay. Over the years, I followed the big data open-source community and Netflix tech blogs closely, and learned a lot about Netflix’s innovative engineering solutions and active contributions to the open-source ecosystem.

Data Engineering 181

Data Engineering Engineering Big Data Software Engineering 181

The Netflix TechBlog

NOVEMBER 29, 2023

In spite of reaching higher qualities than the fixed ladder, the HDR-DO ladder, on average, occupies only 58% of the storage space compared to fixed-bitrate ladder. Yes, we are committed to supporting the open-source community. This is achieved by more efficiently spacing the ladder points, especially in the high-bitrate region.

Open Source 260

Open Source Software Engineering Internet Internet 260

DZone

JULY 29, 2019

One of the top trending open-source data storage that responds to most of the use cases is Elasticsearch. Elasticsearch is a distributed data storage and search engine with fault-tolerance and high availability capabilities. To make the best use of its search feature, Elasticsearch needs to be configured properly.

Big Data 162

Big Data Government Open Source Storage 162

Dynatrace

JUNE 27, 2023

Many AWS services and third party solutions use AWS S3 for log storage. To date, some customers have used open source or community-backed components to forward logs from S3 to Dynatrace. Logs complement out-of-the-box metrics and enable automated actions for responding to availability, security, and other service events.

AWS 246

AWS Cloud Lambda Analytics 246