The Netflix TechBlog

DECEMBER 21, 2020

At this scale, we can gain a significant amount of performance and cost benefits by optimizing the storage layout (records, objects, partitions) as the data lands into our warehouse. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage 208

Storage Latency Efficiency Data Engineering 208

Scalegrid

MAY 13, 2020

In this blog post, we explain what Greenplum is, and break down the Greenplum architecture, advantages, major use cases, and how to get started. It’s architecture was specially designed to manage large-scale data warehouses and business intelligence workloads by giving you the ability to spread your data out across a multitude of servers.

Big Data 321

Big Data Database Artificial Intelligence Open Source 321

Dynatrace

AUGUST 11, 2022

This enables teams to quickly develop and test key functions without the headaches typically associated with in-house infrastructure management. FaaS vs. monolithic architectures. Monolithic architectures were commonplace with legacy, on-premises software solutions. But how does FaaS fit in? Increased testing complexity.

Development 279

Development Serverless Best Practices Lambda 279

Dynatrace

OCTOBER 4, 2022

Grail architectural basics. The aforementioned principles have, of course, a major impact on the overall architecture. A data lakehouse addresses these limitations and introduces an entirely new architectural design. This decoupling ensures the openness of data and storage formats, while also preserving data in context.

Artificial Intelligence 290

Artificial Intelligence Analytics Storage Architecture 290

Dynatrace

JANUARY 11, 2023

With more organizations taking the multicloud plunge, monitoring cloud infrastructure is critical to ensure all components of the cloud computing stack are available, high-performing, and secure. Cloud monitoring is a set of solutions and practices used to observe, measure, analyze, and manage the health of cloud-based IT infrastructure.

Cloud 295

Cloud Monitoring Best Practices Infrastructure 295

Dynatrace

OCTOBER 4, 2022

Log management and analytics is an essential part of any organization’s infrastructure, and it’s no secret the industry has suffered from a shortage of innovation for several years. Modern IT environments — whether multicloud, on-premises, or hybrid-cloud architectures — generate exponentially increasing data volumes.

Analytics 264

Analytics Artificial Intelligence Storage Serverless 264

Dynatrace

OCTOBER 4, 2022

For IT infrastructure managers and site reliability engineers, or SREs , logs provide a treasure trove of data. These traditional approaches to log monitoring and log analytics thwart IT teams’ goal to address infrastructure performance problems, security threats, and user experience issues. where an error occurred at the code level.

Analytics 264

Analytics Storage Infrastructure Metrics 264

Dynatrace

OCTOBER 4, 2022

While data lakes and data warehousing architectures are commonly used modes for storing and analyzing data, a data lakehouse is an efficient third way to store and analyze data that unifies the two architectures while preserving the benefits of both. Unlike data warehouses, however, data is not transformed before landing in storage.

Artificial Intelligence 246

Artificial Intelligence Storage Analytics Government 246

Dynatrace

APRIL 20, 2023

With more automated approaches to log monitoring and log analysis, however, organizations can gain visibility into their applications and infrastructure efficiently and with greater precision—even as cloud environments grow. Logs are automatically produced and time-stamped documentation of events relevant to cloud architectures.

Analytics 246

Analytics Infrastructure Storage Architecture 246

Scalegrid

FEBRUARY 8, 2024

A distributed storage system is foundational in today’s data-driven landscape, ensuring data spread over multiple servers is reliable, accessible, and manageable. Understanding distributed storage is imperative as data volumes and the need for robust storage solutions rise.

Storage 130

Storage Systems Big Data Azure 130

The Netflix TechBlog

SEPTEMBER 18, 2024

Vidhya Arvind , Rajasekhar Ummadisetty , Joey Lynch , Vinay Chella Introduction At Netflix our ability to deliver seamless, high-quality, streaming experiences to millions of users hinges on robust, global backend infrastructure. Data Model At its core, the KV abstraction is built around a two-level map architecture.

Latency 256

Latency Storage Cache Servers 256

The Netflix TechBlog

NOVEMBER 15, 2022

But it’s not easy: to pull this off, VFX studios need to build and operate serious technical infrastructure (compute, storage, networking, and software licensing), otherwise known as a “ render farm.”

Cloud 286

Cloud Entertainment AWS Infrastructure 286

Dynatrace

SEPTEMBER 23, 2022

Organizations continue to turn to multicloud architecture to deliver better, more secure software faster. But IT teams need to embrace IT automation and new data storage models to benefit from modern clouds. Moreover, IT pros say that cloud architecture and data repositories thwart achieving better data insight.

Cloud 245

Cloud Artificial Intelligence Innovation Architecture 245

Dynatrace

JANUARY 16, 2023

Findings provide insights into Kubernetes practitioners’ infrastructure preferences and how they use advanced Kubernetes platform technologies. Kubernetes infrastructure models differ between cloud and on-premises. Kubernetes infrastructure models differ between cloud and on-premises. Kubernetes moved to the cloud in 2022.

Open Source 358

Open Source Java Operating System Programming 358

High Scalability

JANUARY 11, 2022

Architecture. FUN FACT : In this talk , Rodrigo Schmidt, director of engineering at Instagram talks about the different challenges they have faced in scaling the data infrastructure at Instagram. After that, the post gets added to the feed of all the followers in the columnar data storage. High Level Design. System Components.

Design 334

Design Media Storage Logistics 334

Dynatrace

JUNE 10, 2024

Infrastructure type In most cases, legacy SIEM tools are on-premises. Security analytics must also contend with the multicomponent architecture of modern IT infrastructure. Dehydrated data has been compressed or otherwise altered for storage in a data warehouse.

Analytics 264

Analytics Network Open Source Hardware 264

The Netflix TechBlog

OCTOBER 8, 2024

In previous blog posts, we introduced the Key-Value Data Abstraction Layer and the Data Gateway Platform , both of which are integral to Netflix’s data architecture. Sharded Infrastructure : Leveraging the Data Gateway Platform , we can deploy single-tenant and/or multi-tenant infrastructure with the necessary access and traffic isolation.

Latency 235

Latency Storage Traffic Tuning 235

Dynatrace

MAY 19, 2023

For cloud operations teams, network performance monitoring is central in ensuring application and infrastructure performance. Network traffic growth is the main reason for increasing spending, largely because of the adoption of hybrid and multi-cloud architectures. Teams also don’t have to maintain normalized schemas to query data.

Network 246

Network Monitoring Performance Traffic 246

Dynatrace

OCTOBER 3, 2024

Trace your application Imagine a microservices architecture with hundreds of dependencies. There is no need to think about schema and indexes, re-hydration, or hot/cold storage. This architecture also means you’re not required to determine your log data use cases beforehand or while analyzing logs within the new logs app.

Performance 290

Performance Architecture Innovation Latency 290

Dynatrace

MARCH 6, 2024

On average, organizations use 10 different observability or monitoring tools to manage applications, infrastructure, and user experience across these environments. Kubernetes architectures make it easier to quickly scale services to new users and drive efficiency gains through dynamic resource provisioning.

Analytics 246

Analytics Innovation Strategy Storage 246

Dynatrace

OCTOBER 14, 2022

Log data provides a unique source of truth for debugging applications, optimizing infrastructure, and investigating security incidents. All this is easier said than done because: Kubernetes-based dynamic architecture is becoming the norm. Dynamic landscape and data handling requirements result in manual work. Try it out yourself.

Storage 278

Storage Government Network Infrastructure 278

Dynatrace

SEPTEMBER 28, 2020

Before we dive into the technical implementation, let me explain the visual concept of this “Global Status Page”: Another requirement for this status page was that it has to be lightweight, with no data storage at all. Lightweight architecture. This is where the consolidated API, which I presented in my last post , comes into play.

Storage 276

Storage Architecture Tuning Efficiency 276

Scalegrid

FEBRUARY 24, 2025

This decoupling is crucial in modern architectures where scalability and fault tolerance are paramount. The architecture of RabbitMQ is meticulously designed for complex message routing, enabling dynamic and flexible interactions between producers and consumers.

Best Practices 147

Best Practices Traffic Strategy Efficiency 147

Dynatrace

JUNE 9, 2023

Research has found that 99% of organizations have embraced a multicloud architecture. When data storage strategies become problematic to DevOps maturity Data warehouse-based approaches add cost and time to analytics projects. Cloud complexity leads to data silos Most organizations are battling cloud complexity.

DevOps 246

DevOps Analytics Storage Metrics 246

Dynatrace

OCTOBER 4, 2022

However, cloud infrastructure has become increasingly complex. Further, the delivery infrastructure that makes this happen has also become complex. Traditionally, though, to gain true business insight, organizations had to make tradeoffs between accessing quality, real-time data and factors such as data storage costs.

Analytics 246

Analytics Software Software Storage 246

Dynatrace

NOVEMBER 15, 2021

Log monitoring, log analysis, and log analytics are more important than ever as organizations adopt more cloud-native technologies, containers, and microservices-based architectures. Log analytics also help identify ways to make infrastructure environments more predictable, efficient, and resilient. Dynatrace news. What are logs?

Analytics 264

Analytics Monitoring DevOps Artificial Intelligence 264

Dynatrace

MARCH 24, 2023

Containers enable developers to package microservices or applications with the libraries, configuration files, and dependencies needed to run on any infrastructure, regardless of the target system environment. How does container orchestration work? And organizations use Kubernetes to run on an increasing array of workloads.

Infrastructure 246

Infrastructure Open Source Operating System Cloud 246

The Morning Paper

MARCH 8, 2020

Building an elastic query engine on disaggregated storage , Vuppalapati, NSDI’20. This paper presents Snowflake design and implementation along with a discussion on how recent changes in cloud infrastructure (emerging hardware, fine-grained billing, etc.) But the ephemeral storage service for intermediate data is not based on S3.

Storage 112

Storage Engineering Cache Serverless 112

Dynatrace

JANUARY 4, 2024

It starts with implementing data governance practices, which set standards and policies for data use and management in areas such as quality, security, compliance, storage, stewardship, and integration. Modern, cloud-native architectures have many moving parts, and identifying them all is a daunting task with human effort alone.

Government 263

Government Analytics Benchmarking Storage 263

Dynatrace

FEBRUARY 9, 2022

Cloud vendors such as Amazon Web Services (AWS), Microsoft, and Google provide a wide spectrum of serverless services for compute and event-driven workloads, databases, storage, messaging, and other purposes. AI-powered automation and deep, broad observability for serverless architectures. Dynatrace news.

Serverless 246

Serverless Azure Lambda AWS 246

Dynatrace

NOVEMBER 10, 2020

As a leader in cloud infrastructure and platform services , the Google Cloud Platform is fast becoming an integral part of many enterprises’ cloud strategies. Google Cloud Storage. The installation process and architecture are well documented and described in the GitHub repository. Dynatrace news. Google Cloud Datastore.

Google 264

Google Cloud Serverless Innovation 264

Dynatrace

APRIL 8, 2024

Optimize the IT infrastructure supporting risk management processes and controls for maximum performance and resilience. The IT infrastructure, services, and applications that enable processes for risk management must perform optimally. Once teams solidify infrastructure and application performance, security is the subsequent priority.

Analytics 246

Analytics Infrastructure Efficiency Technology 246

Uber Engineering

DECEMBER 19, 2019

Every day around the world, millions of trips take place across the Uber network, giving users more reliable transportation through ridesharing, bikes, and scooters, drivers and truckers additional opportunities to earn, employees and employers more convenient business travel, and hungry … The post Uber Infrastructure in 2019: Improving Reliability, (..)

Infrastructure 145

Infrastructure Transportation Network Engineering 145

Dynatrace

JUNE 7, 2023

Container orchestration allows an organization to digitally transform at a rapid clip without getting bogged down by slow, siloed development, difficult scaling, and high costs associated with optimizing application infrastructure. This flexibility helps organizations avoid vendor lock-in. OpenShift Dedicated (OSD). Networking.

Open Source 246

Open Source Social Media Infrastructure Operating System 246

Scalegrid

JUNE 4, 2020

As an open source database, it’s a highly popular choice for enterprise applications looking to modernize their infrastructure and reduce their total cost of ownership, along with startup and developer applications looking for a powerful, flexible and cost-effective database to work with. At a glance – TLDR. Compare Pricing.

Database 230

Database Latency Benchmarking Performance 230

Dynatrace

JULY 29, 2021

Today’s digital businesses run on heterogeneous and highly dynamic architectures with interconnected applications and microservices deployed via Kubernetes and other cloud-native platforms. Common questions include: Where do bottlenecks occur in our architecture? Dynatrace news. How can we optimize for performance and scalability?

Analytics 246

Analytics Open Source Serverless Architecture 246

Dynatrace

DECEMBER 8, 2021

In contrast to modern software architecture, which uses distributed microservices, organizations historically structured their applications in a pattern known as “monolithic.” Monolithic applications earned their name because their structure is a single running application, which often shares the same physical infrastructure.

Cloud 246

Cloud Serverless Hardware Operating System 246