The Netflix TechBlog

DECEMBER 3, 2022

by Jasmine Omeke , Obi-Ike Nwoke , Olek Gorajek Intro This post is for all data practitioners, who are interested in learning about bootstrapping, standardization and automation of batch data pipelines at Netflix. You may remember Dataflow from the post we wrote last year titled Data pipeline asset management with Dataflow.

Best Practices 360

Best Practices Code Testing Data Engineering 360

Dynatrace

NOVEMBER 4, 2024

There’s a goldmine of business data traversing your IT systems, yet most of it remains untapped. To unlock business value, the data must be: Accessible from anywhere. Data has value only when you can access it, no matter where it lies. Agile business decisions rely on fresh data. Easy to access. Contextualized.

Analytics 241

Analytics Airlines Metrics Monitoring 241

Dynatrace

MAY 17, 2022

By taking advantage of native Kubernetes standards, Dynatrace Cloud Native Full Stack injection empowers you to precisely provide the data that your teams need in exceptionally fast and automated ways. These situations apply to organizations that are looking to: Onboard teams to Kubernetes observability data as quickly as possible.

Availability 348

Availability Scalability Cloud Metrics 348

Dynatrace

NOVEMBER 22, 2022

The nirvana state of system uptime at peak loads is known as “five-nines availability.” In its pursuit, IT teams hover over system performance dashboards hoping their preparations will deliver five nines—or even four nines—availability. But is five nines availability attainable? Downtime per year. 90% (one nine).

Infrastructure 246

Infrastructure Availability Systems Retail 246

Dynatrace

JULY 15, 2024

As HTTP and browser monitors cover the application level of the ISO /OSI model , successful executions of synthetic tests indicate that availability and performance meet the expected thresholds of your entire technological stack. Our script, available on GitHub , provides details. Are the corresponding services running on those hosts?

Availability 246

Availability Network Monitoring Infrastructure 246

Dynatrace

OCTOBER 31, 2024

To provide maximum freedom in selecting the service-level indicators that matter most to your business, Dynatrace combines SLOs with the power of Dynatrace Grail™ data lakehouse, the central data platform with heterogeneous and contextually linked data. This is where Grail, the Dynatrace central data platform, excels.

Metrics 277

Metrics Availability Monitoring Scalability 277

Dynatrace

SEPTEMBER 27, 2024

Unrealized optimization potential of business processes due to monitoring gaps Imagine a retail company facing gaps in its business process monitoring due to disparate data sources. Due to separated systems that handle different parts of the process, the view of the process is fragmented.

Processing 263

Processing Retail Analytics Monitoring 263

Dynatrace

OCTOBER 4, 2022

For IT infrastructure managers and site reliability engineers, or SREs , logs provide a treasure trove of data. But on their own, logs present just another data silo as IT professionals attempt to troubleshoot and remediate problems. Data volume explosion in multicloud environments poses log issues.

Analytics 264

Analytics Storage Infrastructure Metrics 264

Dynatrace

MAY 2, 2024

Across the globe, privacy laws grant individuals data subject rights, such as the right to access and delete personal data processed about them. Successful compliance with privacy rights requests involves tracking and verifying requests across the entire data ecosystem, including third-party services.

Tuning 278

Tuning Scalability Efficiency Processing 278

Dynatrace

JULY 24, 2024

IBM Z and LinuxONE mainframes running the Linux operating system enable you to respond faster to business demands, protect data from core to cloud, and streamline insights and automation. Telemetry data, such as traces and metrics, allow you to analyze the end-to-end performance of your deployed applications.

Availability 279

Availability Infrastructure Metrics Hardware 279

Dynatrace

OCTOBER 3, 2024

The end goal, of course, is to optimize the availability of organizations’ software. Dynatrace is widely recognized for its AI capabilities’ ability to predict and prevent issues, and automatically identify root causes, maximizing availability.

Availability 278

Availability DevOps Cloud Analytics 278

Dynatrace

OCTOBER 4, 2022

Some time ago, at a restaurant near Boston, three Dynatrace colleagues dined and discussed the growing data challenge for enterprises. At its core, this challenge involves a rapid increase in the amount—and complexity—of data collected within a company. Work with different and independent data types. Thus, Grail was born.

Artificial Intelligence 290

Artificial Intelligence Analytics Storage Architecture 290

Dynatrace

MAY 23, 2024

The jobs executing such workloads are usually required to operate indefinitely on unbounded streams of continuous data and exhibit heterogeneous modes of failure as they run over long periods. Performance is usually a primary concern when using stream processing frameworks. This significantly increases event latency.

Engineering 245

Engineering Tuning Latency Open Source 245

Dynatrace

OCTOBER 16, 2024

When we launched the new Dynatrace experience, we introduced major updates to the platform, including Grail ™, our innovative data lakehouse unifying observability, security, and business data, and Dynatrace Query Language ( DQL ) for accessing and exploring unified data.

Social Media 263

Social Media Metrics Network Analytics 263

The Netflix TechBlog

NOVEMBER 14, 2023

By Abhinaya Shetty , Bharath Mummadisetty At Netflix, our Membership and Finance Data Engineering team harnesses diverse data related to plans, pricing, membership life cycle, and revenue to fuel analytics, power various dashboards, and make data-informed decisions. We expect complete and accurate data at the end of each run.

Data Engineering 268

Data Engineering Engineering Processing Games 268

Dynatrace

OCTOBER 14, 2022

Log data provides a unique source of truth for debugging applications, optimizing infrastructure, and investigating security incidents. This contextualization of log data enables AI-powered problem detection and root cause analysis at scale. Dynamic landscape and data handling requirements result in manual work.

Storage 278

Storage Government Network Infrastructure 278

Dynatrace

SEPTEMBER 14, 2022

As an industry leader, Dynatrace promotes primarily using software and AI to deal with this complexity at scale instead of just putting data on dashboards. Does that mean that reactive and exploratory data analysis, often done manually and with the help of dashboards, are dead? Why today’s data analytics solutions still fail us.

Metrics 264

Metrics Analytics Monitoring Efficiency 264

Dynatrace

MAY 3, 2024

Log data—the most verbose form of observability data, complementing other standardized signals like metrics and traces—is especially critical. As cloud complexity grows, it brings more volume, velocity, and variety of log data. When trying to address this challenge, your cloud architects will likely choose Amazon Data Firehose.

Cloud 279

Cloud Lambda AWS Analytics 279

Dynatrace

OCTOBER 3, 2024

Considering the latest State of Observability 2024 report, it’s evident that multicloud environments not only come with an explosion of data beyond humans’ ability to manage it. It’s increasingly difficult to ingest, manage, store, and sort through this amount of data. You can find the list of use cases here.

Performance 290

Performance Architecture Innovation Latency 290

Dynatrace

JANUARY 4, 2024

While this approach can be effective if the model is trained with a large amount of data, even in the best-case scenarios, it amounts to an informed guess, rather than a certainty. But to be successful, data quality is critical. Teams need to ensure the data is accurate and correctly represents real-world scenarios. Consistency.

Government 263

Government Analytics Benchmarking Storage 263

Dynatrace

AUGUST 20, 2024

Driven by that value, Dynatrace brings real-time observability, security, and business data into context and makes sense of it so our customers can get answers, automate, predict, and prevent. Executives are sitting on a goldmine of data, and they don’t know it.

Analytics 285

Analytics Latency Processing Systems 285

Dynatrace

JANUARY 31, 2024

I have ingested important custom data into Dynatrace, critical to running my applications and making accurate business decisions… but can I trust the accuracy and reliability?” ” Welcome to the world of data observability. At its core, data observability is about ensuring the availability, reliability, and quality of data.

DevOps 272

DevOps Analytics Airlines Metrics 272

Dynatrace

MAY 21, 2024

Observing complex environments involves handling regulatory, compliance, and data governance requirements. This continuously evolving landscape requires careful management and clarity regarding how sensitive data is used. This is particularly important when dealing with large volumes of data.

Government 277

Government Storage Tuning Monitoring 277

The Netflix TechBlog

MARCH 10, 2023

This platform has evolved from supporting studio applications to data science applications, machine-learning applications to discover the assets metadata, and build various data facts. Hence we built the data pipeline that can be used to extract the existing assets metadata and process it specifically to each new use case.

Media 240

Media Traffic Processing Design 240

Dynatrace

FEBRUARY 4, 2025

It packages the existing Dynatrace capabilities needed by developers in their day-to-day worksuch as logs, distributed traces, profiling data, exceptions, and more. Dashboards are a great tool for gaining real-time insights into applications by transforming complex data into dynamic, interactive visualizations.

Development 235

Development Analytics Code Architecture 235

The Netflix TechBlog

SEPTEMBER 18, 2024

Central to this infrastructure is our use of multiple online distributed databases such as Apache Cassandra , a NoSQL database known for its high availability and scalability. Second, developers had to constantly re-learn new data modeling practices and common yet critical data access patterns.

Latency 256

Latency Storage Cache Servers 256

Dynatrace

AUGUST 29, 2024

In today’s data-driven world, businesses across various industry verticals increasingly leverage the Internet of Things (IoT) to drive efficiency and innovation. Both methods allow you to ingest and process raw data and metrics. The ADS-B protocol differs significantly from web technologies.

IoT 264

IoT Analytics Transportation Metrics 264

Dynatrace

APRIL 19, 2023

Creating an ecosystem that facilitates data security and data privacy by design can be difficult, but it’s critical to securing information. When organizations focus on data privacy by design, they build security considerations into cloud systems upfront rather than as a bolt-on consideration.

Design 246

Design Storage Programming Analytics 246

Dynatrace

OCTOBER 29, 2024

In this blog post, we’ll walk you through a hands-on demo that showcases how the Distributed Tracing app transforms raw OpenTelemetry data into actionable insights Set up the Demo To run this demo yourself, you’ll need the following: A Dynatrace tenant. If you don’t have one, you can use a trial account.

Metrics 246

Metrics Tuning Monitoring Availability 246

Dynatrace

FEBRUARY 28, 2024

In a digital-first world, site reliability engineers and IT data analysts face numerous challenges with data quality and reliability in their quest for cloud control. Increasingly, organizations seek to address these problems using AI techniques as part of their exploratory data analytics practices.

Analytics 246

Analytics Metrics Media Monitoring 246

Dynatrace

NOVEMBER 6, 2023

Recently, the Parliament of India released the Digital Personal Data Protection Act 2023 , which regulates the processing of digital personal data in India and recognizes the right of individuals to protect their data in India.

AWS 264

AWS Azure Tuning Analytics 264

Dynatrace

SEPTEMBER 5, 2023

They handle complex infrastructure, maintain service availability, and respond swiftly to incidents. Predictive AI uses machine learning, data analysis, statistical models, and AI methods to predict anomalies, identify patterns, and create forecasts. This data-driven approach fosters continuous refinement of processes and systems.

Artificial Intelligence 279

Artificial Intelligence DevOps Analytics Engineering 279

Dynatrace

MAY 1, 2023

IT operations analytics is the process of unifying, storing, and contextually analyzing operational data to understand the health of applications, infrastructure, and environments and streamline everyday operations. ITOA automates repetitive cloud operations tasks and streamlines the flow of analytics into decision-making processes.

Analytics 246

Analytics Artificial Intelligence Big Data Open Source 246

Dynatrace

NOVEMBER 6, 2023

Azure observability and Azure data analytics are critical requirements amid the deluge of data in Azure cloud computing environments. As digital transformation accelerates and more organizations are migrating workloads to Azure and other cloud environments, they need observability and data analytics capabilities that can keep pace.

Azure 245

Azure Analytics Artificial Intelligence Innovation 245

Dynatrace

OCTOBER 6, 2023

Grail: Enterprise-ready data lakehouse Grail, the Dynatrace causational data lakehouse, was explicitly designed for observability and security data, with artificial intelligence integrated into its foundation. Tables are a physical data model, essentially the type of observability data that you can store.

Artificial Intelligence 279

Artificial Intelligence Metrics Analytics Storage 279

The Netflix TechBlog

OCTOBER 8, 2024

Rajiv Shringi Vinay Chella Kaidan Fullerton Oleksii Tkachuk Joey Lynch Introduction As Netflix continues to expand and diversify into various sectors like Video on Demand and Gaming , the ability to ingest and store vast amounts of temporal data — often reaching petabytes — with millisecond access latency has become increasingly vital.

Latency 235

Latency Storage Traffic Tuning 235

Dynatrace

FEBRUARY 16, 2023

How do you get more value from petabytes of exponentially exploding, increasingly heterogeneous data? The short answer: The three pillars of observability—logs, metrics, and traces—converging on a data lakehouse. The goal is to turn more data into insights so the whole organization can make data-driven decisions and automate processes.

Analytics 246

Analytics Innovation Metrics Database 246