The Netflix TechBlog

AUGUST 1, 2022

A Data Movement and Processing Platform @ Netflix By Bo Lei , Guilherme Pires , James Shao , Kasturi Chatterjee , Sujay Jain , Vlad Sydorenko Background Realtime processing technologies (A.K.A stream processing) is one of the key factors that enable Netflix to maintain its leading position in the competition of entertaining our users.

Processing 272

Processing Transportation Entertainment Tuning 272

Dynatrace

OCTOBER 23, 2024

Automatic data capture and display: More data, including span attributes, is available for out-of-the-box analysis, with no additional configuration necessary. As soon as the new Distributed Tracing Experience is available for your environment, you’ll see a teaser banner in your classic Distributed Traces app.

Tuning 299

Tuning Website Availability Performance 299

Dynatrace

JANUARY 22, 2025

Dynatrace Simple Workflows make this process automatic and frictionlessthere is no additional cost for workflows. Why manual alerting falls short As your product and deployments scale horizontally and vertically, the sheer volume of data makes it impossible for teams to catch every error quickly using manual processes.

Development 245

Development Processing Monitoring Code 245

Dynatrace

SEPTEMBER 27, 2024

Unrealized optimization potential of business processes due to monitoring gaps Imagine a retail company facing gaps in its business process monitoring due to disparate data sources. Due to separated systems that handle different parts of the process, the view of the process is fragmented.

Processing 263

Processing Retail Analytics Monitoring 263

Dynatrace

OCTOBER 16, 2024

Kickstart your creation journey using ready-made dashboards and notebooks Creating dashboards and notebooks from scratch can take time, particularly when figuring out available data and how to best use it. Kickstarting the dashboard creation process is, however, just one advantage of ready-made dashboards.

Social Media 263

Social Media Metrics Network Analytics 263

Dynatrace

OCTOBER 29, 2024

In addition to service-level monitoring, certain services within the OpenTelemetry demo application expose process-level metrics, such as CPU and memory consumption, number of threads, or heap size for services written in different languages. You can even walk through the same example above.

Metrics 246

Metrics Tuning Monitoring Availability 246

Dynatrace

JULY 24, 2024

By leveraging Dynatrace observability on Red Hat OpenShift running on Linux, you can accelerate modernization to hybrid cloud and increase operational efficiencies with greater visibility across the full stack from hardware through application processes. Dynatrace observability is available for Red Hat OpenShift on IBM Power.

Availability 279

Availability Infrastructure Metrics Monitoring 279

Dynatrace

AUGUST 16, 2022

For example: Infrastructure services might provide data about request timings that can give you a precise overview of system health, but the data is logged in a custom format. For example, Dynatrace recently introduced the extraction of log-based metrics for JSON logs.

Processing 246

Processing Metrics Monitoring Java 246

Dynatrace

FEBRUARY 4, 2025

As every developer knows, logs are crucial for uncovering insights and detecting fundamental flaws, such as process crashes or exceptions. Using Live Debugger, we immediately get insights into the running code, including variable values, process and thread information, and even a trace ID for the captured transaction.

Development 235

Development Analytics Code Architecture 235

Dynatrace

OCTOBER 14, 2024

Access policies for Dynatrace Grail™ data lakehouse are still available as service-related policies; they allow you to control access to the monitoring data on a per-data-source level, for example, logs and metrics. All other default policies on the service level, for example, “AutomationEngine – User” access, are now marked as Legacy.

Monitoring 246

Monitoring Metrics Systems Scalability 246

The Netflix TechBlog

NOVEMBER 12, 2024

Both categories share common requirements, such as high throughput and high availability. Eventually Consistent Global Counter While some users may accept the limitations of a Best-Effort counter, others opt for precise counts, durability and global availability.

Latency 247

Latency Cache Infrastructure Strategy 247

Dynatrace

JANUARY 31, 2024

Organizations choose data-driven approaches to maximize the value of their data, achieve better business outcomes, and realize cost savings by improving their products, services, and processes. Data is then dynamically routed into pipelines for further processing.

Analytics 272

Analytics Processing Transportation Storage 272

The Netflix TechBlog

NOVEMBER 14, 2023

By Abhinaya Shetty , Bharath Mummadisetty In the inaugural blog post of this series, we introduced you to the state of our pipelines before Psyberg and the challenges with incremental processing that led us to create the Psyberg framework within Netflix’s Membership and Finance data engineering team.

Processing 230

Processing Data Engineering Efficiency Analytics 230

Dynatrace

NOVEMBER 11, 2024

However, due to the fact that they boil down selected indicators to single values and track error budget levels, they also offer a suitable way to monitor optimization processes while aligning on single values to meet overall goals. By recognizing the insights provided, you can optimize processes and improve overall efficiency.

Efficiency 263

Efficiency Best Practices Monitoring Cloud 263

Dynatrace

FEBRUARY 4, 2025

In this example use case, we’re investigating a high CPU load issue in an application using OpenTelemetry. Dynatrace OpenPipeline is configured to ingest logs from our environment, including the AdService example service. Dynatrace Live Debugger will be generally available (GA) within the next 90 days.

Performance 147

Performance Code Processing Availability 147

Dynatrace

MARCH 5, 2025

The challenge along the path Well-understood within IT are the coarse reduction levers used to reduce emissions; shifting workloads to the cloud and choosing green energy sources are two prime examples. The certification results are now publicly available.

Energy 223

Energy Analytics Traffic Cloud 223

The Netflix TechBlog

FEBRUARY 14, 2025

It requires a state-of-the-art system that can track and process these impressions while maintaining a detailed history of each profiles exposure. In this multi-part blog series, we take you behind the scenes of our system that processes billions of impressions daily.

Tuning 165

Tuning Latency Efficiency Storage 165

DZone

FEBRUARY 28, 2025

Grafana Loki is a horizontally scalable, highly available log aggregation system. Logs can also be transformed appropriately for presentation, for example, or further pipeline processing. It is designed for simplicity and cost-efficiency. Loki can provide a comprehensive log journey.

Architecture 130

Architecture Scalability Efficiency Cloud 130

Scalegrid

FEBRUARY 6, 2025

RabbitMQ is designed for flexible routing and message reliability, while Kafka handles high-throughput event streaming and real-time data processing. RabbitMQ follows a message broker model with advanced routing, while Kafkas event streaming architecture uses partitioned logs for distributed processing. What is Apache Kafka?

Latency 147

Latency Analytics Architecture Storage 147

DZone

APRIL 21, 2023

Various test tools are available for different types of testing, including unit testing, integration testing, and more. Some test tools are intended for developers during the development process, while others are designed for quality assurance teams or end users.

Testing Tools 196

Testing Tools Best Practices Testing Hardware 196

The Netflix TechBlog

MARCH 4, 2025

As a result, requests are uniformly handled, and responses are processed cohesively. This data is processed from a real-time impressions stream into a Kafka queue, which our title health system regularly polls. Many of the metadata and assets involved in title setup have specific timelines for when they become available to members.

Traffic 160

Traffic Strategy Entertainment Innovation 160

Scalegrid

MAY 13, 2020

Greenplum Database is a massively parallel processing (MPP) SQL database that is built and based on PostgreSQL. When handling large amounts of complex data, or big data, chances are that your main machine might start getting crushed by all of the data it has to process in order to produce your analytics results. Query Optimization.

Big Data 321

Big Data Database Artificial Intelligence Open Source 321

Dynatrace

MARCH 13, 2025

Because it includes examples of 10 programming languages that OpenTelemetry supports with SDKs, the application makes a good reference for developers on how to use OpenTelemetry. In this example, we’ll use Dynatrace. This example illustrates how to pass the token most easily using the terminal. metrics from span data.

Open Source 147

Open Source Metrics Architecture Infrastructure 147

The Netflix TechBlog

DECEMBER 19, 2024

As an example, you can specify a Config that reads a pleasantly human-readable configuration file, formatted as TOML. The standard dictionary subscript notation is also available. Take a look at two interesting examples of this pattern in the documentation. Configs can of course also be used within yourflow.

Best Practices 200

Best Practices Cache Metrics Code 200

DZone

JUNE 21, 2023

Monitoring and observability are two key concepts that facilitate this process, offering valuable visibility into the health and performance of systems. In this article, we will explore the differences between monitoring and observability, provide examples to illustrate their applications and highlight their respective benefits.

DevOps 324

DevOps Systems Monitoring Metrics 324

Dynatrace

JULY 11, 2023

By leveraging the Dynatrace Operator and Dynatrace capabilities on Red Hat OpenShift on IBM Power, customers can accelerate their modernization to hybrid cloud and increase operational efficiencies with greater visibility across the full stack from hardware through application processes.

Architecture 189

Architecture Availability Infrastructure Monitoring 189

Dynatrace

DECEMBER 27, 2022

When organizations implement SLOs, they can improve software development processes and application performance. Stable, well-calibrated SLOs pave the way for teams to automate additional processes and testing throughout the software delivery lifecycle. Instead, they can ensure that services comport with the pre-established benchmarks.

Software 306

Software Software Benchmarking Latency 306

Dynatrace

OCTOBER 2, 2024

The Clouds app provides a view of all available cloud-native services. Logs in context, along with other details, are instantly available after selecting a resource. The reasons are easy to find, looking at the latest improvements that went live along with the general availability of the Logs app.

Analytics 304

Analytics AWS DevOps Cloud 304

Dynatrace

AUGUST 24, 2020

But there are other related components and processes (for example, cloud provider infrastructure) that can cause problems in applications running on Kubernetes. And because Dynatrace can consume CloudWatch metrics, almost all your AWS usage information is available to you within Dynatrace. Dynatrace OneAgent documentation .

Analytics 305

Analytics Infrastructure AWS Operating System 305

Dynatrace

SEPTEMBER 13, 2023

For example, optimizing resource utilization for greater scale and lower cost and driving insights to increase adoption of cloud-native serverless services. This process enables you to continuously evaluate software against predefined quality criteria and service level objectives (SLOs) in pre-production environments.

AWS 305

AWS Efficiency Azure Cloud 305

The Netflix TechBlog

JANUARY 2, 2025

Ideally, we would have causal estimates from an A/B test to use for validation, but since that is not available, we use another causal inference design as one of our ensemble of validation approaches. Each format has a different production process and different patterns of cash spend, called our Content Forecast.

Analytics 187

Analytics Engineering Games Entertainment 187

Dynatrace

MARCH 27, 2024

The automated extraction of ownership information, for example, from Kubernetes annotations, is therefore essential. Secondly, knowing who is responsible is essential but not sufficient, especially if you want to automate your triage process. An example via Monaco can be found in this public GitHub repository.

Best Practices 263

Best Practices Azure Database Code 263

The Netflix TechBlog

DECEMBER 3, 2022

And by “sample” we mean “an example”, like food samples in your local grocery store. Obviously not all tools are made with the same use case in mind, so we are planning to add more code samples for other (than classical batch ETL) data processing purposes, e.g. Machine Learning model building and scoring. test_sparksql_write.py

Best Practices 360

Best Practices Code Testing Data Engineering 360

Dynatrace

FEBRUARY 4, 2025

Were excited to announce that Davis CoPilot Chat is now available across the Dynatrace platform. To help you navigate this and boost your efficiency, we’re excited to announce that Davis CoPilot Chat is now generally available (GA). Davis CoPilot Chat will be available with the release of Dynatrace SaaS version 1.307.

Database 130

Database Efficiency Availability Speed 130

Dynatrace

JUNE 4, 2024

This process, known as auto-adaptive thresholding, eliminates the need to define a static threshold upfront. For example, the team must establish specific thresholds for desired service performance behavior. For instance, if an objective is violated, they’re immediately notified, for example, through a Slack channel.

Metrics 297

Metrics Engineering Code Tuning 297

Dynatrace

MARCH 20, 2024

Deploying software in Kubernetes is often viewed as a straightforward process—just use kubectl or a GitOps solution like ArgoCD to deploy a YAML file, and you’re all set, right? Infrastructure health The underlying infrastructure’s health directly impacts application availability and performance.

Open Source 299

Open Source Hardware DevOps Infrastructure 299

Dynatrace

JUNE 1, 2023

But how do you get started, and what are some service level objective examples? In this post, I’ll lay out five foundational service level objective examples that every DevOps and SRE team should consider. These organizations rely heavily on performance, availability, and user satisfaction to drive sales and retain customers.

Latency 245

Latency Website Traffic DevOps 245