Event and Processing - Technology Performance Pulse

Rapid Event Notification System at Netflix

The Netflix TechBlog

FEBRUARY 18, 2022

To this end, we developed a Rapid Event Notification System (RENO) to support use cases that require server initiated communication with devices in a scalable and extensible manner. In this blog post, we will give an overview of the Rapid Event Notification System at Netflix and share some of the learnings we gained along the way.

Systems

Systems Traffic Architecture Mobile

OpenPipeline: Simplify access to critical business data

Dynatrace

NOVEMBER 4, 2024

Business events: Delivering the best data It’s been two years since we introduced business events , a special class of events designed to support even the most demanding business use cases. Business event ingestion and analysis with log files. OpenPipeline: Simplify access and unify business events from anywhere.

Analytics

Analytics Airlines Metrics Monitoring

Microsoft Azure Event Hubs

DZone

FEBRUARY 23, 2023

Introduction With big data streaming platform and event ingestion service Azure Event Hubs , millions of events can be received and processed in a single second. Any real-time analytics provider or batching/storage adaptor can transform and store data supplied to an event hub.

Azure

Azure Big Data Storage Analytics

How To Collect and Ship Windows Events Logs With OpenTelemetry

DZone

FEBRUARY 17, 2023

If you use Windows, you will want to monitor Windows Events. A recent contribution of a distribution of the OpenTelemetry (OTel) Collector makes it much easier to monitor Windows Events with OpenTel. We will be shipping Windows Event logs to a popular backend: Google Cloud Ops. You can find out more on the GitHub page here.

Operating System

Operating System Google Monitoring Cloud

Business process observability: An IT solution to a business challenge

Dynatrace

FEBRUARY 24, 2025

Business processes support virtually all aspects of an organizations operations. Theyre often categorized by their function; core processes directly create customer value, support processes increase departmental efficiency, and management processes drive strategic goals and compliance.

Processing

Processing Analytics Infrastructure Monitoring

Event Streams Are Nothing Without Action

DZone

FEBRUARY 17, 2023

Each data point in a system that produces data on an ongoing basis corresponds to an Event. Event Streams are described as a continuous flow of events or data points. Event Streams are sometimes referred to as Data Streams within the developer community since they consist of continuous data points.

Processing

Processing Systems Development

Generate security events from Dynatrace Security Investigator via OpenPipeline

Dynatrace

SEPTEMBER 17, 2024

You now want to detect such events automatically by creating a custom Dynatrace security event. Ingest query results as security events The simplest way to do this is to use Dynatrace OpenPipeline. Set up a custom pipeline The best way to set up a security event ingestion to Dynatrace is via Dynatrace OpenPipeline.

Transportation

Transportation AWS Engineering Processing

Netflix’s Distributed Counter Abstraction

The Netflix TechBlog

NOVEMBER 12, 2024

By: Rajiv Shringi , Oleksii Tkachuk , Kartik Sathyanarayanan Introduction In our previous blog post, we introduced Netflix’s TimeSeries Abstraction , a distributed service designed to store and query large volumes of temporal event data with low millisecond latencies. This process can also be used to track the provenance of increments.

Latency

Latency Cache Infrastructure Strategy

New integrations announced at AWS re:Invent enhance cloud performance, security, and automation

Dynatrace

DECEMBER 3, 2024

This integration simplifies the process of embedding Dynatrace full-stack observability directly into custom Amazon Machine Images (AMIs). VMware migration support for seamless transitions For enterprises transitioning VMware-based workloads to the cloud, the process can be complex and resource-intensive.

AWS

AWS Cloud Performance Innovation

Create simple workflows to automate alerts during development

Dynatrace

JANUARY 22, 2025

Dynatrace Simple Workflows make this process automatic and frictionlessthere is no additional cost for workflows. Why manual alerting falls short As your product and deployments scale horizontally and vertically, the sheer volume of data makes it impossible for teams to catch every error quickly using manual processes.

Development

Development Processing Monitoring Code

Business Flow: Why IT operations teams should monitor business processes

Dynatrace

MARCH 12, 2024

The business process observability challenge Increasingly dynamic business conditions demand business agility; reacting to a supply chain disruption and optimizing order fulfillment are simple but illustrative examples. Most business processes are not monitored. First and foremost, it’s a data problem.

Processing

Processing Monitoring Analytics C++

Dynatrace enhances Business Analytics with business events powered by Grail

Dynatrace

NOVEMBER 15, 2022

Business events powered by our new Grail™ data lakehouse and by other Dynatrace platform technologies ensures the real-time precision that business and IT teams need to make data-driven decisions and improve business outcomes. Business events deliver the industry’s broadest, deepest, and easiest access to your critical business data.

Analytics

Analytics Mobile Metrics Games

Leverage logs for an end-to-end view of your business processes via Dynatrace OpenPipeline

Dynatrace

SEPTEMBER 27, 2024

Unrealized optimization potential of business processes due to monitoring gaps Imagine a retail company facing gaps in its business process monitoring due to disparate data sources. Due to separated systems that handle different parts of the process, the view of the process is fragmented.

Processing

Processing Retail Analytics Monitoring

Reliability indicators that matter to your business: SLOs for all data types

Dynatrace

OCTOBER 31, 2024

This lets you build your SLOs around the indicators that matter to you and your customers—critical metrics related to availability, failure rates, request response times, or select logs and business events. Are you experiencing an increase or degradation in certain events that indicate a rising problem?

Metrics

Metrics Availability Monitoring Scalability

Data Mesh?—?A Data Movement and Processing Platform @ Netflix

The Netflix TechBlog

AUGUST 1, 2022

A Data Movement and Processing Platform @ Netflix By Bo Lei , Guilherme Pires , James Shao , Kasturi Chatterjee , Sujay Jain , Vlad Sydorenko Background Realtime processing technologies (A.K.A stream processing) is one of the key factors that enable Netflix to maintain its leading position in the competition of entertaining our users.

Processing

Processing Transportation Entertainment Tuning

Extend business observability: Extract business events from online databases (Part 2)

Dynatrace

SEPTEMBER 8, 2023

There are three high-level steps to set up the database business-event stream. Step-by-step: Set up a custom MySQL database extension Now we’ll show you step-by-step how to create a custom MySQL database extension for querying and pushing business data to the Dynatrace business events endpoint. Don’t rename the file.

Database

Database Artificial Intelligence Metrics Monitoring

Practical business process monitoring for real-time business observability

Dynatrace

FEBRUARY 9, 2024

Recent platform enhancements in the latest Dynatrace, including business events powered by Grail™, make accessing the goldmine of business data flowing through your IT systems easier than ever. Business events can come from many sources, including OneAgent®, external business systems, RUM sessions, or log files.

Processing

Processing Monitoring Analytics Games

Enhanced root cause analysis using events

Dynatrace

NOVEMBER 11, 2022

Getting the information and processes in place to ensure alerts like this example can be organizationally difficult. However, Dynatrace can often miss crucial pieces of the puzzle because humans haven’t told it about whole processes occurring on the “human” side of the environment. Offline processes.

DevOps

DevOps C++ Serverless Processing

The value of business events: How IT can increase business agility

Dynatrace

JANUARY 18, 2023

Business events are a special class of events, new to Business Analytics; together with Grail, our data lakehouse, they provide the precision and advanced analytics capabilities required by your most important business use cases. What are business events? This diagram shows a few examples of business events.

Analytics

Analytics Retail Storage Monitoring

Improving customer experience with business process monitoring

Dynatrace

DECEMBER 21, 2023

A business process is a collection of related, usually structured tasks or steps, performed in sequence, that achieve a defined business goal. Tasks may be manual or automatic, and many business processes will include a combination of both. Make better decisions by providing managers with real-time data about the business.

Processing

Processing Monitoring Retail Analytics

Extract metrics from business events to increase the value of business analytics

Dynatrace

FEBRUARY 2, 2023

To close these critical gaps, Dynatrace has defined a new class of events called business events. Dynatrace OneAgent ® prioritizes business events over observability metrics to ensure the lossless precision you need to support demanding business use cases.

Analytics

Analytics Metrics DevOps Storage

Extend business observability: Extract business events from online databases (Part 1)

Dynatrace

AUGUST 10, 2023

But existing business intelligence (BI) tools often lack the broad context, ease of data access, and real-time insights needed to understand and improve customer experience and complex business processes. The key challenges include: Business data is often difficult to access, resulting in fragile data pipelines.

Database

Database Metrics Retail Analytics

Metric events: Set up anomaly detection based on your business needs

Dynatrace

OCTOBER 17, 2022

Davis is the causational AI from Dynatrace that processes billions of events and dependencies and constantly analyzes your IT infrastructure. Dynatrace metric events offer the flexibility needed to customize your anomaly detection configuration. You can use placeholders to create meaningful event-message titles and descriptions.

Metrics

Metrics Strategy Infrastructure Monitoring

2. Diving Deeper into Psyberg: Stateless vs Stateful Data Processing

The Netflix TechBlog

NOVEMBER 14, 2023

By Abhinaya Shetty , Bharath Mummadisetty In the inaugural blog post of this series, we introduced you to the state of our pipelines before Psyberg and the challenges with incremental processing that led us to create the Psyberg framework within Netflix’s Membership and Finance data engineering team.

Processing

Processing Data Engineering Efficiency Analytics

Monitoring GitHub-hosted runners with Dynatrace

Dynatrace

JANUARY 22, 2025

That’s where Dynatrace business events and automation workflows come into play to provide a comprehensive view of your CI/CD pipelines. By integrating Dynatrace with GitHub Actions, you can proactively monitor for potential issues or slowdowns in the deployment processes.

Monitoring

Monitoring DevOps Metrics Virtualization

Accelerate resolution of network issues with AI-powered event reporting based on SNMP traps

Dynatrace

NOVEMBER 30, 2022

The volume of data and events grows in tandem with the rising complexity of IT infrastructure. While SNMP allows you to query monitored devices for performance information, SNMP traps are used to proactively report certain types of events. These can range from routine state transition events to critical problem reports.

Network

Network Infrastructure Metrics Monitoring

DevOps automation: From event-driven automation to answer-driven automation [with causal AI]

Dynatrace

JULY 24, 2023

They need event-driven automation that not only responds to events and triggers but also analyzes and interprets the context to deliver precise and proactive actions. We will also explore the evolution of DevOps automation and the significance of data-driven answers in unlocking streamlined, automated DevOps and SRE processes.

DevOps

DevOps Traffic Efficiency Servers

Introducing Impressions at Netflix

The Netflix TechBlog

FEBRUARY 14, 2025

It requires a state-of-the-art system that can track and process these impressions while maintaining a detailed history of each profiles exposure. In this multi-part blog series, we take you behind the scenes of our system that processes billions of impressions daily.

Tuning

Tuning Latency Efficiency Storage

How executives reveal breakthrough insights into customer experiences with Dynatrace to accelerate business growth

Dynatrace

OCTOBER 23, 2024

Consolidate real-user monitoring, synthetic monitoring, session replay, observability, and business process analytics tools into a unified platform. Real-time customer experience remediation identifies and informs the organization about any issues and prevents them in the experience process sooner.

Analytics

Analytics Logistics IoT Mobile

Dynatrace OpenPipeline: Stream processing data ingestion converges observability, security, and business data at massive scale for analytics and automation in context

Dynatrace

JANUARY 31, 2024

Organizations choose data-driven approaches to maximize the value of their data, achieve better business outcomes, and realize cost savings by improving their products, services, and processes. Data is then dynamically routed into pipelines for further processing. Understanding the context. Addressing security requirements.

Analytics

Analytics Processing Transportation Storage

Helping customers unlock the Power of Possible

Dynatrace

OCTOBER 29, 2024

The Dynatrace platform automatically captures and maps metrics, logs, traces, events, user experience data, and security signals into a single datastore, performing contextual analytics through a “power of three AI”—combining causal, predictive, and generative AI. What’s behind it all? With over 2.5

Innovation

Innovation Cloud Strategy AWS

RabbitMQ vs. Kafka: Key Differences

Scalegrid

FEBRUARY 6, 2025

RabbitMQ is designed for flexible routing and message reliability, while Kafka handles high-throughput event streaming and real-time data processing. Kafka is optimized for high-throughput event streaming , excelling in real-time analytics and large-scale data ingestion. What is Apache Kafka?

Latency

Latency Analytics Architecture Storage

Ingest and enrich AWS Security Hub findings with Dynatrace

Dynatrace

FEBRUARY 20, 2025

Add context to AWS Security Hub findings The Dynatrace platform, powered by OpenPipeline , provides unified security event ingest and analysis across tools and cloud environments. You can consume the ingested events via native Dynatrace Apps, such as Dashboards, Notebooks, Workflows, and more.

AWS

AWS Efficiency Infrastructure Cloud

Better dashboarding with Dynatrace Davis AI: Instant meaningful insights

Dynatrace

JANUARY 21, 2025

Ensuring smooth operations is no small feat, whether you’re in charge of application performance, IT infrastructure, or business processes. Static Threshold: This approach defines a fixed threshold suitable for well-known processes or when specific threshold values are critical.

Traffic

Traffic Metrics Analytics Monitoring

Build resilient IT systems and manage regulatory requirements with compliance and resilience capabilities from Dynatrace

Dynatrace

JANUARY 15, 2025

Smartscape topology visualizes the relationships between applications, services, processes, hosts, and data centers, highlighting problems and vulnerabilities. Workflows assembles a series of actions to build processes in graphical representations.

Systems

Systems DevOps Analytics Monitoring

Incremental Processing using Netflix Maestro and Apache Iceberg

The Netflix TechBlog

NOVEMBER 20, 2023

by Jun He , Yingyi Zhang , and Pawan Dixit Incremental processing is an approach to process new or changed data in workflows. The key advantage is that it only incrementally processes data that are newly added or updated to a dataset, instead of re-processing the complete dataset.

Processing

Processing Big Data Efficiency Engineering

Level up your strategic IT management with fully cost-transparent, fine-grained Dynatrace Cost Allocation

Dynatrace

NOVEMBER 27, 2024

Figure 4: Set up an anomaly detector for peak cost events. You can also create individual reports using Notebooks —or export your data as CSV—and share it with your financial teams for further processing.

Best Practices

Best Practices Strategy Cloud Efficiency

Ingest and enrich Harbor vulnerability findings with Dynatrace

Dynatrace

MARCH 25, 2025

The Dynatrace solution Dynatrace addresses these issues by providing unified security event ingest and analysis for security findings across tools and products. Harbor vulnerability findings and audit logs are fetched periodically and pushed to Dynatrace via a dedicated security event ingest endpoint.

Internet

Internet Internet Efficiency Monitoring

5 powerful use cases beyond debugging for Dynatrace Live Debugger

Dynatrace

MARCH 25, 2025

This powerful tool can be leveraged across various environments, including production, to enhance development processes and ensure robust application performance. Many developers attempt to mitigate this challenge with logs, but thats a tedious and error-prone process. Distributed services involve multiple processes and runtimes.

Benchmarking

Benchmarking Code Open Source Engineering

Transform data into insights with Dynatrace Dashboards and Notebooks

Dynatrace

OCTOBER 16, 2024

Kickstarting the dashboard creation process is, however, just one advantage of ready-made dashboards. This allows platform engineers to focus on high-value tasks like resolving issues and optimizing performance rather than spending time on data discovery and exploration.

Social Media

Social Media Metrics Network Analytics

How Red Hat and Dynatrace intelligently automate your production environment

Dynatrace

MAY 6, 2024

A tight integration between Red Hat Ansible Automation Platform, Dynatrace Davis ® AI, and the Dynatrace observability and security platform enables closed-loop remediation to automate the process from: Detecting a problem. Managing incidents in corresponding tools. Identifying the root cause and proper countermeasures.

DevOps

DevOps Software Engineering Games Infrastructure

Debug complex performance issues in production with ease

Dynatrace

FEBRUARY 4, 2025

Upon detecting a high CPU load, Davis AI generates a problem event and populates it with a direct link to Live Debugger. Using this data, developers can inspect local variables, server-process details, thread information, and trace data to identify the root cause of issues.

Performance

Performance Code Processing Availability

Observability throughout the software development lifecycle increases delivery performance

Dynatrace

OCTOBER 4, 2024

Today, development teams suffer from a lack of automation for time-consuming tasks, the absence of standardization due to an overabundance of tool options, and insufficiently mature DevSecOps processes. This process begins when the developer merges a code change and ends when it is running in a production environment.

Software

Software Software Development Performance

Title Launch Observability at Netflix Scale

The Netflix TechBlog

DECEMBER 17, 2024

As Netflix expanded globally and the volume of title launches skyrocketed, the operational challenges of maintaining this manual process became undeniable. Metadata and assets must be correctly configured, data must flow seamlessly, microservices must process titles without error, and algorithms must function as intended.

Traffic

Traffic Scalability Strategy Monitoring

Rapid Event Notification System at Netflix

OpenPipeline: Simplify access to critical business data

Trending Sources

Microsoft Azure Event Hubs

How To Collect and Ship Windows Events Logs With OpenTelemetry

Business process observability: An IT solution to a business challenge

Event Streams Are Nothing Without Action

Generate security events from Dynatrace Security Investigator via OpenPipeline

Netflix’s Distributed Counter Abstraction

New integrations announced at AWS re:Invent enhance cloud performance, security, and automation

Create simple workflows to automate alerts during development

Business Flow: Why IT operations teams should monitor business processes

Dynatrace enhances Business Analytics with business events powered by Grail

Leverage logs for an end-to-end view of your business processes via Dynatrace OpenPipeline

Reliability indicators that matter to your business: SLOs for all data types

Data Mesh?—?A Data Movement and Processing Platform @ Netflix

Extend business observability: Extract business events from online databases (Part 2)

Practical business process monitoring for real-time business observability

Enhanced root cause analysis using events

The value of business events: How IT can increase business agility

Improving customer experience with business process monitoring

Extract metrics from business events to increase the value of business analytics

Extend business observability: Extract business events from online databases (Part 1)

Metric events: Set up anomaly detection based on your business needs

2. Diving Deeper into Psyberg: Stateless vs Stateful Data Processing

Monitoring GitHub-hosted runners with Dynatrace

Accelerate resolution of network issues with AI-powered event reporting based on SNMP traps

DevOps automation: From event-driven automation to answer-driven automation [with causal AI]

Introducing Impressions at Netflix

How executives reveal breakthrough insights into customer experiences with Dynatrace to accelerate business growth

Dynatrace OpenPipeline: Stream processing data ingestion converges observability, security, and business data at massive scale for analytics and automation in context

Helping customers unlock the Power of Possible

RabbitMQ vs. Kafka: Key Differences

Ingest and enrich AWS Security Hub findings with Dynatrace

Better dashboarding with Dynatrace Davis AI: Instant meaningful insights

Build resilient IT systems and manage regulatory requirements with compliance and resilience capabilities from Dynatrace

Incremental Processing using Netflix Maestro and Apache Iceberg

Level up your strategic IT management with fully cost-transparent, fine-grained Dynatrace Cost Allocation

Ingest and enrich Harbor vulnerability findings with Dynatrace

5 powerful use cases beyond debugging for Dynatrace Live Debugger

Transform data into insights with Dynatrace Dashboards and Notebooks

How Red Hat and Dynatrace intelligently automate your production environment

Debug complex performance issues in production with ease

Observability throughout the software development lifecycle increases delivery performance

Title Launch Observability at Netflix Scale

Stay Connected