DZone

AUGUST 8, 2024

The decision between batch and real-time processing is a critical one, shaping the design, architecture, and success of our data pipelines. Understanding the key distinctions between these two processing paradigms is crucial for organizations to make informed decisions and harness the full potential of their data.

Processing 275

Processing Architecture Design 275

DZone

NOVEMBER 2, 2023

It’s an underlying web application architecture that makes this process possible. In this blog, you’ll discover what a web app architecture is, how a web application architecture diagram looks, and how you can design the right architecture for web apps.

Architecture 272

Architecture Mobile Website Servers 272

DZone

JANUARY 8, 2025

My experiences include tackling challenges like handling service failures in distributed architectures and mitigating the cascading effects of outages in high-demand systems. The Evolution of API Architecture Over the years, API architecture has evolved to address the gaps in its previous designs and keep up with the ever-pressing demands.

Best Practices 239

Best Practices Design Scalability Architecture 239

DZone

FEBRUARY 28, 2024

Mainframe architecture refers to the design and structure of a mainframe computer system, which is a powerful and reliable computing platform used for large-scale data processing, transaction processing, and enterprise applications.

Architecture 263

Architecture Design Processing Systems 263

DZone

FEBRUARY 20, 2023

Over the last few years, microservice architecture emerged to be on top of conventional SOA (Service Oriented Architecture). This much more precise and smaller architecture brought in many benefits. have backed their performance based on this architecture. have backed their performance based on this architecture.

Architecture 259

Architecture Testing Strategy Scalability 259

DZone

FEBRUARY 14, 2025

In the realm of modern software architecture, middleware plays a pivotal role in connecting various components of distributed systems. This is crucial because middleware often serves as the bridge between client applications and backend databases, handling a high volume of requests and data processing tasks.

Database 222

Database Performance Software Architecture Latency 222

DZone

FEBRUARY 17, 2023

For this, the only solution is to build a multi-tenant architecture SaaS application. If this is you, you might have assumed they have a repeatable process to run thousands of apps across all their customers. The real solution is a multi-tenant architecture on AWS for a SaaS application.

AWS 283

AWS Architecture Cloud Servers 283

Dynatrace

MAY 23, 2022

As a result, organizations are weighing microservices vs. monolithic architecture to improve software delivery speed and quality. Traditional monolithic architectures are built around the concept of large applications that are self-contained, independent, and incorporate myriad capabilities. What is monolithic architecture?

Architecture 263

Architecture DevOps Monitoring Programming 263

Dynatrace

JULY 7, 2022

Many organizations are taking a microservices approach to IT architecture. However, in some cases, an organization may be better suited to another architecture approach. Therefore, it’s critical to weigh the advantages of microservices against its potential issues, other architecture approaches, and your unique business needs.

Architecture 246

Architecture Serverless Azure Programming 246

Dynatrace

DECEMBER 5, 2023

Without observability, the benefits of ARM are lost Over the last decade and a half, a new wave of computer architecture has overtaken the world. ARM architecture, based on a processor type optimized for cloud and hyperscale computing, has become the most prevalent on the planet, with billions of ARM devices currently in use.

Efficiency 290

Efficiency Architecture Energy Monitoring 290

Dynatrace

NOVEMBER 29, 2024

Protect data in multi-tenant architectures To bring you the most value by unifying observability and security in one analytics and automation platform powered by AI, Dynatrace SaaS leverages a multitenancy architecture, enabling efficient and scalable data ingestion, querying, and processing on shared infrastructure.

Storage 246

Storage AWS Azure Architecture 246

Dynatrace

DECEMBER 3, 2024

This integration simplifies the process of embedding Dynatrace full-stack observability directly into custom Amazon Machine Images (AMIs). VMware migration support for seamless transitions For enterprises transitioning VMware-based workloads to the cloud, the process can be complex and resource-intensive.

AWS 298

AWS Cloud Performance Innovation 298

DZone

AUGUST 13, 2024

Our company uses artificial intelligence (AI) and machine learning to streamline the comparison and purchasing process for car insurance and car loans. We innovatively use its snapshot feature to implement a primary-replica architecture for ClickHouse. As our data grew, we had problems with AWS Redshift which was slow and expensive.

Artificial Intelligence 243

Artificial Intelligence Architecture AWS Storage 243

DZone

NOVEMBER 16, 2023

This article is intended for data scientists, AI researchers, machine learning engineers, and advanced practitioners in the field of artificial intelligence who have a solid grounding in machine learning concepts, natural language processing , and deep learning architectures.

Artificial Intelligence 264

Artificial Intelligence Architecture Network Engineering 264

DZone

NOVEMBER 28, 2022

Early this year, the book Software Architecture Metrics: Case Studies to Improve the Quality of Your Architecture was published. He wrote a chapter that is particularly useful in contexts where the architecture and environment still have many opportunities for improvement. Intro and Problem Statement.

Processing 223

Processing Software Architecture Architecture Metrics 223

The Netflix TechBlog

JANUARY 10, 2024

Future blogs will provide deeper dives into each service, sharing insights and lessons learned from this process. The Netflix video processing pipeline went live with the launch of our streaming service in 2007. The Netflix video processing pipeline went live with the launch of our streaming service in 2007.

Processing 233

Processing Media Latency Innovation 233

The Netflix TechBlog

AUGUST 1, 2022

A Data Movement and Processing Platform @ Netflix By Bo Lei , Guilherme Pires , James Shao , Kasturi Chatterjee , Sujay Jain , Vlad Sydorenko Background Realtime processing technologies (A.K.A stream processing) is one of the key factors that enable Netflix to maintain its leading position in the competition of entertaining our users.

Processing 272

Processing Transportation Entertainment Tuning 272

DZone

AUGUST 28, 2023

One of those beautiful complications was the introduction of Agile methodologies, which have become a standard in software development by shifting how we develop software into a more responsive and collaborative process. Fortunately, the concept of Architectural Observability steps in to help.

Architecture 246

Architecture Retail Software Architecture Infrastructure 246

DZone

FEBRUARY 26, 2025

With the evolution of modern applications serving increasing needs for real-time data processing and retrieval, scalability does, too. However, the process for effectively scaling Elasticsearch can be nuanced, since one needs a proper understanding of the architecture behind it and of performance tradeoffs.

Scalability 147

Scalability Open Source Latency Architecture 147

Dynatrace

FEBRUARY 4, 2025

As every developer knows, logs are crucial for uncovering insights and detecting fundamental flaws, such as process crashes or exceptions. Additionally, Dynatrace integrates relevant trace data, providing full visibility into complex, microservices-based architectures.

Development 235

Development Analytics Code Architecture 235

Dynatrace

JANUARY 31, 2022

Key takeaways from this article on modern observability for serverless architecture: As digital transformation accelerates, organizations need to innovate faster and continually deliver value to customers. Companies often turn to serverless architecture to accelerate modernization efforts while simplifying IT management.

Serverless 245

Serverless Architecture Open Source Innovation 245

DZone

NOVEMBER 7, 2023

In the labyrinth of data-driven architectures, the challenge of data integration—fusing data from disparate sources into a coherent, usable form — stands as one of the cornerstones. However, it would be a mistake to dismiss batch processing as an antiquated approach. Systems were not equipped to handle multiple tasks simultaneously.

Processing 189

Processing Architecture Technology Technology 189

Scalegrid

FEBRUARY 6, 2025

RabbitMQ is designed for flexible routing and message reliability, while Kafka handles high-throughput event streaming and real-time data processing. This article outlines the key differences in architecture, performance, and use cases to help determine the best fit for your workload. What is RabbitMQ? What is Apache Kafka?

Latency 147

Latency Analytics Architecture Storage 147

The Netflix TechBlog

FEBRUARY 14, 2025

It requires a state-of-the-art system that can track and process these impressions while maintaining a detailed history of each profiles exposure. In this multi-part blog series, we take you behind the scenes of our system that processes billions of impressions daily.

Tuning 165

Tuning Latency Efficiency Storage 165

The Netflix TechBlog

MARCH 4, 2025

Part 3: System Strategies and Architecture By: VarunKhaitan With special thanks to my stunning colleagues: Mallika Rao , Esmir Mesic , HugoMarques This blog post is a continuation of Part 2 , where we cleared the ambiguity around title launch observability at Netflix. The request schema for the observability endpoint.

Traffic 160

Traffic Strategy Entertainment Innovation 160

DZone

NOVEMBER 30, 2023

It uses Python as its programming language and offers a flexible architecture suited for both small-scale and large-scale data processing. One of the key features of Apache Airflow is its ability to schedule and trigger batch jobs, making it a popular choice for processing large volumes of data.

Processing 173

Processing Open Source Programming Architecture 173

The Netflix TechBlog

NOVEMBER 20, 2023

by Jun He , Yingyi Zhang , and Pawan Dixit Incremental processing is an approach to process new or changed data in workflows. The key advantage is that it only incrementally processes data that are newly added or updated to a dataset, instead of re-processing the complete dataset.

Processing 214

Processing Big Data Efficiency Engineering 214

Dynatrace

FEBRUARY 25, 2025

This blog post dissects the vulnerability, explains how Struts processes file uploads, details the exploit mechanics, and outlines mitigation strategies. Introduction Apache Struts 2 is a widely used Java framework for web applications, valued for its flexibility and Model-View-Controller (MVC) architecture.

Servers 214

Servers Processing Java Tuning 214

Scalegrid

MAY 13, 2020

Greenplum Database is a massively parallel processing (MPP) SQL database that is built and based on PostgreSQL. In this blog post, we explain what Greenplum is, and break down the Greenplum architecture, advantages, major use cases, and how to get started. The Greenplum Architecture. The Greenplum Architecture.

Big Data 321

Big Data Database Artificial Intelligence Open Source 321

Dynatrace

JANUARY 17, 2025

The Texas Risk and Authorization Management Program (TX-RAMP) provides a standardized approach for security assessment, certification, and continuous monitoring of cloud computing services that process the data of Texas state agencies. Complex IT environments that house these services are often built on hybrid and multicloud architectures.

Government 246

Government Education Efficiency Cloud 246

Scalegrid

FEBRUARY 24, 2025

This decoupling is crucial in modern architectures where scalability and fault tolerance are paramount. The architecture of RabbitMQ is meticulously designed for complex message routing, enabling dynamic and flexible interactions between producers and consumers. Erlang is the backbone of RabbitMQ clustering.

Best Practices 147

Best Practices Traffic Strategy Efficiency 147

DZone

FEBRUARY 25, 2025

The amount of data we capture in any field is increasing exponentially, which requires a technology that can process large amounts of data in a short duration. One such technology would be Apache Spark.

Java 130

Java Architecture Technology Technology 130

Dynatrace

OCTOBER 7, 2021

To get a better understanding of AWS serverless, we’ll first explore the basics of serverless architectures, review AWS serverless offerings, and explore common use cases. Serverless architecture: A primer. Serverless architecture shifts application hosting functions away from local servers onto those managed by providers.

Serverless 325

Serverless AWS Lambda Storage 325

DZone

SEPTEMBER 16, 2024

By efficiently capturing, processing, and transmitting logs, metrics, and traces, observability agents provide a comprehensive view of system health and behavior. This data is then sent to centralized observability platforms where it can be analyzed to gain valuable insights into system performance, identify issues, and optimize operations.

Architecture 162

Architecture Network Infrastructure Metrics 162

DZone

OCTOBER 8, 2024

To fully leverage its capabilities and improve efficient data processing, it's crucial to optimize query performance. Understanding Snowflake Architecture Let’s briefly cover Snowflake architecture before we deal with data modeling and optimization techniques. Snowflake’s architecture consists of three main layers:

Best Practices 282

Best Practices Performance Architecture Scalability 282

DZone

MAY 15, 2023

In today's fast-paced software development landscape, microservices have emerged as a popular architectural pattern. This architectural style enables teams to develop and deploy services independently, offering flexibility and scalability to the software development process. But what exactly are microservices?

Best Practices 312

Best Practices Architecture Scalability Software 312

The Netflix TechBlog

FEBRUARY 18, 2022

Motivation With the rapid growth in Netflix member base and the increasing complexity of our systems, our architecture has evolved into an asynchronous one that enables both online and offline computation. To avoid processing old events, a staleness filter is applied as a gating check.

Systems 334

Systems Traffic Architecture Mobile 334