DZone

FEBRUARY 27, 2025

Multimodal data processing is the evolving need of the latest data platforms powering applications like recommendation systems, autonomous vehicles, and medical diagnostics. Handling multimodal data spanning text, images, videos, and sensor inputs requires resilient architecture to manage the diversity of formats and scale.

Efficiency 130

Efficiency Processing Latency Storage 130

High Scalability

JUNE 15, 2020

Over the last 3 years, the TripleLift data pipeline scaled from processing millions of events per day to processing billions. This processing can be summed up as the continuous aggregation and delivery of reporting data to users in a cost efficient manner.

Processing 206

Processing Data Engineering Engineering Efficiency 206

DZone

MAY 21, 2024

Introductory note : This article has been co-authored by Federico Trotta and Karin Wolok. Introduction In a previous article , we presented the fundamentals of stream processing.

Processing 130

Processing Systems 130

DZone

MARCH 21, 2024

This article is made especially for beginners to make the ideas and steps of adding NCache to your Java applications clear and easy to understand. It doesn't matter if you've been developing for years or if you're new to caching , this article will help you get a good start with NCache Java Edition.

Java 319

Java Cache Technology Technology 319

DZone

SEPTEMBER 9, 2024

Efficient data processing is crucial for businesses and organizations that rely on big data analytics to make informed decisions. One key factor that significantly affects the performance of data processing is the storage format of the data.

Big Data 278

Big Data Storage Analytics Benchmarking 278

DZone

SEPTEMBER 21, 2020

While it's well-received in the community with its rich fault injection types and easy-to-use dashboard, it was difficult to use Chaos Mesh with end-to-end testing or the continuous integration (CI) process. In this article, I will share how we use chaos-mesh-action, a GitHub action to integrate Chaos Mesh into the CI process.

Engineering 338

Engineering Cloud Testing Processing 338

DZone

SEPTEMBER 21, 2020

Comparing two strings to see if they’re equal takes processing power, but not all comparisons are the same. In a previous article, we looked at How to compare strings in Go and did some benchmarking. Optimizing string comparisons in Go can improve your application’s response time and help scalability.

Benchmarking 354

Benchmarking Programming Scalability Processing 354

DZone

FEBRUARY 17, 2023

In this article, we will be using observIQ’s distribution of the collector. Windows Events logs record many different operating system processes, application activity, and account activity. A recent contribution of a distribution of the OpenTelemetry (OTel) Collector makes it much easier to monitor Windows Events with OpenTel.

Operating System 358

Operating System Google Monitoring Cloud 358

DZone

AUGUST 14, 2024

When building ETL data pipelines using Azure Data Factory (ADF) to process huge amounts of data from different sources, you may often run into performance and design-related challenges. This article will serve as a guide in building high-performance ETL pipelines that are both efficient and scalable.

Azure 306

Azure Cloud Scalability Efficiency 306

DZone

MARCH 29, 2023

Data migration is the process of moving data from one location to another, which is an essential aspect of cloud migration. This article discusses the challenges and best practices of data migration when transferring on-premise data to the cloud. Data migration involves transferring data from on-premise storage to the cloud.

Best Practices 315

Best Practices Cloud Data Engineering Storage 315

DZone

JANUARY 26, 2023

Real-time data processing with Apache Kafka became the de facto standard to correlate and prevent fraud continuously before it happens. Fraud detection becomes increasingly challenging in a digital world across all industries.

Games 324

Games Technology Technology Processing 324

DZone

OCTOBER 3, 2023

This is an article from DZone's 2023 Data Pipelines Trend Report. To keep up, we require real-time analytics (RTA), which provides the immediacy that every user of data today expects and is based on stream processing. Stream processing is used to query a continuous stream of data and immediately process events in that stream.

Analytics 298

Analytics Database Cloud Processing 298

DZone

OCTOBER 11, 2024

This three-part article series will take you through the process of developing a network anomaly detection system using the Spring Boot framework in a robust manner. The series is organized as follows: Part 1: We’ll concentrate on the foundation and basic structure of our detection system, which has to be created.

Network 289

Network Systems Monitoring Technology 289

DZone

FEBRUARY 17, 2023

Event Stream Processing refers to the action taken on generated Events. Event Streams are sometimes referred to as Data Streams within the developer community since they consist of continuous data points.

Processing 285

Processing Systems Development 285

DZone

AUGUST 4, 2024

This level of protection is essential as it lays the foundation for the following testing processes: integration tests, acceptance testing, and finally manual testing, including exploratory testing. Unit testing is the first line of defense against bugs.

Testing 288

Testing Software Software Code 288

DZone

NOVEMBER 7, 2023

With businesses constantly in the race to stay ahead, the process of integrating this data becomes crucial. However, it's no longer enough to assimilate data in isolated, batch-oriented processes. Businesses were content with accumulating data over defined intervals and then processing it in scheduled batches.

Systems 292

Systems Analytics Architecture Engineering 292

DZone

JUNE 21, 2023

Monitoring and observability are two key concepts that facilitate this process, offering valuable visibility into the health and performance of systems. In this article, we will explore the differences between monitoring and observability, provide examples to illustrate their applications and highlight their respective benefits.

DevOps 324

DevOps Systems Monitoring Metrics 324

DZone

NOVEMBER 16, 2023

This article is intended for data scientists, AI researchers, machine learning engineers, and advanced practitioners in the field of artificial intelligence who have a solid grounding in machine learning concepts, natural language processing , and deep learning architectures.

Artificial Intelligence 264

Artificial Intelligence Architecture Network Engineering 264

DZone

NOVEMBER 29, 2023

As I mentioned in my previous article Part 1 , let's explore how such a transition can be effectively managed. This system is tasked with performing reimbursement calculations, typically running overnight through a batch process scheduled in SQL Server. This is particularly relevant in the domain of reimbursement calculation systems.

Systems 299

Systems C++ Scalability Architecture 299

DZone

NOVEMBER 16, 2023

This is an article from DZone's 2023 Observability and Application Performance Trend Report. It allows teams to better identify areas of improvement, enabling them to make informed decisions about their development processes. Telemetry, being a critical part of observability, refers to the continuous nature of data collection.

Software 290

Software Software Systems Processing 290

DZone

MARCH 17, 2021

You might have seen many articles on the internet regarding Nginx and how we can use Nginx for load balancing and reverse proxy. In this article, I would like to share my experience of setting up Nginx for load balancing and reverse proxy with SSL termination in Nginx. Are Reverse Proxy and Load Balancer Similar?

Internet 261

Internet Internet Servers Processing 261

DZone

NOVEMBER 28, 2021

In the process of testing a software application, test plans and test strategies are quite crucial. We will learn about Test Plans and Test Strategies in this article. We will learn about Test Plans and Test Strategies in this article. A strong test plan and strategy will always prevent errors in the application.

Strategy 278

Strategy Testing Software Software 278

DZone

JANUARY 24, 2025

While an earlier version of Hibernate had support for multi-tenancy, its implementation required significant manual configuration and custom strategies to handle tenant isolation, which resulted in higher complexity and slower processes, especially for applications with a number of tenants. This article talks about how Hibernate 6.3.0

Strategy 252

Strategy Processing Performance 252

DZone

NOVEMBER 25, 2023

In a detailed article, we've discussed how to align a NoSQL database with specific business needs. But if your application primarily revolves around batch processing of large datasets, then focusing on write speed could mislead your selection process.

Database 278

Database Latency Speed Metrics 278

DZone

MAY 10, 2023

This article aims to demystify the subject by providing a basic understanding of how class loading works in the JVM. What Are Classloaders In the Java Virtual Machine (JVM), classes are loaded dynamically and found through a process called class loading.

Java 263

Java Virtualization Processing Development 263

DZone

JUNE 15, 2023

It is worth noting that this data collection process does not impact the performance of the application. Moreover, the process of collecting these profiles introduces overhead during application runtime and necessitates the storage and visualization of significantly large datasets.

Latency 253

Latency Storage Strategy Metrics 253

DZone

OCTOBER 2, 2024

In this article, we’ll dive deep into the concept of database sharding, a critical technique for scaling databases to handle large volumes of data and high levels of traffic. This section will provide insights into the architecture and strategies to ensure efficient query processing in a sharded environment.

Database 279

Database Systems Scalability Traffic 279

DZone

SEPTEMBER 18, 2023

This is an article from DZone's 2023 Automated Testing Trend Report. The demand for faster, more reliable, and efficient testing processes has grown exponentially with the increasing complexity of modern applications.

Artificial Intelligence 279

Artificial Intelligence Software Software Testing 279

DZone

JUNE 13, 2024

Relational Databases are the bedrock of any FinTech application, especially for OLTP (Online transaction Processing). This article will explore the FinTech case study, which has built a Lending Platform. This foundational component in any application architecture usually poses challenges around scaling as the business expands rapidly.

FinTech 262

FinTech Tuning AWS Database 262

Scalegrid

MAY 13, 2020

Greenplum Database is a massively parallel processing (MPP) SQL database that is built and based on PostgreSQL. When handling large amounts of complex data, or big data, chances are that your main machine might start getting crushed by all of the data it has to process in order to produce your analytics results. Query Optimization.

Big Data 321

Big Data Database Artificial Intelligence Open Source 321

Software Testing Help

DECEMBER 25, 2024

In this article, we give a detailed account of the software product testing process with the help of examples Software Products need their own unique approach to test adequately and. Read more The post How to Perform Software Product Testing – Detailed Process and Methods with Examples appeared first on Software Testing Help.

Processing 98

Processing Software Software Testing 98

DZone

OCTOBER 23, 2023

In most financial firms, online transaction processing (OLTP) often relies on static or infrequently updated data, also called reference data. In this article, I am not going to determine which AWS NoSQL database is better: the concept of a better database only exists within a specific purposeful context.

AWS 275

AWS Performance Database Architecture 275

DZone

APRIL 11, 2023

Data processing in the cloud has become increasingly popular due to its scalability, flexibility, and cost-effectiveness. This article will explore how these technologies can be used together to create an optimized data pipeline for data processing in the cloud.

Azure 246

Azure Analytics Storage Cloud 246

DZone

FEBRUARY 19, 2023

The theme of their discussion was redefining the boundaries of people, processes, and platforms. There were so many great insights in this discussion; I will cover it in much greater detail in a separate article. My key takeaways from their discussion:

Performance 259

Performance DevOps Google Cloud 259

DZone

MARCH 24, 2023

This article compares QuestDB with one of the most popular databases on the market, MongoDB. It is used in a wide variety of industries and use cases (if you are new to this space, read our article 'What is Time-Series Data' ). Time series data is often processed with dedicated tooling.

Benchmarking 243

Benchmarking Database Processing Performance 243

DZone

NOVEMBER 22, 2022

This is an article from DZone's 2022 Performance and Site Reliability Trend Report. Every company has a relatively similar process to manage incidents, mitigate risks, and analyze root causes. Every company has a relatively similar process to manage incidents, mitigate risks, and analyze root causes. For more: Read the Report.

Servers 240

Servers Engineering Processing Systems 240

DZone

MARCH 11, 2024

This article delves deep into the essence of Istio, illustrating its pivotal role in a Kubernetes (KIND) based environment, and guides you through a Helm-based installation process, ensuring a comprehensive understanding of Istio's capabilities and its impact on microservices architecture.

Open Source 264

Open Source Traffic Architecture Monitoring 264