DZone

MARCH 31, 2023

With this data comes the challenge of processing it in a timely and efficient way. Companies worldwide are investing in technologies that can help them better process, analyze, and use the data they are collecting to better serve their customers and stay ahead of their competitors. Let’s recap some of the basics first.

Processing 242

Processing Social Media IoT Media 242

Dynatrace

FEBRUARY 9, 2024

One of the more popular use cases is monitoring business processes, the structured steps that produce a product or service designed to fulfill organizational objectives. By treating processes as assets with measurable key performance indicators (KPIs), business process monitoring helps IT and business teams align toward shared business goals.

Processing 203

Processing Monitoring Analytics Games 203

DZone

MARCH 29, 2023

The theme of their discussion was redefining the boundaries of people, processes and platforms. Day two of Dynatrace Perform began with a great discussion between Kelsey Hightower , Distinguished Developer Advocate at Google Cloud Platform and Andi Grabner , DevOps Evangelist at Dynatrace.

Processing 321

Processing DevOps Google Cloud 321

Dynatrace

SEPTEMBER 27, 2024

Unrealized optimization potential of business processes due to monitoring gaps Imagine a retail company facing gaps in its business process monitoring due to disparate data sources. Due to separated systems that handle different parts of the process, the view of the process is fragmented.

Processing 211

Processing Retail Analytics Monitoring 211

The Netflix TechBlog

NOVEMBER 20, 2023

by Jun He , Yingyi Zhang , and Pawan Dixit Incremental processing is an approach to process new or changed data in workflows. The key advantage is that it only incrementally processes data that are newly added or updated to a dataset, instead of re-processing the complete dataset.

Processing 215

Processing Big Data Efficiency Engineering 215

DZone

JUNE 5, 2024

Batch processing is a capability of App Connect that facilitates the extraction and processing of large amounts of data. Sometimes referred to as data copy , batch processing allows you to author and run flows that retrieve batches of records from a source, manipulate the records, and then load them into a target system.

Best Practices 239

Best Practices Processing Design Systems 239

DZone

SEPTEMBER 14, 2023

In today's data-driven world, efficient data processing plays a pivotal role in the success of any project. Apache Spark , a robust open-source data processing framework, has emerged as a game-changer in this domain. Optimizing Data Input Make Use of Data Forma t In most cases, the data being processed is stored in a columnar format.

Big Data 279

Big Data Processing Games Open Source 279

DZone

MAY 8, 2023

Batch request processing is a powerful technique used in web development to improve the performance of APIs. In this article, we'll explore how to implement batch request processing in Apache APISIX and look at some use cases where it can be beneficial.

Processing 202

Processing Servers Development Performance 202

The Netflix TechBlog

NOVEMBER 14, 2023

By Abhinaya Shetty , Bharath Mummadisetty In the inaugural blog post of this series, we introduced you to the state of our pipelines before Psyberg and the challenges with incremental processing that led us to create the Psyberg framework within Netflix’s Membership and Finance data engineering team.

Processing 231

Processing Data Engineering Efficiency Analytics 231

DZone

NOVEMBER 30, 2023

It uses Python as its programming language and offers a flexible architecture suited for both small-scale and large-scale data processing. One of the key features of Apache Airflow is its ability to schedule and trigger batch jobs, making it a popular choice for processing large volumes of data.

Processing 173

Processing Open Source Programming Architecture 173

DZone

APRIL 4, 2023

Apache Kafka is a battle-tested distributed stream-processing platform popular in the financial industry to handle mission-critical transactional workloads. Processed market data and orders may then be saved to a time series database for further analysis.

Processing 271

Processing Database Infrastructure Testing 271

DZone

SEPTEMBER 24, 2021

Process Automation is defined as “a centerpiece of digitalization efforts” – where workflow engines are used as “a vital building block in modern architectures.” We also dive deep into Robotic Process Automation, how organizations are finding success with it, the risks involved, and why he considers it as a "short-term painkiller".

Processing 246

Processing Architecture Cloud Engineering 246

DZone

NOVEMBER 28, 2023

Using open-source technology is a good idea when dealing with Automated Clearing House (ACH) batch processing, the primary system that agencies use for electronic funds transfer (EFT). One way to do this is by changing from proprietary tools-driven software development to open-source technology and automation, which eliminates licensing fees.

Open Source 237

Open Source Technology Technology Processing 237

The Netflix TechBlog

AUGUST 1, 2022

A Data Movement and Processing Platform @ Netflix By Bo Lei , Guilherme Pires , James Shao , Kasturi Chatterjee , Sujay Jain , Vlad Sydorenko Background Realtime processing technologies (A.K.A stream processing) is one of the key factors that enable Netflix to maintain its leading position in the competition of entertaining our users.

Processing 273

Processing Transportation Entertainment Tuning 273

DZone

NOVEMBER 20, 2023

As we continue to transition into an era where data is not just an asset but the currency of the digital age, the pressures on ETL processes have increased exponentially. We're not merely dealing with more data; we're also confronting a dramatic uptick in complexity and velocity. What makes ETL performance such an imperative?

Processing 124

Processing Performance Analytics Speed 124

DZone

MAY 24, 2022

The process involved in software development is long and endless. Whether your company is small or big, having fast software development will always keep you ahead of the competition. Concerning fast development, you should never compromise the quality of the software. It will possess a high threat to the company's growth.

Processing 264

Processing Software Software Strategy 264

DZone

MAY 21, 2024

Introduction In a previous article , we presented the fundamentals of stream processing. Introductory note : This article has been co-authored by Federico Trotta and Karin Wolok.

Processing 130

Processing Systems 130

DZone

NOVEMBER 28, 2022

He emphasizes the importance of private builds in the development process, with the underlying principle of his reasoning being easily understandable — you should make use of private builds and run their tests on your local machine, in an environment as close as possible to production, to obtain feedback on your build as soon as possible.

Processing 216

Processing Software Architecture Architecture Metrics 216

DZone

JULY 13, 2023

In today's data-driven world, organizations need efficient and scalable data pipelines to process and analyze large volumes of data. Medallion Architecture provides a framework for organizing data processing workflows into different zones, enabling optimized batch and stream processing.

Azure 246

Azure Architecture Efficiency Processing 246

DZone

FEBRUARY 8, 2022

While increasing both the precision and the recall of our secrets detection engine, we felt the need to keep a close eye on speed. In a gearbox, if you want to increase torque, you need to decrease speed. So it wasn’t a surprise to find that our engine had the same problem: more power, less speed.

Processing 243

Processing Benchmarking Speed Engineering 243

Dynatrace

AUGUST 16, 2022

Advanced processing on your observability platform unlocks the full value of log data. Dynatrace now includes powerful log-processing capabilities for all types of log data. Log data is processed on ingest, whether coming from a fleet of OneAgents deployed across your enterprise hosts or generic API ingest from cloud services.

Processing 201

Processing Metrics Monitoring Java 201

DZone

AUGUST 10, 2023

Many modern applications have a batch processing aspect to them and regularly run high-volume, repetitive data jobs. Here are the four tactics for cost-efficient and resilient batch processing: Here are the four tactics for cost-efficient and resilient batch processing:

Efficiency 130

Efficiency Processing Cloud Technology 130

Dynatrace

JANUARY 31, 2024

Organizations choose data-driven approaches to maximize the value of their data, achieve better business outcomes, and realize cost savings by improving their products, services, and processes. Data is then dynamically routed into pipelines for further processing. Commitment to privacy.

Analytics 203

Analytics Processing Transportation Storage 203

KeyCDN

SEPTEMBER 3, 2020

KeyCDN has significantly simplified the way images are transformed and delivered with our Image Processing service. Our Image Processing service makes it easy to do that. Our Image Processing service will automatically optimize the image quality and reduce the size of the image if no query string is provided.

Processing 127

Processing Cache Website Servers 127

KeyCDN

APRIL 7, 2021

Signing Image Processing requests is an efficient way to ensure that no unwanted image operation can be executed. Secure Token and Image Processing is a powerful setup to transform and deliver images in a secure way. Signing Image Processing requests is easy. Every request needs a valid token in order to be delivered.

Processing 107

Processing Efficiency 107

DZone

JANUARY 12, 2021

There are different options to terminate a process in Unix/Linux flavor of operating systems. You can use the kill command to terminate a process by passing the process id. PID is the process ID of the process that you want to terminate. This article intends to list and provide examples of each option.

Processing 140

Processing Operating System Systems Open Source 140

DZone

OCTOBER 23, 2021

The worst evil in Nginx is if when used in location context. Much has been written about this, including posts on nginx.com. Let’s take a quote: The only 100% safe things which may be done inside if in a location context are:

Processing 246

Processing DevOps Performance 246

DZone

SEPTEMBER 6, 2019

Application performance Review (also known as Application Performance Walkthrough or Application Performance Assessment) is the process of review of an existing application (in production) to evaluate its performance and scalability attributes.

Processing 140

Processing Performance Scalability Best Practices 140

Dynatrace

APRIL 13, 2021

In order to streamline this process even more and reduce the time needed to perform this, we decided to release a solution similar to what we already provide for OneAgents: a mechanism which assures completely hands-free updates of ActiveGates, at the earliest possible convenience.

Processing 258

Processing Monitoring Performance Availability 258

Uber Engineering

DECEMBER 16, 2021

It is currently used by hundreds of … The post Cadence Multi-Tenant Task Processing appeared first on Uber Engineering Blog. Cadence is a multi-tenant orchestration framework that helps developers at Uber to write fault-tolerant, long-running applications, also known as workflows.

Processing 104

Processing Engineering Development Open Source 104

Percona

APRIL 3, 2024

One of the most common tasks for database administrators is checking logs; you can either work directly with the file or process it using another tool. Either way, regularly checking the logs remains essential.

Processing 80

Processing Database Servers Performance 80

High Scalability

JUNE 15, 2020

Over the last 3 years, the TripleLift data pipeline scaled from processing millions of events per day to processing billions. This processing can be summed up as the continuous aggregation and delivery of reporting data to users in a cost efficient manner.

Processing 206

Processing Data Engineering Engineering Efficiency 206

DZone

JULY 8, 2020

Now you need to organize a testing process the right way. Testing is a precise process and each stage of it is extremely important as it can affect the results. This is only a part of the solution. As a client, you may think that the testing provider defines everything about your testing project.

Processing 156

Processing Testing Performance Testing Development 156

DZone

MAY 31, 2019

The new release brings with it the questions around how to get started with the various Red Hat integration, data, and process automation product toolsets that are not installed out of the box.

Processing 147

Processing Cloud Development 147

DZone

JUNE 20, 2023

WeBank is China’s first privately-owned online bank backed by Tencent. WeBank offers accessible and high-quality financial services to underbanked individuals as well as small- and medium-sized enterprises. So far, WeBank has served over 250 million individual customers, 20 million individual business customers, and 1.5 million corporate customers.

Processing 250

Processing Open Source Database 250

DZone

AUGUST 5, 2019

In the previous posts in this series, we've seen some examples of long-running processes , how to model them, and where to store the state. So how can we ensure that our long-running process doesn't get into an inconsistent state if something fails along the way? But building distributed systems is hard.

Processing 100

Processing Strategy Systems Performance 100

InfoQ

NOVEMBER 29, 2023

HubSpot adopted routing messages over multiple Kafka topics (called swimlanes) for the same producer to avoid the build-up in the consumer group lag and prioritize the processing of real-time traffic.

Processing 84

Processing Traffic Data Engineering Scalability 84