The Netflix TechBlog

OCTOBER 27, 2020

By Tianlong Chen and Ioannis Papapanagiotou Netflix has more than 195 million subscribers that generate petabytes of data everyday. Data scientists and engineers collect this data from our subscribers and videos, and implement data analytics models to discover customer behaviour with the goal of maximizing user joy.

Latency 254

Latency Storage Big Data Tuning 254

The Netflix TechBlog

JULY 15, 2021

Data Engineers of Netflix?—?Interview Interview with Kevin Wylie This post is part of our “Data Engineers of Netflix” series, where our very own data engineers talk about their journeys to Data Engineering @ Netflix. Kevin Wylie is a Data Engineer on the Content Data Science and Engineering team.

Data Engineering 200

Data Engineering Engineering Entertainment Big Data 200

Dynatrace

MAY 1, 2023

IT operations analytics is the process of unifying, storing, and contextually analyzing operational data to understand the health of applications, infrastructure, and environments and streamline everyday operations. ITOA collects operational data to identify patterns and anomalies for faster incident management and near-real-time insights.

Analytics 201

Analytics Artificial Intelligence Big Data Open Source 201

DZone

APRIL 15, 2022

Recently, I faced an issue related to a very high load on the database layer. The database was having too many connections in parallel. I had to review my application’s database connection pool (DBCP) properties very closely.

Database 245

Database Code Big Data Performance 245

The Netflix TechBlog

DECEMBER 21, 2020

By Anupom Syam Background At Netflix, our current data warehouse contains hundreds of Petabytes of data stored in AWS S3 , and each day we ingest and create additional Petabytes. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage 214

Storage Latency Efficiency Data Engineering 214

All Things Distributed

AUGUST 18, 2011

Driving down the cost of Big-Data analytics. The Amazon Elastic MapReduce (EMR) team announced today the ability to seamlessly use Amazon EC2 Spot Instances with their service, significantly driving down the cost of data analytics in the cloud. However, this cannot be done without efficient, scalable data analytics.

Big Data 94

Big Data Analytics AWS Cloud 94

Dynatrace

JANUARY 16, 2023

The study analyzes factual Kubernetes production data from thousands of organizations worldwide that are using the Dynatrace Software Intelligence Platform to keep their Kubernetes clusters secure, healthy, and high performing. The strongest Kubernetes growth areas are security, databases, and CI/CD technologies. Java, Go, and Node.js

Open Source 351

Open Source Java Operating System Programming 351

Highly Scalable

MAY 1, 2012

Statistical analysis and mining of huge multi-terabyte data sets is a common task nowadays, especially in the areas like web analytics and Internet advertising. Analysis of such large data sets often requires powerful distributed data stores like Hadoop and heavy data processing with techniques like MapReduce.

Analytics 191

Analytics Traffic Big Data Efficiency 191

Dynatrace

APRIL 20, 2023

Modern organizations ingest petabytes of data daily, but legacy approaches to log analysis and management cannot accommodate this volume of data. based financial services group, discussed how the bank uses log monitoring on the Dynatrace platform with an emphasis on observability and security data.

Analytics 200

Analytics Infrastructure Storage Architecture 200

Dynatrace

FEBRUARY 16, 2023

How do you get more value from petabytes of exponentially exploding, increasingly heterogeneous data? The short answer: The three pillars of observability—logs, metrics, and traces—converging on a data lakehouse. To solve this problem, Dynatrace launched Grail, its causational data lakehouse , in 2022.

Analytics 198

Analytics Innovation Metrics Database 198

The Netflix TechBlog

OCTOBER 18, 2022

by Jun He , Akash Dwivedi , Natallia Dzenisenka , Snehal Chennuru , Praneeth Yenugutala , Pawan Dixit At Netflix, Data and Machine Learning (ML) pipelines are widely used and have become central for the business, representing diverse use cases that go beyond recommendations, predictions and data transformations.

Java 214

Java Scalability Traffic Architecture 214

DZone

JANUARY 20, 2023

Having a distributed and scalable graph database system is highly sought after in many enterprise scenarios. Do Not Be Misled Designing and implementing a scalable graph database system has never been a trivial task.

Scalability 210

Scalability Big Data Hardware Internet 210

Dynatrace

JANUARY 11, 2023

As cloud and big data complexity scales beyond the ability of traditional monitoring tools to handle, next-generation cloud monitoring and observability are becoming necessities for IT teams. With agent monitoring, third-party software collects data and reports from the component that’s attached to the agent.

Cloud 233

Cloud Monitoring Best Practices Infrastructure 233

Dynatrace

JULY 6, 2020

In addition to providing visibility for core Azure services like virtual machines, load balancers, databases, and application services, we’re happy to announce support for the following 10 new Azure services, with many more to come soon: Virtual Machines (classic ones). Effortlessly optimize Azure database performance.

Azure 153

Azure Cloud Big Data Virtualization 153

Highly Scalable

MARCH 10, 2015

Retail is one of the most important business domains for data science and data mining applications because of its prolific data and numerous optimization problems such as optimal prices, discounts, recommendations, and stock levels that can be solved using data analysis methods.

Retail 152

Retail C++ Analytics Metrics 152

Dynatrace

SEPTEMBER 15, 2020

Stefano started his presentation by showing how much cost and performance optimization is possible when knowing how to properly configure your application runtimes, databases, or cloud environments: Correct configuration of JVM parameters can save up to 75% resource utilization while delivering same or better performance!

Performance 195

Performance Java Metrics Cloud 195

Scalegrid

FEBRUARY 8, 2024

A distributed storage system is foundational in today’s data-driven landscape, ensuring data spread over multiple servers is reliable, accessible, and manageable. Understanding distributed storage is imperative as data volumes and the need for robust storage solutions rise.

Storage 130

Storage Systems Big Data Azure 130

Dynatrace

MARCH 11, 2021

In the fourth part of the series, I’ll show you how I used Dynatrace’s raw problem and event data to find the best fit for optimized anomaly detection settings. I took a big-data-analysis approach, which started with another problem visualization. Statistically analyzing Dynatrace’s event and problem data.

Tuning 264

Tuning Architecture Monitoring Big Data 264

All Things Distributed

SEPTEMBER 5, 2013

Over the past few years, two important trends that have been disrupting the database industry are mobile applications and big data. The explosive growth in mobile devices and mobile apps is generating a huge amount of data, which has fueled the demand for big data services and for high scale databases.

Big Data 78

Big Data Mobile Latency Database 78

Dynatrace

JUNE 1, 2020

The variables that can impact the performance of an application vary; from coding errors or ‘bugs’ in the software, database slowdowns, hosting and network performance, to operating system and device type support. And I’m sure we’ve all experienced frustration when an application crashes, is slow to load, or doesn’t load at all.

Artificial Intelligence 185

Artificial Intelligence Social Media Monitoring IoT 185

All Things Distributed

JANUARY 11, 2011

DROAM - Dreaming about Cheap Data Roaming. The one thing that I have always struggled with during my travels are the data plans of the cell phone companies. This international data mess has been a frequent conversation topic with fellow travelers and no one has a good, simple and reliable solution. Comments ().

Wireless 86

Wireless AWS Internet Internet 86

Dynatrace

JUNE 29, 2022

Hybrid cloud architecture is a computing environment that shares data and applications on a combination of public clouds and on-premises private clouds. A hybrid cloud, however, combines public infrastructure and services with on-premises resources or a private data center to create a flexible, interconnected IT environment.

Infrastructure 194

Infrastructure Cloud Azure AWS 194

Dynatrace

APRIL 11, 2023

Log4Shell required many organizations to take devices and applications offline to prevent malicious attackers from gaining access to IT systems and sensitive data. As a result, organizations need to be vigilant in identifying and addressing vulnerabilities to protect their systems and data.

Cloud 199

Cloud DevOps Open Source Retail 199

Scalegrid

MARCH 28, 2024

In this comparison of Redis vs Memcached, we strip away the complexity, focusing on each in-memory data store’s performance, scalability, and unique features. Redis is better suited for complex data models, and Memcached is better suited for high-throughput, string-based caching scenarios.

Cache 130

Cache Storage Architecture Scalability 130

DZone

MARCH 16, 2020

This article compares different options for the in-memory maps and their performances in order for an application to move away from traditional RDBMS tables for frequently accessed data. The migration will enable the application to quickly lookup in the map and vet the physician rather than querying the database table for vetting.

Cache 147

Cache Java Performance Database 147

All Things Distributed

OCTOBER 7, 2015

We live in a world where massive volumes of data are generated from websites, connected devices and mobile apps. In such a data intensive environment, making key business decisions such as running marketing and sales campaigns, logistic planning, financial analysis and ad targeting require deriving insights from these data.

Cloud 116

Cloud Big Data AWS Analytics 116

All Things Distributed

AUGUST 19, 2011

Job Openings in AWS - Senior Leader in Database Services. This week it is an opening for senior leaders with AWS Database Services. AWS Database Services is responsible for setting the database strategy and delivering distributed structured storage services to our AWS customers. Comments (). Contact Info. Werner Vogels.

AWS 65

AWS Database Storage Scalability 65

The Morning Paper

OCTOBER 26, 2020

On the surface this is a paper about fast data ingestion from high-volume streams, with indexing to support efficient querying. Helios also serves as a reference architecture for how Microsoft envisions its next generation of distributed big-data processing systems being built. PVLDB’20. Emphasis mine ).

Cloud 104

Cloud Big Data Latency Architecture 104

The Netflix TechBlog

FEBRUARY 16, 2021

This critical insight helped us re-envision the SKU catalog as a seamless, scalable platform that empowers our stakeholders to make rapid changes with confidence while the platform ensures suitable guardrails for data accuracy and integrity. SKUDB: SKU catalog data was migrated from the metadata configuration files to a relational database.

Mobile 220

Mobile Engineering Infrastructure Scalability 220

The Morning Paper

MAY 14, 2019

Seer: leveraging big data to navigate the complexity of performance debugging in cloud microservices Gan et al., Seer uses a lightweight RPC-level tracing system to collect request traces and aggregate them in a Cassandra database. ASPLOS’19. Seer in action.

Big Data 45

Big Data Cloud Performance Hardware 45

Dynatrace

JUNE 1, 2020

The variables that can impact the performance of an application vary; from coding errors or ‘bugs’ in the software, database slowdowns, hosting and network performance, to operating system and device type support. And I’m sure we’ve all experienced frustration when an application crashes, is slow to load, or doesn’t load at all.

Monitoring 133

Monitoring Performance Social Media Artificial Intelligence 133

All Things Distributed

APRIL 27, 2011

To our shareowners: Random forests, naïve Bayesian estimators, RESTful services, gossip protocols, eventual consistency, data sharding, anti-entropy, Byzantine quorum, erasure coding, vector clocks. To do so, weve leaned heavily on the core principles from the distributed systems and database research communities and invented from there.

Technology 132

Technology Technology AWS Storage 132

ScaleOut Software

JULY 19, 2021

Incoming data is saved into data storage (historian database or log store) for query by operational managers who must attempt to find the highest priority issues that require their attention. The following diagram illustrates a typical workflow. What’s missing in this picture?

IoT 78

IoT Big Data Analytics Architecture 78

All Things Distributed

FEBRUARY 17, 2011

a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. Countdown to What is Next in AWS. Expanding the Cloud â?? Introducing the AWS South America (Sao Paulo) Region.

AWS 123

AWS Website Storage Servers 123

Scalegrid

MARCH 14, 2024

Implementing a hybrid cloud solution involves careful decision-making regarding application and data placement, migration strategies, and choosing compatible cloud service providers while ensuring seamless integration and addressing security and compliance challenges.

Strategy 130

Strategy Cloud Artificial Intelligence Infrastructure 130

Abhishek Tiwari

DECEMBER 27, 2017

ETL refers to extract, transform, load and it is generally used for data warehousing and data integration. ETL is a product of the relational database era and it has not evolved much in last decade. There are several emerging data trends that will define the future of ETL in 2018. Unified data management architecture.

Big Data 49

Big Data Artificial Intelligence Storage Hardware 49

All Things Distributed

AUGUST 16, 2011

For example a number of our European customers are subject to data residency requirements when it comes to PII data and they use the EU Region to meet to those requirements. Government and Big Data. One particular early use case for AWS GovCloud (US) will be massive data processing and analytics. More information.

AWS 90

AWS Government Big Data Cloud 90