The Netflix TechBlog

OCTOBER 19, 2020

In our previous blog post we introduced Edgar, our troubleshooting tool for streaming sessions. If we had an ID for each streaming session then distributed tracing could easily reconstruct session failure by providing service topology, retry and error tags, and latency measurements for all service calls.

Infrastructure 235

Infrastructure Transportation Storage Open Source 235

Dynatrace

APRIL 8, 2024

This blog explores how vertically integrated risk management solutions that use AI and automation enable unparalleled visibility, control, and efficiency for risk management in banking. Collect data automatically and pre-processed from a range of sources: application programming interfaces, integrations, agents, and OpenTelemetry.

Analytics 195

Analytics Infrastructure Efficiency Technology 195

Percona

MARCH 13, 2023

There is a section in our Documentation ( Introduction to Serverless PostgreSQL ) and a short overview of the primary components: Page Server The storage server with the primary goal of storing all data pages and WAL records Safe Keeper A component to store WAL records in memory (to reduce latency). 5454 --listen-http=0.0.0.0:7676

Serverless 120

Serverless C++ Storage Servers 120

The Netflix TechBlog

SEPTEMBER 18, 2024

Second, developers had to constantly re-learn new data modeling practices and common yet critical data access patterns. These include challenges with tail latency and idempotency, managing “wide” partitions with many rows, handling single large “fat” columns, and slow response pagination.

Latency 239

Latency Storage Cache Efficiency 239

All Things Distributed

JANUARY 23, 2012

Expanding the Cloud - The AWS Storage Gateway. Today Amazon Web Services has launched the AWS Storage Gateway, making the power of secure and reliable cloud storage accessible from customersâ?? With the launch of the AWS Storage Gateway our customers can now integrate their on-premises IT environment with AWSâ??s

Storage 111

Storage AWS Virtualization Cloud 111

Dynatrace

SEPTEMBER 13, 2023

In this blog post, we’ll demonstrate how Dynatrace automation and the Dynatrace Site Reliability Guardian can help you implement your applications according to all six AWS Well-Architected pillars by integrating them into your software development lifecycle (SDLC).

AWS 242

AWS Efficiency Azure Cloud 242

The Netflix TechBlog

DECEMBER 21, 2020

By Anupom Syam Background At Netflix, our current data warehouse contains hundreds of Petabytes of data stored in AWS S3 , and each day we ingest and create additional Petabytes. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage 206

Storage Latency Efficiency Data Engineering 206

The Morning Paper

NOVEMBER 5, 2019

File systems unfit as distributed storage backends: lessons from 10 years of Ceph evolution Aghayev et al., In this case, the assumption that a distributed storage backend should clearly be layered on top of a local file system. What is a distributed storage backend? SOSP’19. This is not surprising in hindsight.

Storage 64

Storage Systems Hardware Efficiency 64

Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on. Ford, et al., “TCP

Performance 140

Performance Latency Hardware Storage 140

Dynatrace

JUNE 8, 2020

Dynatrace Managed operates as a SaaS solution that keeps your data and data analysis on-premise in your own data center, thereby fulfilling the highest privacy and regulatory needs. Access your cluster health data in Dynatrace Managed. An illustration of the cluster overview dashboard is shown below.

Software 164

Software Software Programming Metrics 164

Percona

JUNE 22, 2023

In this blog, we will explore various MySQL KPIs that are basic and essential to track using monitoring tools like PMM. Disk space usage Monitor the disk space usage of MySQL data files, log files, and temporary files. The point we’re making here is they are critical for MySQL Performance.

Performance 106

Performance Monitoring Traffic Database 106

Dynatrace

APRIL 26, 2022

Digital experience monitoring enables companies to respond to issues more efficiently in real time, and, through enrichment with the right business data, understand how end-user experience of their digital products significantly affects business key performance indicators (KPIs). Endpoint monitoring (EM). Endpoints can be physical (i.e.,

Monitoring 198

Monitoring Social Media IoT Metrics 198

Uber Engineering

MARCH 12, 2017

With the evolution of storage formats like Apache Parquet and Apache ORC and query engines like Presto and Apache Impala , the Hadoop ecosystem has the potential to become a general-purpose, unified serving layer for workloads that can tolerate latencies … The post Hudi: Uber Engineering’s Incremental Processing Framework on Apache Hadoop appeared (..)

Processing 105

Processing Latency Storage Engineering 105

VoltDB

JUNE 26, 2024

Using real-time streaming data and analytics, manufacturers can optimize workflows in the moment, reducing bottlenecks and minimizing downtime. AI-driven quality control systems continuously analyze data from production lines to identify variations and anomalies as soon as they emerge.

IoT 52

IoT Analytics Artificial Intelligence Energy 52

Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on. Ford, et al., “TCP

Performance 52

Performance Latency Hardware Storage 52

The Netflix TechBlog

MARCH 7, 2023

These media focused machine learning algorithms as well as other teams generate a lot of data from the media files, which we described in our previous blog , are stored as annotations in Marken. But we cannot search or present low latency retrievals from files Etc. We refer the reader to our previous blog article for details.

Media 267

Media Latency Architecture Database 267

Dynatrace

AUGUST 20, 2024

Driven by that value, Dynatrace brings real-time observability, security, and business data into context and makes sense of it so our customers can get answers, automate, predict, and prevent. Executives are sitting on a goldmine of data, and they don’t know it. Common business analytics incur too much latency.

Analytics 206

Analytics Latency Processing Storage 206

All Things Distributed

DECEMBER 14, 2011

This new Region has been highly requested by companies worldwide, and it provides low-latency access to AWS services for those who target customers in South America. The new Sao Paulo Region provides better latency to South America, which enables AWS customers to deliver higher performance services to their South American end-users.

AWS 113

AWS Latency Storage Cloud 113

All Things Distributed

MARCH 2, 2011

Japanese companies and consumers have become used to low latency and high-speed networking available between their businesses, residences, and mobile devices. The advanced Asia Pacific network infrastructure also makes the AWS Tokyo Region a viable low-latency option for customers from South Korea. blog comments powered by Disqus.

AWS 108

AWS Cloud Games Latency 108

All Things Distributed

SEPTEMBER 25, 2012

Following the huge success of being able to provision a consistent, user-requested I/O rate for DynamoDB and Elastic Block Store (EBS), the AWS Database Services team has now released Provisioned IOPS, a new high performance storage option for the Amazon Relational Database Service (Amazon RDS). Read more about on their blog.

Cloud 112

Cloud AWS Storage Database 112

Dynatrace

JUNE 25, 2020

The network latency between cluster nodes should be around 10 ms or less. With Dynatrace actively managing business-critical applications, some of our globally distributed enterprise customers require Dynatrace Managed to continue operating even when an entire data center goes down. Minimized cross-data center network traffic.

Availability 242

Availability Hardware Latency Traffic 242

All Things Distributed

JULY 13, 2010

Further computationally intensive, highly parallel workloads have found their way to Amazon EC2 as businesses have explored using HPC types of algorithms for other application categories, for example to to process very large unstructured data sets for Business Intelligence applications. blog comments powered by Disqus. Contact Info.

Cloud 114

Cloud AWS Automotive Latency 114

The Netflix TechBlog

JUNE 13, 2023

Our previous blog post presented replay traffic testing — a crucial instrument in our toolkit that allows us to implement these transformations with precision and reliability. This blog post will delve into the techniques leveraged at Netflix to introduce these changes to production.

Traffic 281

Traffic Metrics Systems Strategy 281

All Things Distributed

JULY 11, 2011

Spot Instances are ideal for use cases like web and data crawling, financial analysis, grid computing, media transcoding, scientific research, and batch processing. To get started using Spot or for more details visit the Amazon EC2 Spot Instance web page, the AWS developer blog , and the EC2 Release Notes. Contact Info. Werner Vogels.

AWS 80

AWS Storage Cloud Big Data 80

All Things Distributed

DECEMBER 3, 2009

This new Region consists of multiple Availability Zones and provides low-latency access to the AWS services from for example the Bay Area. blog comments powered by Disqus. he posts material that doesnt belong on this blog or on twitter. Driving Storage Costs Down for AWS Customers. Contact Info. Werner Vogels.

AWS 60

AWS Cloud Latency Storage 60

All Things Distributed

AUGUST 16, 2011

There are different considerations when deciding where to allocate resources with latency and cost being the two obvious ones, but compliance sometimes plays an important role as well. Government and Big Data. One particular early use case for AWS GovCloud (US) will be massive data processing and analytics. More information.

AWS 107

AWS Government Big Data Cloud 107

All Things Distributed

APRIL 28, 2010

Customers can now store their data and run their applications from our Singapore location in the same way they do from our other U.S. There are four main reasons to do so: Performance - For many applications and services, data access latency to end users is important. You can also find more information on the AWS developer blog.

AWS 61

AWS Cloud Latency Storage 61

The Netflix TechBlog

SEPTEMBER 24, 2021

Our previous tech blog Packaging award-winning shows with award-winning technology detailed our packaging technology deployed on the streaming side. As described by the white paper Apple ProRes ( link ), the target data rate of the Apple ProRes HQ for 1920x1080 at 29.97 is 220 Mbps.

Cloud 239

Cloud Media Storage Cache 239

All Things Distributed

DECEMBER 5, 2010

Low-latency query resolution The query resolution functionality of Route 53 is based on anycast, which will route the request automatically to the DNS server that is the closest. This achieves very low-latency for queries which is crucial for the overall performance of internet applications. blog comments powered by Disqus.

Cloud 116

Cloud Internet Internet AWS 116

All Things Distributed

NOVEMBER 19, 2010

There were blog posts by Jeff Barr The Cluster GPU Instance and James Hamilton HPC in the Cloud with GPGPUs , as well as my background posting: Expanding the Cloud - Adding the Incredible Power of the Amazon EC2 Cluster GPU Instances. blog comments powered by Disqus. he posts material that doesnt belong on this blog or on twitter.

AWS 65

AWS Cloud Benchmarking Storage 65

Percona

MARCH 29, 2023

Compression in any database is necessary as it has many advantages, like storage reduction, data transmission time, etc. Storage reduction alone results in significant cost savings, and we can save more data in the same space. In this blog, we will discuss both data and network-level compression offered in MongoDB.

Storage 110

Storage Network Open Source Latency 110

Adrian Cockcroft

APRIL 18, 2018

Juvenal Photo taken in Lisbon Portugal by Adrian Cockcroft The documentation for most monitoring tools describes how to use that tool in isolation, often as if no other tools exist, but sometimes with ways to import or export some of the data to other tools. When auto-scaling is used monitoring is also part of a critical feedback loop.

Monitoring 40

Monitoring Systems Virtualization Metrics 40

The Netflix TechBlog

SEPTEMBER 2, 2020

In an earlier blog post, we discussed Telltale , our health monitoring system. With request tracing and additional data from logs, events, metadata, and analysis, Edgar is able to show the flow of a request through our distributed system?—?what But, there are a few substantial differences in how Edgar approaches its data and its users.

Latency 297

Latency Transportation Engineering Traffic 297

Percona

AUGUST 10, 2018

We provided an introduction to VACUUM and bloat in an earlier blog post. avi@percona:~$ps -eaf | egrep "/post|autovacuum" postgres 2862 1 0 Jun17 pts/0 00:00:11 /usr/pgsql-10/bin/postgres -D /var/lib/pgsql/10/data postgres 2868 2862 0 Jun17 ? To know more about dead tuples and bloat, please read our previous blog post.

Tuning 45

Tuning Cache Database Storage 45

Percona

DECEMBER 11, 2023

A Dedicated Log Volume (DLV) is a specialized storage volume designed to house database transaction logs separately from the volume containing the database tables. DLVs are particularly advantageous for databases with large allocated storage, high I/O per second (IOPS) requirements, or latency-sensitive workloads.

AWS 99

AWS Benchmarking Performance Traffic 99

Percona

MARCH 23, 2023

This blog is in reference to our previous ones for ‘Innodb Performance Optimizations Basics’ 2007 and 2013. Although there have been many blogs about adjusting MySQL variables for better performance since then, I think this topic deserves a blog update since the last update was a decade ago, and MySQL 5.7

Performance 127

Performance Hardware Tuning Storage 127

The Netflix TechBlog

NOVEMBER 3, 2022

This blog post presents how our current iteration of Titus deals with high API call volumes by scaling out horizontally. We introduce a caching mechanism in the API gateway layer, allowing us to offload processing from singleton leader elected controllers without giving up strict data consistency and guarantees clients observe.

Cache 227

Cache Latency Traffic Systems 227