The Netflix TechBlog

OCTOBER 19, 2020

Reconstructing a streaming session was a tedious and time consuming process that involved tracing all interactions (requests) between the Netflix app, our Content Delivery Network (CDN), and backend microservices. A second job taps the data feed from the first job, does tail sampling of data and writes traces to the storage system.

Infrastructure 238

Infrastructure Transportation Storage Open Source 238

Dynatrace

MAY 6, 2021

Contact Dynatrace ONE if you wish to enable Cluster-side screenshot storage on pre-1.216 fresh-installed Clusters. On the “Network overview” page, `Host group` filter suggestions are now properly displayed when a management zone is selected. APM-292756). APM-288329). APM-290353). APM-292404). APM-293415). APM-295166).

Operating System 243

Operating System AWS Metrics Storage 243

Percona

APRIL 26, 2023

When you do a database restore, you want to have it done as soon as possible. And the database is unusable until the restore is done. Our tests showed physical database restores up to 5x faster than the logical ones. The speed of the physical restoration comes down to how fast we can copy (download) data from the remote storage.

Speed 84

Speed Storage AWS Azure 84

All Things Distributed

AUGUST 20, 2012

Managing Cold Storage with Amazon Glacier. With the introduction of Amazon Glacier , IT organizations now have a solution that removes the headaches of digital archiving and provides extremely low cost storage. Amazon Glacier integrates seamlessly with other AWS services such as Amazon S3 and the different AWS Database services.

Storage 121

Storage Cloud AWS Media 121

HammerDB

APRIL 19, 2023

Among the different components of modern software solutions, the database is one of the most critical. There are many times we get asked why some cloud instance performed poorly for their database application and almost always turned out to be some configuration error. For storage, FIO is generally used.

Performance 54

Performance Benchmarking Cache Storage 54

Scalegrid

JANUARY 8, 2024

AI-driven cloud solutions like ScaleGrid offer a diverse range of database hosting options, robust infrastructure optimized for scalability and security, and enable significant cost reductions, supporting businesses in efficient growth and improved ROI. These services are tailored to meet various business requirements.

Artificial Intelligence 147

Artificial Intelligence Cloud Scalability Analytics 147

Dynatrace

NOVEMBER 23, 2020

Nevertheless, there are related components and processes, for example, virtualization infrastructure and storage systems (see image below), that can lead to problems in your Kubernetes infrastructure. Configuring storage in Kubernetes is more complex than using a file system on your host. Logs can also be used to represent event data.

Virtualization 169

Virtualization Infrastructure Monitoring Cloud 169

All Things Distributed

JULY 7, 2011

Expanding the Cloud - AWS Import/Export Support for Amazon EBS. The AWS Import/Export team has announced today that they have expanded their functionality significantly by adding Import into Amazon EBS. AWS Import/Export transfers data off of storage devices using Amazons high-speed internal network and bypassing the Internet.

AWS 90

AWS Cloud Storage Internet 90

All Things Distributed

MARCH 2, 2011

Expanding the Cloud - Introducing the AWS Asia Pacific (Tokyo) Region. Today Amazon Web Services is expanding its world-wide coverage with the launch of a new AWS Region located in Tokyo, Japan. The advanced Asia Pacific network infrastructure also makes the AWS Tokyo Region a viable low-latency option for customers from South Korea.

AWS 108

AWS Cloud Games Latency 108

All Things Distributed

JULY 16, 2012

Register for AWS re: Invent. The first annual AWS user and partner conference will be held November 27-29 at The Venetian in Las Vega. General registration opens up on July 25, 2012, however active AWS customers will get a chance to jump the line and start registering on July 20. All Things Distributed. Comments ().

AWS 78

AWS Big Data Media Storage 78

All Things Distributed

AUGUST 16, 2011

Expanding the Cloud - The AWS GovCloud (US) Region. Today AWS announced the launch of the AWS GovCloud (US) Region. The concept of regions gives AWS customers control over the placement of their resources and services. All Things Distributed. Werner Vogels weblog on building scalable and robust distributed systems.

AWS 107

AWS Government Big Data Cloud 107

All Things Distributed

JANUARY 19, 2011

AWS Elastic Beanstalk: A Quick and Simple Way into the Cloud. Flexibility is one of the key principles of Amazon Web Services - developers can select any programming language and software package, any operating system, any middleware and any database to build systems and applications that meet their requirements. Comments ().

AWS 112

AWS Cloud Java Operating System 112

The Netflix TechBlog

MAY 21, 2019

Lerner, as a web-service, relies on Amazon Web Services (AWS) and Netflix’s Open Source Software (OSS) tools. Lerner uses AWS services to store binary versions of the agents, agent configurations, and training data. Lerner uses AWS services to store binary versions of the agents, agent configurations, and training data.

Testing 164

Testing AWS Lambda Network 164

High Scalability

APRIL 16, 2019

Scenario 3 – Network Partition – Network Connectivity Breaks Down Between Master and Slave Nodes. This is a classical problem in any distributed system where each node thinks the other nodes are down, while in reality, only the network communication between the nodes is broken.

Availability 149

Availability Network Azure AWS 149

The Morning Paper

AUGUST 29, 2019

If you’re moving an OLAP workload to the cloud (AWS in the context of this paper), what DBMS setup should you go with? We focused on OLAP-oriented parallel data warehouse products available for AWS and restricted our attention to commercially available systems. Choosing a cloud DBMS: architectures and tradeoffs Tan et al.,

Architecture 76

Architecture Cloud Storage Serverless 76

All Things Distributed

APRIL 5, 2012

From tax preparation to safe social networks, Amazon RDS brings new and innovative applications to the cloud. Empowering innovation is at the heart of everything we do at Amazon Web Services (AWS). Intelligent Social network - Facilitate topical Q&A conversations among employees, customers and our most valued super contributors.

Innovation 138

Innovation AWS Education Network 138

The Netflix TechBlog

DECEMBER 21, 2020

By Anupom Syam Background At Netflix, our current data warehouse contains hundreds of Petabytes of data stored in AWS S3 , and each day we ingest and create additional Petabytes. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage 212

Storage Latency Efficiency Data Engineering 212

High Scalability

JUNE 22, 2018

using them to respond to storage events on s3 or database events or auth events is super easy and powerful. OpenConnect, the ability to deploy the CDN directly into the internal network of these ISPs served multiple purposes--not the least of which was to expose the fact that they were holding Netflix for ransom.

Internet 113

Internet Internet Scalability Artificial Intelligence 113

All Things Distributed

AUGUST 17, 2011

As some of you may remember I was pretty excited when Amazon Simple Storage Service (S3) released its website feature such that I could serve this weblog completely from S3. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Countdown to What is Next in AWS.

Servers 123

Servers Social Media AWS Website 123

The Netflix TechBlog

JULY 30, 2019

The Netflix Media Database (NMDB) is one such example. Our cluster size ranges from 12–18 instances of AWS EC2 m4.4xlarge instances, typically running at ~30% capacity. *?—?Cassandra This opens the route to using a managed Elasticsearch cluster (a la AWS) as part of the Conductor deployment.

Lambda 191

Lambda Media Open Source Metrics 191

Scalegrid

MAY 13, 2020

Greenplum Database is a massively parallel processing (MPP) SQL database that is built and based on PostgreSQL. Greenplum Database is an open-source , hardware-agnostic MPP database for analytics, based on PostgreSQL and developed by Pivotal who was later acquired by VMware. What is an MPP Database?

Big Data 321

Big Data Database Artificial Intelligence Open Source 321

Adrian Cockcroft

JUNE 21, 2022

The early days at Sun Cambridge were special, I absorbed a lot about networking and the technical side of the role from my fellow systems engineer Martin Baines, and we were driving all over the region in cool company cars (I had a Citroen BX 16V) selling a really hot product. It was the first database sizing guide Sun had ever produced.

Tuning 52

Tuning Benchmarking Engineering C++ 52

All Things Distributed

JULY 13, 2010

Customers with complex computational workloads such as tightly coupled, parallel processes, or with applications that are very sensitive to network performance, can now achieve the same high compute and networking performance provided by custom-built infrastructure while benefiting from the elasticity, flexibility and cost advantages of Amazon EC2.

Cloud 114

Cloud AWS Automotive Latency 114

Percona

DECEMBER 11, 2023

There are a wealth of options on how you can approach storage configuration in Percona Operator for PostgreSQL , and in this blog post, we review various storage strategies — from basics to more sophisticated use cases. For example, you can choose the public cloud storage type – gp3, io2, etc, or set file system.

Storage 108

Storage Strategy Cloud Azure 108

All Things Distributed

DECEMBER 5, 2010

Route 53 has the business properties that you have come to expect from an AWS service: fully self-service and programmable, with transparent pay-as-you-go pricing and no minimum usage commitments. Route 53 provides Authoritative DNS functionality implemented using a world-wide network of highly-available DNS servers. No lock-in.

Cloud 116

Cloud Internet Internet AWS 116

Scalegrid

JULY 29, 2019

MySQL is a free open source relational database management system that is leveraged across a majority of WordPress sites, and allows you to query your data such as posts, pages, images, user profiles, and more. Managing a database is hard, as it needs continuous updating, tuning, and monitoring to ensure the performance of your website.

Database 166

Database Cloud Website Azure 166

Percona

MAY 2, 2023

That’s why it’s essential to implement the best practices and strategies for MongoDB database backups. Why are MongoDB database backups important? Regular database backups are essential to protect against data loss caused by system failures, human errors, natural disasters, or cyber-attacks.

Best Practices 52

Best Practices Database Storage Servers 52

Percona

FEBRUARY 24, 2023

In the Home Dashboard of PMM, a low CPU utilization on any of the database services that are being monitored could mean that the server is inactive or over-provisioned. From AWS documentation , Amazon EBS is an easy-to-use, scalable, high-performance block-storage service designed for Amazon EC2.

Cloud 123

Cloud Serverless AWS Storage 123

Brendan Gregg

NOVEMBER 29, 2017

At Netflix, we've been using these technologies as they've been made available for instance types in the AWS EC2 cloud. The latest AWS hypervisor, Nitro, uses everything to provide a new hardware-assisted hypervisor that is easy to use and has near bare-metal performance. AWS called this [enhanced networking]. Impressive!

Virtualization 110

Virtualization AWS Hardware Storage 110

Dynatrace

OCTOBER 2, 2024

For example, if one of your customers unexpectedly uploaded a 1 GB file instead of a 1 MB file, was there an error with the buffer overflowing, or was the network stack unable to handle the unexpected load? With Dynatrace, there is no need to think about schema and indexes, re-hydration, or hot/cold storage concepts.

Azure 197

Azure DevOps Infrastructure Cloud 197

Scalegrid

MARCH 14, 2024

When delving into the networking aspect of a hybrid cloud deployment, complexities arise due to the requirement of linking or expanding existing on-premises network architectures into the cloud sphere. Ready to take your database management to the next level with ScaleGrid’s cutting-edge solutions?

Strategy 130

Strategy Cloud Artificial Intelligence Infrastructure 130

Scalegrid

JANUARY 8, 2024

For example, an application could use on-premises databases while also having access to ‘cloud bursting’ capabilities during times of high demand. The company utilizes public cloud resources such as Amazon Web Services (AWS) for streaming purposes and maintains on-premises infrastructure for content creation and processing.

Strategy 147

Strategy Cloud Scalability Artificial Intelligence 147

Scalegrid

JANUARY 12, 2024

Simply put, it’s the set of computational tasks that cloud systems perform, such as hosting databases, enabling collaboration tools, or running compute-intensive algorithms. Storage is a critical aspect to consider when working with cloud workloads. What is workload in cloud computing?

Cloud 130

Cloud Virtualization Storage Efficiency 130

Dynatrace

AUGUST 27, 2020

Unlike other competitors in the market, the Dynatrace Software Intelligence Platform is purpose-built for dynamic enterprise cloud environments such as AWS, with full automation and AI at the core. Achieve full observability of all AWS services. The latest batch of services cover databases, networks, machine learning and computing.

AWS 277

AWS Cloud IoT Database 277

The Morning Paper

MARCH 3, 2020

Millions of tiny databases , Brooker et al., It takes you through the thinking processes and engineering practices behind the design of a key part of the control plane for AWS Elastic Block Storage (EBS): the Physalia database that stores configuration information. NSDI’20. This paper is a real joy to read.

Database 106

Database AWS Network Design 106

Scalegrid

JULY 8, 2019

Are you looking to get started with the world’s most popular open-source database, and wondering how you should setup your MySQL hosting ? While Microsoft Azure does offer a managed solution, Azure Database, the solution has some major limitations you should know about before migrating your MySQL deployments.

Azure 219

Azure Cloud Virtualization Database 219

All Things Distributed

JUNE 21, 2018

A common question that I get is why do we offer so many database products? To do this, they need to be able to use multiple databases and data models within the same application. Seldom can one database fit the needs of multiple distinct use cases. Seldom can one database fit the needs of multiple distinct use cases.

Database 167

Database AWS Games Latency 167