All Things Distributed

NOVEMBER 12, 2018

Today, I'm happy to announce that the AWS GovCloud (US-East) Region, our 19th global infrastructure Region, is now available for use by customers in the US. With this launch, AWS now provides 57 Availability Zones, with another 12 zones and four Regions in Bahrain, Cape Town, Hong Kong SAR, and Stockholm expected to come online by 2020.

AWS 115

AWS Healthcare Cloud Government 115

Dynatrace

AUGUST 27, 2019

Welcome back to the blog series in which we share how you can easily solve three common problem scenarios by using Dynatrace and xMatters Flow Designer. This is where xMatters Flow Designer comes into play, by automating remediation steps at the touch of a button. Step 5 – xMatters triggers a runbook in Ansible to fix the disk latency.

Systems 208

Systems DevOps Latency Azure 208

Percona

MAY 1, 2023

Want to save money on your AWS RDS bill? I’ll show you some MySQL settings to tune to get better performance, and cost savings, with AWS RDS. Percona Consultants have decades of experience solving complex database performance issues and design challenges. After that, things went back to normal.

AWS 135

AWS Hardware Storage Tuning 135

All Things Distributed

DECEMBER 14, 2011

Introducing the AWS South America (Sao Paulo) Region. Today, Amazon Web Services is expanding its worldwide coverage with the launch of a new AWS Region in Sao Paulo, Brazil. Several prominent South American customers have been using AWS since the early days. RAMA , a Brazilian financial services firm and AWS customer : â??The

AWS 104

AWS Latency Storage Cloud 104

Dynatrace

MAY 17, 2023

The Dynatrace Site Reliability Guardian is designed for this practice; it allows development teams to define quality objectives in their code, which is validated throughout the delivery process before the code reaches production. Ultimately, the result is shared via a Slack notification to report on the current business behavior.

DevOps 246

DevOps Traffic Latency Best Practices 246

The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! To sustain this data growth at Netflix, it has deployed open-source software Ceph using AWS services to achieve the required SLOs of some of the post-production workflows.

AWS 100

AWS Entertainment Open Source Benchmarking 100

The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! To sustain this data growth at Netflix, it has deployed open-source software Ceph using AWS services to achieve the required SLOs of some of the post-production workflows.

AWS 100

AWS Entertainment Open Source Benchmarking 100

All Things Distributed

JANUARY 18, 2012

a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Today is a very exciting day as we release Amazon DynamoDB , a fast, highly reliable and cost-effective NoSQL database service designed for internet scale applications. Amazon DynamoDB offers low, predictable latencies at any scale. Comments ().

Scalability 162

Scalability Database Ecommerce Latency 162

The Netflix TechBlog

SEPTEMBER 24, 2021

Uploading and downloading data always come with a penalty, namely latency. Figure 2: Cloud Resource and Job Sizes This initial architecture was designed at a time when packaging from a list of chunks was not possible and terabyte-sized files were not considered. thus making this design viable. at most few parts at a time?—?thus

Cloud 240

Cloud Media Storage Cache 240

The Netflix TechBlog

FEBRUARY 18, 2022

Scaling Policies To address the thundering herd problem and to keep latencies under acceptable thresholds, the cluster scale-up policies are configured to be more aggressive than the scale-down policies. Event Priority Based Clusters AWS Instance Clusters that subscribe to the corresponding queues with the same priority.

Systems 334

Systems Traffic Architecture Mobile 334

Percona

DECEMBER 11, 2023

A quick configuration change may do the trick in improving the performance of your AWS RDS for MySQL instance. A Dedicated Log Volume (DLV) is a specialized storage volume designed to house database transaction logs separately from the volume containing the database tables. and later v10 versions MySQL: 8.0.28

AWS 117

AWS Benchmarking Performance Traffic 117

All Things Distributed

MARCH 2, 2011

Expanding the Cloud - Introducing the AWS Asia Pacific (Tokyo) Region. Today Amazon Web Services is expanding its world-wide coverage with the launch of a new AWS Region located in Tokyo, Japan. The advanced Asia Pacific network infrastructure also makes the AWS Tokyo Region a viable low-latency option for customers from South Korea.

AWS 92

AWS Cloud Games Latency 92

All Things Distributed

JANUARY 23, 2012

Expanding the Cloud - The AWS Storage Gateway. Today Amazon Web Services has launched the AWS Storage Gateway, making the power of secure and reliable cloud storage accessible from customersâ?? The Amazon Virtual Private Cloud extends on-premises compute with all the power of AWS, making it elastic, scalable and highly reliable.

Storage 96

Storage AWS Virtualization Cloud 96

The Netflix TechBlog

NOVEMBER 3, 2022

In that scenario, the system would need to deal with the data propagation latency directly, for example, by use of timeouts or client-originated update tracking mechanisms. We started seeing increased response latencies and leader servers running at dangerously high utilization. The query rate in this test is set to 1K requests/second.

Cache 228

Cache Latency Traffic Systems 228

The Netflix TechBlog

SEPTEMBER 10, 2024

Where aws ends and the internet begins is an exercise left to the reader. To support this growth, we’ve revisited Pushy’s past assumptions and design decisions with an eye towards both Pushy’s future role and future stability. This initial functionality was built out for FireTVs and was expanded from there.

Latency 230

Latency Cache Tuning Efficiency 230

High Scalability

SEPTEMBER 8, 2018

No, I don’t think that is because AWS is earning a 355x margin on DynamoDB! To meet user-defined goals for performance (request latency) and cost, the monitoring service tracks and adjusts resources to workload changes. In order to implement these mechanisms, we had to make two significant changes to the design of Anna.

Storage 199

Storage Performance AWS Cloud 199

The Netflix TechBlog

OCTOBER 27, 2020

The processed data is typically stored as data warehouse tables in AWS S3. The data warehouse is not designed to serve point requests from microservices with low latency. Therefore, we must efficiently move data from the data warehouse to a global, low-latency and highly-reliable key-value store.

Latency 248

Latency Storage Big Data Tuning 248

All Things Distributed

JUNE 21, 2018

Further, with the growth and scale of Amazon.com, boundless horizontal scale needed to be a key design point--scaling up simply wasn't an option. In fact, this is been proven by our customers as Amazon Aurora remains the fastest growing service in AWS history. The opposite is true. Take Expedia, for example.

Database 167

Database AWS Games Latency 167

All Things Distributed

JULY 14, 2015

DynamoDB Streams is the enabling technology behind two other features announced today: cross-region replication maintains identical copies of DynamoDB tables across AWS regions with push-button ease, and triggers execute AWS Lambda functions on streams, allowing you to respond to changing data conditions. DynamoDB Streams.

Database 158

Database Lambda AWS IoT 158

The Netflix TechBlog

DECEMBER 21, 2020

By Anupom Syam Background At Netflix, our current data warehouse contains hundreds of Petabytes of data stored in AWS S3 , and each day we ingest and create additional Petabytes. This article will list some of the use cases of AutoOptimize, discuss the design principles that help enhance efficiency, and present the high-level architecture.

Storage 208

Storage Latency Efficiency Data Engineering 208

All Things Distributed

JULY 20, 2015

This architecture affords Amazon ECS high availability, low latency, and high throughput because the data store is never pessimistically locked. As you can see, the latency remains relatively jitter-free despite large fluctuations in the cluster size. Hailo was founded in 2011 and has been built on AWS since Day 1.

Latency 156

Latency Architecture AWS Open Source 156

The Netflix TechBlog

MARCH 7, 2024

Since its inception , Metaflow has been designed to provide a human-friendly API for building data and ML (and today AI) applications and deploying them in our production infrastructure frictionlessly. In other cases, it is more convenient to share the results via a low-latency API.

Systems 231

Systems Media Cache Open Source 231

The Netflix TechBlog

OCTOBER 19, 2020

Now let’s look at how we designed the tracing infrastructure that powers Edgar. If we had an ID for each streaming session then distributed tracing could easily reconstruct session failure by providing service topology, retry and error tags, and latency measurements for all service calls.

Infrastructure 236

Infrastructure Transportation Storage Open Source 236

Dynatrace

JULY 10, 2019

With insights from Dynatrace into network latency and utilization of your cloud resources, you can design your scaling mechanisms and save on costly CPU hours. Dynatrace provides out-of-the-box support for VMware, AWS, Azure, Pivotal Cloud Foundry, and Kubernetes.

Java 197

Java Best Practices Metrics Azure 197

All Things Distributed

NOVEMBER 15, 2010

For example, the most fundamental abstraction trade-off has always been latency versus throughput. These trade-offs have even impacted the way the lowest level building blocks in our computer architectures have been designed. The throughput of this pipeline is more important than the latency of the individual operations.

AWS 125

AWS Programming Latency Architecture 125

The Netflix TechBlog

APRIL 26, 2022

We will share how its design has evolved over the years and the lessons learned while building it. To understand Axion’s design, we need to know the various components that interact with it. The motivation has not changed since then; the design has. Design evolution Axion fact store has four components?—?fact

Storage 191

Storage Design Scalability Latency 191

Dynatrace

OCTOBER 1, 2021

When an observability solution also analyzes user experience data using synthetic and real-user monitoring, you can discover problems before your users do and design better user experiences based on real, immediate feedback. The architects and developers who create the software must design it to be observed. Benefits of observability.

Metrics 363

Metrics Open Source Monitoring Cloud 363

Dynatrace

OCTOBER 4, 2022

Data lakehouses take advantage of low-cost object stores like AWS S3 or Microsoft Azure Blob Storage to store and manage data cost-effectively. Data lakehouses deliver the query response with minimal latency. Agent and open technologies make it easy to ingest large volumes of observability, security, and business data. Data management.

Artificial Intelligence 246

Artificial Intelligence Storage Analytics Government 246

The Netflix TechBlog

AUGUST 29, 2023

A brief history of IPC at Netflix Netflix was early to the cloud, particularly for large-scale companies: we began the migration in 2008, and by 2010, Netflix streaming was fully run on AWS. Today we have a wealth of tools, both OSS and commercial, all designed for cloud-native environments.

Traffic 224

Traffic Latency Cloud C++ 224

All Things Distributed

MARCH 11, 2016

The epoch of AWS is the launch of Amazon S3 on March 14, 2006, now almost 10 years ago. Given that AWS is a pioneer in building and operating these services world-wide, these lessons have been of crucial importance to our business. AWS helps its customers do this too. your resource usage. Build security in from the ground up.

AWS 127

AWS Hardware Retail Virtualization 127

All Things Distributed

FEBRUARY 22, 2012

Today AWS launched an exciting new service for developers: the Amazon Simple Workflow Service. By designing autonomous distributed components, developers get the flexibility to deploy and scale out parts of the application independently as load increases. As always The AWS developer blog has additional details. Comments ().

Cloud 117

Cloud AWS Java Scalability 117

Scalegrid

NOVEMBER 4, 2024

Performance Benchmarking of PostgreSQL on ScaleGrid vs. AWS RDS Using Sysbench This article evaluates PostgreSQL’s performance on ScaleGrid and AWS RDS, focusing on versions 13, 14, and 15. Test Environment Setup Instance Types : We used similar cloud instances for AWS RDS and ScaleGrid to ensure a fair comparison.

Benchmarking 130

Benchmarking AWS Tuning Metrics 130

Brendan Gregg

JULY 4, 2021

It's an exciting time for developments in computer performance, not just for the BPF technology (which I often [write about]) but also for processors with 3D stacking and cloud vendor CPUs (e.g., Ford, et al., “TCP on Upcoming Sapphire Rapids CPUs,” [link] Oct 2020 - [Liu 20] Linda Liu, “Samsung QVO vs EVO vs PRO: What’s the Difference?

Performance 121

Performance Latency Hardware Storage 121

The Netflix TechBlog

FEBRUARY 9, 2021

Before designing a solution it’s important to understand the main product requirements for such a feature: The content needs to be new, relevant, and regional (not all countries have the same catalogue). To reduce latency, assets should be generated in an offline fashion and not in real time.

Engineering 168

Engineering Storage Latency Entertainment 168

n0derunner

AUGUST 20, 2020

Each bare-metal instance is in a separate rack by design (for fault tolerance). The post Cross rack network latency in AWS appeared first on n0derunner. The bandwidth is 25GbE however, the response time between the hosts is so high that I need multiple streams to consume that bandwidth.

Latency 40

Latency Network AWS Design 40

All Things Distributed

DECEMBER 5, 2010

Route 53 has the business properties that you have come to expect from an AWS service: fully self-service and programmable, with transparent pay-as-you-go pricing and no minimum usage commitments. We have designed Route 53 to propagate updates very quickly and give the customer the tools to find out when all changes have been propagated.

Cloud 112

Cloud Internet Internet AWS 112

The Morning Paper

FEBRUARY 6, 2020

’ Stateless is fine until you need state, at which point the coarse-grained solutions offered by current platforms limit the kinds of application designs that work well. On the Cloudburst design teams’ wish list: A running function’s ‘hot’ data should be kept physically nearby for low-latency access.

Serverless 98

Serverless Lambda Cache Latency 98