All Things Distributed

NOVEMBER 13, 2018

Today, I am happy to announce our plans to open a new AWS Region in Italy! The AWS Europe (Milan) Region is the 25th AWS Region that we've announced globally. It's the sixth AWS Region in Europe, joining existing regions in France, Germany, Ireland, the UK, and the new Region that we recently announced in Sweden.

AWS 164

AWS Energy Automotive Traffic 164

High Scalability

DECEMBER 7, 2018

It's HighScalability time: This is your 1500ms latency in real life situations - pic.twitter.com/guot8khIPX. Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading).

Internet 163

Internet Internet Scalability Blockchain 163

All Things Distributed

DECEMBER 8, 2020

And because they never stop, their need for security, availability, performance, scalability, and flexibility over how they choose to run their workloads never stop. At AWS, we are customer obsessed, so we never stop building either.

AWS 178

AWS Latency Scalability Infrastructure 178

High Scalability

OCTOBER 29, 2019

AWS is the #1 cloud provider for open-source database hosting, and the go-to cloud for MySQL deployments. As organizations continue to migrate to the cloud, it’s important to get in front of performance issues, such as high latency, low throughput, and replication lag with higher distances between your users and cloud infrastructure.

AWS 133

AWS Latency Performance Open Source 133

High Scalability

MARCH 22, 2019

µs of replication latency on lossy Ethernet, which is faster than or comparable to specialized replication systems that use programmable switches, FPGAs, or RDMA.". They'll learn a lot and love you even more.5 5 billion : weekly visits to Apple App store; $500m : new US exascale computer; $1.7 We achieve 5.5 Yep, there are more quotes.

Internet 134

Internet Internet Scalability Wireless 134

High Scalability

DECEMBER 21, 2018

It's HighScalability time: Have a very scalable Xmas everyone! 2) Cloud will decentralise in terms of provision not power i.e. Amazon will "invade" more of those holdouts with AWS Outpost. Tim Bray : How to talk about [Serverless Latency] · To start with, don’t just say “I need 120ms.” Don't be late.

Internet 137

Internet Internet Scalability Serverless 137

Dynatrace

DECEMBER 22, 2019

AWS offers a broad set of global, cloud-based services including computing, storage, networking, Internet of Things (IoT), and many others. You can use these services in combinations that are tailored to help your business move faster, lower IT costs, and support scalability. Monitor and understand additional AWS services.

AWS 136

AWS Metrics IoT Storage 136

Dynatrace

DECEMBER 22, 2019

AWS offers a broad set of global, cloud-based services including computing, storage, networking, Internet of Things (IoT), and many others. You can use these services in combinations that are tailored to help your business move faster, lower IT costs, and support scalability. Monitor and understand additional AWS services.

AWS 133

AWS Metrics IoT Storage 133

All Things Distributed

DECEMBER 14, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Introducing the AWS South America (Sao Paulo) Region. Today, Amazon Web Services is expanding its worldwide coverage with the launch of a new AWS Region in Sao Paulo, Brazil. RAMA , a Brazilian financial services firm and AWS customer : â??The

AWS 104

AWS Latency Storage Cloud 104

The Netflix TechBlog

NOVEMBER 3, 2022

In that scenario, the system would need to deal with the data propagation latency directly, for example, by use of timeouts or client-originated update tracking mechanisms. We started seeing increased response latencies and leader servers running at dangerously high utilization.

Cache 228

Cache Latency Traffic Systems 228

High Scalability

JULY 20, 2018

A typical example of modern "microservices-inspired" Java application would function along these lines: Netflix : We observed during experimentation that RAM random read latencies were rarely higher than 1 microsecond whereas typical SSD random read speeds are between 100–500 microseconds. There are a few more quotes.

Internet 121

Internet Internet Scalability Automotive 121

All Things Distributed

JUNE 21, 2018

As I have talked about before, one of the reasons why we built Amazon DynamoDB was that Amazon was pushing the limits of what was a leading commercial database at the time and we were unable to sustain the availability, scalability, and performance needs that our growing Amazon.com business demanded. The opposite is true.

Database 167

Database AWS Games Latency 167

All Things Distributed

NOVEMBER 13, 2014

When AWS launched, it changed how developers thought about IT services: What used to take weeks or months of purchasing and provisioning turned into minutes with Amazon EC2. Our answer is a new compute service called AWS Lambda. You can go from code to service in three clicks and then let AWS Lambda take care of the rest.

Lambda 113

Lambda AWS Cloud Scalability 113

The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! To sustain this data growth at Netflix, it has deployed open-source software Ceph using AWS services to achieve the required SLOs of some of the post-production workflows.

AWS 100

AWS Entertainment Open Source Benchmarking 100

The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! To sustain this data growth at Netflix, it has deployed open-source software Ceph using AWS services to achieve the required SLOs of some of the post-production workflows.

AWS 100

AWS Entertainment Open Source Benchmarking 100

Scalegrid

SEPTEMBER 10, 2024

At ScaleGrid, we’re always pushing the boundaries to offer more flexibility and scalability to our customers. Here’s what you need to know: AWS Outposts & Philadelphia Local Zone Support Our customers asked, and we listened.

AWS 130

AWS Latency Storage Tuning 130

The Netflix TechBlog

OCTOBER 19, 2020

If we had an ID for each streaming session then distributed tracing could easily reconstruct session failure by providing service topology, retry and error tags, and latency measurements for all service calls. The next challenge was to stream large amounts of traces via a scalable data processing platform.

Infrastructure 236

Infrastructure Transportation Storage Open Source 236

All Things Distributed

MARCH 2, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Expanding the Cloud - Introducing the AWS Asia Pacific (Tokyo) Region. Today Amazon Web Services is expanding its world-wide coverage with the launch of a new AWS Region located in Tokyo, Japan. For more details on AWS in Japan see [link].

AWS 92

AWS Cloud Games Latency 92

The Netflix TechBlog

MARCH 7, 2024

Compute: Titus Whereas open-source users of Metaflow rely on AWS Batch or Kubernetes as the compute backend , we rely on our centralized compute-platform, Titus. We have talked about the importance of a production-grade workflow orchestrator in the context of Metaflow when we released support for AWS Step Functions years ago.

Systems 231

Systems Media Cache Open Source 231

All Things Distributed

JANUARY 23, 2012

Werner Vogels weblog on building scalable and robust distributed systems. Expanding the Cloud - The AWS Storage Gateway. Today Amazon Web Services has launched the AWS Storage Gateway, making the power of secure and reliable cloud storage accessible from customersâ?? Once the AWS Storage Gatewayâ??s Comments ().

Storage 96

Storage AWS Virtualization Cloud 96

High Scalability

AUGUST 17, 2018

coryodaniel : Rewrote an #AWS APIGateway & #lambda service that was costing us about $16000 / month in #elixir. 12 million requests / hour with sub-second latency, ~300GB of throughput / day. It is time for the world to move to an OS structure appropriate for 21st century security requirements. myelixirstatus !#Serverless.No

Internet 105

Internet Internet Scalability Games 105

The Netflix TechBlog

FEBRUARY 18, 2022

To this end, we developed a Rapid Event Notification System (RENO) to support use cases that require server initiated communication with devices in a scalable and extensible manner. Event Priority Based Clusters AWS Instance Clusters that subscribe to the corresponding queues with the same priority.

Systems 334

Systems Traffic Architecture Mobile 334

All Things Distributed

OCTOBER 12, 2016

Three years ago, as part of our AWS Fast Data journey we introduced Amazon ElastiCache for Redis , a fully managed in-memory data store that operates at sub-millisecond latency. This allows for faster failover times while minimizing latency. Amazon’s enhancements address many day-to-day challenges with running Redis.

Scalability 102

Scalability Cache Analytics AWS 102

All Things Distributed

JULY 20, 2015

To be robust and scalable, this key/value store needs to be distributed for durability and availability, to protect against network partitions or hardware failures. This architecture affords Amazon ECS high availability, low latency, and high throughput because the data store is never pessimistically locked.

Latency 156

Latency Architecture AWS Open Source 156

Dynatrace

OCTOBER 1, 2021

Volume, velocity, variety, and complexity : It’s nearly impossible to get answers from the sheer amount of raw data collected from every component in ever-changing modern cloud environments, such as AWS , Azure , and Google Cloud Platform (GCP). Making observability actionable and scalable for IT teams.

Metrics 363

Metrics Open Source Monitoring Cloud 363

The Netflix TechBlog

OCTOBER 27, 2020

The processed data is typically stored as data warehouse tables in AWS S3. The data warehouse is not designed to serve point requests from microservices with low latency. Therefore, we must efficiently move data from the data warehouse to a global, low-latency and highly-reliable key-value store.

Latency 248

Latency Storage Big Data Tuning 248

All Things Distributed

JULY 14, 2015

DynamoDB Streams is the enabling technology behind two other features announced today: cross-region replication maintains identical copies of DynamoDB tables across AWS regions with push-button ease, and triggers execute AWS Lambda functions on streams, allowing you to respond to changing data conditions. DynamoDB Streams.

Database 158

Database Lambda AWS IoT 158

All Things Distributed

AUGUST 16, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Expanding the Cloud - The AWS GovCloud (US) Region. Today AWS announced the launch of the AWS GovCloud (US) Region. The concept of regions gives AWS customers control over the placement of their resources and services. All Things Distributed.

AWS 90

AWS Government Big Data Cloud 90

All Things Distributed

FEBRUARY 22, 2012

Werner Vogels weblog on building scalable and robust distributed systems. Today AWS launched an exciting new service for developers: the Amazon Simple Workflow Service. Amazon SWF is an orchestration service for building scalable distributed applications. As always The AWS developer blog has additional details.

Cloud 117

Cloud AWS Java Scalability 117

Dynatrace

OCTOBER 10, 2022

Many organizations today rely on cloud-native applications for their scalability and agility, among other benefits. Popular examples include AWS Lambda and Microsoft Azure Functions , but new providers are constantly emerging as this model becomes more mainstream. Serverless benefits include the following: Dynamic scalability.

Serverless 166

Serverless Infrastructure Lambda Latency 166

Scalegrid

NOVEMBER 4, 2024

Performance Benchmarking of PostgreSQL on ScaleGrid vs. AWS RDS Using Sysbench This article evaluates PostgreSQL’s performance on ScaleGrid and AWS RDS, focusing on versions 13, 14, and 15. Test Environment Setup Instance Types : We used similar cloud instances for AWS RDS and ScaleGrid to ensure a fair comparison.

Benchmarking 130

Benchmarking AWS Tuning Metrics 130

High Scalability

SEPTEMBER 8, 2018

No, I don’t think that is because AWS is earning a 355x margin on DynamoDB! To meet user-defined goals for performance (request latency) and cost, the monitoring service tracks and adjusts resources to workload changes. You get all the multicore Anna performance you want, but you don’t pay for what you don’t need.

Storage 199

Storage Performance AWS Efficiency 199

All Things Distributed

NOVEMBER 15, 2010

Werner Vogels weblog on building scalable and robust distributed systems. For example, the most fundamental abstraction trade-off has always been latency versus throughput. The throughput of this pipeline is more important than the latency of the individual operations. Also more details can be found on the AWS Developer blog.

AWS 125

AWS Programming Latency Architecture 125

The Netflix TechBlog

APRIL 26, 2022

We use Keystone as it is easy to use, reliable, scalable, and provides aggregation of facts from different cloud regions into a single AWS region. We plan to split these Keystone streams into multiple streams for horizontal scalability. We needed scalability testing and performance testing as well.

Storage 191

Storage Design Scalability Latency 191

All Things Distributed

AUGUST 20, 2015

When using relational databases, traversing relationships requires expensive table JOIN operations, causing significantly increased latency as table size and query complexity grow. Like many AWS innovations, the desire to build a solution for a scalable graph database came from Amazon’s retail business. Enter graph databases.

Database 120

Database Logistics Availability Storage 120

All Things Distributed

MARCH 11, 2016

The epoch of AWS is the launch of Amazon S3 on March 14, 2006, now almost 10 years ago. Looking back over the past 10 years, there are hundreds of lessons that we’ve learned about building and operating services that need to be secure, reliable, scalable, with predictable performance at the lowest possible cost.

AWS 127

AWS Hardware Retail Virtualization 127

All Things Distributed

DECEMBER 5, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Route 53 has the business properties that you have come to expect from an AWS service: fully self-service and programmable, with transparent pay-as-you-go pricing and no minimum usage commitments. All Things Distributed. Comments (). Some fundamentals on Naming.

Cloud 112

Cloud Internet Internet AWS 112