The Netflix TechBlog

MARCH 25, 2019

Now, imagine yourself in the role of a software engineer responsible for a micro-service which publishes data consumed by few critical customer facing services (e.g. You are about to make structural changes to the data and want to know who and what downstream to your service will be impacted.

Infrastructure 225

Infrastructure Big Data Transportation Architecture 225

All Things Distributed

DECEMBER 12, 2018

In April 2017, Amazon Web Services announced that it would launch a new AWS infrastructure region Region in Sweden. Today, I'm happy to announce that the AWS Europe (Stockholm) Region, our 20th Region globally, is now generally available for use by customers. Public sector.

AWS 119

AWS Cloud Games Serverless 119

All Things Distributed

NOVEMBER 30, 2016

Around 20 years ago, we used machine learning in our recommendation engine to generate personalized recommendations for our customers. Last week, I wrote a blog about helping the machine learning scientist community select the right deep learning framework from among many we support on AWS such as MxNet, TensorFlow, Caffe, etc.

AWS 153

AWS Lambda Artificial Intelligence Mobile 153

All Things Distributed

JUNE 20, 2017

Today, I am very excited to announce our plans to open a new AWS Region in Hong Kong! The new region will give Hong Kong-based businesses, government organizations, non-profits, and global companies with customers in Hong Kong, the ability to leverage AWS technologies from data centers in Hong Kong.

AWS 126

AWS Logistics Cloud Social Media 126

The Netflix TechBlog

OCTOBER 27, 2020

By Tianlong Chen and Ioannis Papapanagiotou Netflix has more than 195 million subscribers that generate petabytes of data everyday. Data scientists and engineers collect this data from our subscribers and videos, and implement data analytics models to discover customer behaviour with the goal of maximizing user joy.

Latency 252

Latency Storage Big Data Tuning 252

Dynatrace

JANUARY 16, 2023

Most Kubernetes clusters in the cloud (73%) are built on top of managed distributions from the hyperscalers like AWS Elastic Kubernetes Service (EKS), Azure Kubernetes Service (AKS), or Google Kubernetes Engine (GKE). Big data : To store, search, and analyze large datasets, 32% of organizations use Elasticsearch.

Open Source 358

Open Source Java Operating System Programming 358

All Things Distributed

APRIL 4, 2017

Today, I am very excited to announce our plans to open a new AWS Region in the Nordics! The new region will give Nordic-based businesses, government organisations, non-profits, and global companies with customers in the Nordics, the ability to leverage the AWS technology infrastructure from data centers in Sweden.

AWS 107

AWS Airlines Latency Games 107

DZone

DECEMBER 27, 2023

Welcome to the first post in our exciting series on mastering offline data pipeline's best practices, focusing on the potent combination of Apache Airflow and data processing engines like Hive and Spark. Working together, they form the backbone of many modern data engineering solutions.

Best Practices 140

Best Practices Data Engineering Big Data Games 140

The Netflix TechBlog

MAY 26, 2020

Service Segmentation: The ease of the cloud deployments has led to the organic growth of multiple AWS accounts, deployment practices, interconnection practices, etc. VPC Flow Logs VPC Flow Logs is an AWS feature that captures information about the IP traffic going to and from network interfaces in a VPC.

Network 154

Network Tuning AWS Big Data 154

All Things Distributed

JANUARY 19, 2011

AWS Elastic Beanstalk: A Quick and Simple Way into the Cloud. Additionally customers are not restricted to AWS services; they can mix-and-match services from other providers to best meet their needs. There are some excellent platforms running on AWS that do precisely this; Ruby on Rails developers have Heroku. Comments ().

AWS 100

AWS Cloud Java Scalability 100

All Things Distributed

OCTOBER 7, 2015

However, the data infrastructure to collect, store and process data is geared toward developers (e.g., In AWS’ quest to enable the best data storage options for engineers, we have built several innovative database solutions like Amazon RDS, Amazon RDS for Aurora, Amazon DynamoDB, and Amazon Redshift.

Cloud 114

Cloud Big Data AWS Analytics 114

All Things Distributed

JUNE 26, 2016

In June 2015, Amazon Web Services announced that it would launch a new AWS infrastructure region in India. Examples of continuous sensing are found in the managed cloud platform built by Rachio on AWS IoT to enable the secure interaction of its connected devices with cloud applications/other devices. The opportunity to revolutionize.

AWS 77

AWS Cloud Healthcare Blockchain 77

The Netflix TechBlog

OCTOBER 18, 2022

by Jun He , Akash Dwivedi , Natallia Dzenisenka , Snehal Chennuru , Praneeth Yenugutala , Pawan Dixit At Netflix, Data and Machine Learning (ML) pipelines are widely used and have become central for the business, representing diverse use cases that go beyond recommendations, predictions and data transformations.

Java 211

Java Scalability Traffic Architecture 211

All Things Distributed

NOVEMBER 15, 2016

As I mentioned, we live in a world where massive volumes of data are being generated, every day, from connected devices, websites, mobile apps, and customer applications running on top of AWS infrastructure. It is the underlying engine that allows QuickSight to deliver blazing fast response times on large data sets.

Analytics 126

Analytics Availability Media Social Media 126

All Things Distributed

APRIL 27, 2011

We use high-performance transactions systems, complex rendering and object caching, workflow and queuing systems, business intelligence and data analytics, machine learning and pattern recognition, neural networks and probabilistic decision making, and a wide variety of other techniques. Driving Storage Costs Down for AWS Customers.

Technology 129

Technology Technology AWS Storage 129

All Things Distributed

AUGUST 20, 2012

Using Amazon Glacier AWS customers no longer need to worry about how to plan and manage their archiving infrastructure, unlimited archival storage is available to them with a familiar pay-as-you-go model, and with storage priced as low as 1 cent per GB it is extremely cost-effective. s largest organizations. A Complete Storage Solution.

Storage 114

Storage Cloud AWS Media 114

All Things Distributed

MARCH 28, 2011

The service is designed to sustain concurrent device failures by quickly detecting and repairing any lost redundancy, for example there may be a concurrent loss of data in two facilities without the customer ever noticing. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. All postings.

AWS 123

AWS Cloud Storage Internet 123

All Things Distributed

AUGUST 17, 2011

Jekyll in written in Ruby and uses YAML for metadata management and uses the Liquid template engine to manipulate the content. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Countdown to What is Next in AWS. Introducing the AWS South America (Sao Paulo) Region.

Servers 111

Servers Social Media AWS Website 111

High Scalability

MAY 26, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! this is going to be a challenging journey for any backend engineer! Learn the stuff they don't teach you in the AWS docs. Learn the Good Parts of AWS. Please apply here.

Education 100

Education AWS Software Engineering Scalability 100

All Things Distributed

NOVEMBER 15, 2010

A third generation of APIs, however, left the graphics specifics interfaces behind and instead focused on exposing the pipeline as a generic highly parallel engine supporting task and data parallelism. Also more details can be found on the AWS Developer blog. Driving Storage Costs Down for AWS Customers. Contact Info.

AWS 125

AWS Programming Latency Architecture 125

All Things Distributed

APRIL 28, 2010

Expanding the Cloud - Opening the AWS Asia Pacific (Singapore) Region. Today Amazon Web Services has taken another important step in serving customers worldwide: the AWS Asia Pacific (Singapore) Region is now launched. The new Singapore Region offers customers in APAC lower-latency access to AWS services. Comments ().

AWS 60

AWS Cloud Latency Storage 60

The Netflix TechBlog

DECEMBER 21, 2020

By Anupom Syam Background At Netflix, our current data warehouse contains hundreds of Petabytes of data stored in AWS S3 , and each day we ingest and create additional Petabytes. Some of the optimizations are prerequisites for a high-performance data warehouse.

Storage 212

Storage Latency Efficiency Data Engineering 212

All Things Distributed

SEPTEMBER 6, 2013

This spring I travelled through Europe for the AWS Global Summit series. Europe is a continent with much diversity and for each country there are great AWS customer examples to tell. To reach those goals, Shell in 2010 began using AWS. These sensors create massive amounts of geological data.

Cloud 102

Cloud Energy AWS Healthcare 102

All Things Distributed

DECEMBER 5, 2010

Route 53 has the business properties that you have come to expect from an AWS service: fully self-service and programmable, with transparent pay-as-you-go pricing and no minimum usage commitments. If you want to learn more about Route 53 visit [link] and read the blog post at the AWS Developer weblog. Countdown to What is Next in AWS.

Cloud 113

Cloud Internet Internet AWS 113

All Things Distributed

JULY 13, 2010

Cluster Computer Instances are similar to other Amazon EC2 instances but have been specifically engineered to provide high performance compute and networking. Jeff Barr in his blog post on the AWS developer blog has additional details and there are some great testimonials of early Cluster Compute Instances customers in the press release.

Cloud 98

Cloud AWS Automotive Latency 98

Scalegrid

MARCH 14, 2024

Workloads from web content, big data analytics, and artificial intelligence stand out as particularly well-suited for hybrid cloud infrastructure owing to their fluctuating computational needs and scalability demands.

Strategy 130

Strategy Cloud Artificial Intelligence Infrastructure 130

All Things Distributed

SEPTEMBER 30, 2010

With existing customers I get a change to dive deep on their AWS usage and understand what works well and where we can do better. There is huge variety in exiting architectures and I am often impressed about the ingenuity of the engineers in how to best transform the application if "Lift & Shift" is not an option. . | Comments ().

AWS 66

AWS Cloud Storage Best Practices 66

All Things Distributed

NOVEMBER 19, 2010

Science & Engineering. an engineering adventure to break the 1,000 mph barrier in a car. The Big Idea: Biomimetic Architecture - The National Geographic came in the mail this week with a beautiful pull-out of Gaudís Sagrada Família, the online version is only a summary. Driving Storage Costs Down for AWS Customers.

AWS 62

AWS Cloud Benchmarking Storage 62

Abhishek Tiwari

JULY 7, 2023

In the era of big data and complex data processing, data pipelines have emerged as a popular solution for managing and manipulating data. They provide a systematic approach to extract, transform, and load (ETL) data from various sources, enabling organizations to derive valuable insights.

Logistics 52

Logistics Transportation Scalability Data Engineering 52

VoltDB

MARCH 13, 2024

In the age of big-data-turned-massive-data, maintaining high availability , aka ultra-reliability, aka ‘uptime’, has become “paramount”, to use a ChatGPT word. A badly engineered system could fail again in this scenario, or requests could be handled out of sequence.

Availability 52

Availability Latency DevOps Systems 52

Scalegrid

NOVEMBER 25, 2019

ScyllaDB is an open-source distributed NoSQL data store, reimplemented from the popular Apache Cassandra database. Released just four years ago in 2015, Scylla has averaged over 220% year-over-year growth in popularity according to DB-Engines. percentile latency is up to 11X better than Cassandra on AWS EC2 bare metal.

Big Data 187

Big Data Database Open Source Azure 187

Adrian Cockcroft

NOVEMBER 18, 2024

Photo by Adrian of my father’s “round tuit” which I’m hoping will inspire AWS to do something… There’s an old saying that any headline that ends in a question mark can be answered with a “no”. Learn from Nasdaq, whose AI-powered environmental, social, and governance (ESG) platform uses Amazon Bedrock and AWS Lambda.

AWS 98

AWS Energy Lambda Government 98