The Netflix TechBlog

JUNE 7, 2021

Challenges The cloud network infrastructure that Netflix utilizes today consists of AWS services such as VPC, DirectConnect, VPC Peering, Transit Gateways, NAT Gateways, etc and Netflix owned devices. After several iterations of the architecture and some tuning, the solution has proven to be able to scale.

Network 327

Network Transportation AWS Cloud 327

The Netflix TechBlog

JANUARY 13, 2022

This blog will explore these two systems and how they perform auto-diagnosis and remediation across our Big Data Platform and Real-time infrastructure. Since the data platform manages keystone pipelines, users expect platform issues to be detected and remediated by the Keystone team without any involvement from their end.

Big Data 242

Big Data Infrastructure Metrics Games 242

All Things Distributed

NOVEMBER 30, 2016

Last week, I wrote a blog about helping the machine learning scientist community select the right deep learning framework from among many we support on AWS such as MxNet, TensorFlow, Caffe, etc. Developers can build, test, and deploy chatbots directly from the AWS Management Console.

AWS 153

AWS Lambda Artificial Intelligence Mobile 153

The Netflix TechBlog

APRIL 29, 2019

takes place in Amazon Web Services (AWS), whereas everything that happens afterwards (i.e., The service that orchestrates failover uses numpy and scipy to perform numerical analysis, boto3 to make changes to our AWS infrastructure, rq to run asynchronous workloads and we wrap it all up in a thin layer of Flask APIs. are you logged in?

Open Source 280

Open Source Network Infrastructure Big Data 280

The Netflix TechBlog

MAY 26, 2020

Service Segmentation: The ease of the cloud deployments has led to the organic growth of multiple AWS accounts, deployment practices, interconnection practices, etc. VPC Flow Logs VPC Flow Logs is an AWS feature that captures information about the IP traffic going to and from network interfaces in a VPC. We named this library Sqooby.

Network 154

Network Tuning AWS Traffic 154

The Netflix TechBlog

OCTOBER 27, 2020

Usually Data scientists and engineers write Extract-Transform-Load (ETL) jobs and pipelines using big data compute technologies, like Spark or Presto , to process this data and periodically compute key information for a member or a video. The processed data is typically stored as data warehouse tables in AWS S3.

Latency 252

Latency Storage Big Data Tuning 252

The Netflix TechBlog

MARCH 25, 2019

Netflix’s diverse data landscape made it challenging to capture all the right data and conforming it to a common data model. Spark is the primary big-data compute engine at Netflix and with pretty much every upgrade in Spark, the spark plan changed as well springing continuous and unexpected surprises for us.

Infrastructure 225

Infrastructure Big Data Transportation Architecture 225