The Netflix TechBlog

MARCH 25, 2019

We adopted the following mission statement to guide our investments: “Provide a complete and accurate data lineage system enabling decision-makers to win moments of truth.” Please share your experience by adding your comments below and stay tuned for more on data lineage at Netflix in the follow up blog posts. .

Infrastructure 225

Infrastructure Big Data Transportation Architecture 225

The Netflix TechBlog

MAY 26, 2020

Service Segmentation: The ease of the cloud deployments has led to the organic growth of multiple AWS accounts, deployment practices, interconnection practices, etc. Cloud Network Insight is a suite of solutions that provides both operational and analytical insight into the Cloud Network Infrastructure to address the identified problems.

Network 154

Network Tuning AWS Traffic 154

VoltDB

DECEMBER 11, 2024

Managing and storing this data locally presents logistical and cost challenges, particularly for industries like manufacturing, healthcare, and autonomous vehicles. Redundancy and inefficiency in data aggregation. High costs of frequent data transmission to the cloud for backup.

IoT 52

IoT Energy Logistics Latency 52

The Netflix TechBlog

JUNE 6, 2022

This approach has also allowed us to build strong relationships with central engineering teams at Netflix (Data Platform, Developer Tools, Cloud Infrastructure, IAM Product Engineering) that will continue to serve as central points of leverage for security in the long term.

Software Engineering 176

Software Engineering Scalability Education Engineering 176

O'Reilly

MARCH 2, 2020

This year’s growth in Python usage was buoyed by its increasing popularity among data scientists and machine learning (ML) and artificial intelligence (AI) engineers. The shift to cloud native design is transforming both software architecture and infrastructure and operations. Still cloud-y, but with a possibility of migration.

Artificial Intelligence 27

Artificial Intelligence Software Architecture Programming C++ 27

The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 202 A day in the life of a Netflix Engineer Dave Hahn , SRE Engineering Manager Abstract : Netflix is a large, ever-changing ecosystem serving millions of customers across the globe through cloud-based systems and a globally distributed CDN. Thursday?—?December

AWS 100

AWS Entertainment Open Source Benchmarking 100

The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 202 A day in the life of a Netflix Engineer Dave Hahn , SRE Engineering Manager Abstract : Netflix is a large, ever-changing ecosystem serving millions of customers across the globe through cloud-based systems and a globally distributed CDN. Thursday?—?December

AWS 100

AWS Entertainment Open Source Benchmarking 100

The Netflix TechBlog

SEPTEMBER 10, 2024

The folks on the Cloud Data Engineering (CDE) team, the ones building the paved path for internal data at Netflix, graciously helped us scale it up and make adjustments, but it ended up being an involved process as we kept growing. Security As the edge of the Netflix cloud, security considerations are always top of mind.

Latency 234

Latency Cache Tuning Efficiency 234

O'Reilly

SEPTEMBER 15, 2021

Cloud certifications, specifically in AWS and Microsoft Azure, were most strongly associated with salary increases. As we’ll see later, cloud certifications (specifically in AWS and Microsoft Azure) were the most popular and appeared to have the largest effect on salaries. Cloud certifications were by far the most popular.

Azure 145

Azure Programming AWS Social Media 145

Abhishek Tiwari

FEBRUARY 26, 2020

Skills++: Induct specialised Frontend, Backend & QA engineers, Technical Leadership 21-30 Engineers Focus: Building quality in products and continue to innovate while proactively planning team structure, incentives, and culture. Start moving existing apps into a DIY cloud environment - VMs and managed DB services.

Technology 82

Technology Technology Architecture Engineering 82

Adrian Cockcroft

FEBRUARY 13, 2023

There was some new sustainability information that was quietly added to Amazon’s Sustainability in the Cloud page in October 2022 that is significant. I’m working with Aerin on a Sustainability in the Cloud Maven class in the UK at the moment, and we’re looking for attendees. But they didn’t go out of their way to promote it.

AWS 64

AWS Energy Architecture Programming 64

Abhishek Tiwari

DECEMBER 27, 2017

Kubernetes has emerged as go to container orchestration platform for data engineering teams. In 2018, a widespread adaptation of Kubernetes for big data processing is anitcipated. Organisations are already using Kubernetes for a variety of workloads [1] [2] and data workloads are up next.

Big Data 63

Big Data Storage Benchmarking Hardware 63

Abhishek Tiwari

JULY 12, 2018

Unfortunately, building data pipelines remains a daunting, time-consuming, and costly activity. Not everyone is operating at Netflix or Spotify scale data engineering function. Often companies underestimate the necessary effort and cost involved to build and maintain data pipelines.

Latency 63

Latency Analytics Scalability Engineering 63

Abhishek Tiwari

DECEMBER 27, 2017

ETL refers to extract, transform, load and it is generally used for data warehousing and data integration. With the arrival of new cloud-native tools and platform, ETL is becoming obsolete. There are several emerging data trends that will define the future of ETL in 2018.

Big Data 49

Big Data Artificial Intelligence Storage Hardware 49

The Symphonia

JUNE 19, 2018

From data engineering, to cost management, via conversations about team dynamics and architecture, we like to get involved with all-things-cloud-and-DevOps related at our clients. to chat about how we might help you, or just to share ideas how the Cloud is helping your team, and your business.

Serverless 40

Serverless AWS DevOps Open Source 40

Abhishek Tiwari

DECEMBER 22, 2017

Cheap storage and on-demand compute in the cloud coupled with the emergence of new big data frameworks and tools are forcing us to rethink the whole ETL and data warehousing architecture. You can also use cloud and SaaS services such as Google BigQuery, Amazon Redshift Spectrum, Amazon Athena, Qubole to implement ELT approach.

Big Data 40

Big Data Retail Storage Google 40

InfoQ

AUGUST 3, 2023

AWS recently announced the general availability (GA) of Amazon EC2 P5 instances powered by the latest NVIDIA H100 Tensor Core GPUs suitable for users that require high performance and scalability in AI/ML and HPC workloads. The GA is a follow-up to the earlier announcement of the development of the infrastructure. By Steef-Jan Wiggers

AWS 75

AWS Availability Scalability Infrastructure 75

InfoQ

APRIL 7, 2021

Recently, Google announced A2 Virtual Machines (VMs)' general availability based on the NVIDIA Ampere A100 Tensor Core GPUs in Compute Engine.

Virtualization 57

Virtualization Google Availability Engineering 57

InfoQ

JULY 14, 2023

Mei-Chin Tsai, Vinod discuss the internal architecture of Azure Cosmos DB and how it achieves high availability, low latency, and scalability. By Mei-Chin Tsai, Vinod Sridharan

Latency 52

Latency Azure Availability Scalability 52

InfoQ

JULY 20, 2023

Microsoft recently announced the general availability (GA) of Azure Managed Lustre, a managed file system for high-performance computing (HPC) and AI workloads. By Steef-Jan Wiggers

Azure 40

Azure Availability Systems Performance 40

All Things Distributed

NOVEMBER 15, 2016

Previously, I wrote about Amazon QuickSight , a new service targeted at business users that aims to simplify the process of deriving insights from a wide variety of data sources quickly, easily, and at a low cost. QuickSight is a very fast, cloud-powered, business intelligence service for the 1/10th the cost of old-guard BI solutions.

Analytics 123

Analytics Availability Media Social Media 123

O'Reilly

NOVEMBER 26, 2024

But many jobs require skills that frequently aren’t taught in traditional CS departments, such as cloud development, Kubernetes, and microservices. Entirely new paradigms rise quickly: cloud computing, data engineering, machine learning engineering, mobile development, and large language models.

Education 110

Education Azure AWS Java 110

The Netflix TechBlog

DECEMBER 17, 2024

By J Han , PallaviPhadnis Context At Netflix, we use Amazon Web Services (AWS) for our cloud infrastructure needs, such as compute, storage, and networking to build and run the streaming platform that we love. The standardized data model and processing promotes scalability and consistency.

Efficiency 206

Efficiency Cloud Analytics Infrastructure 206