The Netflix TechBlog

APRIL 29, 2019

We use and contribute to many open-source Python packages, some of which are mentioned below. takes place in Amazon Web Services (AWS), whereas everything that happens afterwards (i.e., video streaming) takes place in the Open Connect network. If any of this interests you, check out the jobs site or find us at PyCon.

Open Source 280

Open Source Network Infrastructure Big Data 280

Dynatrace

JUNE 20, 2023

Data is proliferating in separate silos from containers and Kubernetes to open source APIs and software to serverless compute services, such as AWS and Azure. Solving custom API observability with OpenTelemetry Because the bank group needed complete coverage for all API traffic, it wanted to standardize tracing on all API calls.

Open Source 246

Open Source Google Serverless Azure 246

Dynatrace

MARCH 16, 2022

Over the last year, Dynatrace extended its AI-powered log monitoring capabilities by providing support for all log data sources. We added monitoring and analytics for log streams from Kubernetes and multicloud platforms like AWS, GCP, and Azure, as well as the most widely used open-source log data frameworks.

Metrics 246

Metrics Lambda Infrastructure Monitoring 246

The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! Netflix shares how Amazon EC2 Auto Scaling allows its infrastructure to automatically adapt to changing traffic patterns in order to keep its audience entertained and its costs on target.

AWS 100

AWS Entertainment Open Source Benchmarking 100

The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! Netflix shares how Amazon EC2 Auto Scaling allows its infrastructure to automatically adapt to changing traffic patterns in order to keep its audience entertained and its costs on target.

AWS 100

AWS Entertainment Open Source Benchmarking 100

Percona

DECEMBER 11, 2023

A quick configuration change may do the trick in improving the performance of your AWS RDS for MySQL instance. Thus, using Dedicated Log Volumes (DLVs), though new in AWS RDS, has been one of the recommended best practices and is a welcome setup improvement for your RDS instance. and later v10 versions MySQL: 8.0.28

AWS 117

AWS Benchmarking Performance Traffic 117

The Netflix TechBlog

OCTOBER 19, 2020

Troubleshooting a session in Edgar When we started building Edgar four years ago, there were very few open-source distributed tracing systems that satisfied our needs. Our tactical approach was to use Netflix-specific libraries for collecting traces from Java-based streaming services until open source tracer libraries matured.

Infrastructure 238

Infrastructure Transportation Storage Open Source 238

Scalegrid

DECEMBER 16, 2019

MongoDB is the #3 open source database and the #1 NoSQL database in the world. So, what makes DigitalOcean a compelling alternative to other popular MongoDB hosting providers like AWS and Azure ? DigitalOcean Advantages for MongoDB. What’s most impressive is that you’re not compromising performance for cost.

Azure 187

Azure AWS Database Latency 187

Dynatrace

NOVEMBER 22, 2022

For retail organizations, peak traffic can be a mixed blessing. While high-volume traffic often boosts sales, it can also compromise uptimes. Include metrics, event logs, distributed traces, metadata, user experience data, and telemetry data from open source technologies and cloud platforms.

Infrastructure 246

Infrastructure Availability Systems Retail 246

The Netflix TechBlog

AUGUST 13, 2020

We also highlight interesting broader events such as regional traffic evacuations and nearby deployments , information that is vital to understanding health holistically. It usually has dependencies, talks to other services, and lives in different AWS regions. Regional traffic evacuations. Especially during an incident.

Monitoring 219

Monitoring Tuning Traffic Metrics 219

The Netflix TechBlog

MARCH 7, 2024

The Machine Learning Platform (MLP) team at Netflix provides an entire ecosystem of tools around Metaflow , an open source machine learning infrastructure framework we started, to empower data scientists and machine learning practitioners to build and manage a variety of ML systems.

Systems 235

Systems Media Cache Open Source 235

The Netflix TechBlog

DECEMBER 17, 2019

There are several open source CDC projects, often using the same underlying libraries, database APIs, and protocols. No locks on tables are ever acquired, which prevent impacting write traffic on the source database. Hence, downstream consumers have confidence to receive change events as they occur on a source.

Database 221

Database Traffic Transportation Open Source 221

The Netflix TechBlog

MARCH 5, 2019

the order of the rows on your Netflix home page, issuing content licenses when you click play, finding the Open Connect cache closest to you with the content you requested, and many more). All these micro-services are currently operated in AWS cloud infrastructure.

Infrastructure 185

Infrastructure Cloud Scalability AWS 185

The Netflix TechBlog

DECEMBER 17, 2019

There are several open source CDC projects, often using the same underlying libraries, database APIs, and protocols. No locks on tables are ever acquired, which prevent impacting write traffic on the source database. Hence, downstream consumers receive change events as they occur on a source.

Database 206

Database Traffic Transportation Open Source 206

Adrian Cockcroft

FEBRUARY 9, 2023

The virtualization and networking platform could be datacenter based, with something like VMware, or cloud based using one of the cloud providers such as AWS EC2. Above that there’s a deployment platform such as Kubernetes or AWS Lambda. Netflix made many contributions to open source projects by it’s cloud platform team.

Engineering 136

Engineering Serverless Lambda AWS 136

Dynatrace

AUGUST 9, 2023

Falco is an open-source, cloud-native security tool that utilizes the Linux kernel technology  eBPF , to generate fine-grained networking, security, and observability events. Install Falco in AWS EKS to gather security-relevant events from all the happenings in Unguard. It also generates  OpenTelemetry  traces.

Analytics 246

Analytics AWS Infrastructure Strategy 246

The Netflix TechBlog

OCTOBER 18, 2022

We started seeing signs of scale issues, like: Slowness during peak traffic moments like 12 AM UTC, leading to increased operational burden. As the usage increased, we had to vertically scale the system to keep up and were approaching AWS instance type limits. Meson was based on a single leader architecture with high availability.

Java 211

Java Scalability Traffic Architecture 211

Dynatrace

APRIL 30, 2019

Keptn is the open source control plane for continuous deployment and automated operations for cloud native applications on Kubernetes. At its heart it uses Istio (for traffic control) and Knative (for event driven tool orchestration) and stores all configuration in Git – following the GitOps approach. It takes your artifacts (e.g:

Architecture 149

Architecture Open Source Azure Metrics 149

Dynatrace

MAY 17, 2023

OpenTelemetry , the open source observability tool, has become the go-to standard for instrumenting custom applications to collect observability telemetry data. OneAgent is the native telemetry data collector and monitoring solution of Dynatrace. On the other hand, if we checked out the process page for our Node.js

Metrics 243

Metrics Database Monitoring Network 243

All Things Distributed

APRIL 11, 2018

Last year at re:Invent, we launched AWS Fargate , and today, I want to explore how Fargate fundamentally changes the landscape of container technology. You should be able to write your code and have it run, without having to worry about configuring complex management tools, open source or not. Getting Started.

Cloud 115

Cloud AWS Innovation Traffic 115

The Netflix TechBlog

JULY 30, 2019

and beyond By Anoop Panicker and Kishore Banala Conductor is a workflow orchestration engine developed and open-sourced by Netflix. Our cluster size ranges from 12–18 instances of AWS EC2 m4.4xlarge instances, typically running at ~30% capacity. *?—?Cassandra Cassandra persistence module is a partial implementation.

Lambda 191

Lambda Media Open Source Metrics 191

Percona

DECEMBER 21, 2023

Percona Monitoring and Management (PMM) is a state-of-the-art piece of software that exists in part thanks to great open source projects like VictoriaMetrics, PostgreSQL, and ClickHouse. This is made to avoid SSL certificates since everything goes through the same VPC (everything runs on ec2 instances on AWS).

Availability 122

Availability Monitoring Open Source Traffic 122

The Netflix TechBlog

MARCH 10, 2023

Elasticsearch Integration Elasticsearch is one of the best and widely adopted distributed, open source search and analytics engines for all types of data, including textual, numerical, geospatial, structured or unstructured data. We also utilize Kafka to process these assets asynchronously without impacting our real time traffic.

Strategy 264

Strategy Cache Storage Analytics 264

All Things Distributed

APRIL 11, 2012

And like most AWS services, Amazon CloudSearch scales automatically as your data and traffic grow, making it an easy choice for applications small to large. Their main motivation is that existing search technologies, both commercial and open source, have proven to be hard to manage and complex to configure. How it Works.

Cloud 134

Cloud AWS Technology Technology 134

All Things Distributed

SEPTEMBER 6, 2013

This spring I travelled through Europe for the AWS Global Summit series. Europe is a continent with much diversity and for each country there are great AWS customer examples to tell. To reach those goals, Shell in 2010 began using AWS. Shell leverages AWS for big data analytics to help achieve these goals.

Cloud 102

Cloud Energy AWS Healthcare 102

High Scalability

APRIL 16, 2019

Application traffic will be redirected to this new master MySQL node and the slave S2 will start replicating from the new master. At ScaleGrid, we offer highly available hosting for MySQL on AWS and MySQL on Azure that is implemented based on the concepts explained in this blog series.

Availability 149

Availability Network Azure AWS 149

All Things Distributed

MAY 19, 2017

Soon after the launch of Amazon Relation Dabase Service (RDS) AWS customers were giving us feedback that they would love to migrate to RDS but what they would love even more was if we could also unshackle them from the high-cost, punitive licensing schemes that came with the proprietary databases.

Database 89

Database Design Cloud Storage 89

KeyCDN

FEBRUARY 22, 2023

With cloud-based infrastructure, organizations can easily scale their web applications to handle increased traffic or demand without the need for expensive hardware upgrades. There are several popular cloud-based platforms for web development and deployment, such as AWS , Azure , and Google Cloud Platform.

Artificial Intelligence 111

Artificial Intelligence Development Serverless Website 111

Percona

OCTOBER 19, 2023

A security group in AWS for PostgreSQL with inbound rules allowing traffic on ports 5432 , 5433 , and 5434 for PostgreSQL. We offer support for open-source databases, managed services, and consulting services, or visit our Percona Community forum ; we are happy to help. This tutorial will use Ubuntu 20.04

Scalability 107

Scalability Database Open Source Traffic 107

Percona

DECEMBER 5, 2023

It was a fascinating read, shedding light on leveraging AWS snapshots’ capabilities to avoid initial data copy within the RDS environment. Then, we need a small downtime window just to move the traffic from the original instance to the upgraded one. Download Percona Distribution for PostgreSQL Today!

Database 103

Database Traffic C++ Servers 103

All Things Distributed

NOVEMBER 21, 2017

Four years ago, as part of our AWS fast data journey, we introduced Amazon ElastiCache for Redis , a fully managed, in-memory data store that operates at microsecond latency. While open source Redis comes with primitives to help reshard a cluster, they are inadequate. Atomic slot migration.

Games 96

Games Retail Latency Education 96

Percona

JULY 1, 2023

Now that Database-as-a-service (DBaaS) is in high demand, there are multiple questions regarding AWS services that cannot always be answered easily: When should I use Aurora and when should I use RDS MySQL ? Amazon Aurora is a proprietary, cloud-native, fully managed relational database service developed by Amazon Web Services (AWS).

AWS 52

AWS Database Serverless Storage 52

The Netflix TechBlog

OCTOBER 18, 2019

For example, a ranking team would ingest live user traffic and subject it to a number of ranking configurations and simulate the event outcomes using predictive models running on canary rankers. To support these efforts, they developed and open sourced some tools along the way. Sam described Ax and BoTorch?—?Ax

Infrastructure 131

Infrastructure Metrics Architecture Efficiency 131

O'Reilly

AUGUST 8, 2023

At its peak, ChatGPT was in very exclusive company: it’s not quite on the level of Python, Kubernetes, and Java, but it’s in the mix with AWS and React, and significantly ahead of Docker. Or should they start with an open source model that can run locally and be trained for their specific application?

Artificial Intelligence 98

Artificial Intelligence Programming Open Source Games 98

KeyCDN

OCTOBER 31, 2018

AWS Cloud 9 AWS Cloud 9 is ideal for developing PHP applications completely in the cloud. Aptana Studio Aptana Studio boasts a surprising amount of features for an open source IDE. Selenium Selenium is a lightweight, open source testing framework that lets anyone create custom UI tests in any language.

Development 57

Development Open Source Java AWS 57

Scalegrid

JUNE 7, 2024

RabbitMQ is an open-source message broker that simplifies inter-service communication by ensuring messages are effectively queued, delivered, and processed across diverse applications. Using a message queue approach, RabbitMQ adeptly handles surges in traffic through data queuing.

IoT 130

IoT Software Architecture Architecture Scalability 130

Smashing Magazine

AUGUST 3, 2023

It increases our visibility and enables us to draw a steady stream of organic (or “free”) traffic to our site. While paid marketing strategies like Google Ads play a part in our approach as well, enhancing our organic traffic remains a major priority. The higher our organic traffic, the more profitable we become as a company.

Performance 78

Performance Cache Traffic Metrics 78