The Netflix TechBlog

DECEMBER 3, 2019

The infrastructure should allow them to exercise their freedom as data scientists but it should provide enough guardrails and scaffolding, so they don’t have to worry about software architecture too much. Metaflow is a cloud-native framework. It leverages elasticity of the cloud by design?—?both both for compute and storage.

Open Source 135

Open Source AWS Infrastructure Energy 135

The Netflix TechBlog

MARCH 6, 2019

Mounting object storage in Netflix’s media processing platform By Barak Alon (on behalf of Netflix’s Media Cloud Engineering team) MezzFS (short for “Mezzanine File System”) is a tool we’ve developed at Netflix that mounts cloud objects as local files via FUSE. Netflix operates in multiple AWS regions. Disk Caching? — ?

Media 218

Media Storage Processing Cache 218

All Things Distributed

MAY 27, 2015

Over a year ago the AWS team opened a "pop-up loft" in San Francisco at 925 Market Street. The goal of opening the loft was to give developers an opportunity to get in-person support and education on AWS, to network, get some work done, or just hang out with peers. Usually $30 each, these labs are offered for free in the AWS loft.

AWS 76

AWS Education Big Data Games 76

All Things Distributed

NOVEMBER 15, 2010

Expanding the Cloud - Adding the Incredible Power of the Amazon EC2 Cluster GPU Instances. Cluster GPU programming in the Cloud with the Amazon Web Services changes of all of that. Configuring kernel execution is not a trivial exercise and requires GPU device specific knowledge. Driving Storage Costs Down for AWS Customers.

AWS 125

AWS Programming Latency Architecture 125

All Things Distributed

OCTOBER 2, 2017

" Of course, no technology change happens in isolation, and at the same time NoSQL was evolving, so was cloud computing. As we began growing the AWS business, we realized that external customers might find our Dynamo database just as useful as we found it within Amazon.com.

Internet 111

Internet Internet AWS Performance 111

Dynatrace

JUNE 1, 2021

Millions of lines of code comprise these apps, and they include hundreds of interconnected digital services and open-source solutions , and run in containerized environments hosted across multiple cloud services. Why cloud-native applications make APM challenging. Cloud-native apps also produce many kinds of data.

Monitoring 165

Monitoring Mobile Social Media Open Source 165

O'Reilly

OCTOBER 19, 2021

, but to reference concrete tooling used today in order to ground what could otherwise be a somewhat abstract exercise. Prior to the cloud, setting up and operating a cluster that can handle workloads like this would have been a major technical challenge. Adapted from the book Effective Data Science Infrastructure.

DevOps 145

DevOps Software Engineering Open Source Infrastructure 145

O'Reilly

JANUARY 17, 2023

You might say that the outcome of this exercise is a performant predictive model. Second, this exercise in model-building was … rather tedious? Especially when you consider how Certain Big Cloud Providers treat autoML as an on-ramp to model hosting. That’s sort of true. Building a Better for() loop for ML. Damn convenient.

Tuning 145

Tuning Open Source Software Software 145

The Netflix TechBlog

SEPTEMBER 10, 2024

Where aws ends and the internet begins is an exercise left to the reader. The folks on the Cloud Data Engineering (CDE) team, the ones building the paved path for internal data at Netflix, graciously helped us scale it up and make adjustments, but it ended up being an involved process as we kept growing.

Latency 234

Latency Cache Tuning Efficiency 234

HammerDB

APRIL 19, 2023

Regardless of whether the computing platform to be evaluated is on-prem, containerized, virtualized, or in the cloud, it is crucial to consider several essential factors. Instead, focus on understanding what the workloads exercise to help us determine how to best use them to aid our performance assessment.

Performance 55

Performance Benchmarking Cache Storage 55

Adrian Cockcroft

NOVEMBER 11, 2019

There are many possible failure modes, and each exercises a different aspect of resilience. This is why most AWS regions have three availability zones. The third team is the infrastructure platform team, who deal with datacenter and cloud based resources. If something fails, there should be another way for the system to succeed.

Latency 53

Latency Systems Engineering Hardware 53

Adrian Cockcroft

NOVEMBER 11, 2019

There are many possible failure modes, and each exercises a different aspect of resilience. This is why most AWS regions have three availability zones. The third team is the infrastructure platform team, who deal with datacenter and cloud based resources. If something fails, there should be another way for the system to succeed.

Latency 52

Latency Systems Engineering Hardware 52

HammerDB

JANUARY 5, 2019

For example when considering moving a database application into the cloud are you going to copy your actual production data into multiple cloud environments for testing? When testing your application however this is a very real consideration.

Benchmarking 40

Benchmarking Database Social Media Scalability 40

O'Reilly Software

MARCH 7, 2018

AWS, Kafka, Google Cloud, Spring, ElasticSearch). Alpakka has a sizeable and growing set of connectors [7] implementing support for most standard enterprise protocols (i.e., HTTP, TCP, FTP, MQTT, JMS), databases (i.e., JDBC, Cassandra, HBase, MongoDB) and infrastructure products (i.e., From one-off scripts to reusable blueprints.

Transportation 72

Transportation Java Programming Architecture 72

O'Reilly

NOVEMBER 26, 2024

But many jobs require skills that frequently aren’t taught in traditional CS departments, such as cloud development, Kubernetes, and microservices. Entirely new paradigms rise quickly: cloud computing, data engineering, machine learning engineering, mobile development, and large language models.

Education 110

Education Azure AWS Java 110