Remove Cache Remove Data Engineering Remove Design
article thumbnail

Supporting Diverse ML Systems at Netflix

The Netflix TechBlog

The Machine Learning Platform (MLP) team at Netflix provides an entire ecosystem of tools around Metaflow , an open source machine learning infrastructure framework we started, to empower data scientists and machine learning practitioners to build and manage a variety of ML systems.

Systems 235
article thumbnail

Pushy to the Limit: Evolving Netflix’s WebSocket proxy for the future

The Netflix TechBlog

To support this growth, we’ve revisited Pushy’s past assumptions and design decisions with an eye towards both Pushy’s future role and future stability. The DeviceToDeviceManager is also responsible for observability, with metrics around cache hits, calls to the data store, message delivery rates, and latency percentile measurements.

Latency 234
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

How LinkedIn Serves Over 4.8 Million Member Profiles per Second

InfoQ

LinkedIn introduced Couchbase as a centralized caching tier for scaling member profile reads to handle increasing traffic that has outgrown their existing database cluster. The new solution achieved over 99% hit rate, helped reduce tail latencies by more than 60% and costs by 10% annually. By Rafal Gancarz

Cache 87
article thumbnail

5 data integration trends that will define the future of ETL in 2018

Abhishek Tiwari

In 2018, we will see new data integration patterns those rely either on a shared high-performance distributed storage interface ( Alluxio ) or a common data format ( Apache Arrow ) sitting between compute and storage. In contrast, Alluxio a middleware for data access - think Alluxio storage layer as fast cache.

article thumbnail

Part 3: A Survey of Analytics Engineering Work at Netflix

The Netflix TechBlog

Dashboard DesignTips Rina Chang , SusieLu What is design, and why does it matter? Often people think design is about how things look, but design is actually about how things work. Everything is designed, because were all making choices about how things work, but not everything is designed well.

Analytics 202