Remove AWS Remove Cache Remove Data Engineering
article thumbnail

Supporting Diverse ML Systems at Netflix

The Netflix TechBlog

Finally, when all matching is done and data is written the new table is committed so it can be read by other jobs. Compute: Titus Whereas open-source users of Metaflow rely on AWS Batch or Kubernetes as the compute backend , we rely on our centralized compute-platform, Titus.

Systems 235
article thumbnail

How Data Inspires Building a Scalable, Resilient and Secure Cloud Infrastructure At Netflix

The Netflix TechBlog

the order of the rows on your Netflix home page, issuing content licenses when you click play, finding the Open Connect cache closest to you with the content you requested, and many more). All these micro-services are currently operated in AWS cloud infrastructure. Give us a holler if you are interested in a thought exchange.

article thumbnail

Pushy to the Limit: Evolving Netflix’s WebSocket proxy for the future

The Netflix TechBlog

Where aws ends and the internet begins is an exercise left to the reader. The folks on the Cloud Data Engineering (CDE) team, the ones building the paved path for internal data at Netflix, graciously helped us scale it up and make adjustments, but it ended up being an involved process as we kept growing.

Latency 234