Remove Latency Remove Network Remove Software Engineering
article thumbnail

Designing Instagram

High Scalability

When a user requests for feed then there will be two parallel threads involved in fetching the user feeds to optimize for latency. FUN FACT : In this talk , Dikang Gu, a software engineer at Instagram core infra team has mentioned about how they use Cassandra to serve critical usecases, high scalability requirements, and some pain points.

Design 334
article thumbnail

Site reliability done right: 5 SRE best practices that deliver on business objectives

Dynatrace

This shift is leading more organizations to hire site reliability engineers to guarantee the reliability and resiliency of their services. How site reliability engineering affects organizations’ bottom line SRE applies the disciplines of software engineering to infrastructure management, both on-premises and in the cloud.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Consistent caching mechanism in Titus Gateway

The Netflix TechBlog

In that scenario, the system would need to deal with the data propagation latency directly, for example, by use of timeouts or client-originated update tracking mechanisms. We started seeing increased response latencies and leader servers running at dangerously high utilization.

Cache 233
article thumbnail

Snap: a microkernel approach to host networking

The Morning Paper

Snap: a microkernel approach to host networking Marty et al., This paper describes the networking stack, Snap , that has been running in production at Google for the last three years+. You need a lot of software engineers and the willingness to rewrite a lot of software to entertain that idea. SOSP’19.

Network 92
article thumbnail

Netflix at AWS re:Invent 2019

The Netflix TechBlog

4:45pm-5:45pm NFX 209 File system as a service at Netflix Kishore Kasi , Senior Software Engineer Abstract : As Netflix grows in original content creation, its need for storage is also increasing at a rapid pace. Technology advancements in content creation and consumption have also increased its data footprint. Wednesday?—?December

AWS 38
article thumbnail

Edge Authentication and Token-Agnostic Identity Propagation

The Netflix TechBlog

A few years ago, we decided to address this complexity by spinning up a new initiative, and eventually a new team, to move the complex handling of user and device authentication, and various security protocols and tokens, to the edge of the network, managed by a set of centralized services, and a single team.

article thumbnail

Evolution of ML Fact Store

The Netflix TechBlog

The first version of our logger library optimized for storage by deduplicating facts and optimized for network i/o using different compression methods for each fact. Pay the storage, network, and compute cost. While “ keep the design simple ” is a frequently shared learning in software engineering, it is not always easy to achieve.

Storage 195