High Scalability

JANUARY 11, 2022

Architecture. There are two major processes which gets executed when a user posts a photo on Instagram. The second process occurs asynchronously by persisting user activity in a columnar data-storage(Cassandra) and triggering the process to pre-compute the feed of followers of non-celebrity users (having few thousand followers).

Design 334

Design Media Storage Logistics 334

Dynatrace

APRIL 20, 2023

At Dynatrace Perform 2023 , Maciej Pawlowski, senior director of product management for infrastructure monitoring at Dynatrace, and a senior software engineer at a U.K.-based Indeed, according to Dynatrace data, 71% of CIOs say the explosion of data from cloud-native architectures is beyond human ability to manage.

Analytics 246

Analytics Infrastructure Storage Architecture 246

Strategic Tech

MARCH 16, 2021

There are a few qualities that differentiate average from high performing software engineering organisations. I believe that attitude towards the design of code and architecture is one of them. In Accelerate , Nicole Forsgren shows a link between well-designed, loosely-coupled architecture and more frequent software delivery.

Software Engineering 80

Software Engineering Design Engineering Software 80

Dynatrace

FEBRUARY 4, 2021

Site reliability engineering (SRE) is the practice of applying software engineering principles to operations and infrastructure processes to help organizations create highly reliable and scalable software systems. Dynatrace news. SRE drives a “shift left” mindset.

Engineering 245

Engineering DevOps Government Latency 245

Dynatrace

DECEMBER 5, 2022

DevOps is a widely practiced set of procedures and tools for streamlining the development, release, and updating of software. In their most basic form, DevOps procedures can result in complicated processes, data silos, and fragmented responsibilities. Manually managing and securing multi-cloud environments is no longer practical.

DevOps 264

DevOps Virtualization Best Practices Innovation 264

Dynatrace

JUNE 15, 2021

The fact is, Reliability and Resiliency must be rooted in the architecture of a distributed system. The email walked through how our Dynatrace self-monitoring notified users of the outage but automatically remediated the problem thanks to our platform’s architecture. Fact #2: No significant impact on Dynatrace Users.

AWS 246

AWS Traffic Architecture Azure 246

DZone

JULY 14, 2021

Fei Xu (Software Engineer at PingCAP). TiDB is a Hybrid Transaction/Analytical Processing (HTAP) database that can efficiently process analytical queries. However, when large amounts of data are involved, the CPU becomes the bottleneck for processing queries that include JOIN statements and/or aggregation functions.

Analytics 223

Analytics Software Engineering Performance Database 223

Dynatrace

NOVEMBER 14, 2023

These small wins, such as implementing a blameless root cause analysis process, can take many forms and don’t necessarily involve numerical metrics. Customer empathy is key to a fully optimized site reliability engineering practice Software engineering can often be an impersonal discipline.

Best Practices 246

Best Practices Engineering DevOps Software Engineering 246

The Netflix TechBlog

JUNE 20, 2024

This “Enterprise Data Model/Architect Agent” employs generative AI techniques for autonomous enterprise data modeling and architecture. Tulika Bhatt, Senior Data Engineer at Netflix, shared how her team manages impression data at scale.

Data Engineering 206

Data Engineering Engineering Entertainment Software Engineering 206

Dynatrace

JULY 18, 2023

As software development grows more complex, managing components using an automated onboarding process becomes increasingly important. This is especially crucial in microservice architectures, where the number of components can be overwhelming. The screenshot below displays such a configuration.

Best Practices 246

Best Practices Code Infrastructure Latency 246

The Netflix TechBlog

MARCH 25, 2019

Now, imagine yourself in the role of a software engineer responsible for a micro-service which publishes data consumed by few critical customer facing services (e.g. To improve data accuracy, we decided to leverage AWS S3 access logs to identify entity relationships not been captured by our traditional ingestion process.

Infrastructure 225

Infrastructure Big Data Transportation Architecture 225

Dynatrace

MARCH 6, 2024

” Because of their versatility, teams can use IDPs for all types of software engineering projects, not just those in cloud-native scenarios. This process is vital to an IDP’s effectiveness. Dynatrace has made the reference IDP architecture available on GitHub for anyone to use.

Engineering 264

Engineering Development DevOps Infrastructure 264

Dynatrace

APRIL 18, 2022

Software engineer Taras Tsugrii of Meta (formerly Facebook) paid Keptn a high compliment, saying it feels like a reference implementation of Google’s SRE principles , which are the search giant’s techniques for ensuring the integrity of its sites and services. SLOs are a great way to define what software should do.

Open Source 246

Open Source DevOps Cloud Metrics 246

Dynatrace

MARCH 28, 2023

Incident management with clearly defined responsibilities Site Reliability Engineers (SRE) are challenged not only to detect problems and identify the root cause quickly but also to remediate problems immediately. Any software engineer can search for monitored entities that relate to specific deployments and their respective teams.

Software 246

Software Software Monitoring Software Engineering 246

The Netflix TechBlog

FEBRUARY 9, 2021

In the process, we changed end-to-end identity propagation within the network of services to use a cryptographically-verifiable token-agnostic identity object. We would need to process authentication tokens (and protocols) further upstream. EAS also covers the read-only processing of tokens to create Passports (more on that later).

Architecture 228

Architecture Latency Servers Website 228

The Netflix TechBlog

SEPTEMBER 13, 2021

Supporting developers through those checklists for edge cases, and then validating that each team’s choices resulted in an architecture with all the desired security properties, was similarly not scalable for our security engineers. With these lofty goals in mind, we turned to our central engineering teams to help get us there.

Internet 227

Internet Internet Cloud Traffic 227

Dynatrace

FEBRUARY 12, 2024

At the 2024 Dynatrace Perform conference in Las Vegas, Michael Winkler, senior principal product management at Dynatrace, ran a technical session exploring just some of the many ways in which Dynatrace helps to automate the processes around development, releases, and operation. Ortner reviewed the process of solving these issues. “We

Engineering 246

Engineering Development Infrastructure Cloud 246

Dynatrace

MARCH 17, 2021

Application security is a software engineering term that refers to several different types of security practices designed to ensure applications do not contain vulnerabilities that could allow illicit access to sensitive data, unauthorized code modification, or resource hijacking. Dynatrace news. That’s where Dynatrace comes in.

Open Source 242

Open Source Cloud Games Java 242

Dynatrace

MAY 31, 2023

This shift is leading more organizations to hire site reliability engineers to guarantee the reliability and resiliency of their services. How site reliability engineering affects organizations’ bottom line SRE applies the disciplines of software engineering to infrastructure management, both on-premises and in the cloud.

Best Practices 246

Best Practices DevOps Latency Metrics 246

Strategic Tech

JANUARY 22, 2024

Architecture modernization initiatives are strategic efforts involving many teams, usually for many months or years. An AMET is an architecture Enabling Team that helps to coordinate and upskill all teams and stakeholders involved in a modernization initiative. They need a more loosely coupled architecture and empowered teams.

Architecture 98

Architecture Metrics Logistics Strategy 98

O'Reilly

OCTOBER 19, 2021

This is both frustrating for companies that would prefer making ML an ordinary, fuss-free value-generating function like software engineering, as well as exciting for vendors who see the opportunity to create buzz around a new category of enterprise software. All ML projects are software projects.

DevOps 145

DevOps Software Engineering Open Source Infrastructure 145

The Netflix TechBlog

DECEMBER 3, 2019

mainly because of mundane reasons related to software engineering. We heard many stories about difficulties related to data access and basic data processing. While a typical machine learning workflow running on Metaflow touches only a small shard of this warehouse, it can still process terabytes of data.

Open Source 238

Open Source AWS Infrastructure Energy 238

The Netflix TechBlog

NOVEMBER 3, 2022

The original assumptions and architectural choices were no longer viable. We introduce a caching mechanism in the API gateway layer, allowing us to offload processing from singleton leader elected controllers without giving up strict data consistency and guarantees clients observe.

Cache 233

Cache Latency Traffic Systems 233

The Netflix TechBlog

JULY 12, 2019

By Drew Koszewnik This is the story about how the Content Setup Engineering team used Hollow, a Netflix OSS technology, to re-architect and simplify an essential component in our content pipeline?—?delivering delivering a large amount of business value in the process. Improved debuggability and visibility into liveness processing.

Cache 184

Cache Architecture Latency Engineering 184

The Netflix TechBlog

OCTOBER 18, 2022

Meson was based on a single leader architecture with high availability. For example, a workflow to backfill hourly data for the past five years can lead to 43800 jobs (24 * 365 * 5), each of which processes data for an hour. Figure 1 shows the high-level architecture. With the high growth of workflows in the past few years?

Java 211

Java Scalability Traffic Architecture 211

Dynatrace

JANUARY 23, 2024

The following resources provide more information on how to get the most out of your AI investment, the importance of data quality for business success, and automating manual IT processes to prioritize innovation. Teams face siloed processes and toolsets, vast volumes of data, and redundant manual tasks.

Performance 246

Performance DevOps Innovation Energy 246

Strategic Tech

FEBRUARY 19, 2020

I’ve been disappointed for a long time with the way in which companies organise software development teams. I remember as a young, naive software developer, I assumed there would be structured processes and patterns similar to those used for designing a software architecture. A clear warning sign.

Architecture 64

Architecture Strategy Design Software Architecture 64

Dynatrace

AUGUST 9, 2019

Microservices are often the best approach to breaking down software. The code is smaller, and every software engineer makes production changes on an ongoing basis. Therefore, keep a close eye on your dependencies—especially when you’re breaking monolithic applications into a microservices architecture.

Speed 147

Speed Innovation DevOps Cloud 147

The Netflix TechBlog

MAY 14, 2021

Managing and operating asynchronous workflows can be difficult without the proper tools and architecture that puts observability, debugging, and tracing at the forefront. We are expected to process 1,000 watermarks for a single distribution in a minute, with non-linear latency growth as the number of watermarks increases.

Traffic 166

Traffic Java Latency Google 166

Dynatrace

JUNE 2, 2022

Site reliability engineering (SRE) continues to gain popularity as organizations embrace hybrid cloud strategies and IT automation at scale. By applying software engineering principles to operations and infrastructure practices, SRE enables organizations to streamline and automate IT processes.

DevOps 147

DevOps Innovation Engineering Benchmarking 147

Strategic Tech

NOVEMBER 1, 2021

The 2010s were a turning-point in the history of software engineering. While MVPs have been mainstream for a long time, the concept of Value Streams and Value Stream Architecture is still in the early adopter phase in the DevOps world. The details can vary quite a lot.

Software Architecture 98

Software Architecture Architecture DevOps Software 98

High Scalability

AUGUST 4, 2020

T riplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Application security products and processes, however, have not kept up with advances in software development. Created by former senior-level AWS engineers of 15 years.

Education 115

Education AWS Scalability Java 115

The Netflix TechBlog

FEBRUARY 9, 2021

Introduction and account creation Highlight our value propositions and begin the account creation process. In particular, we’ll define plans and offers, review the legacy architecture and some of its shortcomings, and dig into our new architecture and some of its advantages. Each step of the flow serves a distinct purpose.

Engineering 145

Engineering Scalability Architecture Innovation 145

ACM Sigarch

FEBRUARY 23, 2018

I had a professor in grad school who used to joke that all architecture is reinvented every 5 years. Both virtualization and power burst onto the architecture community seemingly out of nowhere even though there was a clear historical basis and trend for both. Virtualization, for instance, was being addressed by IBM in the 1960s.

Architecture 48

Architecture Cache Hardware Virtualization 48

The Netflix TechBlog

OCTOBER 16, 2019

By Ammar Khaku Introduction In a microservice architecture such as Netflix’s, propagating datasets from a single source to multiple downstream destinations can be challenging. This post is a high level overview of the design and architecture of Gutenberg. A publisher publishes to a topic and consumers consume from a topic.

Cache 191

Cache Architecture Metrics Java 191

The Netflix TechBlog

JUNE 1, 2021

Interview with Samuel Setegne Samuel Setegne This post is part of our “Data Engineers of Netflix” interview series, where our very own data engineers talk about their journeys to Data Engineering @ Netflix. Samuel Setegne is a Senior Software Engineer on the Core Data Science and Engineering team.

Data Engineering 130

Data Engineering Engineering Big Data Healthcare 130

High Scalability

JUNE 11, 2020

T riplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Application security products and processes, however, have not kept up with advances in software development. Created by former senior-level AWS engineers of 15 years.

Education 105

Education AWS Scalability Java 105