Uber Engineering

AUGUST 3, 2018

Data powers Uber’s global marketplace, enabling more reliable and seamless user experiences across our products for riders, … The post Databook: Turning Big Data into Knowledge with Metadata at Uber appeared first on Uber Engineering Blog.

Big Data 110

Big Data Transportation Engineering Storage 110

The Netflix TechBlog

APRIL 29, 2019

An easy, though imprecise, way of thinking about Netflix infrastructure is that everything that happens before you press Play on your remote control (e.g., Various software systems are needed to design, build, and operate this CDN infrastructure, and a significant number of them are written in Python. are you logged in?

Open Source 280

Open Source Network Infrastructure Big Data 280

Dynatrace

JANUARY 16, 2023

The study analyzes factual Kubernetes production data from thousands of organizations worldwide that are using the Dynatrace Software Intelligence Platform to keep their Kubernetes clusters secure, healthy, and high performing. Kubernetes infrastructure models differ between cloud and on-premises. Kubernetes moved to the cloud in 2022.

Open Source 358

Open Source Java Operating System Programming 358

DZone

JANUARY 15, 2021

It is no surprise that Site Reliability Engineers have risen to prominence in the last decade. Modern IT infrastructure requires robust systems thinking and reliability engineering to keep the show on the road. Downtime is not an option. 88% showed that 60 minutes of downtime costs their business more than $300,000.

Engineering 154

Engineering Infrastructure Cloud Systems 154

Dynatrace

JUNE 26, 2023

Software analytics offers the ability to gain and share insights from data emitted by software systems and related operational processes to develop higher-quality software faster while operating it efficiently and securely. This involves big data analytics and applying advanced AI and machine learning techniques, such as causal AI.

Software 246

Software Software Analytics Big Data 246

Dynatrace

MARCH 24, 2023

Containers enable developers to package microservices or applications with the libraries, configuration files, and dependencies needed to run on any infrastructure, regardless of the target system environment. Mesos supports several container orchestration engines and can launch Docker containers independently of the Docker daemon.

Infrastructure 246

Infrastructure Open Source Operating System Cloud 246

Dynatrace

JULY 6, 2022

As organizations continue to adopt multicloud strategies, the complexity of these environments grows, increasing the need to automate cloud engineering operations to ensure organizations can enforce their policies and architecture principles. This requires significant data engineering efforts, as well as work to build machine-learning models.

Artificial Intelligence 246

Artificial Intelligence Tuning Strategy Big Data 246

Dynatrace

APRIL 20, 2023

With more automated approaches to log monitoring and log analysis, however, organizations can gain visibility into their applications and infrastructure efficiently and with greater precision—even as cloud environments grow. They enable IT teams to identify and address the precise cause of application and infrastructure issues.

Analytics 246

Analytics Infrastructure Storage Architecture 246

The Netflix TechBlog

JUNE 7, 2021

At much less than 1% of CPU and memory on the instance, this highly performant sidecar provides flow data at scale for network insight. Challenges The cloud network infrastructure that Netflix utilizes today consists of AWS services such as VPC, DirectConnect, VPC Peering, Transit Gateways, NAT Gateways, etc and Netflix owned devices.

Network 327

Network Transportation AWS Cloud 327

Dynatrace

OCTOBER 25, 2022

DevOps requires infrastructure experts and software experts to work hand in hand. Thus, NoOps became a loosely defined concept that initially proposed only leveraging cloud-based PaaS and IaaS solutions that freed up operations from provisioning infrastructure and deploying applications. Introduction of AIOps.

DevOps 246

DevOps Big Data Cloud Innovation 246

Dynatrace

OCTOBER 4, 2022

Log management and analytics is an essential part of any organization’s infrastructure, and it’s no secret the industry has suffered from a shortage of innovation for several years. Several pain points have made it difficult for organizations to manage their data efficiently and create actual value.

Analytics 264

Analytics Artificial Intelligence Storage Serverless 264

The Netflix TechBlog

FEBRUARY 16, 2021

Membership Engineering at Netflix is responsible for the plan and pricing configurations for every market worldwide. To solve the challenges mentioned above and meet our rapidly evolving business needs, we re-architected the legacy SKU catalog from the ground up and partnered with the Growth Engineering team to build a scalable SKU platform.

Mobile 218

Mobile Engineering Infrastructure Scalability 218

Dynatrace

JUNE 1, 2020

There are many different types of monitoring from APM to Infrastructure Monitoring, Network Monitoring, Database Monitoring, Log Monitoring, Container Monitoring, Cloud Monitoring, Synthetic Monitoring, and End User monitoring. With our AI engine, Davis, at the core Dynatrace provides precise answers in real-time. AI-Assistance.

Artificial Intelligence 254

Artificial Intelligence Social Media Monitoring IoT 254

Dynatrace

DECEMBER 15, 2022

ITOps is an IT discipline involving actions and decisions made by the operations team responsible for an organization’s IT infrastructure. Besides the traditional system hardware, storage, routers, and software, ITOps also includes virtual components of the network and cloud infrastructure. What is ITOps? ITOps vs. AIOps.

Artificial Intelligence 246

Artificial Intelligence DevOps Hardware Virtualization 246

The Netflix TechBlog

OCTOBER 27, 2020

By Tianlong Chen and Ioannis Papapanagiotou Netflix has more than 195 million subscribers that generate petabytes of data everyday. Data scientists and engineers collect this data from our subscribers and videos, and implement data analytics models to discover customer behaviour with the goal of maximizing user joy.

Latency 252

Latency Storage Big Data Tuning 252

The Netflix TechBlog

SEPTEMBER 10, 2019

Our A/B tests range across UI, algorithms, messaging, marketing, operations, and infrastructure changes. Instead of relying on engineers to productionize scientific contributions, we’ve made a strategic bet to build an architecture that enables data scientists to easily contribute.

Metrics 221

Metrics Architecture Infrastructure Innovation 221

Dynatrace

AUGUST 10, 2021

Vikash Chhaganlal , GM of Engineering and Infrastructure at Kiwibank said it. She dispelled the myth that more big data equals better decisions, higher profits, or more customers. Investing in data is easy but using it is really hard”. The fact is, data on its own isn’t meaningful. And they were.

DevOps 246

DevOps Innovation Big Data Cloud 246

Uber Engineering

APRIL 5, 2018

Three years ago, Uber Engineering adopted Hadoop as the storage ( HDFS ) and compute ( YARN ) infrastructure for our organization’s big data analysis.

Systems 109

Systems Big Data Storage Infrastructure 109

The Netflix TechBlog

NOVEMBER 3, 2023

Democratizing Stream Processing @ Netflix By Guil Pires , Mark Cho , Mingliang Liu , Sujay Jain Data powers much of what we do at Netflix. On the Data Platform team, we build the infrastructure used across the company to process data at scale.

Processing 271

Processing Engineering Infrastructure Latency 271

The Netflix TechBlog

MAY 26, 2020

Cloud Network Insight is a suite of solutions that provides both operational and analytical insight into the Cloud Network Infrastructure to address the identified problems. As with any sustainable engineering design, focusing on simplicity is very important. And excellent logging is needed for debugging purposes and supportability.

Network 154

Network Tuning AWS Traffic 154

Dynatrace

JUNE 1, 2020

There are many different types of monitoring from APM to Infrastructure Monitoring, Network Monitoring, Database Monitoring, Log Monitoring, Container Monitoring, Cloud Monitoring, Synthetic Monitoring and End User monitoring. With our AI engine, Davis, at the core Dynatrace provides precise answers in real-time. AI-Assistance.

Monitoring 173

Monitoring Performance Social Media Artificial Intelligence 173

Uber Engineering

AUGUST 14, 2019

Once identified, … The post Less is More: Engineering Data Warehouse Efficiency with Minimalist Design appeared first on Uber Engineering Blog. In our experience, optimizing for operational efficiency requires answering one key question: for which tables does the maintenance cost supersede utility?

Efficiency 110

Efficiency Engineering Design Storage 110

All Things Distributed

OCTOBER 7, 2015

In such a data intensive environment, making key business decisions such as running marketing and sales campaigns, logistic planning, financial analysis and ad targeting require deriving insights from these data. However, the data infrastructure to collect, store and process data is geared toward developers (e.g.,

Cloud 114

Cloud Big Data AWS Analytics 114

The Netflix TechBlog

MARCH 2, 2021

At Netflix, our data scientists span many areas of technical specialization, including experimentation, causal inference, machine learning, NLP, modeling, and optimization. Together with data analytics and data engineering, we comprise the larger, centralized Data Science and Engineering group.

Analytics 216

Analytics C++ Innovation Engineering 216

Dynatrace

MARCH 14, 2022

AIOps (or “AI for IT operations”) uses artificial intelligence so that big data can help IT teams work faster and more effectively. Learn more about Dynatrace’s approach to AIOps and explore Davis , our innovative AI engine at the core of our observability platform. Gartner introduced the concept of AIOps in 2016.

Healthcare 246

Healthcare Artificial Intelligence Innovation Strategy 246

All Things Distributed

NOVEMBER 15, 2016

As I mentioned, we live in a world where massive volumes of data are being generated, every day, from connected devices, websites, mobile apps, and customer applications running on top of AWS infrastructure. Put simply, data is not always readily available and accessible to organizational end users. Enter Amazon QuickSight.

Analytics 126

Analytics Availability Media Social Media 126

Dynatrace

OCTOBER 14, 2021

Gartner defines AIOps as the combination of “big data and machine learning to automate IT operations processes, including event correlation, anomaly detection, and causality determination.” The second challenge with traditional AIOps centers around the data processing cycle. But what is AIOps, exactly? What is AIOps?

Artificial Intelligence 246

Artificial Intelligence DevOps Innovation Metrics 246

Scalegrid

MARCH 14, 2024

Key Takeaways A hybrid cloud platform combines private and public cloud providers with on-premises infrastructure to create a flexible, secure, cost-effective IT environment that supports scalability, innovation, and rapid market response. The architecture usually integrates several private, public, and on-premises infrastructures.

Strategy 130

Strategy Cloud Artificial Intelligence Infrastructure 130

Alex Podelko

DECEMBER 19, 2019

– Performance engineering as it done at Alibaba – which emerging as a major cloud provider. – Clearly a hot topic – and the most interesting point here would be how it is changing performance engineering. Meeting of the Minds: Performance Engineering. – Optimizing IT infrastructure – with specific use cases.

Efficiency 100

Efficiency Artificial Intelligence Scalability Performance 100

All Things Distributed

DECEMBER 8, 2016

Earlier this year, Amazon Web Services (AWS) announced it would launch a new AWS infrastructure region in Montreal, Quebec. The new Canada (Central) Region offers a robust suite of infrastructure, management, and developer services that can enable innovators to deploy market-leading applications. in the coming year. Performance.

AWS 131

AWS Cloud Lambda Innovation 131

All Things Distributed

JUNE 20, 2017

9GAG has a small team of nine people, including three engineers to support the business, and uses AWS to service their global visitors. They chose to use AWS in order to focus on developing their platform, instead of managing infrastructure. They believe that they have reduced development time from 20 to 30 percent by having done so.

AWS 126

AWS Logistics Cloud Social Media 126

All Things Distributed

APRIL 4, 2017

The new region will give Nordic-based businesses, government organisations, non-profits, and global companies with customers in the Nordics, the ability to leverage the AWS technology infrastructure from data centers in Sweden. After migrating, database queries that took six seconds now take three seconds in their AWS infrastructure.

AWS 107

AWS Airlines Latency Games 107

The Netflix TechBlog

DECEMBER 21, 2020

Some of the optimizations are prerequisites for a high-performance data warehouse. Sometimes Data Engineers write downstream ETLs on ingested data to optimize the data/metadata layouts to make other ETL processes cheaper and faster.

Storage 212

Storage Latency Efficiency Data Engineering 212

Uber Engineering

OCTOBER 30, 2018

Cluster management, a common software infrastructure among technology companies, aggregates compute resources from a collection of physical hosts into a shared resource pool, amplifying compute power and allowing for the flexible use of data center hardware.

Hardware 103

Hardware Infrastructure Engineering Technology 103

High Scalability

MARCH 24, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! this is going to be a challenging journey for any backend engineer! Who's Hiring? Please apply here. Apply here. Apply here. Make your job search O (1), not O ( n ). Apply here.

Education 105

Education Software Engineering Engineering Big Data 105

High Scalability

APRIL 14, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! this is going to be a challenging journey for any backend engineer! They also do live system design discussions every week. Try out their platform. Please apply here. Apply here.

Education 105

Education Software Engineering Scalability Engineering 105

High Scalability

MARCH 30, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! this is going to be a challenging journey for any backend engineer! They also do live system design discussions every week. Try out their platform. Please apply here. Apply here.

Education 105

Education Software Engineering Engineering Big Data 105