The Netflix TechBlog

JANUARY 13, 2022

This has led to a dramatic reduction in the time it takes to detect issues in hardware or bugs in recently rolled out data platform software. One example where it can dramatically help is Spark jobs, where memory tuning is a significant challenge. Expand Pensive with Machine Learning classifiers.

Big Data 242

Big Data Infrastructure Metrics Games 242

All Things Distributed

MARCH 19, 2018

At AWS, we continue to strive to enable builders to build cutting-edge technologies faster in a secure, reliable, and scalable fashion. Now, thousands of customers are trying Amazon SageMaker and building ML models on top of their data lakes in AWS. Post-training model tuning and rich states. Acceleration and distribution.

Scalability 127

Scalability Hardware AWS Tuning 127

All Things Distributed

OCTOBER 8, 2014

We built DynamoDB as a fully-managed service because we wanted to enable our customers, both internal and external, to focus on their application rather than being distracted by undifferentiated heavy lifting like dealing with hardware and software maintenance. The DynamoDB team is launching document model support.

Availability 122

Availability Performance Scalability AWS 122

Brendan Gregg

SEPTEMBER 25, 2021

As a Xen guest, this profile was gathered using perf(1) and the kernel's software cpu-clock soft interrupts, not the hardware NMI. Aftermath I provided details to AWS and Canonical, and then moved onto the other performance issues as part of the migration. Other EC2 instance types, such as C5 or M5, use the AWS Nitro Hypervisor.

Speed 103

Speed Java AWS Virtualization 103

O'Reilly

NOVEMBER 28, 2023

Even with cloud-based foundation models like GPT-4, which eliminate the need to develop your own model or provide your own infrastructure, fine-tuning a model for any particular use case is still a major undertaking. That pricing won’t be sustainable, particularly as hardware shortages drive up the cost of building infrastructure.

Open Source 135

Open Source Google Artificial Intelligence Infrastructure 135

Percona

NOVEMBER 3, 2023

PostgreSQL Cluster One coordinator node citus-coord-01 Three worker nodes citus1 citus2 citus3 Hardware AWS Instance Ubuntu Server 20.04, SSD volume type 64-bit (x86) c5.xlarge Redundancy can potentially decrease overall performance.

Database 114

Database Benchmarking Latency C++ 114

Brendan Gregg

SEPTEMBER 5, 2017

Linux has been adding tracing technologies over the years: kprobes (kernel dynamic tracing), uprobes (user-level dynamic tracing), tracepoints (static tracing), and perf_events (profiling and hardware counters). But there's another factor at play: jobs are also migrating from both Solaris and Linux to cloud jobs instead, specifically AWS.

Virtualization 75

Virtualization AWS Engineering Hardware 75

Dotcom-Montior

NOVEMBER 11, 2020

those resources now belong to cloud providers, such as AWS Lambda, Google Cloud Platform, Microsoft Azure, and others. Developers don’t have to put in additional time to fine-tuning the system, or rely on other teams for support, as it’s done automatically with the cloud provider. Focus on Application Development.

Serverless 57

Serverless Monitoring Lambda Latency 57

Dotcom-Montior

OCTOBER 12, 2017

EasyTravel demo application on AWS server. LoadView scripting tool Easystep on AWS server. Firewall on AWS server. AWS server security policy. Setup Demo App EasyTravel on AWS box. Connect demo application hosted on AWS to Dynatrace. Setup Demo App EasyTravel on AWS Box. Verify hardware sizing.

AWS 40

AWS Monitoring Tuning Servers 40

HammerDB

APRIL 19, 2023

For example, if you are buying the latest Amazon memory-optimized EC2 instance (R7iz), the AWS page ( [link] ) tells us the following: Up to 3.9 Please refer to this tuning guide to tune the system for HammerDB: Open Source Database Tuning Guide on 3rd Generation Intel® Xeon® Scalable Processors Based Platform.

Performance 55

Performance Benchmarking Cache Storage 55

HammerDB

JANUARY 5, 2019

As the chart shows because we know that both HammerDB and the implementation of the TPC-C workload scales then we can determine that with this particular database engine both the software and hardware scales as well. If you only test your own application (and if you have more than one application which one will you use for benchmarking?)

Benchmarking 40

Benchmarking Database Social Media Scalability 40

Percona

APRIL 17, 2023

The main objective of this post is to share my experience over the past years tuning MongoDB and centralize the diverse sources that I crossed in this journey in a unique place. systemctl stop tuned $ systemctl disable tuned Dirty ratio The dirty_ratio is the percentage of total system memory that can hold dirty pages.

Best Practices 52

Best Practices Design Tuning Database 52

Percona

JANUARY 17, 2019

On multi-core machines – which is the majority of the hardware nowadays – and in the cloud, we have multiple cores available for use. AWS Aurora (based on MySQL 5.6) AWS Aurora (based on MySQL 5.6) I will compare AWS Aurora with MySQL (Percona Server) 5.6 With faster disks (i.e.

Cache 72

Cache C++ AWS Airlines 72

Brendan Gregg

SEPTEMBER 25, 2021

As a Xen guest, this profile was gathered using perf(1) and the kernel's software cpu-clock soft interrupts, not the hardware NMI. Aftermath I provided details to AWS and Canonical, and then moved onto the other performance issues as part of the migration. Other EC2 instance types, such as C5 or M5, use the AWS Nitro Hypervisor.

Speed 40

Speed Java AWS Virtualization 40

Adrian Cockcroft

NOVEMBER 13, 2023

In the simplest case, you have a growing workload, and you optimize it to run more efficiently so that you don’t need to buy or rent additional hardware, so your carbon footprint stays the same, but the carbon per transaction or operation is going down. I’ve written before about how to tune out retry storms.

Energy 52

Energy Cloud Traffic Transportation 52

Dynatrace

OCTOBER 28, 2021

In 2009, the purveyor of online videos migrated to AWS cloud infrastructure to deliver its entertainment to a growing audience. A disruption spanning hardware, networks, and cloud infrastructure can require input and participation from network and infrastructure architects, risk experts, security teams, and even procurement officers.

Engineering 264

Engineering Entertainment Cloud Infrastructure 264

Adrian Cockcroft

JUNE 21, 2022

I became the Sun UK local specialist in performance and hardware, and as Sun transitioned from a desktop workstation company to sell high end multiprocessor servers I was helping customers find and fix scalability problems. We had specializations in hardware, operating systems, databases, graphics, etc. He hasn’t changed.

Tuning 52

Tuning Benchmarking Engineering C++ 52

All Things Distributed

NOVEMBER 30, 2016

Last week, I wrote a blog about helping the machine learning scientist community select the right deep learning framework from among many we support on AWS such as MxNet, TensorFlow, Caffe, etc. Developers can build, test, and deploy chatbots directly from the AWS Management Console.

AWS 152

AWS Lambda Artificial Intelligence Mobile 152

Adrian Cockcroft

NOVEMBER 18, 2024

Photo by Adrian of my father’s “round tuit” which I’m hoping will inspire AWS to do something… There’s an old saying that any headline that ends in a question mark can be answered with a “no”. Learn from Nasdaq, whose AI-powered environmental, social, and governance (ESG) platform uses Amazon Bedrock and AWS Lambda.

AWS 98

AWS Energy Lambda Government 98

High Scalability

NOVEMBER 25, 2019

Egnyte is a secure Content Collaboration and Data Governance platform, founded in 2007 when Google drive wasn't born and AWS S3 was cost-prohibitive. AWS for builds. We did this as AWS was cost-prohibitive. We use nodes in different AWS regions to test bandwidth performance consistently. Ubuntu for server teams.

Architecture 52

Architecture Cache Azure Storage 52