Scalegrid

FEBRUARY 8, 2024

A distributed storage system is foundational in today’s data-driven landscape, ensuring data spread over multiple servers is reliable, accessible, and manageable. Understanding distributed storage is imperative as data volumes and the need for robust storage solutions rise.

Storage 130

Storage Systems Big Data Azure 130

Dynatrace

AUGUST 11, 2022

Cloud providers then manage physical hardware, virtual machines, and web server software management. Infrastructure as a service (IaaS) handles compute, storage, and network resources. Microservices, on the other hand, make it possible to quickly scale up a single aspect of an application, such as storage or compute use.

Development 279

Development Serverless Best Practices Lambda 279

DZone

JULY 19, 2023

To address this need, the integration of cloud computing and virtualization has emerged as a groundbreaking solution as these technologies boast scalability and flexibility, entirely transforming the operational landscape. Alongside the transition to the cloud, Enel embraced virtualization to maximize the utilization of its IT resources.

Virtualization 130

Virtualization Scalability Cloud Energy 130

Dynatrace

JANUARY 16, 2023

Accordingly, the remaining 27% of clusters are self-managed by the customer on cloud virtual machines. On-premises data centers invest in higher capacity servers since they provide more flexibility in the long run, while the procurement price of hardware is only one of many cost factors.

Open Source 358

Open Source Java Operating System Programming 358

Dynatrace

SEPTEMBER 27, 2024

Carbon Impact uses host utilization metrics from OneAgents to report the estimated energy consumption for CPU, storage I/O, memory, and network. The app automatically builds baselines, important reference points for analyzing the environmental impact of individual hardware or software instances.

Energy 246

Energy Infrastructure Cloud Best Practices 246

All Things Distributed

JANUARY 23, 2012

Expanding the Cloud - The AWS Storage Gateway. Today Amazon Web Services has launched the AWS Storage Gateway, making the power of secure and reliable cloud storage accessible from customersâ?? With the launch of the AWS Storage Gateway our customers can now integrate their on-premises IT environment with AWSâ??s

Storage 97

Storage AWS Virtualization Cloud 97

The Morning Paper

MARCH 8, 2020

Building an elastic query engine on disaggregated storage , Vuppalapati, NSDI’20. This paper presents Snowflake design and implementation along with a discussion on how recent changes in cloud infrastructure (emerging hardware, fine-grained billing, etc.) But the ephemeral storage service for intermediate data is not based on S3.

Storage 112

Storage Engineering Cache Serverless 112

DZone

MARCH 20, 2024

Understanding KVM Kernel-based Virtual Machine (KVM) stands out as a virtualization technology in the world of Linux. Embedded within the Linux kernel, KVM empowers the creation of VMs with their virtualized hardware components, such as CPUs, memory, storage, and network cards, essentially mimicking a machine.

Virtualization 124

Virtualization Hardware Open Source Storage 124

Dynatrace

AUGUST 5, 2022

Consumers store messages in a queue — usually in a buffer or on a storage medium — until they can process and delete them. In this scenario, message queues coordinate large numbers of microservices, which operate autonomously without the need to provision virtual machines or allocate hardware resources.

Monitoring 246

Monitoring Programming Serverless Speed 246

Dynatrace

AUGUST 5, 2022

Consumers store messages in a queue — usually in a buffer or on a storage medium — until they can process and delete them. In this scenario, message queues coordinate large numbers of microservices, which operate autonomously without the need to provision virtual machines or allocate hardware resources.

Monitoring 246

Monitoring Programming Serverless Speed 246

SQL Performance

DECEMBER 17, 2019

This removes the burden of purchasing and maintaining your hardware, storage and networking infrastructure, while still giving you a very familiar experience with Windows and SQL Server itself. Microsoft currently has eight main types of virtual machines designed for different types of workloads. Azure VM Types and Series.

Azure 72

Azure Virtualization Servers Storage 72

Dynatrace

AUGUST 5, 2019

Lift & Shift is where you basically just move physical or virtual hosts to the cloud – essentially you just run your host on somebody else’s hardware. Optimize Query Performance and Data Storage Cost. Extract less critical data into a cheaper database storage option. Optimize the performance of key queries.

Cloud 38

Cloud Traffic Database Network 38

All Things Distributed

MARCH 11, 2016

This is a given, whether you are using the highest quality hardware or lowest cost components. This becomes an even more important lesson at scale: for example, as S3 processes trillions and trillions of storage transactions, anything that has even the slightest probability of error will become realistic. Primitives not frameworks.

AWS 127

AWS Hardware Retail Virtualization 127

Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on. I also wrote about these topics in detail for my recent [Systems Performance 2nd Edition] book.

Performance 121

Performance Latency Hardware Storage 121

Scalegrid

AUGUST 30, 2024

Authorization and Access Control In RabbitMQ, authorization dictates the operations a user may execute on given virtual hosts. Virtual Hosts and Resource Permissions In RabbitMQ, virtual hosts craft distinct isolated environments that upgrade security and resource segregation by restricting inter-vhost communication.

Transportation 130

Transportation Virtualization Strategy Storage 130

KeyCDN

FEBRUARY 22, 2023

Chatbots and virtual assistants Chatbots and virtual assistants are becoming more common on websites and web applications as they provide an efficient and convenient way for users to interact with a business. These technologies can answer questions, provide customer support, or even complete transactions.

Artificial Intelligence 111

Artificial Intelligence Development Serverless Website 111

Brendan Gregg

NOVEMBER 28, 2017

Hardware virtualization for cloud computing has come a long way, improving performance using technologies such as VT-x, SR-IOV, VT-d, NVMe, and APICv. The latest AWS hypervisor, Nitro, uses everything to provide a new hardware-assisted hypervisor that is easy to use and has near bare-metal performance. I'd expect between 0.1%

Virtualization 40

Virtualization AWS Hardware Storage 40

Percona

JUNE 22, 2023

Defining high availability In general terms, high availability refers to the continuous operation of a system with little to no interruption to end users in the event of hardware or software failures, power outages, or other disruptions. Without enough infrastructure (physical or virtualized servers, networking, etc.),

Availability 132

Availability Database Open Source Hardware 132

Percona

AUGUST 22, 2023

Some of the most important elements include: No single point of failure (SPOF): You must eliminate any SPOF in the database environment, including any potential for an SPOF in physical or virtual hardware. Without enough infrastructure (physical or virtualized servers, networking, etc.), there cannot be high availability.

Availability 98

Availability Hardware Open Source Database 98

All Things Distributed

JANUARY 18, 2012

In response, we began to develop a collection of storage and database technologies to address the demanding scalability and reliability requirements of the Amazon.com ecommerce platform. s pricing is simple and predictable: Storage is $1 per GB per month. The growth of Amazonâ??s Domain scaling limitations. Amazon DynamoDBâ??s

Scalability 162

Scalability Database Ecommerce Latency 162

ACM Sigarch

NOVEMBER 1, 2019

This blog post gives a glimpse of the computer systems research papers presented at the USENIX Annual Technical Conference (ATC) 2019, with an emphasis on systems that use new hardware architectures. Intel Quick Assist Technology (QAT) was the focus of the QZFS paper which used this new hardware device to speed up file system compression.

Architecture 12

Architecture Hardware Cache Storage 12

Brendan Gregg

FEBRUARY 28, 2023

Make sure your system can handle next-generation DRAM,” [link] Nov 2011 - [Hruska 12] Joel Hruska, “The future of CPU scaling: Exploring options on the cutting edge,” [link] Feb 2012 - [Gregg 13] Brendan Gregg, “Blazing Performance with Flame Graphs,” [link] 2013 - [Shimpi 13] Anand Lal Shimpi, “Seagate to Ship 5TB HDD in 2014 using Shingled Magnetic (..)

Performance 103

Performance Latency Cache Virtualization 103

Percona

JULY 7, 2023

HA in PostgreSQL databases delivers virtually continuous availability, fault tolerance, and disaster recovery. Also, in general terms, a high availability PostgreSQL solution must cover four key areas: Infrastructure: This is the physical or virtual hardware database systems rely on to run. there cannot be high availability.

Availability 98

Availability Open Source Architecture Database 98

The Morning Paper

MAY 27, 2019

Last week we saw the benefits of rethinking memory and pointer models at the hardware level when it came to object storage and compression ( Zippads ). The protections are hardware implemented and cannot be forged in software. At hardware reset the boot code is granted maximally permissive architectural capabilities.

C++ 61

C++ Hardware Virtualization Benchmarking 61

SQL Performance

APRIL 27, 2020

The basic tier provides up to 5 DTUs with standard storage. The standard tier supports from 10 up to 3000 DTUs with standard storage and the premium tier supports 125 up to 4000 DTUs with premium storage, which is orders of magnitude faster than standard storage. New Hardware Configuration for Provisioned Compute Tier.

Azure 55

Azure Database Serverless Hardware 55

Abhishek Tiwari

DECEMBER 27, 2017

Containerized data workloads running on Kubernetes offer several advantages over traditional virtual machine/bare metal based data workloads including but not limited to. faster access to external storage and data locality (I/O, bandwidth). Storage provisioning. But Kubernetes storage is evolving quite quickly.

Big Data 63

Big Data Storage Benchmarking Hardware 63

The Morning Paper

FEBRUARY 18, 2020

Here are the three big directional bets that align with the three main areas cited by the authors: We will train in the cloud , where its possible to take advantage of managed infrastructure well suited to large amounts of data, spiky resource usage, and access to the latest hardware.

Government 95

Government Software Engineering Storage Database 95

The Morning Paper

DECEMBER 8, 2019

Byte-addressable non-volatile memory,) NVM will fundamentally change the way hardware interacts, the way operating systems are designed, and the way applications operate on data. Traditional pointers address a memory location (often virtual of course). At least, the nature of pointers that we want to make persistent.

Hardware 56

Hardware Programming Media Storage 56

Percona

SEPTEMBER 1, 2023

Vertical scaling is also often discussed, which involves increasing the resources of a single server, which can have limitations in hardware capabilities and become costly as demands grow. 2) Hardware limitations Disk and memory are inexpensive nowadays. An example is running MongoDB on Mesos.

Best Practices 98

Best Practices Traffic Database Servers 98

HammerDB

JUNE 27, 2022

It enables the user to measure database performance and make comparative judgements about database hardware and software. These factors meant that often when looking for database performance information, the results for a particular combination of software and hardware were not available. What is HammerDB? Why HammerDB was developed.

Benchmarking 63

Benchmarking Open Source C++ Cache 63

HammerDB

APRIL 19, 2023

Regardless of whether the computing platform to be evaluated is on-prem, containerized, virtualized, or in the cloud, it is crucial to consider several essential factors. As database performance is heavily influenced by the performance of storage, network, memory, and processors, we must understand the upper limit of these key components.

Performance 55

Performance Benchmarking Cache Storage 55

Alex Russell

APRIL 29, 2021

Chrome has missed several APIs for 3+ years: Storage Access API. Now in development in WebKit after years of radio silence, WebXR APIs provide Augmented Reality and Virtual Reality input and scene information to web applications. is access to hardware devices. Where Chrome Has Lagged. Device APIs: The Final Frontier.

Media 145

Media Games Education Engineering 145

Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on. I also wrote about these topics in detail for my recent [Systems Performance 2nd Edition] book.

Performance 52

Performance Latency Hardware Storage 52

Testsigma

JULY 3, 2021

More control: While performing on-premise testing, organizations have more control over configurations, setup, hardware, and software. The security and data storage infrastructure has to meet certain security compliances and standards; only then, the infrastructure is good to go for testing. All of these require a lot of capital.

Cloud 57

Cloud Testing Storage DevOps 57

Percona

AUGUST 27, 2018

Essentially, you can assign a thread to a specific virtual CPU. assigning to a specific CPU) is a manageable resource, represented by the concept of “virtual CPU” as a term that includes CPU cores, hyperthreads, hardware threads, and so forth. MySQL determines, at startup, how many virtual CPUs are available. References.

Tuning 69

Tuning Virtualization Testing Servers 69

HammerDB

OCTOBER 12, 2018

As is also the case this limitation is at the database level (especially the storage engine) rather than the hardware level. InnoDB is the storage engine that will deliver the best OLTP throughput and should be chosen for this test. . driver: intel_pstate CPUs which run at the same hardware frequency: 0 .

Best Practices 40

Best Practices Scalability Performance C++ 40

Brendan Gregg

SEPTEMBER 5, 2017

Linux has been adding tracing technologies over the years: kprobes (kernel dynamic tracing), uprobes (user-level dynamic tracing), tracepoints (static tracing), and perf_events (profiling and hardware counters). What happens if processes really do try to populate all that virtual memory?

Virtualization 75

Virtualization AWS Engineering Hardware 75