The Morning Paper

OCTOBER 26, 2020

Helios also serves as a reference architecture for how Microsoft envisions its next generation of distributed big-data processing systems being built. What follows is a discussion of where big data systems might be heading, heavily inspired by the remarks in this paper, but with several of my own thoughts mixed in.

Cloud 104

Cloud Big Data Latency Architecture 104

All Things Distributed

DECEMBER 5, 2010

There are two main types of DNS servers: authoritative servers and caching resolvers. But the real robustness of the DNS system comes through the way lookups are handled, which is what caching resolvers do. Caching techniques ensure that the DNS system doesnt get overloaded with queries. Countdown to What is Next in AWS.

Cloud 111

Cloud Internet Internet AWS 111

All Things Distributed

JUNE 20, 2017

Beyond running their web properties and applications, Next Digital also uses Amazon RDS (database), Amazon ElastiCache (caching), and Amazon Redshift (data warehousing). Next Digital operates on AWS in a more highly available and fault-tolerant environment than their previous colocation solution.

AWS 125

AWS Logistics Cloud Social Media 125

InfoQ

JULY 3, 2023

LinkedIn introduced Couchbase as a centralized caching tier for scaling member profile reads to handle increasing traffic that has outgrown their existing database cluster. The new solution achieved over 99% hit rate, helped reduce tail latencies by more than 60% and costs by 10% annually. By Rafal Gancarz

Cache 86

Cache Latency Traffic Database 86

All Things Distributed

APRIL 27, 2011

We use high-performance transactions systems, complex rendering and object caching, workflow and queuing systems, business intelligence and data analytics, machine learning and pattern recognition, neural networks and probabilistic decision making, and a wide variety of other techniques. Driving down the cost of Big-Data analytics.

Technology 127

Technology Technology AWS Storage 127

All Things Distributed

AUGUST 17, 2011

But while this blog happily runs out of S3, the process of creating and updating the content still required a server to run my Moveable Type installation and hold the database. My templates and blog posts are now located in DropBox and thus locally cached at each machine I use. Job Openings in AWS - Senior Leader in Database Services.

Servers 109

Servers Social Media AWS Website 109

The Morning Paper

JUNE 23, 2019

Coupled with stateless application servers to execute business logic and a database-like system to provide persistent storage, they form a core component of popular data center service archictectures. If you want to store time-expiring data that should be shared across application processes, used Memcached or Redis.

Cache 79

Cache Latency Google Network 79

The Morning Paper

MAY 14, 2019

Seer: leveraging big data to navigate the complexity of performance debugging in cloud microservices Gan et al., Seer uses a lightweight RPC-level tracing system to collect request traces and aggregate them in a Cassandra database. ASPLOS’19. Distributed tracing and instrumentation.

Big Data 45

Big Data Cloud Performance Hardware 45

Abhishek Tiwari

DECEMBER 27, 2017

ETL refers to extract, transform, load and it is generally used for data warehousing and data integration. ETL is a product of the relational database era and it has not evolved much in last decade. There are several emerging data trends that will define the future of ETL in 2018. Machine learning meets data integration.

Big Data 49

Big Data Artificial Intelligence Storage Hardware 49

The Morning Paper

SEPTEMBER 19, 2019

MongoDB is an important database, and this paper explains the tunable (per-operation) consistency models that MongoDB provides and how they are implemented under the covers. Microsoft have a paper describing their new recovery mechanism in Azure SQL Database , the key feature being that it can recovery in constant time.

Blockchain 46

Blockchain Hardware Google Speed 46

ScaleOut Software

NOVEMBER 24, 2020

It sends messages over the cell network to the telematics system, which uses its compute servers (that is, web and application servers) to store incoming messages as snapshots in an in-memory data grid , also known as a distributed cache. The results of batch analysis are typically produced after an hour’s delay or more.

Analytics 52

Analytics Architecture Scalability Software Architecture 52

The Netflix TechBlog

OCTOBER 15, 2019

Part I: Overview Andreas Andreakis , Falguni Jhaveri , Ioannis Papapanagiotou , Mark Cho , Poorna Reddy , Tongliang Liu Overview It is a commonly observed pattern for applications to utilize multiple datastores where each is used to serve a specific need such as storing the canonical form of data (MySQL etc.), caching (Memcached etc.),

Transportation 199

Transportation Architecture Processing Storage 199