Remove Big Data Remove Data Engineering Remove Virtualization
article thumbnail

What is IT automation?

Dynatrace

IT admins can automate virtually any time-consuming task that requires regular application. This requires significant data engineering efforts, as well as work to build machine-learning models. Big data automation tools. What is IT automation? AI that is based on machine learning needs to be trained.

article thumbnail

Kubernetes for Big Data Workloads

Abhishek Tiwari

Kubernetes has emerged as go to container orchestration platform for data engineering teams. In 2018, a widespread adaptation of Kubernetes for big data processing is anitcipated. Organisations are already using Kubernetes for a variety of workloads [1] [2] and data workloads are up next. Key challenges.

article thumbnail

A case for ELT

Abhishek Tiwari

Cheap storage and on-demand compute in the cloud coupled with the emergence of new big data frameworks and tools are forcing us to rethink the whole ETL and data warehousing architecture. There is a strong argument for ELT i.e. extract, load, and transform model. Classic ETL.