Rutgers Classifieds>Rutgers Online Courses>Building ETL and Data Pipelines with Bash, Airflow and Kafka

Building ETL and Data Pipelines with Bash, Airflow and Kafka

About this Course

Well-designed and automated data pipelines and ETL processes are the foundation of a successful Business Intelligence platform. Defining your data workflows, pipelines and processes early in the platform design ensures the right raw data is collected, transformed and loaded into desired storage layers and available for processing and analysis as and when required. This course is designed to provide you the critical knowledge and skills needed by Data Engineers and Data Warehousing specialists to create and manage ETL, ELT, and data pipeline processes. Upon completing this course you’ll gain a solid understanding of Extract, Transform, Load (ETL), and Extract, Load, and Transform (ELT) processes; practice extracting data, transforming data, and loading transformed data into a staging area; create an ETL data pipeline using Bash shell-scripting, build a batch ETL workflow using Apache Airflow and build a streaming data pipeline using Apache Kafka. You’ll gain hands-on experience with practice labs throughout the course and work on a real-world inspired project to build data pipelines using several technologies that can be added to your portfolio and demonstrate your ability to perform as a Data Engineer. This course pre-requisites that you have prior skills to work with datasets, SQL, relational databases, and Bash shell scripts.

Created by: IBM

Level: Introductory


Related Online Courses

Modern software is under constant attack, but many software developers have never been told how to effectively counter those attacks. This course works to solve that problem, by explaining the... more
Site Reliability Engineers must have the right tools and strategies to perform in a fast-paced technical environment. Nine competency areas guide the successful practice of IBM Cloud SREs. ● A... more
The Chief Information Security Officer (CISO) in any given organization serves a leadership position, protecting the data and digital systems that a company’s employees as well as its customers d... more
How do you protect the critical data that is increasingly being stored in the cloud? Learn how to build a security strategy that keeps data safe and mitigates risk. In this course, part of the... more
"Introduction to WebAssembly" is designed for web, cloud, and blockchain developers, architects, and CTOs interested in learning about WebAssembly’s strengths and limitations, and its potential a... more

CONTINUE SEARCH

FOLLOW COLLEGE PARENT CENTRAL