Apache Spark for Data Engineering and Machine Learning
About this Course
Apache® Spark™ is a fast, flexible, and developer-friendly open-source platform for large-scale SQL, batch processing, stream processing, and machine learning. Users can take advantage of its open-source ecosystem, speed, ease of use, and analytic capabilities to work with Big Data in new ways. In this short course, you explore concepts and gain hands-on skills to use Spark for data engineering and machine learning applications. You'll learn about Spark Structured Streaming, including data sources, output modes, operations. Then, explore how Graph theory works and discover how GraphFrames supports Spark DataFrames and popular algorithms. Organizations can acquire data from structured and unstructured sources and deliver the data to users in formats they can use. Learn how to use Spark for extract, transform and load (ETL) data. Then, you'll hone your newly acquired skills during your "ETL for Machine Learning Pipelines" lab. Next, discover why machine learning practitioners prefer Spark. You'll learn how to create pipelines and quickly implement features for extraction, selections, and transformations on structured data sets. Discover how to perform classification and regression using Spark. You'll be able to define and identify both supervised and unsupervised learning. Learn about clustering and how to apply the k-mean s clustering algorithm using Spark MLlib. You'll reinforce your knowledge with focused, hands-on labs and a final project where you will apply Spark to a real-world inspired problem. Prior to taking this course, please ensure you have foundational Spark knowledge and skills, for example, by first completing the IBM course titled "Big Data, Hadoop and Spark Basics."Created by: IBM
Level: Intermediate
Related Online Courses
About the Database Series of Courses "Databases" was one of Stanford's three inaugural massive open online courses in the fall of 2011. It has been offered in synchronous and then in self-paced... more
Source code management systems are where code, ci-scripts, and Infrastructure as Code (IaC) scripts are stored and managed. That means that properly protecting the SCM is an important step towards... more
Please Note: Learners who successfully complete this IBM course can earn a skill badge — a detailed, verifiable and digital credential that profiles the knowledge and skills you’ve acquired in thi... more
The modern data analysis pipeline involves collection, preprocessing, storage, analysis, and interactive visualization of data. The goal of this course, part of the Analytics: Essential Tools and... more
Want to learn how to create a digital product that is truly ready for the world? One that can be released in many markets and for different languages without any alterations to its code? This is... more