1 Spark & Scala Foundation

What is Spark 

Difference between Spark vs MapReduce

Spark Jobs 

Development & Runtime Environment

Spark Installation on Windows Introduction 

Java 8 Installation 

Scala Installation 

Spark Installation 

Spark and Scala Programming Introduction 

Interactive mode introduction

First Spark Example in Interactive Mode 

What is Script Mode

First Spark Example in Script Mode 

Differences between Interactive and Script Mode 

2 Scala for Spark

Introduction 

Advantages 

Installation 

Programming Introduction 

Script Mode 

Variables

Datatypes

Simple Datatypes 

Scala Typecasting 

Scala Strings 

Scala Complex Datatypes

Scala Complex Datatypes  Array

Tuple 

List 

Set

MutableSet 

Map

Scala Functions

Scala OOPs 

Scala Inheritance 

3. Spark Core

Spark Core Introduction 

First Spark Core Program

What is RDD 

RDD Programming Introduction 

What is SparkContext 

How to create RDD 

Creating RDD by loading local file 

Creating RDD by loading HDFS file 

Creating RDD by using makeRDD 

Creating RDD by using parallelize 

Operations on RDD 

What are Transformations and Actions 

map Transformation

filter Transformation

flatMap Transformation 

reduceByKey Transformation

Spark WebUI

Spark Partitions 

— What is Persistence

— How to Persistence

Resilience vs Persistence 

Spark Set Operators 

— Union 

— Intersection

— Subtraction

— Cartesion

Spark Aggregation Operators 

— reduce aggregation 

— fold aggregation 

— aggregate aggregation

Pair RDD 

MapPartition RDD and Shuffled RDD 

4. Spark SQL

What is SparkSQL 

SparkSQL Programming Introduction

What is DataFrame 

Create DataFrame by loading csv 

Spark Datatypes

Custom Schema

DataFrame by Loading JSON 

DataFrame by Loading XML

hdfs integration 

RDBMS Integration

SparkSession 

Hive Integration 

DSL Introduction 

DSL Examples

DSL  filter 

DSL withColumn 

Native SQL Introduction

Native SQL Examples 

5. Spark Streaming

What is Spark Streaming 

Batch Processing vs Stream Processing 

Important Points of Spark Streaming

How to Write Spark Streaming Application 

Task Reading Data from Network Sockets 

Read files from Specified directory