PySpark

1. PySpark Foundation

     Pyspark 

1_2. PySpark Installation Introduction 

1_3. Java 8 Installation on Windows 

1_4. python installation on windows 

1_5. Spark Installation part 4 

1_6. Script Mode Introduction 

1_7. First PySpark Application in Sript Mode 

1_8. How to Start Interactive Mode in IDE 

1_9. Difference between Interactive Mode and Script 

2 Python for PySpark

 1 What is Python 

 What is Scripting Oriented Programming 

 What is Functional Oriented Programming 

 What is Object Oriented Programming 

 Advantages of Python 

 Python Installation on Windows

 Anaconda Installation on Windows

 PyCharm IDE Installation 

 Python Programming Introduction 

 Python Programming Introduction

 Python Programming Syntax 

 Python Datatypes Introduction 

 Python Simple Datatypes 

 Python Collection Datatypes

 Python Control Statements

 Python functions

3. PySpark Core

PySpark Core Introduction

SparkContext

RDD Introduction 

Different ways of Creating RDD 

Create RDD by Loading File

Creating RDD using Local Collection 

Creating RDD by Performing Transformations 

Creating RDD by Loading HDFS File

RDD Operations

Transformations Introduction 

Actions Introduction 

map() transformation 

flatMap() transformation 

reduceByKey() 

transformation

Spark Architecture part 1 

Spark Architecture

 — Local Mode part 

Spark Web UI 

Allocating Cores to application 

Partitions in Spark 

Persistence Spark 

Set Operations in Spark Introduction 

— Union 

— Intersection

— Subtraction

— Cartesian

— Aggregation Operations

— reduce 00:11:2132 

 — fold 00:08:3033

 — aggregate 00:12:0834

4  RDD 

SparkSession 

SparkSession using PyCharm IDE 

 Persistence Spark 

5.PySpark SQL

what is spark sql

How to write PySpark SQL Scripts 

What is DataFrame 

Processing in PySparkSQL 

Creating DataFrame by Loading csv example1 

Creating Our Own Schema using Spark SQL Datatypes

Creating DataFrame by loading HDFS File 

Create DataFrame by Loading JSON File 

Create DataFrame by reading RDBMS table 

DSL Language 

Native SQL introduction 

Temporary Table 

Native SQL Permanent Table 

Hive Introduction

Hive Architecture 

Hive Spark Integration Introduction 

Hive Integration Example

6.PySpark Streaming

What is PySpark Streaming 

 Batch Processing vs Streaming Processing 

 Important points of Spark Streaming

 How to write PySpark Streaming Applications 

 Task1 read streams from network port

 Task read files from directory 

 Task Process files

PySpark Windowing 

 Integrating PySpark 

submitting spark application standalone master slave