Module 1: Cloud Computing Concepts

• What is the “Cloud”?
• Why cloud services
• Types of cloud models
  • Deployment Models
  • Private Cloud deployment model
  • Public Cloud deployment model
  • Hybrid cloud deployment model
  • Microsoft Azure,
  • Amazon Web Services,
  • Google Cloud Platform
• Characteristics of cloud computing
  • On-demand self-service
  • Broad network access
  • Multi-tenancy and resource pooling
  • Rapid elasticity and scalability
  • Measured service
• Cloud Data Warehouse Architecture
• Shared Memory architecture
• Shared Disk architecture
• Shared Nothing architecture

Module 2: Core Azure services

• Core Azure Architectural components
• Core Azure Services and Products
• Azure solutions
• Azure management tools

Module 3: Security, Privacy, Compliance

• Securing network connectivity
• Core Azure identity services
• Security tools and features
• Azure Governance methodologies
• Monitoring and reportingS
• Privacy, compliance, and data protection standards

Module 4:Azure Pricing and Support

• Azure subscriptions
• Planning and managing costs
• Azure support options
• Azure Service Level Agreements (SLAs)
• Service Lifecycle in Azure

Module 5: Introduction to Azure Databricks

• Introduction to Databricks
• Azure Databricks Architecture
• Azure Databricks Main Concepts

Module 6:Azure Databricks Account Creation

• Azure Free Account
• Free Subscription for Azure Databricks
• Create Databricks Community Edition Account

Module 7:Databricks Cluster Types and Notebook Options

• Creating and configuring clusters
• Create Notebook
• Quick tour on notebook options

Module 8:Databricks Utilities and Notebook Parameters

• Dbutils commands on files, directories
• Notebooks and libraries
• Databricks Variables
• Widget Types
• Databricks notebook parameters

Module 9:Databricks CLI

• Azure Databricks CLI Installation
• Databricks CLI – DBFS, Libraries and Jobs

Module 10:Databricks Integration with Azure Blob Storage

• Read data from Blob Storage and Creating Blob mount point

Module 11:Databricks Integration with Azure Data Lake Storage Gen2

• Reading files from Azure Data Lake Storage Gen2

Module 12:Databricks Integration with Azure Data Lake Storage Gen1

• Reading Files from data lake storage Gen1

Module 12:Databricks Integration with Azure Data Lake Storage Gen1

• Reading Files from data lake storage Gen1

Module 13:Reading and Writing CSV files in Databricks

• Read CSV Files
• Read TSV Files and PIPE Seperated CSV Files
• Read CSV Files with multiple delimiter in spark 2 and spark 3
• Reading different position Multidelimiter CSV files

Module 14:Reading and Writing Parquet files in Databricks

• Read Parquet files from Data Lake Storage Gen2
• Reading and Creating Partition files in Spark

Module 15:Parsing Complex Json FilesL

• Reading and Writing JSON Files
• Reading, Transforming and Writing Complex JSON files

Module 16:Reading and Writing ORC and Avro Files

• Reading and Writing ORC and Avro Files

Module 17:Databricks Integration with Azure Synapse

• Reading and Writing Azure Synapse data from Azure Databricks

Module 18:Databricks Integration with Amazon Redshift(Redshift)

• Read and Write data from Redshift using databricks

Module 19:Databricks Integration with Snowflake

• Reading and Writing data from Snowflake

Module 20:Databricks Integration with CosmosDB SQL API

• Reading and Writing data from Azure CosmosDB Account

Module 21:Python Introduction

• Python Introduction
• Installation and setup
• Python Data Types for Azure Databricks

Module 22:Python Data Types

• Deep dive into String Data Types in Python for Azure Databricks
• Deep dive into python collection list and tuple
• Deep dive on set and dict data types in python

Module 23:Python Functions and Arguments

• Python Functions and Arguments
• Lambda Functions

Module 24:Python Modules and Packages

• Python Modules and Packages

Module 25:Python Flow Control

• Python Flow Control
• For-Each
• While

Module 25:Python Flow Control

• Python Flow Control
• For-Each
• While

Module 26:Python File Handling

• Python File Handling

Module 27:Python Logging Module

• Python Logging Module

Module 28:Python Exception Handling

• Python Exception Handlings

Module 29:Pyspark Introduction

• Pyspark Introduction
• Pyspark Components and Features

Module 30:Spark Architecture and Internals

• Apache Spark Internal architecture
• jobs stages and tasks
• Spark Cluster Architecture Explained

Module 31:Spark RDD

• Different Ways to create RDD in Databricks
• Spark Lazy Evaluation Internals & Word Count Program
• RDD Transformations in Databricks & coalesce vs repartition
• RDD Transformation and Use Cases

Module 32:Spark SQL

• Spark SQL Introduction
• Different ways to create DataFrames

Module 33:Spark SQL Intenals

• Catalyst Optimizer and Spark SQL Execution Plan
• Deep dive on Sparksession vs sparkcontext
• spark SQL Basics part-1
• RDD Transformation and Use Cases

Module 34:Spark SQL Basics

• Spark SQL Basics Part-2
• Joins in Spark SQL

Module 35:Spark SQL Functions and UDFs

• Spark SQL Functions part-1
• Spark SQL Functions part-2
• Spark SQL Functions Part-3
• Spark SQL UDFs
• Spark SQL Temp tables and Joins

Module 36:Databricks Delta and Implementing Dimensions SCD1 and SCD2

• Implementing SCD Type1 and Apache Spark Databricks Delta
• Delta Lake in Azure Databricks
• Implementing SCD Type with and without Databricks Delta

Module 37:Databricks Integration with Azure Data Factory

• Azure Data Factory Integration with Azure Databricks

Module 38:Databricks Streaming

• Delta Streaming in Azure Databricks
• Data Ingestion with Auto Loader in Azure Databricks

Module 39:Azure Databricks Projects

• Azure Databricks Project-1
• Azure Databricks Project-2

Module 40:Databricks Integration with Azure Devops

• Azure Databricks CICD Pipelines