In this module, we will implement a batch data processing project for music streaming data. You'll learn to use Airflow for orchestration and Redshift Serverless for storage and querying, culminating in a full pipeline execution. The focus is on understanding the interaction between orchestration tools and AWS services.

In this module, we will process music stream data using a distributed system that combines PySpark and DynamoDB. You'll use Airflow to orchestrate the workflow and execute jobs using the AWS Glue Docker image locally. This project introduces scalable and parallel data processing techniques.

In this module, we will build a robust ETL pipeline for rental apartment data. You will set up MySQL in AWS Aurora, use Glue for data transformation, and orchestrate the workflow using Step Functions and EventBridge. This lab emphasizes automation and modular pipeline execution.

In this module, we will create a datalake for a rental vehicle store using scalable services like EMR and Athena. You'll execute PySpark both locally and on the cloud, integrate metadata using Glue crawlers, and automate the pipeline using Step Functions.

In this module, we will develop an event-driven data pipeline tailored for an e-commerce application. You'll containerize Python apps, deploy them using ECS, and automate workflows using Step Functions and EventBridge. This lab blends DevOps and data pipeline principles.

In this module, we will build a lakehouse architecture combining the flexibility of data lakes and the performance of data warehouses. You will use PySpark with Delta Lake, manage metadata with Glue Catalog, and query data through Athena and Redshift.

In this module, we will implement real-time processing of taxi trip data using a serverless approach. You'll set up Kinesis streams, deploy Lambda functions, and execute a complete pipeline. This lab reinforces serverless computing and event-driven design.

In this module, we will process mobile network logs using real-time technologies and deliver interactive insights via Streamlit. You'll build and deploy dashboards to ECS, leveraging Spark for streaming data and Glue Catalog for metadata management.

In this module, we will set up CI/CD pipelines to automate deployment of AWS Glue jobs, ECS tasks, and Lambda functions using GitHub Actions. You'll learn how to build and manage version-controlled workflows for repeatable deployments.

In this module, we will ingest real-time clickstream data using Kinesis Firehose and enrich it using Lambda before storing it in Redshift. You'll build a robust pipeline suitable for web analytics or behavioral tracking applications.

In this module, we will challenge you to independently set up a MySQL database on AWS Aurora. This assignment reinforces database fundamentals and AWS RDS deployment skills.

In this module, you will independently implement a lakehouse architecture for a commercial flights dataset. This assignment consolidates your understanding of data lakes, delta tables, and metadata integration with Glue.

In this module, you'll build a real-time system that dynamically adjusts pricing for e-commerce users based on events. This assignment emphasizes practical business applications of event-driven data processing.

In this module, you'll build a real-time streaming job to process Spotify metrics. This assignment helps you apply PySpark and AWS Glue in real-world streaming scenarios.

In this final module, you'll implement CI/CD automation for Lambda functions using GitHub Actions. This assignment solidifies your DevOps knowledge and prepares you for real-world deployment automation.