This program equips you with the open-source tools and architectural thinking used by professional data engineers to build scalable, reliable data systems from the ground up. You will work hands-on with Apache Spark for distributed data processing, dbt for modular SQL-based transformation, and Apache Airflow for workflow orchestration — the same stack powering data infrastructure at leading technology and data-driven organizations worldwide.
Across the courses, you will gain practical expertise in designing dimensional data models, implementing incremental load strategies, optimizing Spark job performance, enforcing data quality with automated testing frameworks, and deploying pipelines through CI/CD workflows. You will also develop foundational skills in cloud storage provisioning, containerization with Docker, and version control best practices that mirror real production environments.
By the end of this Program, you will be able to design and deploy end-to-end data pipelines that ingest from diverse sources, transform data through well-tested models, and deliver analytics-ready datasets to downstream consumers — demonstrating job-ready engineering skills valued across analytics engineering, data platform, and data infrastructure roles.
应用的学习项目
Throughout this Program, you will complete hands-on projects that mirror real production data engineering challenges — from building modular ETL pipelines that ingest CRM and streaming data into a cloud data warehouse, to authoring Airflow DAGs with retry logic and SLA monitoring, to diagnosing Spark performance bottlenecks and implementing Delta Lake versioning. Each project asks you to work in your own development environment, producing portfolio-ready artifacts that demonstrate your ability to design, optimize, and deploy reliable data infrastructure using open-source tools.
