Building Modern Data Applications Using Databricks Lakehouse

Building Modern Data Applications Using Databricks Lakehouse

位教师：Packt - Course Instructors

访问权限由 Coursera Learning Team 提供

10个模块

深入了解一个主题并学习基础知识。

初级等级

推荐体验

1 周完成

在 10 小时一周

灵活的计划

自行安排学习进度

10个模块

深入了解一个主题并学习基础知识。

初级等级

推荐体验

1 周完成

在 10 小时一周

灵活的计划

自行安排学习进度

您将学到什么

Deploy near-real-time data pipelines using Delta Live Tables
Orchestrate data pipelines with Databricks workflows
Implement data validation and monitor data quality

您将获得的技能

要了解的详细信息

可分享的证书

添加到您的领英档案

作业

10 项作业

授课语言：英语（English）

了解顶级公司的员工如何掌握热门技能

了解关于 Coursera for Business 的更多信息

Petrobras, TATA, Danone, Capgemini, P&G 和 L'Oreal 的徽标

该课程共有10个模块

In today’s data-driven world, building scalable and efficient data applications is crucial for staying ahead in business and technology. This course explores the power of Databricks Lakehouse, a unified platform for managing and analyzing large volumes of data, and guides you through essential skills to create modern data applications.

Throughout the course, you’ll learn to work with Delta Live Tables for data transformation, management, and quality assurance. You will also dive deep into Databricks’ Unity Catalog for enhanced governance, data lineage, and location management. The hands-on experience with deploying and maintaining DLT pipelines using Terraform prepares you for real-world data infrastructure challenges. This course stands out by combining theoretical understanding with practical, real-world applications. You’ll gain a robust set of skills in data pipeline management, governance, and monitoring, preparing you for building production-level data applications with Databricks Lakehouse. Designed for professionals looking to deepen their expertise in modern data architecture, this course is suitable for data engineers, data scientists, and IT professionals who want to leverage Databricks to solve real-world data problems.

In this section, we explore real-time data pipelines with Delta Live Tables (DLT), analyze Delta Lake architecture, and design scalable streaming solutions for lakehouse environments

涵盖的内容

2个视频6篇阅读材料1个作业

2个视频总计2分钟

Course Overview 1分钟
An Introduction to Delta Live Tables - Overview Video 1分钟

6篇阅读材料总计60分钟

Introduction 10分钟
The Databricks Lakehouse 10分钟
Views 10分钟
Unity Catalog 10分钟
Supporting Concurrent Table Reads and Writes 10分钟
Tracking Table Changes Using Change Data Feed 10分钟

1个作业总计10分钟

Delta Live Tables Fundamentals 10分钟

In this section, we cover ingesting data with DLT, applying changes, and configuring pipelines for scalability.

涵盖的内容

1个视频6篇阅读材料1个作业

1个视频总计1分钟

Applying Data Transformations Using Delta Live Tables - Overview Video 1分钟

6篇阅读材料总计70分钟

Introduction 10分钟
Using Auto Loader with DLT 10分钟
Publishing Datasets to Unity Catalog 10分钟
Data Pipeline Settings 10分钟
Pipeline Cluster Types 10分钟
Loading External Dependencies 20分钟

1个作业总计10分钟

Mastering Delta Live Tables Pipelines 10分钟

In this section, we explore implementing data quality expectations in DLT pipelines, validating data integrity with temporary datasets, and quarantining poor-quality data for correction.

涵盖的内容

1个视频4篇阅读材料1个作业

In this section, we cover scaling DLT pipelines through cluster optimization, autoscaling, and Delta Lake techniques.

涵盖的内容

1个视频4篇阅读材料1个作业

In this section, we explore implementing data governance in a lakehouse using Unity Catalog, focusing on access controls, data discovery, and lineage tracking for compliance and security.

涵盖的内容

1个视频7篇阅读材料1个作业

1个视频总计1分钟

Mastering Data Governance in the Lakehouse with Unity Catalog - Overview Video 1分钟

7篇阅读材料总计55分钟

Introduction 5分钟
A Problem Worth Solving 10分钟
Unity Catalog-Enabled Cluster Types 5分钟
Enabling Unity Catalog on an Existing Databricks Workspace 5分钟
Data Discovery and Cataloging 10分钟
Observability with System Tables 10分钟
Hands-on Example Data Masking Healthcare Datasets 10分钟

1个作业总计10分钟

Data Governance in the Lakehouse with Unity Catalog 10分钟

In this section, we cover managing data storage locations in Unity Catalog with secure governance and access control.

涵盖的内容

1个视频3篇阅读材料1个作业

In this section, we explore data lineage in Unity Catalog, tracing origins, visualizing transformations, and identifying dependencies to ensure data integrity and proactive issue detection.

涵盖的内容

1个视频3篇阅读材料1个作业

In this section, we cover deploying and managing DLT pipelines using Terraform in Databricks.

涵盖的内容

1个视频4篇阅读材料1个作业

In this section, we explore Databricks Asset Bundles (DABs) for streamlining data pipeline deployment, emphasizing GitHub integration, version control, and cross-team collaboration.

涵盖的内容

1个视频4篇阅读材料1个作业

1个视频总计1分钟

Leveraging Databricks Asset Bundles to Streamline Data Pipeline Deployment - Overview Video 1分钟

4篇阅读材料总计60分钟

Introduction 20分钟
Databricks Asset Bundles in Action 20分钟
Hands-on Exercise Deploying Your First DAB 10分钟
Hands-on Exercise Simplifying Cross-Team Collaboration with GitHub Actions 10分钟

1个作业总计10分钟

Streamlining Data Pipeline Deployment with Databricks 10分钟

In this section, we explore monitoring data pipelines using Databricks, focusing on health, performance, and data quality. Techniques include DBSQL alerts and webhook triggers for real-time issue resolution.