Coursera Instructor Network
Building Smarter Data Pipelines: SQL, Spark, Kafka & GenAI 专项课程
Coursera Instructor Network

Building Smarter Data Pipelines: SQL, Spark, Kafka & GenAI 专项课程

Build Scalable Data Engineering Systems. Learn to design, implement, and optimize data pipelines using industry-standard tools and frameworks

Caio Avelino
Starweaver
Soheil Haddadi

位教师:Caio Avelino

包含在 Coursera Plus

深入学习学科知识
4.6

(5 条评论)

中级 等级

推荐体验

4 周 完成
在 10 小时 一周
灵活的计划
自行安排学习进度
深入学习学科知识
4.6

(5 条评论)

中级 等级

推荐体验

4 周 完成
在 10 小时 一周
灵活的计划
自行安排学习进度

您将学到什么

  • Design and implement scalable data ingestion, processing, and storage systems using Apache Kafka and Spark

  • Build high-performance data pipelines integrating cloud platforms, databases, and generative AI technologies

  • Apply data engineering best practices for enterprise-scale analytics, optimization, and real-time processing

要了解的详细信息

可分享的证书

添加到您的领英档案

授课语言:英语(English)
最近已更新!

August 2025

了解顶级公司的员工如何掌握热门技能

Petrobras, TATA, Danone, Capgemini, P&G 和 L'Oreal 的徽标

精进特定领域的专业知识

  • 向大学和行业专家学习热门技能
  • 借助实践项目精通一门科目或一个工具
  • 培养对关键概念的深入理解
  • 通过 Coursera Instructor Network 获得职业证书

专业化 - 8门课程系列

您将学到什么

  • Analyse the architecture and components of data pipelines to understand their impact on data flow and processing efficiency.

  • Implement robust ETL processes, for scalability and maintainability.

  • Analyze big data challenges and introduce Hadoop ecosystem tools (HDFS, MapReduce, Hive, Pig, and Spark) for data processing tasks.

您将获得的技能

类别:Extract, Transform, Load
类别:Big Data
类别:Data Pipelines
类别:Apache Hadoop
类别:Apache Spark
类别:Data-Driven Decision-Making
类别:Data Processing
类别:Data Warehousing
类别:Apache Hive
类别:Data Management
类别:Data Transformation
类别:Data Analysis
类别:Data Integration
类别:Scalability

您将学到什么

  • Identify and describe the components and importance of data ecosystems.

  • Understand the basic structure and function of data pipelines.

  • Recognize the steps involved in ETL workflows and their role in data handling.

  • Gain an introductory knowledge of big data and the application of Apache Spark.

您将获得的技能

类别:Extract, Transform, Load
类别:Apache Spark
类别:Data Pipelines
类别:Data Management
类别:Dataflow
类别:Data Architecture
类别:Data Integration
类别:Big Data
类别:Scalability
类别:Data Infrastructure
类别:Data Processing

您将学到什么

  • Explain the importance of data warehousing in business intelligence.

  • Design and implement effective schema designs for data warehouses.

  • Implement ETL processes to load and transform data into a data warehouse.

  • Apply performance optimization techniques to enhance data warehouse efficiency.

您将获得的技能

类别:Extract, Transform, Load
类别:Star Schema
类别:Snowflake Schema
类别:Data Warehousing
类别:Performance Tuning
类别:Data Management
类别:Databases
类别:Data Integration
类别:Data Modeling
类别:Database Design
类别:Scalability
类别:Data Transformation
类别:Business Intelligence

您将学到什么

  • Analyze and tune SQL queries to enhance SQL performance and reduce application latency.

  • Evaluate effective database index and maintenance task strategies to improve efficiency.

  • Monitor the performance of troubleshooting techniques used for resolving common SQL server issues.

  • Apply best practices for SQL Server performance to ensure consistent and reliable operations.

您将获得的技能

类别:Microsoft SQL Servers
类别:Performance Tuning
类别:Database Design
类别:Stored Procedure
类别:SQL
类别:System Monitoring
类别:Application Performance Management
类别:Database Management
类别:Scalability
类别:Network Troubleshooting
Cloud Architecture Design Patterns

Cloud Architecture Design Patterns

第 5 门课程3小时

您将学到什么

  • Show understanding of the fundamentals of cloud architecture, including key components like virtual machines, storage, and networking.

  • Identify and implement core cloud design patterns such as Load Balancer, Circuit Breaker, and Auto-Scaling to ensure scalability and reliability.

  • Demonstrate advanced cloud design patterns, including Microservices Architecture, Event-Driven Architecture, and Serverless Computing.

您将获得的技能

类别:Cloud Computing Architecture
类别:Load Balancing
类别:Microservices
类别:Serverless Computing
类别:Infrastructure As A Service (IaaS)
类别:Cloud Services
类别:Cloud Computing
类别:Cloud Platforms
类别:Software Design Patterns
类别:Software Architecture
类别:Cloud Security
类别:Cloud Applications
类别:Cloud Infrastructure
类别:Event-Driven Programming
类别:Scalability

您将学到什么

  • Identify the capabilities of GenAI for basic role specific, Data Engineer functions.

  • Examine real-world applications to leverage GenAI for streamlining work and fostering innovation in Data Engineering functions.

  • Deploy strategies and tactics to responsibly integrate GenAI into data engineering practices, while maintaining human oversight and accountability.

您将获得的技能

类别:Generative AI
类别:Data Pipelines
类别:SQL
类别:Data Modeling
类别:Data Quality
类别:Responsible AI
类别:Prompt Engineering
类别:Data Ethics
类别:Data Transformation
类别:AI Product Strategy
类别:Database Management
Apache Kafka - An Introduction

Apache Kafka - An Introduction

第 7 门课程3小时

您将学到什么

  • Describe Apache Kafka's architecture and its components, enhancing data pipeline efficiency.

  • Configure and manage Kafka clusters, ensuring high availability and fault tolerance.

  • Apply (Create and use) topics, publishers, and subscribers to facilitate real-time data exchange.

  • Implement basic stream processing applications using Kafka Streams, addressing real-world data challenges.

您将获得的技能

类别:Apache Kafka
类别:Scalability
类别:Real Time Data
类别:Data Pipelines
类别:Data Processing
类别:Performance Tuning
类别:Operational Databases
类别:System Monitoring

您将学到什么

您将获得的技能

类别:Generative AI
类别:Data Cleansing
类别:Data Quality
类别:Automation
类别:Artificial Intelligence
类别:Responsible AI
类别:Alteryx
类别:Data Transformation
类别:Data Validation
类别:Data Processing
类别:OpenAI
类别:Tensorflow

获得职业证书

将此证书添加到您的 LinkedIn 个人资料、简历或履历中。在社交媒体和绩效考核中分享。

位教师

Caio Avelino
6 门课程6,127 名学生
Starweaver
Coursera Instructor Network
446 门课程818,636 名学生
Soheil Haddadi
Coursera Instructor Network
5 门课程2,670 名学生

提供方

人们为什么选择 Coursera 来帮助自己实现职业发展

Felipe M.
自 2018开始学习的学生
''能够按照自己的速度和节奏学习课程是一次很棒的经历。只要符合自己的时间表和心情,我就可以学习。'
Jennifer J.
自 2020开始学习的学生
''我直接将从课程中学到的概念和技能应用到一个令人兴奋的新工作项目中。'
Larry W.
自 2021开始学习的学生
''如果我的大学不提供我需要的主题课程,Coursera 便是最好的去处之一。'
Chaitanya A.
''学习不仅仅是在工作中做的更好:它远不止于此。Coursera 让我无限制地学习。'
Coursera Plus

通过 Coursera Plus 开启新生涯

无限制访问 10,000+ 世界一流的课程、实践项目和就业就绪证书课程 - 所有这些都包含在您的订阅中

通过在线学位推动您的职业生涯

获取世界一流大学的学位 - 100% 在线

加入超过 3400 家选择 Coursera for Business 的全球公司

提升员工的技能,使其在数字经济中脱颖而出

常见问题