Managing a Data Mesh with Dataplex
Dataplex is an intelligent data fabric that enables organizations to centrally discover, manage, monitor, and govern their data across data lakes, data warehouses, and data marts. You can use Dataplex to build a data mesh architecture to decentralize data ownership among domain data owners. In this course, you will learn how to discover, manage, monitor, and govern your data across data lakes, data warehouses, and data marts through guided lectures and independent exercises using sample data.
What you will learn
- Identify the importance of a modern data platform
- Configure and set up Dataplex
- Secure data lakes, zones, and assets
- Implement tagging for resources and use tags to search for assets
- Process data using Dataplex tasks
- Design, execute and report on data quality processes
Prerequisites
- Completion of the "Modernizing Data Lakes and Data Warehouses with Google Cloud" and "Building Batch Data Pipelines on Google Cloud" courses in the "Data Engineer" learning path or equivalent experience using Google Cloud.
Target audience
- Customers
Training Program
7 modules to master the fundamentals
Objectives
- Identify the importance of a modern data platform
- Explain the role of Dataplex on Google Cloud
Topics covered
- →Modern Data Platforms and Data-Oriented Design
- →Pillars of Data Governance
- →What is Dataplex?
- →Dataplex Capabilities
- →Dataplex compared with other products on Google Cloud
Objectives
- Define key Dataplex concepts
- Configure and set up Dataplex
Topics covered
- →What is a data mesh?
- →Dataplex concepts
- →Creating data lakes and zones
- →Assets in Dataplex
Activities
Lab: Provision a Data Mesh using Dataplex
Objectives
- Understand different data processing options in Dataplex
- Configure and run data preparation tasks on Dataplex
Topics covered
- →Processing data on Dataplex
- →Data preparation tasks
- →Ingestion jobs
- →Dataflow and Spark tasks
Activities
Lab: Standardize Data using Dataplex Tasks
Objectives
- Secure data lakes, zones, and assets in Dataplex
Topics covered
- →IAM permissions and roles
- →Securing your data lake
- →Policy management
- →Metadata security
Activities
Lab: Manage Data Security using Dataplex
Objectives
- Implement tagging for resources and use tags to search for assets
Topics covered
- →Introduction to Data Catalog
- →Technical metadata vs. business metadata
- →Tags and tag templates
- →Entries and entry groups
- →Data lineage
Activities
Lab: Data Catalog and Data Lineage
Objectives
- Design, execute and report on data quality processes
Topics covered
- →Data quality tasks and AutoDQ
- →Reporting on data quality
- →Data profiling
Activities
Lab: Data Quality and Profiling your Data in BigQuery
Objectives
- Implement best practices for Dataplex
Topics covered
- →Best practices
- →End-to-end demo
Activities
Challenge Lab: Managing a Data Mesh with Dataplex
Quality Process
SFEIR Institute's commitment: an excellence approach to ensure the quality and success of all our training programs. Learn more about our quality approach
- Lectures / Theoretical Slides — Presentation of concepts using visual aids (PowerPoint, PDF).
- Technical Demonstration (Demos) — The instructor performs a task or procedure while students observe.
- Guided Labs — Guided practical exercises on software, hardware, or technical environments.
- Quiz / MCQ — Quick knowledge check (paper-based or digital via tools like Kahoot/Klaxoon).
The achievement of training objectives is evaluated at multiple levels to ensure quality:
- Continuous Knowledge Assessment : Verification of knowledge throughout the training via participatory methods (quizzes, practical exercises, case studies) under instructor supervision.
- Progress Measurement : Comparative self-assessment system including an initial diagnostic to determine the starting level, followed by a final evaluation to validate skills development.
- Quality Evaluation : End-of-session satisfaction questionnaire to measure the relevance and effectiveness of the training as perceived by participants.
Train multiple employees
- Volume discounts (multiple seats)
- Private or custom session
- On-site or remote