Adopt Data Mesh in Microsoft Fabric

This blog post explains how to adopt the data mesh principal in Microsoft Fabric & Power BI. Let me start explaining what data mesh is first and then we can check how Microsoft Fabric can help adopt data mesh principals in the organizations.

We used to build IT centric architecture for the organization where the data across different sources, domains are managed and governed centrally. The main motive of this approach was to maintain a single version of the truth and avoid data silos. This approach was widely followed by different organizations and that was good on initial stages but later organizations were started facing challenges an accessing the right data at right time because the people who need to access the data from centralized data architecture should have data + domain knowledge. Getting the right date for analytics was time taken process as they need to connect with different people to understand and identify the data sources.

Data mesh architecture was introduced to solve the problems of IT centric data architecture. It helps the organization to build a decentralized data architecture that organizes the data by BU, LOB and domains such as sales, human resources, marketing etc. It enables the organization to provide full control to domain experts who can manage and maintain their data which again helps to build analytics faster than the before. This federated model can be organized according to business needs.

Data Mesh architecture comes with following principals,

1. Domain centric architecture – Domain owners are owners who can decide the data architecture of their domain on how to manage and maintain a platform, databases, pipelines etc.
2. Data as a Product – Domain owners are responsible for the data (product) who should ensure the quality, availability and etc.
3. Self Service Access – It should enable easy self service through BI tools, APIs and etc. At the same time, data should be accessible every from other domains whenever is required
4. Data Governance – Security & governance should be applied to ensure right people can access right data.

Today, organizations are handling huge amounts of data and there is a high need to able to organize and manage the data in a logical way of grouping that should facilities targeted and efficient use and governance.

Let’s talk about how Microsoft Fabric can handle the data mesh architecture.

Microsoft Fabric has released a feature called “Domains”. This is in public preview now (July 2023). As of now, it supports organizing the data and artifacts into logical grouping called domains and enabling the data consumers to filter and find the content by domain.

In future, we can see more governance and security level settings released at domain level. This will enable organisations to control the access at domains level.

Refer – https://learn.microsoft.com/en-us/fabric/governance/domains

How Microsoft Fabric Domain Work?

Domains are nothing but a logical grouping and it allows to group multiple workspaces. For example, if you are a sales domain expert and you and your team has multiple workspaces created in Power BI and hosted datasets and reports related to the Sales domain. All your workspaces can go under one domain called “Sales”. In the same way different domains can be created and assign the workspaces.

Follow the Microsoft documentation on how to create a domains.

https://learn.microsoft.com/en-us/fabric/governance/domains#create-a-domain

One of the main objective of the data mesh architecture is to enable the data for self-service users on the domain and across domain.

Once we created the domains and assign the workspaces, the same will be available under the “OneLake data hub”

Users can select different domains if they are part of different workspaces across domains.

Users will start see the contents which are available on workspaces grouped under the select domains.

OneLake data hub will display all the below artifacts for users to access.

1. Datasets
2. Lakehouse
3. SQL endpoint
4. Warehouse

Recently added the video version of Microsoft Fabric domain for everyone to understand the concepts easily.

Share your feedback to help enhance my blog. Thanks. Happy Learning!!