Executive Summary
An organization is entitled to identify, define, model, and standardize its data as part of a data management framework. However, defining industry-specific and standardized data dictionaries can be an excruciating task. Many organizations hesitate to establish a good data management program without a good business case and, in turn, lack support from leadership. A clean and standardized data template can jump start a data management and governance program and lay the foundation for an Enterprise data architecture.
Definian has assembled a retail industry-specific template with Microsoft Azure Purview. This template includes a standardized set of data definitions, regulations, and policies to kick-start a data governance program with Azure Purview. The retail Industry template can be enhanced and molded as per an organization’s requirements to build a solid foundation and is also easy to adapt, customize, and implement.
Along with the retail industry-specific template, a common theme we have heard from many of our customers is an interest in having Azure Purview co-exist with other data governance platforms. We have developed an integration layer for data management platforms (Meta-mesh), which harmonizes information across various platforms, including Azure Purview and Collibra Data Intelligence Cloud Platform. This solution is built on Azure and can integrate other enterprise-wide data sources.
Retail Industry Templates
Inventory of Retail Specific Business Terms
We have built an inventory with over 100 retail glossary terms listed, defined, and enriched with attributes to carry out data governance in Azure Purview. The glossary is described in business terms with a standard format for ease of understanding, whether the consumer of this data is from a business or technical background. Critical Data Elements (CDE's) and Personally Identifiable Information (PII) are identified and tagged accordingly within the glossary (see Figure 1).
Data Quality Rules
This template includes a set of data quality rules for completeness and validity. You can extend this template to cover rules to measure accuracy, uniqueness, integrity, and consistency (see Figure 2).
Figure 3 illustrates the association of the data quality rule for checking the completeness of the Data Quality Rule to the business terms from the PII and Retail Industry Glossaries.
Inventory of Governance Policies
This template includes an inventory of the policies as per the standard Data Governance framework. These policies are categorized based on specific domains like data quality and stewardship (see Figure 4).
Figure 5 illustrates how a Data Governance Policy for 'Sensitive Data Management' governs the business terms from the respective parent glossaries.
Inventory of California Consumer Privacy Act (CCPA) Regulation
The most critical aspect of data governance is to ensure compliance with relevant regulations. Retail industry templates have defined around 100 privacy clauses for the California Consumer Privacy Act (CCPA) (see Figure 6).
Figure 7 illustrates the glossary terms from Retail industry that are regulated by the CCPA regulation citation (1798-100)(b).
Metadata Exchange Platform (Meta-mesh)
Definian has built a Metadata Exchange Platform, an integration layer for data management platforms. This platform is built on Azure and uses the Azure Event hub to seamlessly move metadata from one platform to another. This platform also integrates Azure Purview and Collibra Data Intelligence, providing a unidirectional flow of metadata such as a glossary, data quality rules, policies, and regulations from Collibra to Azure Purview. This solution is available as a single-app service in Azure. The meta-mesh platform is flexible to use the Kafka data pipeline should an organization choose. Near real-time visualization of data in these pipelines is achievable using Stream Analytics on Azure or integrating third-party tools if an organization chooses to track and report the metadata quality moving between data management platforms.
Figure 8 shows an inventory of approved business terms in Collibra Data Intelligence cloud platform.
Figure 9 shows the migration of the glossary terms from Collibra to Azure Purview using Meta-mesh. The platform provides the ability to map out-of-the-box as well the custom operating model (attributes and relationships) in Collibra to Purview.
