Since the beginning of the 21st century, we’ve been experiencing a true digital revolution. The world is constantly being digitized and human activity is increasingly structured around data and network services. The manufacturing, leisure, administration, service, medical, and so many other industries are now organized around complex and interconnected information systems. As a result, more and more data is continuously collected by the devices and technologies present in our daily lives (Web, Smartphone, IoT) and transited from system to system. It has become central for any company that provides products or services to do everything possible to protect the data of their customers. The best approach to do so is through Privacy by Design.

In this article, we explain what Privacy by Design is, how we applied this approach in the design of our data catalog, as well as how a data catalog can help companies implement Privacy by Design.

Data Protection: A Key Issue for Enterprises

Among all the various data mentioned above, some allow the direct or indirect identification of physical persons. These are known as personal data, as defined by the CNIL. It is of paramount importance in the modern world because of its intrinsic value.

On a daily basis, huge volumes of personal data pass between individuals, companies, and governments. There is a real risk of their misuse, as the Cambridge Analytica scandal in 2015 showed, for example. Cybercriminals can also make substantial gains from it, via account hacking, reselling data to other cybercriminals, identity theft, or attacking companies via phishing or president scams. For example, a real estate developer was recently robbed of several tens of millions of euros in France.

The need to protect data has never been so important.

States have quickly become aware of this issue to protect individuals from the abuses related to the exploitation of their data. In Europe, for example, the GDPR (the General Data Protection Regulation) has been in effect since 2016 and is already well established in the daily activities of companies. In the rest of the world, regulations are constantly evolving and are a concern for nearly every country. Recently, California passed a consumer data privacy law, a U.S. equivalent of the GDPR. Even China has just legislated on this topic.

Privacy by Design: Defining a Key Concept for Data Protection

While many legislations rely heavily on the notion of Privacy by Design, it was conceptualized by Ann Cavoukian in the late 1990s when she was the Information and Privacy Commissioner of the Province of Ontario in Canada. The essence of this idea is to include the issue of personal data protection right from the design of a computer system.

In this sense, Privacy by Design lists seven fundamental principles:

Proactivity: Any company must put in place the necessary provisions for data protection upstream, and must not rely on a reactive policy;

Personal data protection as a default setting: Any system must take as a default setting the highest possible level of protection for the sensitive data of its users;

Privacy by design: Privacy should be a systematically studied and considered aspect of the design and implementation of new functionality;

Full functionality: No compromise should be made with security protocols or with the user experience;

End-to-end security: The system must ensure the security of data throughout its lifecycle, from collection to destruction (including if the data is outsourced);

Visibility and transparency: The system and the company must document and communicate personal data protection procedures and actions taken in a clear, consistent and transparent manner;

Respect for user privacy: Every design and implementation decision must be made with the user’s interest at the center.

The Application of Privacy by Design

We’ve built our product on the foundations of Privacy by Design.

The Treatment of Users’ Personal Data

First of all, we have anchored data protection at the heart of our architecture. Each of our customer’s data is segregated into different tenants, each encrypted with their own key. User authentication is managed through a specialized third-party system. We encourage identity federation among our customers, which allows them to maintain control over the data needed for user identification and authentication.

We have also included the concept of Privacy by Design in the design of our application. For example, we collect only the bare minimum of information, all system outputs are anonymized (logs, application errors, APIs).

Processing Customer Business Data

Our main mission being to document the data, our solution contains by essence metadata. By design, the Actian Data Intelligence Platform does not extract any data from our customers’ systems. Indeed, the risk is intrinsically less on the metadata than on the data.

Nevertheless, we offer within the platform, several features allowing us to provide information on the data present in the client systems (statistics, sampling, etc.). Because of our architecture, the calculations are always done on the client’s infrastructure, as close as possible to the data and its security. And in compliance with principle #2 of Privacy by Design, we have set the protection of personal data as a default setting. Thus, all these features are disabled by default and can only be activated by the customer.

How Our Data Catalog Helps Companies Implement Privacy by Design

Our data catalog can help your company implement Privacy by Design, especially on the control and verification aspects. Taking the 7 principles described earlier, the data catalog can effectively participate in two of them: the visibility and transparency principle, and the end-to-end security principle. The data catalog also enables the automation of the identification of sensitive data.

Visibility and Transparency via the Data Catalog

The objective of a data catalog is to centralize a company’s data assets, document them, and share them with as many people as possible. This centralization allows each employee to know what data is collected by the CRM, and the marketing and customer success teams to process this information in the acquisition and churn tracking reports.

Once this inventory has been established, the catalog can be used to document certain additional information that is necessary for the company’s proper functioning. This is notably the case for the sensitive or non-sensitive nature of the documented information, the rules of governance, the processing, or the access procedures that must be applied.

In the context of a Privacy by Design approach, the data catalog can be used to add a business term corresponding to sensitive data (a social security number, a telephone number, etc.). This business term can then be easily associated with the tables or physical fields that contain the data, thus allowing its easy identification. This initiative contributes to the principle of visibility and transparency of Privacy by Design.

End-to-End Security via the Data Catalog

The data catalog also provides data lineage capabilities. Automatic data lineage ensures that the processes applied to data identified as sensitive comply with what is defined by the company’s data governance. It is then simple with the data catalog to fill in the governance rules to be applied to sensitive data.

Moreover, the lineage allows us to follow the whole life cycle of the data, from its creation to its final use, including its transformations. This makes it easy to check that all the stages of this life cycle comply with the rules and correct any errors.

The data catalog, via the data lineage, thus contributes to the principle of the end-to-end security of Privacy by Design.

With that said, we remain convinced that a data catalog is not a compliance solution, but rather a tool for the acculturation of teams to sensitive data and its particularities of use.

Identifying Sensitive Data via the Data Catalog

In a rapidly changing data environment, the data catalog must reflect reality as much as possible in order to maintain the trust of its users. Without this, the entire adoption of the data catalog project is put into question.

We are firmly convinced that the data catalog must be automated as much as possible to be scalable and efficient. This starts with the inventory of available data. In this sense, our inventory is automated and is responsible for passing on all modifications to the original system (source) of the data directly into the catalog. Thus, at any time, the customer has an exhaustive list of the data present in its systems.

And to help our customers identify which of the inventoried data deserve special treatment because of their sensitive data status, the automation does not stop at the inventory. We now offer a system that suggests the tagging of new data inventoried with a sensitive data profile. This makes it easier to bring this data to the forefront and to spread the information faster and more easily throughout the company.

Conclusion

In the past few years, personal data has become a real concern for most consumers. More and more countries are setting up regulations to guarantee their citizens maximum protection. One of the major principles governing all these regulations is Privacy by Design.

We have from the start included the reflection around personal data at the heart of our product. Both in our technical development and in the processing of our users’ data, as well as in our reflection on the data that our clients process via our catalog. 

We believe that a data catalog can be a significant asset in the implementation and monitoring of Privacy by Design policies. We also heavily rely on automation and AI to bring many more improvements in the upcoming months: automatic construction of technical data lineage, improved detection of sensitive data in the catalog objects to better document them, quality control of processes applied to sensitive data, etc. The possibilities are numerous.

To learn more about the advantages of the catalog in the management of your sensitive and personal data, don’t hesitate to schedule a meeting with one of our experts.

About Actian Corporation

Actian empowers enterprises to confidently manage and govern data at scale, streamlining complex data environments and accelerating the delivery of AI-ready data. The Actian data intelligence approach combines data discovery, metadata management, and federated governance to enable smarter data usage and enhance compliance. With intuitive self-service capabilities, business and technical users can find, understand, and trust data assets across cloud, hybrid, and on-premises environments. Actian delivers flexible data management solutions to 42 million users at Fortune 100 companies and other enterprises worldwide, while maintaining a 95% customer satisfaction score.

Data quality refers to an organization’s ability to maintain the quality of its data in time. If we were to take some data professionals at their word, improving Data quality is the panacea to all our business woes and should therefore be the top priority.

We believe this should be nuanced: Data quality is a means amongst others to limit the uncertainties of meeting corporate objectives.

In this series of articles, we will go over everything data professionals need to know about Data Quality Management (DQM):

1. The nine dimensions of data quality
2. The challenges and risks associated with data quality
3. The main features of Data Quality Management tools
4. The data catalog contribution to DQM

A Data Catalog is not a DQM Tool

An essential element is that a data catalog should not be considered a Data Quality Management tool, per se.

First of all, one of the core principles at the heart of Data Quality is that controls should ideally take place in the source system. Running these controls solely in the data catalog – rather than at the source and the data transformation flow – increases the global cost of the undertaking.

Furthermore, a data catalog must be both comprehensive and less intrusive to facilitate its rapid deployment within the company. This is simply incompatible with the complex nature of data transformation and the multitude of tools used to carry out these transformations.

Lastly, a data catalog must remain a simple tool to understand and use.

How Does a Data Catalog Contribute to DQM?

While the data catalog isn’t a Data Quality tool, its contribution to the upkeep of Data Quality is nonetheless substantial. Here is how:

• A data catalog enables data consumers to easily understand metadata and avoid hazardous interpretations of the data. It echoes the clarity dimension of quality;
• A data catalog gives a centralized view of all the available enterprise data. Data Quality information is therefore metadata like any other that carries value and should be made available to all. They are easy to interpret and extract, an echo of the dimensions of accuracy, validity, consistency, uniqueness, completeness, and timeliness.
• A data catalog has data traceability capacities (Data Lineage), echoing the traceability dimension of quality;
• A data catalog usually allows direct access to the data sources, echoing the availability dimension of quality.

The Implementation Strategy of the DQM

The following table details how Data Quality is taken into account depending on the different solutions on the market:

As stated above, quality testing should by default take place directly in the source system. Quality test integration in a data catalog can improve user experience, but it isn’t a must in light of its limitations – as Data Quality isn’t integrated into the transformation flow.

That said, when the systems stacks become too complex and we need, for example, to consolidate data from different systems with different functional rules, a Data Quality tool becomes unavoidable.

The implementation strategy will depend on use cases and company objectives. It is nonetheless apropos to put Data Quality in place incrementally to:

1. Ensure the source systems have put in place the relevant quality rules;
2. Implement a data catalog to improve quality on the dimensions of clarity, traceability, and/or availability;
3. Integrate Data Quality in the transformation flows with a specialized tool while importing this information automatically into the data catalog via APIs.

Conclusion

Data Quality refers to the ability of a company to maintain the sustainability of its data over time. We define it through the prism of nine of the sixty dimensions described by DAMA International: completeness, accuracy, validity, uniqueness, consistency, timeliness, traceability, clarity, and availability.

As a data catalog provider, we reject the idea that a data catalog is a full-fledged quality management tool. In fact, it is only one of several ways to contribute to the improvement of Data Quality, notably through the dimensions of clarity, availability, and traceability.

Get our Data Quality Management Guide for Data-Driven Organizations

For more information on Data Quality and DQM, download our free guide: “A Guide to Data Quality Management” now! Download the eBook.

About Actian Corporation

Actian empowers enterprises to confidently manage and govern data at scale, streamlining complex data environments and accelerating the delivery of AI-ready data. The Actian data intelligence approach combines data discovery, metadata management, and federated governance to enable smarter data usage and enhance compliance. With intuitive self-service capabilities, business and technical users can find, understand, and trust data assets across cloud, hybrid, and on-premises environments. Actian delivers flexible data management solutions to 42 million users at Fortune 100 companies and other enterprises worldwide, while maintaining a 95% customer satisfaction score.

To remain competitive, organizations must make decisions quickly, as the slightest mistake can lead to a waste of precious time in the race for success. Defining the company’s reason for being, its direction, and its strategy makes it possible to build a solid foundation for creating an alignment – subsequently facilitating decisions that impact product development. Aligning all stakeholders in product development is a real challenge for Product Managers. Yet, it is an essential mission to bring up a successful product and an obvious prerequisite to motivate teams who need to know why they get up each morning to go to work.

The Foundations of a Shared Product Vision Within the Company

Various frameworks (NorthStar, OKR, etc.) have been developed over the last few years to enable companies and their product teams to lay these foundations, disseminate them within the organization, and build a roadmap that creates cohesion. These frameworks generally define a few key artifacts and have already given rise to a large body of literature. Although versions may differ from one framework to another, the following concepts are generally found:

• Vision: The dream, the true North of a team. The vision must be inspiring and create a common sense of purpose throughout the organization.
• The Mission: It represents an organization’s primary objective and must be measurable and achievable.
• The Objectives: These define measurable short and medium-term milestones to accomplish the mission.
• The Roadmap: A source of shared truth – it describes the vision, direction, priorities, and progress of a product over time.

With a clear and shared definition of these concepts across the company, product teams have a solid foundation for identifying priority issues and effectively ordering product backlogs.

Product Values: The Key to Team Buy-in and Alignment Over Time

Although well-defined at the beginning, these concepts described above can nevertheless fall into oblivion after a while or become obsolete! Indeed, the company and the product evolve, teams change, and consequently the product can lose its direction… A work of reconsideration and acculturation must therefore be carried out continuously by the product teams in order for it to last.

Indeed, product development is both a sprint and a marathon. One of the main difficulties for product teams is to maintain this alignment over time. In this respect, another concept in these frameworks is often under-exploited when it is not completely forgotten by organizations: product values.

Jeff Steiner, Executive Chairman at LinkedIn, particularly emphasized the importance of defining company values through the Vision to Values framework. LinkedIn defines values as “The principles that guide the organization’s day-to-day decisions; a defining element of your culture”. For example “be honest and constructive”, “demand excellence”, etc.

Defining product values in addition to corporate values can be a great way for product teams to create this alignment over time and this is exactly what we do at the Actian Data Intelligence Platform.

From Corporate Vision to Product Values: A Focus on a Data Catalog

Organization & Product Consistency

We have a shared vision – “Be the first step of any data journey” – and a clear mission – “To help data teams accelerate their initiatives by creating a smart & reliable data asset landscape at the enterprise level”.

We position ourselves as a data catalog pure-player and we share the responsibility of a single product between several Product Managers. This is why we have organized ourselves into feature teams. This way, each development team can take charge of any new feature or evolution according to the company’s priorities, and carry it out from start to finish.

If we prioritize the backlog and delivery by defining and adapting our strategy and organization according to the objectives, three problems remain:

• How do we ensure that the product remains consistent over time when there are multiple pilots onboard the plane?
• How do we favor one approach over another?
• How do we ensure that a new feature is consistent with the rest of the application?

Indeed, each product manager has his or her own sensitivity, his or her own background. And if the problems are clearly identified, there are usually several ways to solve them. This is where product values come into play…

Actian Data Intelligence Platform’s Product Values

If the vision and the mission help us to answer the “why?”, the product values allow us to remain aligned with the “how?”. It is a precious tool that challenges the different possible approaches to meet customer needs. And each Product Manager can refer to these common values to make decisions, prioritize a feature or reject it, and ensure a unified & unique user experience across the product.

Thus, each new feature is built with the following 5 product values as guides:

Simplicity

This value is at the heart of our convictions. The objective of a Data Catalog is to democratize data access. To achieve this, facilitating catalog adoption for end users is key. Simplicity is clearly reflected in the way each functionality is proposed. Many applications end up looking like Christmas trees with colored buttons all over the place that no one knows how to use; others require weeks of training before the first button is clicked. The use of the Data Catalog should not be reserved to experts and should therefore be obvious and fluid regardless of the user’s objective. This value was reflected in our decision to create two interfaces for our Data Catalog: one dedicated to search and exploration, and the other for the management and monitoring of the catalog’s documentation.

Empowering

Documentation tasks are often time-consuming and it can be difficult to motivate knowledgeable people to share and formalize their knowledge. In the same way, the product must encourage data consumers to be autonomous in their use of data. This is why we have chosen not to offer rigid validation workflows, but rather a system of accountability. This allows Data Stewards to be aware of the impacts of their modifications. Coupled with an alerting and auditing system after the fact, it ensures better autonomy while maintaining traceability in the event of a problem.

Reassuring

It is essential to allow end-users to trust in the data they consume. The product must therefore reassure the user by the way it presents its information. Similarly, Data Stewards who maintain a large amount of data need to be reassured about the operations for which they are responsible: have I processed everything correctly? How can I be sure that there are no inconsistencies in the documentation? What will really happen if I click this button? What if it crashes? The product must create an environment where the user feels confident using the tool and its content. This value translates into preventive messages rather than error reports, a language type, idempotency of import operations, etc.

Flexibility

Each client has their own business context, history, governance rules, needs, etc. The data catalog must be able to adapt to any context to facilitate its adoption. Flexibility is an essential value to enable the catalog to adapt to all current technological contexts and to be a true repository of data at enterprise level. The product must therefore adapt to the user’s context and be as close as possible to their uses. Our flat and incremental modeling is based on this value, as opposed to the more rigid hierarchical models offered on the market.

Deep Tech

This value is also very important in our development decisions. Technology is at the heart of our product and must serve the other values (notably simplicity and flexibility). Documenting, maintaining, and exploiting the value of enterprise-wide data assets cannot be done without the help of intelligent technology (automation, AI, etc.). The choice to base our search engine on a knowledge graph or our positioning in terms of connectivity are illustrations of this “deep tech” value.

The Take Away

Creating alignment around a product is a long-term task. It requires Product Managers – in synergy with all stakeholders – to define from the very beginning: the vision, the mission, and the objectives of the company. This enables product management teams to effectively prioritize the work of their teams. However, to ensure the coherence of a product over time, the definition and use of product values  are essential. With the Actian Data Intelligence Platform, our product values are simplicity, autonomy, trust, flexibility and deep-tech. They are reflected in the way we design and enhance our Data Catalog and allow us to ensure a better customer experience over time.

About Actian Corporation

Actian empowers enterprises to confidently manage and govern data at scale, streamlining complex data environments and accelerating the delivery of AI-ready data. The Actian data intelligence approach combines data discovery, metadata management, and federated governance to enable smarter data usage and enhance compliance. With intuitive self-service capabilities, business and technical users can find, understand, and trust data assets across cloud, hybrid, and on-premises environments. Actian delivers flexible data management solutions to 42 million users at Fortune 100 companies and other enterprises worldwide, while maintaining a 95% customer satisfaction score.