Data Governance
      Back to home
      1. Zetaris Support - Knowledge Base
      2. Zetaris Technical Use-Cases Library
      3. Data Governance

      Creating automated consent mechanisms with metadata tagging

      A data catalog is a very powerful tool in the data world for classifying and organizing data. We at Zetaris have recently unleashed the power of our metadata tagging and search mechanism by incorporating data redaction principles into the way metadata is tagged in the Zetaris data catalog

      User Story

      Data collection in the research industry follows a strict principle of collecting and holding data that has been consented by participants. There is a legal obligation in the industry to remove and discontinue usage of and personal data that has not been consented to or has undergone a withdrawal of consent. To align to strict data privacy rules, time is essential in managing the dynamic nature of the consent associated with the participant’s personal data.

      The process of dynamic consent in a data ecosystemThe process of dynamic consent in a data ecosystem

      Implementing a simple scalable and flexible solution

      Below are the steps that need to be implemented for the Withdrawal system

      1. Creation of a virtual Data Filter
        1. Purpose of the view is to record the IDs/Primary Keys of all the unconsented records from incoming tables/file sources
      2. Deletion Script to remove all unconsented data from the incoming source files
        1. Purpose of the script is to isolate and delete the unconsented records from the Zetaris Lightning File source system as soon as it gets registered in the system.
        2. Input of the script will be virtual Data Filter(Step 1).
        3. Output Script will delete all the unconsented records from the File source as soon as it is registered so that unconsented data will not be accessible within the system.
      3. Modification in existing pipelines(Data Store)
        1. Purpose of the pipeline modification is to include virtual Data Filter(Step 1) to block the unconsented data (if any). It is acting as a precaution to block the unconsented data towards the downward systems.
        2. Input will be virtual Data Filter(Step 1). The view will be used as an inner join in each Data Store pipeline to block the unconsented records (if any).
        3. Output Only consented data gets transformed and flows into the Data Store
      4. History View Data Filter
        1. Purpose of the view is to store all the unconsented records of the Data Store history tables with their corresponding DML statements.
        2. Input of the View will be the data from the Data Store history tables.
        3. Output View will be used by the data repository administrator to execute the DML statements to delete the unconsented records.
      5. Audit Table
        1. Purpose of the audit table is to store all the unconsented deleted records when the deletion is being done by the data repository administrator.
        2. Input of the Audit table will be History View Data Filter(Step 4) & Zetaris Lightning Audit table(to record the user deletion information).
        3. Output The table can be used for admin and audit reporting.

      Implementation of Auto Consent Mechanism

      Once this guide has been followed, please click the link below for a guide to implement the auto consent mechanism