Email Marketing Data Mart


Email Marketing Data Mart

Over the last year, our team noticed a need for a solution that helped us scale our data requests and our lifecycle marketing tactics.

Enter the email marketing data mart (also known as Email Marketing Database), which integrates several disparate data sources to help us reach our users, customers, and prospects in a more intelligent and scalable way.

In partnership with enterprise applications and marketing teams, we have created a solution that will enable Denomas to more quickly communicate critical customer and user updates, understand the overlap between prospects and users, and enable us to market to them more efficiently.

The goal of this page:

  • Help you to use the TD: Email Marketing Data Mart to generate email campaigns.
  • Help you understand the data models used to create the TD: Email Marketing Data Mart.
  • Coming Soon Assess your understanding by taking a certification most applicable to your role at Denomas.
  • How to add new fields to the Data Mart and Marketo
  • And overall help everyone contribute!









Getting Started

To get started we want to make sure you understand:

  • Key terms that will explain how we account for the metrics
  • The data sources behind the database

Key Terms, Key Fields, and Business Logic

Key Terms Dimensions:
  • Sales Segment
  • Sales Region
  • Product Tier
  • Product Delivery
  • User Role
  • Subscription Start and End Dates
  • Trial Start and End Dates
Key Metrics, KPIs, and PIs Facts:
  • Usage Ping
  • Days until trial ends
Key Fields and Business Logic * The grain of the Email Marketing Data Mart is one row per unique email address. There is a one to many relationship between each email address and different dimensions such as product tier, product delivery, subscriptions and namespaces. Therefore, in the Data Mart, we have used aggregation logic to create fields to describe the email addresses. For example, there are fields named `group_owner_of_saas_premium_tier` and `group_owner_of_saas_ultimate_tier` which are boolean fields. It is possible that a group owner could have a TRUE value for both of these fields if they are owners of both a Premium and Ultimate plan.

For more information around what fields are in the Email Marketing Data Mart, please refer to the documentation page.

Understanding the Data Sources and Data Models


Data Sources:

  • Zuora
  • Salesforce
  • Denomas.com
  • Customers DB
  • Marketo

Entity Relationship Diagram (ERD):

Data Lineage * The dbt solution generates a dimensional model from RAW source data. The documentation and SQL for mart_marketing_contact can be found here , and the complete data lineages can be found at dbt mart_marketing_contact lineage chart
Example Queries Let's pull some email lists. You can use these queries in Snowflake.
``` --Pull ALL owners and individual namespaces for Paid SaaS Plans SELECT email_address FROM "PREP"."SENSITIVE"."MART_MARKETING_CONTACT" WHERE GROUP_OWNER_OF_SAAS_BRONZE_TIER = TRUE OR GROUP_OWNER_OF_SAAS_PREMIUM_TIER = TRUE OR GROUP_OWNER_OF_SAAS_ULTIMATE_TIER = TRUE OR INDIVIDUAL_NAMESPACE_IS_SAAS_BRONZE_TIER OR INDIVIDUAL_NAMESPACE_IS_SAAS_PREMIUM_TIER OR INDIVIDUAL_NAMESPACE_IS_SAAS_ULTIMATE_TIER

–Pull ALL contacts related to paid Self Managed instances SELECT email_address FROM “PREP”.“SENSITIVE”.“MART_MARKETING_CONTACT” WHERE is_self_managed_delivery

–Pull ALL customers associated with a trial SELECT email_address FROM “PREP”.“SENSITIVE”.“MART_MARKETING_CONTACT” WHERE days_until_saas_trial_ends IS NOT NULL


Since this data is classified as **ORANGE** as it is customer / email data, in Sisense we only expose a non-sensitive view of it [mart_marketing_contact_no_pii](https://gitlab-data.gitlab.io/analytics/#!/model/model.gitlab_snowflake.mart_marketing_contact_no_pii). You can use substitude the table above and the `email_address` field with the `dim_marketing_contact_id` to have an idea on how big an email pull will be.

</details>
<br>

<style> #headerformat {
background-color: #6666c4; color: black; padding: 5px; text-align: center;
}
</style>
<h1 id="headerformat">Adding new columns to DataMart and Pump to Marketo </h1>

The process to add fields to the DataMart and Pump is below. There are several teams and systems involved.

1. [Create an issue](https://code.denomas.com/denomas-com/marketing/marketing-operations/-/issues/new?issuable_template=new_field) with Marketing Operations team to add a new field to Marketo (and SFDC, if applicable)
    - If field is existing, you can skip this step.
1. Marketing Operations creates a changeset in SFDC marketing sandbox and [creates an issue](https://code.denomas.com/denomas-com/sales-team/field-operations/systems/-/issues/new) to deploy the changeset. 
1. Marketing Operations creates a [new issue](https://code.denomas.com/denomas-data/analytics/-/issues/new?issuable_template=email_data_mart_pump_new_column) in Data Team project to add a column to the datamart.
1. Data team triages and asks questions as neeeded.
    -  If you have the reference SQL snippet, please add to the issue
1. Data team creates an MR to build column in
1. Data team [creates an issue](https://code.denomas.com/denomas-com//business-technology/enterprise-apps/integrations/platypus/-/issues/new?issuable_template=Change) to build out the integration in Platypus from the Data Mart to Marketo.


<style> #headerformat {
background-color: #6666c4; color: black; padding: 5px; text-align: center;
}
</style>
<h1 id="headerformat">Making an email pull request </h1>

To make an email list pull request, please open an issue in [the Marketing Performance Project](https://code.denomas.com/denomas-com/marketing/marketing-strategy-performance/-/issues/new) with the list use case and requirements so the request can be prioritized.


<style> #headerformat {
background-color: #6666c4; color: black; padding: 5px; text-align: center;
}
</style>
<h1 id="headerformat">Updating The Marketing Do Not Contact List </h1>

The Marketing Do Not Contact (DnC) list contains a set of emails that have been identified as `undeliverable`, `do_not_send`, ... in Marketo. This information is then used in the Email Marketing Database to create the field `wip_is_valid_email_address` which defines whether an email can or cannot be contacted through email.

This list is uploaded through a process called Driveload. The Driveload process goes to a Google Drive folder and uploads the CSV files found in said folder to a table in the data warehouse. The Marketing Do Not Contact driveload process is set to `append = 1`, this means that any new CSV files that are added to the Google Drive folder will be appended to the past CSV files in the data warehouse table.

To update/add another CSV file to the list you need to:

1. Go to [the Marketing Do Not Contact List GDrive folder](https://drive.google.com/drive/folders/1EUYXgGYAcGEWGemu_SbYz3eCQjk57vRq)
2. Upload a CSV file containing the following fields: `address, is_role_address, is_disposable_address, did_you_mean, result, reason, risk, root_address`.

    a. Make sure to use the same casing as shown here.
    
    b. At the very least the fields `address` (email address) and `result` (the field containing whether the email address is `undeliverable`, `unknown`, `do_not_send`, ...) need to be filled as the Marketing Database data models expect these fields to have valid information in them.

3. The driveload process runs everyday and it will automatically pick up the new CSV file dropped in the GDrive folder without any further manual inputs. So you just need to wait for this process to complete.

Note: If an email address is found more than once in the list then the information that will be taken will be from the latest uploaded CSV file.

In case a contact was first uploaded as `undeliverable`, `do_not_send`, ... and this is no longer the case and wants to be changed to deliverable, you only need to upload a new CSV file with said email address and the field `result` with a value of `deliverable` and follow the above steps.

<style> #headerformat {
background-color: #6666c4; color: black; padding: 5px; text-align: center;
}
</style>
<h1 id="headerformat">Additional Resources </h1>

<details>
<summary markdown='span'>
  Email Marketing Data Pump
</summary>

For more information around the Email Marketing Data Pump, which is the technology in charge of moving a selected set of data from the email marketing database in Snowflake to Marketo, please reference [this page](/handbook/business-technology/enterprise-applications/integrations/wiki/integrations-list/marketing-data-pump/).

</details>



<details>
<summary markdown='span'>
  Trusted Data Solution
</summary>

Coming Soon

</details>

<details>
<summary markdown='span'>
  EDM Enterprise Dimensional Model Validations
</summary>
The [(WIP) Enterprise Dimensional Model Validation Dashboard](https://app.periscopedata.com/app/gitlab/760445/WIP:-Enterprise-Dimensional-Model-Validation-Dashboard) reports on latest Enterprise Dimensional model test and runs.
</details>

<details>
<summary markdown='span'>
  RAW Source Data Pipeline validations
</summary>
[Data Pipeline Health Validations](https://app.periscopedata.com/app/gitlab/715938/Data-Pipeline-Health-Dashboard)
</details>

<details>
<summary markdown='span'>
  Data Security Classification
</summary>


**ORANGE**

- Description: Personal data at the row or record level. Given this, this data is masked in `PROD.COMMON_MART_MARKETING`, but is available in `PREP.SENSITIVE`. Customer Metadata (such as customer company name) is available in `PROD.COMMON_MART_MARKETING`
- Objects: 
  - `PREP.SENSITIVE.DIM_MARKETING_CONTACT`
  - `PREP.SENSITIVE.MART_MARKETING_CONTACT`
  - `PROD.COMMON_MART_MARKETING.MART_CRM_TOUCHPOINT`
  - `PROD.COMMON_MART_MARKETING.MART_CRM_ATTRIBUTION_TOUCHPOINT`
 
</details>

<details>
<summary markdown='span'>
  Solution Ownership
</summary>
* Source System Owner:
  * Salesforce: `@jbrennan1`
  * Zuora:
  * Denomas.com
  * Customers DB
  * Marketo: `@amy.waller`
* Source System Subject Matter Expert:
  * Salesforce: `@jbrennan1`
  * Zuora:
  * Denomas.com
  * Customers DB
  * Marketo `@amy.waller`
* Data Team Subject Matter Expert: `@jeanpeguero` `@jjstark` `@iweeks`
</details>
Last modified December 1, 2023: bulk update (176cf9ec)