Campaign Insights

A tool to allow you to investigate campaign and model performance.

Campaign Insights provides a detailed analysis of your campaigns by combining campaign and/or audience data with category, channel and model data.

How to Use Campaign Insights

Landing Page

Navigate to CI by selecting Reports -> Campaign Insights on the left hand navigation bar. There you will be met with the “new report” landing page. To refine the report, use the 'Filters' button in the top right corner of the page.

Apply Filters to select your Audience

Clicking on the Filters button reveals the filters that allow you to select an Audience, a Model / report type to view, a Date Range to restrict (more on this too) and finally define the Categories and Channels to include. Regardless of the filters selected, the reports sections outlined below will be broadly consistent.
1. Selecting an Audience: In Campaign Insights, there are really only 2 distinct types of audiences:
  1. Campaigns: A campaign audience reports on a specific campaign from your CRM / data source and filters the data observed in the report to by both:
    1. the campaigns that the revenue or communication was related to; and
    2. the constituents included in the campaign
    To be included in the report, in your CRM / data source the constituents must be included in the Campaign Members (communication sent) AND the revenue must be related to the selected campaign.
  2. Saved Audience: These audiences filter the data observed in the report only by the Constituents included, and does not reduce the revenue nor communications to a specified campaign from your CRM/data source. The saved audience (from Audience Builder), saved list (uploads) or Smart Audience (pre-defined audiences from Dataro) are all treated the same way. To upload a saved list you will be directed to the saved list feature.
  3. These two audience types (Campaign & Save Audience) can be used in unison, however this will have the following consequences:
    1. The data observed in the report will firstly be restricted to only constituents included in the campaigns selected. Then this will be reduced to include only those constituents also featured in the saved audience, however, if a constituent is featured in the saved audience but not in the campaign they will still be counted as a communication**
      ** This implementation was used to accomodate clients with missing communications data.
      PLEASE NOTE** To use and uploaded list in CI, you must upload it via the saved audience page. It is expected for these files to contain a contact_id column and can have "experiment_group_name" or "pickup" to specify experiment files or telemarketing responses.
2. Selecting a Model (Report Type):
  This filter is used to determine what kind of report will be generated (e.g. an appeal report or a churn report). There are 3 main reporting types:
  1. Binary Actions: These are reports measure if some action did or did not happen. In these reports we are less interested in how many times something happened and rather just interested if it did or did not happen. Examples include: RG Upgrade Report, RG Reactivate Report, RG Convert Report.
  2. Multiple Actions: These are reports where we are interested in how many times a donor completed an action over a specified period. Examples of this kind of report are: DM Appeal Report
  3. Interventions: These are reports where we want to measure the prevelance of an event in one group vs another. This could be reports where we want to compare the rate of churn in donors who you did and did not contact. Example of this kind of report are: RG Churn Report
  The 'Model' you select also specifies which predictor should be assessed in the Dataro prediction evaluation sections. Currently on the following models are implemented:
  1. DM Appeal
  2. RG Upgrades
  3. RG Convert
  4. RG Reactivation
3. Selecting a Date Range: Choose the date range for your report. If you are running a campaign report, then make sure that your start date is the date you selected your campaign audience and your end date includes the latest donations.
4. Selecting Categories & Channels: If you wish to restrict your report only to transactions and communications from specific Categories and Channels (as shown in your tagging), you can filter accordingly using these Category and Channel filters.
Regardless of the filters selected, the following report sections will be present for the majority of reporting types. The exact fields shown will be dynamic based off of the type of report selected.
1. At a glance: Highlights key performance metrics for the selected model / report type. This may be, the number of recipients or successful contacts, the total return, estimated returns etc.
2. Overall results: A table of results aggregated as an overall collection. The exact fields displayed in this section is also dynamic based off of the report type / model selected.
3. Dataro Predictions Score Chart: The score chart will show both the response rate and net return for donors aggregated by the score from their Dataro predictions. Note: this is not always the best measure of effectiveness as it is possible that there were very few constituents with high scores, in which case the Rank chart is more helpful
4. Dataro Predictions Rank Chart: Similar to the score chart the rank chart shows the response rate and net return of donors aggregated by the Dataro ranks predicted. Unlike the score, we would expect the highest response / conversion rates (and usually the largest net returns) to come from the best ranked constituents.
5. Donation Chart: The donation chart section is dynamic to the reporting type being viewed but it essentially always displays the number of communications (or successful contacts) and the number of events, where an event is dynamic to the reporting type [donation, upgrade, reactivation, conversion etc].
6. Score & Rank Table: The score and rank tables outline more detailed statistics aggregated across the Dataro predicted scores and ranks.
7. Campaign Segments: This section displays similar statistics to the score and rank tables but this time the statistics are aggregated across the campaigns / sub-campaigns from your CRM / primary data source.
8. Additional Information Section: The final section of the report is the additional information, which displays all of the campaigns featured in this report along with the tagged channel and categories and costs of communications. This is where you can make edits to the campaign costs.

Troubleshooting

Missing Campaigns: If a campaign doesn’t show up, check that it’s properly tagged in your campaign tagging. Sometimes, data issues can prevent channels or campaigns from appearing.

Key Concepts: Estimates & Costs

Estimates*
Depending on the report type selected there may be Estimates displayed within the report. The details of these estimates should be present in the tool tip, however as a rule of thumb this will always be the average amount of the event (for upgrades this is only the upgraded amount, for reactivation or convert it is the full gift amount), multiplied by the number of constituents who completed the event, multiplied by 18 months. For example if the average upgrade was $10 and 90 donors did this, the estimated return would be $10 * 90 * 18 = $16,200 which should be read as an estimated return based off of the upgrades within this campaign.

Costs *
There are two distinct types of costs in Campaign Insights and both are applied quite differently depending on the reporting type (model selected).

DM Appeal: For the DM Appeals, a cost is accrued any time a communication is sent regardless of if that communication was successfully received. So if there were a total of 1,000 communications sent and each cost $1 the total cost becomes 1000 * $1 = $1,000.00
RG Upgrade, Reactivate & Convert: For these reports a cost is only accrued if the communication is successful in reaching the constituent, in other words if the constituent doesn't answer a telephone call the charity is usually not charged by their calling agency. So if a campaign had 5,000 calls but only 500 were successfully answered, and the cost per call was $12, then the total cost would be 500 * $12 = $6000.00 as only 500 constituents answered.