Dataro Predict
      Back to home
      1. Dataro Help Centre
      2. Dataro Predict

      Running an experiment

      How to design a fundraising campaign experiment using Dataro

      The following examples provide a step-by-step guide to running an experiment or A/B test to compare Dataro's campaign selections to your existing campaign selections. The example considers a DM (Mail) appeal, but it can be applied to any mass fundraising campaign.

      Step 1: Generate Your Own Campaign List Using Your Old Method

      • In order to create a fair comparison between your old approach (e.g. RFM analysis) and Dataro's method (Propensity Scoring), you first need to generate a campaign list using your old method (let's call this the 'RFM List'). Typically, this will involve creating a query that selects a number of 'segments' and applies a series of 'exclusions' (e.g. Do Not Mail) in order to arrive at a final list of donors to include.

      For a fair and impartial test, it is critical that you create your campaign list without referring to Dataro's propensity scores and ranks.

      Step 2: Generate a Second List Using Dataro's Propensity Scores & Ranks

      • Now that you have the RFM List, you need to generate a second list using Dataro's Propensity Scores and Ranks (the 'Dataro List'). To do this using Raiser's Edge, follow the steps contained in this article. Salesforce instructions are here.
      • When it comes to selecting your list size, login into your Dataro Account and review our current list size recommendation for a DM Appeal. You can also consider these rules of thumb:
        • The Dataro List will almost always be smaller than the RFM List (often 10-20% smaller). This is due to the superior targeting using our propensity modelling.
        • A DM Appeal Score of 0.01 (i.e. a 1% chance of giving) tends to be a reasonable cut off point.
        • You can also make an educated guess using the results from the same appeal last year. At what campaign size did the campaign turn from a positive ROI to a negative ROI?
      • Apply the same exclusions to the Dataro List as you applied to the RFM List.

      Tip: Remember, you need to apply the exact same exclusions to both the RFM List and the Dataro List. For more help on list size, contact us.

      Step 3: Compare your list to the Dataro List

      • Now that you have created two lists, you can compare them. This allows you to identify:
        • Donors that appear in both lists (let's call them the 'Both Lists' segment)
        • Donors that only appear in your list only (let's call them the 'Us Only' segment)
        • Donors that only appear in the Dataro list only (let's call them the 'Dataro Only' segment)
      • You can imagine this comparison as a Venn Diagram. The 'Both + Us-Only' segment which is your List and the 'Both + Dataro-only' segment which is the Dataro List.
        Untitled design (2)-1

      Step 4: Add the extra Dataro donors (the Dataro Only Segment) to the campaign and store the results

      • In order to measure the results, you need to send the campaign to every donor appearing in either the RFM List or the Dataro List. In essence, this means adding the 'extra' donors from the Dataro List (i.e. the Dataro Only Segment) to the RFM List. This gives you your final campaign data file.
      • It is important to store the results correctly for easy analysis later. This means you need to store the Experiment Segment (Both, Dataro Only, Us Only) for every constituent who received the appeal.
        • In Salesforce this can be done with an Experiment field on the Campaign Members Object.
        • For Raiser's Edge this can be done using Comments on the AssignedAppeals.
        • For more information on our recommended data-storing practices, see this article.

      Please let your friendly Dataro team know if you are running an experiment so we can work with you to get those results into the app.