The Scientific Way to Compare Your CRM Tactic
Matan Block-Temin & Jonathan Inbar
The Motivation for Comparing Tactics
- //Optimizing tactics for each customer persona
- //Different campaigns have different objectives
- //Objectives may change over time
- //Testing and validating
What is a Journey?
A journey is a sequence of campaigns put together to implement a CRM tactic
Measuring Campaign Uplift
Total campaign uplift = (Test results + Control results) – (Estimated organic results)
- //Test results = test responses X test AVG KPI
- //Control results = control responses X control AVG KPI
- //Estimated organic results = (test group + control group) X control response rate X control AVG KPI
Comparing Tactics, Scientifically – Two analysis methods:
- Measuring a tactic – Super Control Group (SCG)
- SCG – What is it?
Excluding percent of total population from all journey communication
- SCG – Pros and cons
- SCG – How to measure?
Apply same methodology as the campaign analysis:
Total journey uplift = (Journey results + Control results) – (Estimated organic results)
- Streams – comparing journeys
- Stream – What is it?
Dividing total population into two different communication journeys in order to define the optimal tactic
- Streams – Pros and cons
- Streams – How to measure?
Compare between both stream results to define the wining tactic.
Always Have Something for Everyon
Hadas Avrech & Scott Shapiro
- 1. Use assets from your website in your email marketing
- 2.Cross-hit your active users! Distribute the same content across all relevant channels
- 3.Refresh and republish your best performing historical pieces
- 4.Use technology to reuse content
- 5.Use your customers as content generators
Personalize Email Like You Personalize Your Website
Ella Aharon & Shefa Weinstein
Examples of dynamic content
Real Time Content:
- //Real time pricing / Inventory
- //Loyalty points
- //Forms / Surveys
- //Pop ups
Personalizing emails with Dynamic Content
- //Scrape the homepage banner
- //Show latest bestsellers
- //Get them into your store
2nd Transaction Emails:
- //Recommended products
- //Complementary products
- //Similar deals
Monthly loyalty / Account summary emails:
- //Current status level
- //Offers for their specific level and how to redeem them
- //Summary of winnings
Email Headers and Footers:
- //Loyalty points
- //News feeds
- //Social feeds
- //Closest store
04:DIY Hack for Advanced Campaign Analysis
Measuring Campaign Effectiveness and Long-term Impact
Nitay Alon & Yohai Sabag
Business question: Do we make a change in the behavior of our customers?
Relevant distribution region: All
Scientific tool: Subsample
How to accomplish this in Excel:
- 1. Add a random column using RAND() function next to your data column
- a. Create random values for all the observations
- b. Copy and paste the random values (as values) – this fixes the value of the random number
- 2. Sort the data by the random variable column
- 3. Select the (control size) top observations
- a. Compute the average of the sample
- b. Compare it to the average of the control group
- c. Mark 1 if the sample average exceeds the control and mark 0 if else
- 4. Repeat steps 1 – 3 as needed
- 5. Compute the proportion of times that the test subsample average is bigger than the control average
Data based decision: Look at the winning odds of each group. If it’s close to 50-50, keep exploring if you’re in an A/B setting. Otherwise, consider stopping if you’re in a Test/Control setting.
Business question: Do we make a change for most of the target population?
Relevant distribution region: The central mass (the Filet Mignon)
Scientific tool: Trimmed mean
How to accomplish this in Excel:
- 1. Use excel TRIMMEAN() function
- a. First select the range of the data
- b. Set the percent of trimming, say 0.2 to trim 20% of the data
Data based decision: If a trimmed mean t-test is significant, then there’s a statistically significant effect on the majority.
Business question: How much money is expected to be seen in the next iteration?
Relevant distribution region: The average and standard deviation of the logged data
Scientific tool: Log normal distribution
How to accomplish this in excel:
- 1. Apply the LN() function over the data you wish to simulate from
- 2. Compute the average and standard deviation of the post-transformation data
- 3. Using RAND() function create 10,000 new values
- a. Copy and paste as values
- 4. Use NORM.INV() function on the values from step (2) and (3) to sample data from normal distribution
- 5. Using EXP() function transform the data from step (4) into Log-Normal data
Data based decision: Take a sample of known size from the simulated data and compute the sum. Compare sums between alternatives (Automate this process and repeat 10K times to get a better feeling).
Methodologies to Define Your Most Valuable Customers
Roni Cohen & Nadav Yekutiel
VIP Segmentation: Methodologies to better define your most valuable customers
Every business should know who their VIP customers are to ensure the satisfaction and loyalty of those customers.
There are several questions we should answer to accurately define the VIP segment to promise optimal results.
What attributes should you look at?
- //Different metrics and timeframes to be considered
- //Correlation matrix to find attributes with highest correlation to customer’s future value
What is the ideal segment size?
- //Pareto analysis shows us which % of customers generates a mass of the income
Fixed or dynamic threshold?
- //Fixed thresholds are easier to implement, track and measure
- //Dynamic thresholds capture customer behavioral changes
How often do we recalculate?
- //Daily update promises most updates VIP list
- //Periodic update is more stable and easy to work with
- //Optimal solution: Grace Period – Recalculate list on a daily basis but hold close customers in segment for a certain period of time
Should we split our VIP into sub segments?
- //Allows more personal messages for each sub segment
- //Understand customer potential by monitoring highest achieved tier
Build a Marketing Tech Stack to Meet Your Business Objectives
Leigh Noy & Andrew Samuels
A marketing technology stack is the layering and interconnection of different technologies that make marketing happen.
What comprises a stack?
- //Data Layer
- //Planning Layer
- //Execution Layer
Building your stack:
Two main approaches:
- //Single Vendor Suite
- //Best of Breed
Things to bear in mind
- //Be thoughtful
- //Visualize your stack
- //Focus on the big picture
- //Think long-term
- //Company’s stage of maturity
- //Team and talents
- //Your existing tools
- //The scope of the funnel and customer experience you’re trying to create
- //Your business model
More to consider
- //Align with IT
- //Create a use case wishlist
- //Test vendors
- //Understand R&R
“Architecture is about aging well, about precision and authenticity.” – Annabell Selldorf