I worked under the head of marketing and our big data expert in developing the survey. We combined our skillsets to create the survey and analyze the data.
- Devin Shelley, Manager, Research and Analytics
- Thomas Larsen, User Experience Researcher
- Ryan Turner, Lead, Big Data
1. Survey Development
Designed the survey to test several ad variants in:
Implemented major randomization to avoid bias. Asked questions to test specific zones of the ad in order to provide specific, element by element feedback.
2. Survey Administration
- Worked with Qualtrics’ panel team to get participants.
- Filtered participants by those who had gone to Hawaii in the past year or would be going in the next year.
- More than 600 participants responded.
3. Data Analysis
Looked at the results ad by ad. Filtered results by:
- Which ad was seen first
- Income bracket
Analyzed heat maps and section click data to learn which specific sections were liked or disliked and why.
Findings and Recommendations
- The new ad was more memorable, more engaging, and more likely to increase interest in going to the PCC than the old ad.
- Participants liked the pictures on the new ad better.
- Participants liked showing authentic Polynesians more than Caucasians.
- Switch to the new ad.
- Keep Polynesian photos.
- The new ad had a 13% increase in conversion (5% higher than estimated.)