My Role:

I worked under the head of marketing and our big data expert in developing the survey. We combined our skillsets to create the survey and analyze the data.

Team

  • Devin Shelley, Manager, Research and Analytics
  • Thomas Larsen, User Experience Researcher
  • Ryan Turner, Lead, Big Data

1. Survey Development

Designed the survey to test several ad variants in:

  • Memorability
  • Engagement
  • Appeal
  • Investment

Implemented major randomization to avoid bias. Asked questions to test specific zones of the ad in order to provide specific, element by element feedback.

49
Complex survey flow

2. Survey Administration

  • Worked with Qualtrics’ panel team to get participants.
  • Filtered participants by those who had gone to Hawaii in the past year or would be going in the next year.
  • More than 600 participants responded.
50
Survey questions

3. Data Analysis

Looked at the results ad by ad. Filtered results by:

  • Which ad was seen first
  • Age
  • Income bracket

Analyzed heat maps and section click data to learn which specific sections were liked or disliked and why.

51
Click heat map and quantitative regional data

Findings and Recommendations

KEY FINDINGS

  • The new ad was more memorable, more engaging, and more likely to increase interest in going to the PCC than the old ad.
  • Participants liked the pictures on the new ad better.
  • Participants liked showing authentic Polynesians more than Caucasians.

RECOMMENDATIONS

  • Switch to the new ad.
  • Keep Polynesian photos.

OUTCOMES

  • The new ad had a 13% increase in conversion (5% higher than estimated.)