Bounce and Completion Rate

In the latest release of CloudResearch.com we added many new features and fixes among them 2 additional metrics for every survey:

• Bounce Rate
• Completion Rate

The Bounce Rate is the percentage of Amazon workers who previewed your survey but decided not to accept it. An open question is whether and how this self-selection of participants affects the representativeness of the participant pool. In addition, a high bounce rate may be an indicator that there is something wrong with your survey

We have already discussed the importance of the Completion Rate:

Completion rate is an important indicator of data quality. A low completion rate indicates that there is a selection bias which may be influencing the representativeness of the results. A very high dropout rate may also mean that there is something wrong with the study. It is typically good practice to report completion rate in the method/results section of a paper. Indeed, some editors require authors to use the CHERRIES checklist for survey research (Eysenbach, 2004), which asks about a study’s completion rate.

Under the Hood

Bounce Rate is 100% – ;[Accepted Assignments] / [Previewed Assignments]. Previewed Assignments is set to be the total of unique IP addresses that previewed this assignment. Since nearly all workers have unique IP addresses this is a reasonable estimate of the total number of previews.