What is CloudResearch? A Rundown of CloudResearch’s Products and the Best Use-Case for Each One
Whether you’re a seasoned researcher or just getting started, it can be tough to stay up-to-date with changes in online research. New tools emerge, best practices change, and everything moves fast!
CloudResearch, formerly known as TurkPrime, is a New York-based company founded by academics in 2015. We offer several tools for online research—Connect, Prime Panels, Managed Research, the MTurk Toolkit, and Sentry—and it isn’t always easy to know which tool is best suited to different situations. Of course, you can always contact us to discuss your project, but if you just want clarity about our products, this blog is for you!
| Connect | Prime Panels | Managed Research | MTurk Toolkit | |
| Participant Source | CloudResearch | Market Research Panels | Varies by Study | Amazon MTurk |
| Data Quality | Outstanding | Good | Great | Great |
| Price Range | $ | $$ | $$$ | $$ |
| Utility | – Best General-Purpose Sample – Easy to Use – Team Collaboration | – Niche Demographics – Largest Sampling Pool – International | – Full Service from Study Design to Reporting, as Needed – Complex, Niche, and International Samples | – Legacy Users of MTurk |
| Participant Payment | Determined by Researchers | Set by Market Research Panels | Project-based Quote | Determined by Researchers |
| Invoicing | Prepaid or Invoiced | Prepaid or Invoiced | Invoiced | – Prepaid CloudResearch Fees – Separate Amazon MTurk Account Needed |
The New Kid on the Block: Connect
CloudResearch’s newest product, ConnectTM, is a participant recruitment platform that we built from the ground up. That means, unlike with the MTurk Toolkit (see below), CloudResearch controls the source of participants for Connect. Connect has no relationship with MTurk.
In creating Connect, we wanted to build a platform that placed data quality at the forefront while giving researchers the tools to make their online studies better. Every one of the participants on Connect has been vetted to make sure they provide quality data using the same methods CloudResearch pioneered to bring quality to our other industry-leading research tools. Because of its targeted focus, Connect can offer high engagement and low attrition rates, allowing researchers to be confident in the quality of their data on anything from a simple survey to a complex online experiment. And because it was built with researchers’ needs in mind, our Connect Teams feature allows seamless transfer of funds, collaboration on projects, and simple logistics. See our Connect FAQ for more details.
When to Use CloudResearch Connect
Connect is perfect for complex studies that require high attention, especially engaged participants, or harder-than-typical tasks. Connect’s intuitive interface and simple logistics mean that you can spend your time focusing on your study design and analysis instead of trying to set up and integrate several platforms. Our service fees are the lowest in the industry, which makes Connect the best choice for large-scale projects or those seeking a Census-Matched sample.
Also, Connect is the best choice for teams and labs looking to collaborate. The Teams feature allows everyone to view and manage shared projects and a team wallet makes paying for studies and sharing funds easier than ever before.
The Selective Sampler: Prime Panels
If you need very large or very specific samples, then Prime Panels is what you’re looking for. Especially when your goal is to show that a finding generalizes, you might want participants who are like the general population: people with lots of diverse personalities, demographics, and interests. Prime Panels can provide these participants because it aggregates from many participant panels, where people from all walks of life sign up to take surveys in exchange for gift cards or monetary rewards. Aggregation across panels also allows researchers to be highly selective in their sampling, such as people in specific US zip codes, occupations, or interest groups.
When to Use Prime Panels
Through Prime Panels, you have access to millions of diverse respondents around the world and studies can be conducted in various languages. Because there are so many people available to survey, it’s also easier to reach very niche populations in case you happen to be studying parents of 8–10-year-olds with an ADHD diagnosis, for example.
Prime Panels is best suited to studies that are short (<20-30 minutes) and straight forward. In addition, to help manage data quality all Prime Panels studies are integrated with our patented solution: Sentry®.
Managed Research
CloudResearch offers a Managed Research service which is useful for everything from helping you recruit hard-to-reach samples to full project design and execution.
Whenever a project is run with Managed Research, our team of expert project managers help researchers collect the data they need in an efficient and time effective manner. Managed Research can be customized to your team and project’s needs. We can, for example, help design and manage complex studies, perform data analysis, and write reports, or we can let you do that work and simply deliver the high-quality data you need from hard-to-reach groups of participants. With every Managed Research project, we guarantee a dedicated project manager and high-quality data.
When to Use Managed Research
Managed Research makes sense when: 1) you’re interested in collecting data from a group of participants that are “hard-to-reach” and therefore would not be efficient using our do-it-yourself services, or 2) your team lacks the time, expertise, or knowledge to conduct a complicated online project. Because our team has years of experience and expertise, we can execute on the most complicated and demanding online projects like the incredibly intense longitudinal project we conducted with The Office of the Director of National Intelligence.
The OG: Our MTurk Toolkit
Another service CloudResearch offers, the MTurk Toolkit, pulls participants from Amazon Mechanical Turk (MTurk), but adds various layers of quality protections and advanced features. When CloudResearch began in 2015, the MTurk Toolkit was our flagship product. With it we aimed to transform MTurk’s microtask site into a platform for behavioral and social research. In many respects, that is still the aim of the MTurk Toolkit today.
Since 2018, however, there have been serious data quality issues on MTurk. To confront these issues and ensure MTurk remains a source of quality data, CloudResearch has introduced a series of innovations. The largest of these innovations came in 2020 when we added the CloudResearch “Approved Participants” group to our Toolkit.
The CloudResearch Approved Participants group is a subset of participants on MTurk who have been vetted by CloudResearch. Our team of highly trained PhDs have created a set of patented data quality measures based on years of research and experience. We vet participants by combing through data from hundreds of thousands of respondents and identifying people who consistently provide high-quality data. We also block those who provide poor-quality data or whom we suspect of fraud. Research conducted by our team and independent researchers attests to the quality of this solution.
When to Use the MTurk Toolkit
When paired with the CloudResearch Approved Participants group, our MTurk Toolkit is great for collecting high-quality data from engaged participants willing to complete a wide variety of tasks.
With over 100,000 participants, there are enough people to target specific populations and to conduct all kinds of studies (e.g., video interviews, longitudinal projects, UX studies). However, even with the improvements our MTurk Toolkit brings to the MTurk environment, there is still a steep learning curve to using MTurk effectively. There are also logistical challenges posed by merging the MTurk and CloudResearch systems. So, for anyone starting out with online research or looking for the most efficient way to conduct studies, we recommend CloudResearch Connect.
The Data Defender: Sentry®
Sentry is CloudResearch’s patented data quality solution. Although many platforms offer tools to block duplicate IPs or use reCAPTCHA’s, Sentry is the gold-standard for ensuring high data quality using behavioral measures in addition to technological detection.
Sentry works as a pre-survey instrument that puts participants through a series of check questions in under 30 seconds, using a large database of validated questions, to ensure that participants are attentive and honest. Sentry can also automatically score open ends, and measure whether participants are duplicates, copy-pasting, going off screen, or translating questions into another language. Sentry can be applied to filter data coming from any sample source and it is the backbone of quality across our products.
When to Use Sentry
When should you use Sentry? Whenever you can! It is already included with all studies run through Prime Panels, but it is designed to integrate with any source of online participants. Adding Sentry to your survey is recommended any time there are concerns about poor data quality. Even if you seek your sample outside of CloudResearch, Sentry can be easily applied to filter all the data you collect. Because it prevents bad actors from ever entering your survey, it cuts down on the time needed to reconcile participants or clean data. We also have special Sentry modules (available by request) designed for many niche B2B, consumer, and medical target populations so that you can be confident the people in your survey are who they say they are.
So, there you have it, the main products CloudResearch has to offer and the best way to use each one. If you’d like more details, request a demo or check out the resources below!
How to Cite CloudResearch in Your Research Papers
How are Participants on Prime Panels Compensated?
Survey Screening Questions: Good & Bad Examples
The Good, Bad, and Ugly of Attention Check Questions: Guidelines and Examples
How to Fairly Identify Low-Quality Research Participants
(Example of Sentry in use): No, Americans are not gargling bleach: How bad data inflated estimates in the latest CDC report, and how to prevent this from happening in the future
