Read our latest articles for tips, tricks, advice and news about research techniques and solutions.
Participants call it "bubble hell"—pages of tiny circles that make surveys miserable. Learn what survey methodology research says about designing better studies.
Read More
Learn how Completion Branches on Connect pay participants fairly for partial completes, screen-outs, and failed attention checks.
Read More
Before you analyze survey data, you need to clean it. A practical guide to reviewing attention checks, flagging bad responses, and making exclusion decisions.
Read More
A Verisoul-hosted webinar reveals how fraud farms escalate tactics to evade detection — and why layered, behavior-based defenses matter most.
Read More
Most researchers use attention checks without ever validating them. Learn how benchmark validation, spectrum thinking, and a fit-for-purpose framework can improve your online survey data quality.
Read More
A new CloudResearch and Siena exploratory study asked New Yorkers about their finances, then let them explain, in their own words, why they feel worse off than a year ago.
Read More
AI agents can now complete surveys, passing 99.8% of attention checks. Part 2 of the data quality crisis explores AI as both threat and tool.
Read More
I've spent 8 years at the center of the data quality crisis. Here's what survey fraud actually looks like—from click farms to video interviews with fraudsters.
Read More
Where do researchers find online participants? Compare market research panels, MTurk, Prolific, and CloudResearch to find the right fit for your study.
Read More
The Singularity isn't a single moment — it's a rolling frontier crossing one domain at a time. Chess, Go, protein folding. Survey research is next.
Read More
Qualitative research captures depth. Quantitative research captures scale. AI is making it possible to have both. Learn how conversational AI is transforming surveys.
Read More
Panel companies have known about the data quality problem for years — and kept laughing about it. CloudResearch explains why, and what accountability actually looks like.
Read More
A surgeon performed fake knee surgeries to test if the real ones worked. They didn't. Learn how experiments separate what works from what we believe works.
Read More
A CloudResearch study shows that pairing Sentry's behavioral screening with Verisoul's device-level fraud detection flagged 84% of low-quality respondents.
Read More
CloudResearch detected autonomous AI agents attempting to complete surveys — less than 0.1% of traffic, but a clear signal. Here's what we're seeing and what it means.
Read More
AI agents can now take surveys with one prompt — and 30–40% of online surveys are already fraudulent. The Bot Olympics is the industry's $50K answer.
Read More
How did scientists prove smoking causes cancer without running an experiment? Learn the logic of causal inference and how to build a case when experiments aren't possible.
Read More
CloudResearch is retiring the MTurk Toolkit by end of 2026. Learn why we're sunsetting the product, what it meant to us, and how Connect, Engage, and Sentry are the future of high-quality survey research.
Read More
Not every data quality problem in survey research is a bot problem. Learn the four-part taxonomy — Bots, Cyborgs, Zombies, and Humans — and why the distinction matters.
Read More
Post-collection bot detection hands researchers a magnifying glass. CloudResearch explains why preventing AI infiltration before data collection is the only real fix.
Read More
Learn how behavioral scientists use correlational research to find real patterns in human behavior—and avoid the traps our brains set for us.
Read More
What are reliability and validity in research? Learn the foundations of measurement through Thomas Edison's famous failure—and a 2023 polling disaster.
Read More
Discover why participants love Connect. Read real testimonials about earning extra income, engaging studies, and a better research experience.
Read More
What is descriptive research and why does it matter? Learn how behavioral scientists describe the world systematically—and what Civil War battles teach us about getting it right.
Read More
CloudResearch is coming to a kids channel near you with Panel Patrol — a new show about defending surveys from bots. (Okay, not really… but data quality matters.)
Read More
In this workshop recap, learn how to launch high-quality online studies—from choosing participant platforms to detecting survey fraud.
Read More
AI agents can now take surveys with ease, but the biggest threat to online research data quality is still human fraud. Our Insights Association webinar explored both challenges and the layered defenses needed to protect research integrity.
Read More
A reflection on the moment when scientific researchers shift from submitting papers and navigating peer review to being recognized as authorities—when leading journals and science journalists come seeking your expertise.
Read More
From steam engines to AI, technology has shaped how scientists study the mind. Explore the modern tools of behavioral research and how they've transformed the field.
Read More
Without a ground truth, you're not doing quality control — you're guessing. Learn why Sentry is the only data quality solution with face validity, and why evidence should always win over authority.
Read More
What is behavioral science? Learn how one researcher's unexpected journey from construction work to CloudResearch reveals why the scientific study of human behavior matters — and how you can start learning the methods for free.
Read More
A ChatGPT conversation about figure skating turned into an unexpected mirror, revealing blind spots about how facts alone don't move people.
Read More
New research reveals that 39% of Americans would press a button to stop all AI development or would seriously consider doing it if they could.
Read More
On December 17th, 2025, CloudResearch hosted an MRII webinar in response to a paper in the Proceedings of the National Academy of Sciences (PNAS) reporting that AI agents passed 99.8% of attention checks, examining the implications for survey research and how CloudResearch is staying ahead of this emerging threat with purpose-built data integrity solutions.
Read More
As LLMs show hidden and deceptive behavior, new risks are emerging. Learn how CloudResearch is building defensive AI to protect data quality and research integrity.
Read More
CloudResearch has developed the industry's first comprehensive, platform-integrated solution for detecting and preventing AI agent participation in online surveys. In controlled testing, our system achieved a 100% success rate in detecting AI agents while maintaining a false positive rate of less than 1%.
Read More
Discover why the AI bot threat to survey research is real but not widespread yet, what new PNAS findings mean, and how CloudResearch Engage and Sentry protect data quality.
Read More
Discover why true leadership means anticipating future threats before they emerge. Learn how CloudResearch uses Red Team/Blue Team strategies to protect data quality long before crises hit.
Read More
New data reveals Americans are prioritizing harmony at Thanksgiving tables despite political tensions. Discover the five key strategies families use to maintain peace during holiday gatherings, based on a nationally representative survey of 1,065 Americans.
Read More
OpenAI and MIT partnered with CloudResearch Connect to conduct a longitudinal study examining how regular ChatGPT use affects emotional well-being. Learn why OpenAI chose CloudResearch for this ambitious 28-day research project involving 981 participants and over 32,000 surveys.
Read More
Beacon Research partnered with CloudResearch to survey over 600 New York City voters using the AI-powered Engage platform, uncovering an election driven by two ...
Read More
CEO Jonathan Robinson explores the unstoppable advance of AI, its risks and promises, and CloudResearch's responsibility to shape an ethical path forward—balanc...
Read More
CEO Jonathan Robinson, PhD, reflects on AI misjudging reality, emphasizing his belief that truth, clarity, and independent thinking—free from ideology and socia...
Read More
CloudResearch's AI-powered Engage platform surveyed Americans after Charlie Kirk's murder. Most expressed sympathy, yet others justified it, revealing generatio...
Read More
With expanded teams and proven leadership across Connect, Engage, and Sentry, CloudResearch is positioned to deliver quarterly innovation—combining data quality...
Read More
CEO Jonathan Robinson, PhD, explores how quantum computing will transform discovery, revolutionize medicine, supercharge AI, and solve once-impossible problems ...
Read More
Co-founder Jonathan Robinson explores how nuclear fusion is advancing rapidly, offering clean, limitless energy—and why this breakthrough could transform global power structures, economies, and our AI-driven future.
Read More
AI is advancing faster than the public realizes. Within two years, most knowledge work may be automated—forcing companies, workers, and society to confront dram...
Read More
CloudResearch's AI-powered Engage platform together with a Siena College poll, capture the rich, human stories behind New Yorkers' concerns about the economy th...
Read More
Five years ago, our team sketched out plans for an AI-powered conversational research platform. The vision was clear, but the technology wasn't there yet. We wa...
Read More
Explore data quality challenges in online experiments, attention checks and how the Connect platform improves participant attentiveness, reducing noise and enhancing experimental reliability.
Read More
Traditional attention checks are easy to spot and beat. CloudResearch's new fraud detection approach catches acquiescence bias (the tendency to say "Yes" to everything) using questions about fictional products and impossible scenarios that blend naturally into surveys.
Read More
CloudResearch used AI to gauge employee satisfaction at major big box retailers in the US.
Read More
CloudResearch, a leading online participant recruitment and data quality platform, today announced the launch of its innovative new feature called DoubleCheck.
Read More
The session "Cultural Blind Spots: The Hidden Factor in Fraud Prevention" at Quirks LA 2025 highlighted the often-overlooked role of cultural nuances in fraud detection.
Read More
At CloudResearch, we understand both the opportunities and the challenges, which is why we created Waves—a tool designed to streamline and automate multi-stage studies.
Read More
Starting January 1, 2025, the standard service fee on Connect increased to 40%. This adjustment supports our ongoing investment in delivering the industry's most advanced research platform, while maintaining our steadfast commitment to academic and non-profit research by keeping their service fee at 25%
Read More
CloudResearch conducted a study to better understand the people who support Luigi Mangione's decision to murder UnitedHealthcare CEO Brian Thompson
Read More
A postelection study from CloudResearch looks at why Trump won the 2024 election, exploring factors that drew Black, Latino, young, and Independent voters to his policies on the economy, immigration, and national security.
Read More
A new study using Connect participants reveals how users adapt their behavior to influence future recommendations, shedding light on the strategic interaction between users and recommendation algorithms.
Read More
CloudResearch uses its AI survey platform to explore why voters remain undecided and why some undecideds finally made up their minds.
Read More
A new study using Connect participants explores how common-sense beliefs about wealth creation influence retirement savings behavior.
Read More
A new study using Connect participants explores how Obsessive-Compulsive Disorder (OCD) influences Problematic Smartphone Use (PSU). Learn about the psychological factors contributing to excessive smartphone use and their implications.
Read More
CloudResearch conducted a first of its kind, longitudinal, online poll using AI-driven interviews to explore how voters' perceptions changed before and after the September 10th debate.
Read More
Siena College Research Institute and CloudResearch partner to use AI for polling, and explore how persuadables view Kamala Harris
Read More
Exploring political polarization and national unity through interviews with 80 Americans, this blog delves into the impact of recent political violence and the potential for bipartisan cooperation to heal the nation's divisions using Engage AI conversational surveys.
Read More
Explore key insights and quotes from persuadable voters interviewed using Engage, our AI Conversation Interview platform, following the presidential debate between Joe Biden and Donald Trump. Understand their opinions on candidates, policies, and media coverage.
Read More
Siena College Research Institute and CloudResearch partner to use AI for polling insights, and explore how double hater voters will impact the 2024 election.
Read More
A new study using Connect participants investigates the psychological mechanisms behind support for political violence in the U.S., revealing key correlates such as need for chaos, dehumanization, and social dominance orientation, while also discussing the role of political variables.
Read More
The GAN Face Database (GANFD) provides a free resource of realistic, computer-generated face stimuli with race and ethnicity manipulations. Developed by Art Marsden and colleagues, GANFD aids researchers in conducting more controlled and generalizable studies, balancing internal and external validity
Read More
A new study using Connect participants demonstrates an interesting phenomenon—buyers are willing to pay more for products and services that sellers enjoy creating, while sellers are willing to charge less for the same services.
Read More
A new study using Connect participants explores how age and gender biases affect young leaders, revealing critical insights into workplace discrimination.
Read More
CloudResearch's "Re-Engage" platform emphasizes human conversation in survey research, moving away from AI to restore authentic interactions. Learn more about this exciting shift!
Read More
CloudResearch unveils Engage-Link, a cortex implant, enables behavioral scientists to continuously collect data on human mental processes from diverse participants!
Read More
Discover how SOC2 compliance, alongside CloudResearch's commitment to security through certifications and best practices, ensures data security and customer trust in the digital age.
Read More
A new study using Connect participants explores the effects of the shift to remote work on employee performance and well-being, offering insights into the hybrid work era.
Read More
A new study using Connect participants explores tech and expertise in fashion for sustainability, focusing on fit and style preferences.
Read More
A new study examines how social media algorithms amplify angry, partisan, and outgroup hostile content contributing to affective polarization.
Read More
A new study using CloudResearch participants reveals how dual-time sales messaging boosts sales, offering key insights for effective promotions
Read More
A newly published study investigates the integration of democratic values into AI algorithms to reduce partisan animosity on social media platforms.
Read More
A new study using Connect participants links US democracy quality to vacation choices, showing modest impact on travel to states like Florida and Georgia due to political climate.
Read More
A new study using Connect participants explores the impact of coaches' and student athletes' misconduct on event attendance, highlighting the complex dynamics of ethical conduct and its implications in collegiate sports.
Read More
A new study using Connect participants reveals that the habitual checking of smartphones, driven by goals like staying informed or connected, can lead to both beneficial and harmful consequences, affecting productivity, safety, and social interactions.
Read More
A new series of studies using CloudResearch participants investigates how highlighting the impact of bias on victims can prompt individuals who benefit from positive bias to recognize and actively counteract these unfair advantages.
Read More
A new study examines two unexplored dimensions of causality related to holistic vs analytic thinking styles and the implication for corporate social responsibility
Read More
Connect offers access to high-quality survey participants in 5 new countries – Australia, Canada, Ireland, New Zealand, and the United Kingdom.
Read More
There are use cases for LLMs in online research but behavioral scientists have growing concerns about the impact large language models might have on data collection.
Read More
Learn more about Connect's capabilities as well as the vision and dedication that drives its success in our new white paper.
Read More
A recent time perception study examines the relationship between activity and mental time and explores the potential link to detecting depression and inspiring potential new treatments. Learn more from CloudResearch grant winners Yrjö Rinne & Michiel Spapé.
Read More
Market research panels are a great source of participants for studies that require larger samples, and highly-specific or international targeting. Our latest paper serves as a guide for how to effectively use market research panels for online research.
Read More
Are businesses losing billions to market research fraud? CloudResearch's Chief Research Officer, Leib Litman, uncovers the real-world impact fraudulent market research data has on the research community and society at large.
Read More
Learn more about Sentry, the secret to data quality across CloudResearch's products. Discover how Sentry improves online data quality from any sample source.
Read More
Recent peer-reviewed research articles demonstrate that CloudResearch consistently ranks best among participant recruitment platforms for data quality and cost. Learn more about why CloudResearch is the top choice!
Read More
Learn more about CloudResearch's products, including the best use case for each. Contact us for a demo today!
Read More
CloudResearch launches groundbreaking online research innovation using recent advancements in AI from products like ChatGPT. Learn more about virtual participant recruitment!
Read More
CloudResearch examines new Rasmussen poll on racial disparities which led to Scott Adam's Dilbert getting cancelled
Read More
Learn more about Prime Panels international sampling capabilities, which offers access to diverse high-quality participants in 19 different countries.
Read More
Humans have used information about other people's reputations for tens of thousands if not hundreds of thousands of years. Given people's fluency with tracking others' reputations it is not surprising that many tech platforms rely on reputation systems to help regulate transactions between strangers (think Airbnb reviews). For behavioral researchers who use Amazon Mechanical Turk (MTurk), the story is no different: many researchers rely on reputation metrics when sampling participants.
Read More
Data quality is a known issue within online panels. Considering the time and money research requires, gathering bad data is not a mistake you want to make. To avoid the consequences of bad data, you must take a proactive approach.
Read More
In March 2020, health authorities around the globe were quick to release recommendations to avoid COVID-19 infections. As many things about COVID-19 were uncertain, several recommendations were based on research on other infectious diseases, some of which are transmitted via smear infections. One of these recommendations that remains on the CDC's website to this date was to "avoid touching your eyes, nose, and mouth with unwashed hands."
Read More
Introducing Connect, a new participant platform for paid online surveys. Sign up now to start earning money for participating in interesting tasks and surveys!
Read More
CloudResearch is thrilled to introduce Connect – our new platform for online research! Connect is dedicated to delivering high-quality data, making collaboration among researchers easy, and pushing the boundaries of what is possible with online research. Sign Up Now!
Read More
Despite widespread media attention in recent years, polling conducted by CloudResearch in June of 2021 and again in 2022 shows that most people know little about the holiday Juneteenth. Only about half of the US population, for example, knows that Juneteenth marks the end of slavery. And only one fifth of people have plans to celebrate the holiday this year.
Read More
When it comes to data quality in market research, our team at CloudResearch thrives on creating solutions and leading important conversations. What better way to “talk shop” than joining experienced UX Researcher and industry thought leader, Matt Valle on his...
Read More
Psychologists have spent a lot of time studying how parenting affects children. Attachment theory, for example, may be one of the widest studied topics in all of psychology. But despite a longstanding focus on parenting and children, relatively little time has been spent thinking about how parenthood affects parents themselves. This is in many ways surprising.
Read More
CloudResearch and InnovateMR jointly announce a strategic partnership to improve the most pressing challenge facing the online survey industry: data quality.
Read More
Several years ago, we introduced the option for researchers to create a dynamic completion code when setting up MTurk Toolkit studies. While this system has worked well in the past, we are discontinuing its use over concerns about security and utility.
Read More
The never-ending research discussion: data quality. Hear from Cheskie Rosenzweig on the importance of data quality and how the industry can do more.
Read More
Learn more about the largest longitudinal online research project ever conducted, including tips for retention in longitudinal studies
Read More
a new study explores the effects of memes on COVID-related stress
Read More
a recently published study examines perceptions regarding the relevance of social psychology in society and the importance of social psychology in education
Read More
When decisions are made based on brand tracking surveys with low quality data it leads to false conclusions and costly mistakes.
Read More
Online data quality is a huge deal. You need to know how much online survey fraud exists, how it's measured, and what you can do to avoid it.
Read More
Learn more about MTurk participant data based on tens of millions of HITs facilitated via CloudResearch.
Read More
Learn more about why CloudResearch is retiring the Block Low-Quality Participants feature and continuing with the CloudResearch Approved Participants option.
Read More
Filtering through bad data in hopes to preserve the validity of online research can threaten your project, and just be plain time-consuming.
Read More
A new study published in Nature Human Behaviour examines the impact of the pandemic on paranoia and belief updating!
Read More
Starting with the methods section of your academic paper can make the process easier—we're here to help you accurately cite research samples.
Read More
Learn how to verify participant age in online studies using a new method from CloudResearch.
Read More
Learn how to construct survey screening questions. We also demonstrate examples of good and bad survey screening questions.
Read More
Learn about the CloudResearch internship experience, helping students make informed decisions about staying in academia vs. going to industry
Read More
We are thrilled to announce our Innovations in Online Research Conference and our snippet: >- 0,000 CloudResearch Grant Award!
Read More
Understanding survey reliability and validity is important for people who conduct and consume research. Learn more about the details of both constructs here.
Read More
Nonresponse bias can impact even well-run research. Learn how you can avoid this fundamental risk to data quality in your online surveys.
Read More
CloudResearch updated data examines if there have been any changes in the composition of people taking studies on MTurk.
Read More
How much is your audience willing to pay for your product or service? Brands often struggle to answer this crucial question; online pricing surveys offer a fast, efficient solution.
Read More
The COVID-19 pandemic has affected everything about how we work. New research from the CloudResearch team dives into its effects on MTurk worker demographics
Read More
Participant retention is a key factor in any longitudinal study, particularly for researchers operating online. Learn 4 key strategies to improve your retention rate.
Read More
In 2020, the Covid-19 pandemic initiated an unprecedented shift in the American workforce. In a matter of weeks a significant portion of the labor force shifted to working online, a change likely to remain in place for the foreseeable future. Although many of these workers are continuing to perform their pre-pandemic jobs from home, an ever-expanding number are participants in the gig-economy.
Read More
Reply to MTurk, Prolific or panels? Choosing the right audience for online research
Read More
Measuring brand awareness is vital in today's competitive digital landscape. Learn more about how to conduct online brand awareness surveys here
Read More
Learn how and why the election polls in the 2020 presidential race underestimated the Trump campaign. Read more here.
Read More
Keep these five common misconceptions and mistakes in mind when reading polling results to develop a more critical eye for interpreting poll data.
Read More
Join CloudResearch to discuss common pitfalls when working with tight timelines and learn best practices to ensure quick fielding without sacrificing data quality.
Read More
Attention check questions are a valuable tool for ensuring data quality in surveys, but only if done right. See specific examples of good (and bad) attention checks here.
Read More
A majority of Republicans believe unsubstantiated claims about election fraud and voting corruption, and that Trump is the real winner of the 2020 election
Read More
The country is divided on whether the election was conducted fairly. Vast majority of Republicans believe that “President Trump is being cheated out of winning the election.” Vast majority of Democrats think the election was fair.
Read More
Discover 2020 retail shopping trends through Prime Panels, an online survey platform offered by CloudResearch. More here.
Read More
In this article, we take a deeper dive into whether or not the oft-debated “shy Trump voters” exist, and how they might be affecting poll results in the crucial swing state of Pennsylvania.
Read More
According to an October poll, the majority of Americans are anticipating a contested 2020 Presidential Election result. See how attitudes toward this event, and the potential effects of it, differ by party.
Read More
Learn why survey data cleaning - separating high-quality from low-quality responses - is important in order for researchers to draw accurate conclusions from data.
Read More
As the COVID-19 pandemic keeps behavioral researchers out of the lab, you may be wondering how to take your study online. Follow our simple guide to get started.
Read More
CloudResearch's MTurk Toolkit is now fully compatible with AWS billing. See how you can connect your account and make funding your research simple.
Read More
The ‘shy Trump voter' was a common topic of poll analysts following the 2016 presidential upset – but do they exist? New research helps shed light on polling accuracy and honesty.
Read More
Academic researchers have a major ethical decision to make when choosing whether to reject research participants. Learn how to be certain you're making a fair choice.
Read More
CloudResearch users are now able to quickly and easily sample respondents in cities identified as COVID-19 hotspots, giving researchers new insight into the human effects of the pandemic.
Read More
Amazon's MTurk has given researchers a powerful tool for connecting to survey respondents online. See how six simple best practices can help improve your data quality.
Read More
Can using Amazon's MTurk to conduct online research cause distress in survey respondents? A new study from CloudResearch shows that for most workers, the answer is no.
Read More
New solutions from CloudResearch help researchers obtain high-quality data on MTurk while blocking inattentive and otherwise low-quality respondents from participating.
Read More
Surprisingly little is known about the psychological impact of quarantine. Read our research on the consequences of prolonged isolation.
Read More
Explore the numbers behind superworkers on Amazon mturk and learn how these professional survey respondents can bias your behavioral research sample.
Read More
A recent CDC report made the alarming claim that 39% of Americans engaged in high-risk behavior to avoid COVID. A new study from CloudResearch questions these claims.
Read More
Take a behind the scenes look at the Amazon Mturk worker pool. CloudResearch investigates turker trends by gender, age, and other key demographics.
Read More
Conducting Online Research on Amazon Mechanical Turk and Beyond is a comprehensive resource for any researcher looking to source data from online platforms.
Read More
To support academic researchers during the ongoing COVID-19 pandemic, CloudResearch is waiving service fees for graduate students until the end of May.
Read More
The popularity of Amazon's Mturk as a source of research participants has raised recent concerns about naivety. See how researchers can still utilize the platform.
Read More
The speed of modern online data collection platforms introduces a new wrinkle for survey-based research: time of day bias. See how to control for it using CloudResearch.
Read More
While it seems like the gender pay gap would close in an anonymous, online environment, a study from CloudResearch sheds new light on wages and gender on Amazon MTurk.
Read More
A data quality analysis of Mturk and our Prime Panels tool. See how this alternative online research platform stacks up against Amazon's Mturk.
Read More
In this blog, we describe how to rotate API keys for security on Amazon Mechanical Turk.
Read More
Amazon Mturk has become a go-to platform for researchers sourcing survey participants, but how many workers use it? See estimates from our recent study here
Read More
One feature of Mechanical Turk is the ability of researchers to give workers bonus payments. Bonuses may be issued for various reasons such as exemplary performance on a task, answering extra questions in a study, or as a reward for...
Read More
By now, most people have heard of the gig economy and have some idea of how it works. In the gig economy, people perform short-term jobs or tasks to earn money. Gig economy jobs are considered independent or contract work, meaning people who work in the gig economy often trade the benefits and stability of traditional employment for the freedom and flexibility to decide when and how much they work.
Read More
A persistent cause of concern for researchers who conduct studies online is understanding what participants might be doing while completing their study. When participants are outside the lab, they cannot be observed and distracting aspects of the environment cannot be controlled by the research team. As a result, researchers are left to wonder: how much attention are participants giving my survey?
Read More
Last month, we published a blog titled, “Five Things You Should Not Be Doing in Online Data Collection.” Among the things we identified that you should not be doing was launching your study without piloting it first. As a way...
Read More
Academic research is a collaborative endeavor. Faculty members work with post-docs, grad students, and undergrads. Sometimes one lab collaborates with another. During the course of such work, resources sometimes need to be shared or redistributed. At CloudResearch, we have sought...
Read More
One reason Amazon Mechanical Turk has become so popular among researchers is the speed with which data can be collected. Compared to more traditional research methods — lab-based experiments, field studies, ethnographic interviews, etc. — MTurk is exceptionally fast, making...
Read More
Overview of CloudResearch Dashboard changes and navigation tips for improved user experience.
Read More
Three weeks ago, we published a blog explaining five things you should be doing in your online data collection. In this blog, we follow up with five things you should NOT be doing when collecting data on MTurk.
Read More
What Is a Survey Group? Survey Groups are one of the most powerful and dynamic tools on CloudResearch for controlling which workers are eligible and ineligible for your study. A Survey Group is exactly what it sounds like: a collection...
Read More
A new year represents the opportunity to consider priorities, set goals, chart new courses of action, and decide how to move forward in the coming months. At CloudResearch, we're moving into 2019 looking for ways to expand the tools and...
Read More
Researchers are responsible for being an expert, or at least knowledgeable, in several areas. There's the topic of your research, the methods common within your discipline, best practices for open science, and the mediums used to communicate about your work—just to name a few. For many researchers, online data collection has been revolutionary, helping collect data faster and more affordably than ever before.
Read More
In this blog, we explain everything you could ever want to know about including and excluding participants from studies while using CloudResearch. In last week's blog on longitudinal studies, we described our Include Workers feature, but this blog digs into the nitty-gritty and explains what our features are, when you might want to use them, and how they work.
Read More
In this blog, we describe how to run a longitudinal study on MTurk, using CloudResearch. We also provide tips for maximizing worker participation and minimizing attrition.
Read More
In this blog, we outline the history of our Block Duplicate Geolocations tool, provide an overview of what geolocations are and the information they convey, and present the results of a study that examined the quality of data obtained from repeated geolocations that are not linked to server farms. We conclude by outlining the steps we are taking to change the default options on our Block Duplicate Geolocations feature.
Read More
Most social science research relies on convenience sampling of participants, meaning few samples look like let alone represent the general population. For many research questions, convenience samples are not a problem. Yet, for other questions, being able to capture and represent the opinions of people from different groups is essential. Because most researchers do not routinely gather these kinds of samples, knowing where to find one when it's needed can be difficult.
Read More
People of different ages vary greatly in their beliefs and behaviors. For example, a recent Pew report outlines wide generational gaps in people's opinions on several political issues like presidential job approval, perceptions of racism, views on immigration, and political ideology (Pew Research Center, 2018). Furthermore, some issues, like the use of Medicare, depend on age and therefore are more relevant to older adults than younger ones.
Read More
At CloudResearch, we advocate for requesters to treat workers fairly when posting HITs on Amazon's Mechanical Turk (MTurk). Workers are, after all, the people who make the research possible. Sometimes situations arise in which an MTurk worker is unable to receive payment, despite having completed a survey. Below are two common scenarios in which a worker may not be paid, despite completing a survey:
Read More
Studying pairs of people (e.g., married couples, friends, coworkers, etc) is becoming increasingly common in the social and behavioral sciences. Online participant populations, such as Mechanical Turk and other online panels, can potentially serve as a rich source of dyadic...
Read More
About a month ago, we published After the Bot Scare blog on workers providing bad quality data on Amazon's Mechanical Turk. This month, we followed up with our “farmers” to assess the effectiveness of the tools we created to deal with the problem. In this blog, we present data from our follow-up study and evidence to suggest our tools are working.
Read More
A Case Study From a Recent JESP Article A new study appearing in the Journal of Experimental Social Psychology suggests Americans strongly believe in economic mobility because they fail to appreciate how vast wealth inequality really is. In this blog,...
Read More
By now, even casual users of MTurk have heard about recent concerns of “bots” or low quality data. We've written about the topic here and laid out evidence that suggests “bots” are actually foreign workers using tools to obscure their true location (here). Perhaps most importantly, we've created two tools to help keep these workers out of your studies. In this blog, we introduce a third tool: the Universal Exclude List.
Read More
In this blog, we review recent concerns about bots, along with general data quality issues, on Mechanical Turk and report the results of a study we conducted to investigate the problem.
Read More
Last week, the research community was struck with concern that “bots” were contaminating data collection on Amazon's Mechanical Turk (MTurk). We wrote about the issue and conducted our own preliminary investigation into the problem using the CloudResearch database. In this blog, we introduce two new tools CloudResearch is launching to help researchers combat suspicious activity on MTurk and reiterate some of the important takeaways from this conversation so far.
Read More
When researchers collect data online, it's natural to be concerned about data quality. Participants aren't in the lab, so researchers can't see who is taking their survey, what those participants are doing while answering questions, or whether participants are who they say they are. Not knowing is unsettling.
Read More
CloudResearch is announcing a change in our pricing for the MicroBatch feature. MicroBatch is now included as a Pro feature, with a fee of 2 cents + 5% per complete. This will also provide users with access to all other...
Read More
When researchers learn about conducting research online, it can sometimes be difficult to understand quite how all the tools that are available can actually be applied to a project. This post is about how specific research ideas can be carried...
Read MoreSome workers on MTurk are extremely active, and take the majority of posted HITs. This can lead to many issues, some of which are outlined in our previous post. Although MTurk has over 100,000 workers who take surveys each year,...
Read MoreIt is important to consider how many highly experienced workers there are on Mechanical Turk. As discussed in previous posts, there is a population pool of active workers in the thousands, but this is far from exhaustible. A small group...
Read MoreThe internet has the reputation of being a place where people can hide in anonymity, and present as being very different people than who they actually are. Is this a problem on Mechanical Turk? Is the self-reported information provided by...
Read MoreHundreds of academic papers are published each year using data collected through Mechanical Turk. Researchers have gravitated to Mechanical Turk primarily because it provides high quality data quickly and affordably. However, Mechanical Turk has strengths and weaknesses as a platform...
Read MoreRun Studies Targeting Specific Big Five Personality Types CloudResearch introduces a new Big Five personality types qualification: Now social science researchers can run studies targeted to the Big Five: Extraversion, Agreeableness, Conscientiousness, Openness, and Neuroticism. Each personality type can be specified...
Read More“Restart HIT” Using HyperBatch The CloudResearch “Restart” feature (which Restarts HITs that have become sluggish; see our blog post on Restarts to see all the benefits of this feature) can now be used in conjunction with HyperBatch. This was a...
Read MoreMany researchers wish to target participants from specific states or regions of the United States like the Northeast or the West. The problem they often encounter is that using MTurk’s Geographic Qualification to specify a particular state is often not...
Read MoreCloudResearch now supports dynamic completion codes for SurveyMonkey and Alchemer (formerly SurveyGizmo) studies. Dynamic codes ensure each MTurk worker receives a unique completion code, eliminating code sharing and enabling auto-approval...
Read MoreProblem: Requesters may observe that some workers, even those with high Approval ratings, may not perform to their expectations on a study. Sometimes this may result in rejecting their work which affects the Worker approval rating. But, often the work...
Read MoreProblem: You are running a longitudinal study and have identified 1000 workers who you want to allow to take your second phase studies. How do you easily group those workers for easy access. Or you want to exclude certain workers...
Read MoreStudies with Panels for just $0.15 – 0.75 / complete Now you can run Mechanical Turk studies using your own Requester account and specify over two dozen demographic traits!. The traits include gender, ethnicity, age, marital status and sexual orientation....
Read MoreHave you ever wanted to run multiple studies simultaneously and make sure that each worker only takes a single study? On MTurk, there is no simple way to block workers who completed one study from accepting and completing another study being run at the same time....until now.
Read MoreGoogle Forms can be used to deliver a study with CloudResearch in a similar manner to other survey platforms (like Qualtrics and SurveyMonkey). In Google Forms set up your survey and then set up the secret completion code display: Click...
Read MoreEver wonder if workers are being honest with you when they answer a survey? Or, if you specify that your study should be taken only by women, whether some workers take the study even though they are not women?
Read MoreMany researchers set up their studies to use the TurkPrime AutoApprove feature so that they do not need to manually approve worker assignments based on the secret code that workers enter. On occasion, a researcher may set up his study...
Read MoreWe recently launched a ground-breaking feature that helps protect Mechanical Turk worker identities. It has been reported in the literature that Mechanical Turk Worker IDs can be used to identify the worker. This is because Amazon uses the same value...
Read MoreOur most popular feature request has been to allow the blocking of multiple workers from the same IP address.
Read MoreYou can now run your Amazon Mechanical Turk studies at Hyper speeds. We worked hard to make the experience identical to our classic MicroBatch option so that you simply launch your study and TurkPrime does all the heavy lifting for you.
Read MoreCloudResearch can now automatically add Worker ID, HIT ID, and Assignment ID to data files, including CSV and SPSS files. These variables can be very useful when you want to match Workers across multiple data files. For example, in longitudinal studies,...
Read MoreTurkPrime has just released a Copy feature which makes duplicating past TurkPrime HITs simple and fast. To copy the settings from an old survey to a new one, follow these simple steps: Go to the HIT you want to copy...
Read MoreWe have deployed a micro-batching feature which allows Requesters to break up their studies into smaller segments, and to include time intervals between the segments. We developed this feature to increase sample representation. This feature can also save 50% in MTurk fees.
Read MoreIn the latest release of CloudResearch.com we added many new features and fixes among them 2 additional metrics for every survey: Bounce Rate Completion Rate The Bounce Rate is the percentage of Amazon workers who previewed your survey but decided...
Read More
Problem: How can a Requester who rejected an assignment in error undo his mistake? A rejected assignment affects the Worker negatively and will often impact the Requester with negative feedback which can damage the Requester’s online reputation which lowers Worker...
Read MoreProblem: How can you increase Amazon Mechanical Turk HIT Worker participation rates and speed completion of a HIT? This is particularly an issue with HITs that have a large number of required participants or have Qualifications that limit the number...
Read MoreRun Qualtrics Surveys on MTurk using CloudResearch Running Qualtrics surveys on CloudResearch streamlines several major pain-points researchers using MTurk have dealt with: Researchers can setup a dynamic secret code so that each Worker receives a unique code upon completing the study....
Read MoreProblem: Suppose you’re running a Mechanical Turk survey and need to exclude workers who took a previous survey. How can you quickly set this up. Some of the currently used solutions require following multiple steps to set things up and...
Read MoreProblem: Suppose you need to run a survey on Mechanical Turk and follow up with the same workers a week, month or year later. A few issues come up: How can I limit the follow up surveys to survey takers...
Read MoreProblem: Suppose you need to run a HIT with 1000 Workers. Or a HIT that is only open to Workers who have an approval Rating of 95% or more and have completed 500 HITs or more. Although when you launched...
Read MoreProblem: Suppose you need to run a group of HITs open only to participants who are women under 50. You previously ran a HIT and know the Worker IDs that you want to reach, but have no way to email...
Read MoreAn IRB will generally request a description of how participants will be recruited, reimbursed and interacted with. Additionally IRBs always request information about how anonymity of the participants is protected. Members of the IRB board may not be familiar with Amazon Turk, and it may be helpful to include a brief description of MTurk in your IRB application.
Read MoreIn this blog, we report on the demographics of Mechanical Turk workers by reviewing 75 studies conducted with US-based Workers in 2013 and 2014. From a total of 32,595 Workers, we found 15,324 were female (47%).
Read MoreWe describe a general formula for predicting the time it takes Workers to complete survey studies on MTurk. The average Worker takes 10.3 seconds to answer a single question. This means that a study with 60 questions should take approximately 10 minutes.
Read MoreIt is generally thought that pay rate does not affect data quality on Mechanical Turk. For example (Buhrmester, Kwang, & Gosling, 2011) showed that whether Workers are paid 5 cents or one dollar for a survey study, the internal reliability of the surveys does not change. They did show however that fewer Workers will take the surveys that pay less. We recently replicated these findings for both US and India-based Workers (Litman et al, 2014).
Read MoreWhat is the completion rate and dropout rate? Dropout rate is defined as the percentage of participants who start taking a study but do not complete it. Dropout rate is sometimes referred to as attrition rate, and is the opposite...
Read MoreMTurk Requesters are often interested in studying specific groups of people. For example, a researcher may be interested in men over 40, Republicans, people who are concerned about the cleanliness of sponges, or cancer survivors. CloudResearch Panels utilizes various techniques...
Read MoreAmazon just announced that their Worker Qualification now supports US State locations. It is currently available through their API and is also available through their Web Interface. It is great to see that Amazon is adding features to their API;...
Read MoreShowing 10 of 223 articles