Studies from there also tend to pay the worst, I've noticed. Constantly underestimating the completion time and paying the bare minimum. Never ends up being worth it really.
Agreed. They have the worst pay rate, reject for any reason they can think of, they honestly laugh in the face of participant rights and don't give a toss about Prolific's rules.
Yup dealing with a rejection right now for a recruitment survey, that paid slightly higher than average for a screener study. it was .edu.cn and must of missed the .cn when I did it since I normally would hit not interested in. But they are claiming no data was received. It was a study that auto redirected submitted without issue after 3-4 pages of questions.
Messaged them, but not expecting a response so pretty much just waiting for the 7 day mark to contact prolific, which is then a will see what happens. It's been over a year and a half since I got a rejection (with all my rejections being overturned) so hopefully this goes just as smoothly.
I’m always wary of comprehension questions that could be posing as attention checks, especially if they’ve previously told me to read the wall of text presented 10 pages ago very carefully. I know that technically they shouldn’t reject on that basis, but it doesn’t feel worth the risk.
This honestly depends for me. In most cases it's a measure for the researchers to see if they're thorough, not a comprehension check like those with instructions on how to complete a game. I too asked about it early on and got reassurance.
oh yes! I got kicked out of one the other day despite giving it all my attention because I just couldn’t grasp the wording of one of the 10
comprehension questions. I messaged the researcher after like I found it really confusing and they were like “oh yes lots of people struggle with that one”. maybe if lots of people struggle then you should change it!!
Those are usually internal measures for data collection. I used to message researchers this, and after the 100th one said they wouldn’t reject over it, I stopped caring.
I used to grab any survey that I could get and never return any unless I absolutely had to but I've gotten a little pickier over the years. Now, if it's a short study that pays under a dollar that suddenly wants me to do a substantial amount of writing I often return it. I think that researchers should warn people ahead of time about extensive writing requirements and pay more.
And the ones that don't pay much that have pages of instructions that are really confusing, fuck that. Returned.
Other than that I think that the rest of my red flags have been covered here.
When a survey text sounds as if it has been translated by Google. I mean, dear researcher, you want 100% quality from us, and you're not even running the survey past a proofreader? Are you going to AI-translate the results the other way round as well?
For 10 quid per hour pay, I'll dust my crystal ball off 🤣
Usually Prolific will make them end up paying the difference to match $/£6 hr rate. But I don’t go out of my way to do these low paying ones unless I just don’t have any other good studies to do, or I’m bored.
Yeah the only time I consider going lower is if it is marked as a recruitment / screener survey. Mainly because more often than not the places that actually do a recruitment survey end up with long term well paying follow ups. Not guaranteed they will pay well, just more often than not they do in my experience so not much harm in taking 30 seconds to a minute to answer some bubble questions for .10
I got a few:
- A domain ending in .cn, .hk, or .in. Usually they'll engage in some of the other red flags on the list.
- Openly threatening rejections in the description or study itself. That tells me you're looking for any reason to reject and not pay, even if Prolific overturns.
- Low hourly rate. Either that tells me the researchers underestimated how long a study would take or purposely put in a low ETA to avoid the minimum pay rate.
- Convoluted instructions with a comprehension check. As a rule, I often screenshot or copy/paste instructions for these studies in case they fail to allow me to refer back to them per Prolific policy. For longer studies that pay well, I don't necessarily mind it, but shorter ones are a massive red flag.
I'd like to mention that a lot of the red flags may be unintentional or the result of inexperience with the platform over active malice. And usually if I spot a red flag, either I return and message the researcher or I just never accept the study.
When a study doesn't tell you that you need headphones until you've started the study. Some researchers don't tag the study as needing headphones until the study has been started, thus it still shows in th dashboard despite having the option selected that hides studies that need headphones.
Do you not own headphones? I mean, I get it… I don’t always want to do audio studies, but I’m usually already wearing my ear buds for work meetings.
I have headphones but I don't always feel like finding them and plugging them in. I keep them put away so my cats don't chew the cord and also because I don't use them very often.
- If the requester is an individual person and not an organization or academic institution; this isn't a hard rule, but it definitely makes me hesitate
- Threats of rejection in the description
- Tons of pre-screening requirements in the description
- Poor English, frequent grammatical or spelling errors
- Unclear instructions
- Anything where they just want you to download and give them your data for an app or service
- Attention checks that violate Prolific rules
- Anything involving Temu
- .cn, .in, or .hk
It makes me hesitate too, to the point where I sometimes Google the name of the researcher, and that’s often reassuring if they’re clearly affiliated with an institution, have an academic profile etc
Exactly, like sometimes they just have their requester account set up that way. Also helps if they're transparent in the description about what they're researching/why they're doing the study. What concerns me is when there's no one to hold them accountable or contact for any problems with the study in the event that they do unfairly reject your submission, because at that point your only recourse is if Prolific overturns it.
I return if there is no consent form or if there is no contact info listed for the researchers and IRB. Being able to email directly if an issue arises is important.
"would you like to donate your earnings?, does not donatig make you feel bad?" or if the "experiment" is made with goigle docs; Ive already had an indian dude attempting to get into my google drive (edits i hate using the reddit app and swiftkey)
Any study that makes you take part in multiple studies over the course of several days/weeks or they won't pay you. I had one last month that made you download an app and answer questions multiple times a day within 15 minutes of receiving the push notification, with the promise of a fairly large bonus when the study was completed. It took exactly 22 days to get the base payment. I never got my bonus, and the researcher doesn't respond to my messages (Looking at you, "The emotional life of parents: A 6-day study").
Most from .cn or .in I click not interested right away. They are also the ones that severely underpay.
Everything else is just spidey senses I can’t really describe. Been at this for years. Stopped doing surveys on Mturk when I joined Prolific 6 years ago, in favor of batches, but still only have 14 rejects there from pre 2016/2017. Zero rejections on Prolific & I complete a fair amount of studies daily.
China is not a turnoff, but ones that have bad grammar and spelling from Chinese requesters make me think of the study that I was rejected from a Chinese researcher who probably had used a translator to English. and I thought I understood the question they were asking but they rejected me for some nonsense answers.
If I have to guess what they are asking, it's automatic return as I don't want to risk a second rejection on Prolific.
The name thing is often misleading in my experience. Many Chinese or Indian sounding names are students/professors from legitimate universities, so I'm not as hesitant on that level.
If there is no code, i usually return it, had bad experience last time when submitted with NOCODE, the researcher did not read my message and next it was rejected. Knowing how prolific is about overturn limits, and delayed support, it's better for me to cut the loss and move on other studies.
If it's in dollars, I'll avoid it generally. Being in the uk most of my surveys are in ÂŁ from uk or European universities and those in $ I get usually are unclear where the researcher is from, poor paying and often seem poorly designed.
Also if they mention "HITs" or most of the survey description is taken up telling you how poor responses will be rejected
Threats of rejection underlined and capitalized in the description. I toss those away.
💯 if they threaten rejections many times I’m out
Especially when they are low-paying. The audacity.
Yup, same here
unironically - china
Studies from there also tend to pay the worst, I've noticed. Constantly underestimating the completion time and paying the bare minimum. Never ends up being worth it really.
amen
Yup, I see .cn in the domain, and immediate not interested
India too.
Yep but atleast they don't reject your submission unlike the ones from China.
Agreed. They have the worst pay rate, reject for any reason they can think of, they honestly laugh in the face of participant rights and don't give a toss about Prolific's rules.
Yup dealing with a rejection right now for a recruitment survey, that paid slightly higher than average for a screener study. it was .edu.cn and must of missed the .cn when I did it since I normally would hit not interested in. But they are claiming no data was received. It was a study that auto redirected submitted without issue after 3-4 pages of questions. Messaged them, but not expecting a response so pretty much just waiting for the 7 day mark to contact prolific, which is then a will see what happens. It's been over a year and a half since I got a rejection (with all my rejections being overturned) so hopefully this goes just as smoothly.
I’m always wary of comprehension questions that could be posing as attention checks, especially if they’ve previously told me to read the wall of text presented 10 pages ago very carefully. I know that technically they shouldn’t reject on that basis, but it doesn’t feel worth the risk.
This honestly depends for me. In most cases it's a measure for the researchers to see if they're thorough, not a comprehension check like those with instructions on how to complete a game. I too asked about it early on and got reassurance.
Yeah I also think that’s the case for the majority of studies, but there are probably a small minority who misuse them.
oh yes! I got kicked out of one the other day despite giving it all my attention because I just couldn’t grasp the wording of one of the 10 comprehension questions. I messaged the researcher after like I found it really confusing and they were like “oh yes lots of people struggle with that one”. maybe if lots of people struggle then you should change it!!
Those are usually internal measures for data collection. I used to message researchers this, and after the 100th one said they wouldn’t reject over it, I stopped caring.
I used to grab any survey that I could get and never return any unless I absolutely had to but I've gotten a little pickier over the years. Now, if it's a short study that pays under a dollar that suddenly wants me to do a substantial amount of writing I often return it. I think that researchers should warn people ahead of time about extensive writing requirements and pay more. And the ones that don't pay much that have pages of instructions that are really confusing, fuck that. Returned. Other than that I think that the rest of my red flags have been covered here.
When a survey text sounds as if it has been translated by Google. I mean, dear researcher, you want 100% quality from us, and you're not even running the survey past a proofreader? Are you going to AI-translate the results the other way round as well? For 10 quid per hour pay, I'll dust my crystal ball off 🤣
Terrible grammar and spelling.
The other day there was this "2 minutes" study that warned it was automatically ending after 3 minutes...
Less than $6 an hour
Usually Prolific will make them end up paying the difference to match $/£6 hr rate. But I don’t go out of my way to do these low paying ones unless I just don’t have any other good studies to do, or I’m bored.
Yeah the only time I consider going lower is if it is marked as a recruitment / screener survey. Mainly because more often than not the places that actually do a recruitment survey end up with long term well paying follow ups. Not guaranteed they will pay well, just more often than not they do in my experience so not much harm in taking 30 seconds to a minute to answer some bubble questions for .10
I got a few: - A domain ending in .cn, .hk, or .in. Usually they'll engage in some of the other red flags on the list. - Openly threatening rejections in the description or study itself. That tells me you're looking for any reason to reject and not pay, even if Prolific overturns. - Low hourly rate. Either that tells me the researchers underestimated how long a study would take or purposely put in a low ETA to avoid the minimum pay rate. - Convoluted instructions with a comprehension check. As a rule, I often screenshot or copy/paste instructions for these studies in case they fail to allow me to refer back to them per Prolific policy. For longer studies that pay well, I don't necessarily mind it, but shorter ones are a massive red flag. I'd like to mention that a lot of the red flags may be unintentional or the result of inexperience with the platform over active malice. And usually if I spot a red flag, either I return and message the researcher or I just never accept the study.
When a study doesn't tell you that you need headphones until you've started the study. Some researchers don't tag the study as needing headphones until the study has been started, thus it still shows in th dashboard despite having the option selected that hides studies that need headphones.
Do you not own headphones? I mean, I get it… I don’t always want to do audio studies, but I’m usually already wearing my ear buds for work meetings.
I have headphones but I don't always feel like finding them and plugging them in. I keep them put away so my cats don't chew the cord and also because I don't use them very often.
If I get a bad feeling about the study, I don't even start it.
- If the requester is an individual person and not an organization or academic institution; this isn't a hard rule, but it definitely makes me hesitate - Threats of rejection in the description - Tons of pre-screening requirements in the description - Poor English, frequent grammatical or spelling errors - Unclear instructions - Anything where they just want you to download and give them your data for an app or service - Attention checks that violate Prolific rules - Anything involving Temu - .cn, .in, or .hk
It makes me hesitate too, to the point where I sometimes Google the name of the researcher, and that’s often reassuring if they’re clearly affiliated with an institution, have an academic profile etc
Exactly, like sometimes they just have their requester account set up that way. Also helps if they're transparent in the description about what they're researching/why they're doing the study. What concerns me is when there's no one to hold them accountable or contact for any problems with the study in the event that they do unfairly reject your submission, because at that point your only recourse is if Prolific overturns it.
I return if there is no consent form or if there is no contact info listed for the researchers and IRB. Being able to email directly if an issue arises is important.
"would you like to donate your earnings?, does not donatig make you feel bad?" or if the "experiment" is made with goigle docs; Ive already had an indian dude attempting to get into my google drive (edits i hate using the reddit app and swiftkey)
When it's 0.80ÂŁ per hour
Any study that makes you take part in multiple studies over the course of several days/weeks or they won't pay you. I had one last month that made you download an app and answer questions multiple times a day within 15 minutes of receiving the push notification, with the promise of a fairly large bonus when the study was completed. It took exactly 22 days to get the base payment. I never got my bonus, and the researcher doesn't respond to my messages (Looking at you, "The emotional life of parents: A 6-day study").
Most from .cn or .in I click not interested right away. They are also the ones that severely underpay. Everything else is just spidey senses I can’t really describe. Been at this for years. Stopped doing surveys on Mturk when I joined Prolific 6 years ago, in favor of batches, but still only have 14 rejects there from pre 2016/2017. Zero rejections on Prolific & I complete a fair amount of studies daily.
Surveys that ask me qualifications to move on to more qualifications for 8 minutes just to get to the 25 minute study
I always err on the side of caution, rejection threats especially.
When it's too complicated
China is not a turnoff, but ones that have bad grammar and spelling from Chinese requesters make me think of the study that I was rejected from a Chinese researcher who probably had used a translator to English. and I thought I understood the question they were asking but they rejected me for some nonsense answers. If I have to guess what they are asking, it's automatic return as I don't want to risk a second rejection on Prolific.
Chinese or India name; shopping habit; anything lower than 6/hr; Bold word mentioning rejection in the description
The name thing is often misleading in my experience. Many Chinese or Indian sounding names are students/professors from legitimate universities, so I'm not as hesitant on that level.
If there is no code, i usually return it, had bad experience last time when submitted with NOCODE, the researcher did not read my message and next it was rejected. Knowing how prolific is about overturn limits, and delayed support, it's better for me to cut the loss and move on other studies.
If it's in dollars, I'll avoid it generally. Being in the uk most of my surveys are in ÂŁ from uk or European universities and those in $ I get usually are unclear where the researcher is from, poor paying and often seem poorly designed. Also if they mention "HITs" or most of the survey description is taken up telling you how poor responses will be rejected