Unreliable research

“I SEE a train wreck looming,” warned Daniel Kahneman, an eminent psychologist, in an open letter last year. The premonition concerned research on a phenomenon known as “priming”. Priming studies suggest that decisions can be influenced by apparently irrelevant actions or events that took place just before the cusp of choice. They have been a boom area in psychology over the past decade, and some of their insights have already made it out of the lab and into the toolkits of policy wonks keen on “nudging” the populace.

Dr Kahneman and a growing number of his colleagues fear that a lot of this priming research is poorly founded. Over the past few years various researchers have made systematic attempts to replicate some of the more widely cited priming experiments. Many of these replications have failed. In April, for instance, a paper in PLoS ONE, a journal, reported that nine separate experiments had not managed to reproduce the results of a famous study from 1998 purporting to show that thinking about a professor before taking an intelligence test leads to a higher score than imagining a football hooligan.


The idea that the same experiments always get the same results, no matter who performs them, is one of the cornerstones of science’s claim to objective truth. If a systematic campaign of replication does not lead to the same results, then either the original research is flawed (as the replicators claim) or the replications are (as many of the original researchers on priming contend). Either way, something is awry.

To err is all too common

Academic scientists readily acknowledge that they often get things wrong. But they also hold fast to the idea that these errors get corrected over time as other scientists try to take the work further. Evidence that many more dodgy results are published than are subsequently corrected or withdrawn calls that much-vaunted capacity for self-correction into question. There are errors in a lot more of the scientific papers being published, written about and acted on than anyone would normally suppose, or like to think.

Various factors contribute to the problem. Statistical mistakes are widespread. The peer reviewers who evaluate papers before journals commit to publishing them are much worse at spotting mistakes than they or others appreciate. Professional pressure, competition and ambition push scientists to publish more quickly than would be wise.

read more here

Many papers and findings from scientists are being published and accepted as reliable and proven but the system does not guarantee that to be true at all. It isn’t that they are falsifying or trying to get over on us or each other as many laymen think it’s that they are not perfect the system isn’t infallible but it is self-correcting or at least it’s supposed to be and or working toward that goal.

reply to post by Grimpachi


Any theory has to be considered possibly wrong. But a given theory may only stop existing after it was irrevocably falsified by experimentation.

That's basically it.

S&F

One of the things i'd imagine is that these research projects are not of interest enough to other scientists to check their results beyond a quick.."hmm that makes reasonable sense" and then going onto the jobs section of the journal, plus its in psychology where even the slightest thing can change things such as while you select a random group of people its from a flawed pool for an experiment and its not really repeatable in a lot of situations without much effort and that effort itself can change the results

Research evidence can be steered depending on the parameters that are imposed on it. You can make evidence fit many things. This screws up verification of evidence also, the same parameters apply. What is excluded from the testing though? What other theories can this evidence back up. Is the theory made from analyzing the evidence correct or is it based on belief in only what we already know. What about what we don't know or have not thought of yet. Lots of theories are screwed up but they are real if they are accepted by enough of the prestigious members of the science they are related to.

reply to post by Grimpachi


Please provide a link to your source.

Never mind: the new format has confused me.

reply to post by DJW001


Assigning numerical data to variables, and designing questions, the answers to which may likely be eventually represented as a numerical figure, are always going to invite critical analysis regarding research results.

An example might be, code the answer to "how are you feeling today". Essentially researchers will try and digitize analogue responses in an experiment. For example emotional responses which need to be quantified as in the quote above. Designing questions and surveys is an art form in itself and are very often used in psychological research.

All you can hope for is a repeat of an experiment by others and look for similar results, so in fairness psychology research is hampered by these factors, and necessary repeats perhaps difficult to obtain funding for.

Daniel Kahneman (Hebrew: דניאל כהנמן‎) (born March 5, 1934) is an Israeli-American psychologist and winner of the 2002 Nobel Memorial Prize in Economic Sciences. He is notable for his work on the psychology of judgment and decision-making, behavioral economics and hedonic psychology.

Awards and recognition[edit]
In 2002, Kahneman received the Nobel Memorial Prize in Economics, despite being a research psychologist, for his work in Prospect theory. Kahneman states he has never taken a single economics course – that everything that he knows of the subject he and Tversky learned from their collaborators Richard Thaler and Jack Knetsch.
Kahneman, co-recipient with Amos Tversky, earned the 2003 University of Louisville Grawemeyer Award for Psychology.[6]
In 2005, he was voted the 101st-greatest Israeli of all time, in a poll by the Israeli news website Ynet to determine whom the general public considered the 200 Greatest Israelis.[7]

In 2007, he was presented with the American Psychological Association's Award for Outstanding Lifetime Contributions to Psychology.[8]

On November 6, 2009, he was awarded an honorary doctorate from the department of Economics at Erasmus University in Rotterdam, the Netherlands. In his acceptance speech Kahneman said, "when you live long enough, you see the impossible become reality." He was referring to the fact that he would never have expected to be honored as an economist when he started his studies into what would become Behavioral Economics.[9]

In both 2011 and 2012, he made the Bloomberg 50 most influential people in global finance.[10]

On November 9, 2011, he was awarded the Talcott Parsons Prize by the American Academy of Arts and Sciences. To see his lecture, click the link below.[11]

His book, Thinking, Fast and Slow was the winner of the 2011 Los Angeles Times Book Award for Current Interest.[12]
In 2012 his book, Thinking, Fast and Slow, was awarded the National Academy of Sciences Communication Award for the best book published in 2011.[13]

In 2012 he was accepted as corresponding academician at the Real Academia Española (Economic and Financial Sciences).[14]

On August 8, 2013, President Barack Obama announced that Daniel Kahneman would be a recipient of the Presidential Medal of Freedom.[15]

Source

Today Psychologist are really uncomfortable with materialist definitions to consciousness.

Any thoughts?

It seems that this revelation shines light on a bigger problem. A lot of these studies are conducted in an academic setting and the state of academia is really poor right now when it comes to the acceptance of ideas that refute the status quo.

Lots of higher education institutions are overrun with liberal professors and what seems to happen is instead of challenging the theory and looking to prove the theory wrong, they accept the theory and look to prove the theory correct.

So what is really happening is one huge circle of group-think. The "debate" on Man-Made Global Warming is another topic that seems to follow the same premise that you are laying out here.

You get a bunch of people that follow the same ideology and suddenly your "peer-review" system is just a bunch of "peers" patting each other on the back saying good job for whatever they conjure up, and people that point out the errors and prove the theory wrong are outcast.

The scariest part of any institution is that its membership refuses to acknowledge lessons related to history. While today many conservative scientist insist that materialist modeling with respect to consciousness is still valid, it is
not.

The idea that there actually exist random event is reality actually could relate exclusively to the scope, we comprehend of reality.

Otherwise that sort of thinking could actually be completely irrelevant.

Any thoughts?