Is Peer Review an enemy of Progress?

The Sokal Affair

reply to post by Ectoterrestrial

This feedback loop tends to help journals, magazines, conferences from decaying too far into social cliques. But on occasion, when they do...

Such decay speaks to the true meaning of the Sokal affair to which the OP makes erroneous reference.

The purpose of Alan Sokal's little jape was not to expose the deficiencies of the peer-review system in science, but to reveal the ignorance and intellectual bankruptcy of postmodernist critics of science who treat it as only one explanatory 'narrative' among others, no more true or trustworthy than Norse myths, Marxist dialectics, Freud's theories of the unconscious or the belief that aliens control our brains. In fact, rather than 'privileging' science above these other 'narratives', postmodernists decry it as sexist, elitist, patriarchal and controlling. They prefer Marx and Derrida to Darwin and Einstein.

Alan Sokal submitted his paper, a piece of learned-sounding nonsense with an equally nonsenical title, to the journal Social Text. This is not a scientific journal but a publication for scholars of 'cultural studies'. You can tell by that suffix 'studies' that Social Text is a bastion of postmodernism. Its editors fell on Sokal's submission like the Children of Israel on the morning manna -- they thought a 'real' scientist was actually coming over to their side! They rushed the paper into print, and ended up with pie on their faces.

The history of the Sokal Affair is both entertaining and instructive; you can read about it from the horse's mouth here. But what is most important from the point of view of this thread is that, as Alan Sokal was well aware, Social Text had no peer review process at the time he submitted his paper.

If it had, the paper would never been published; the first fellow-physicist who looked at it would have rejected it. The fact that it did get published is an argument for peer review, not against it.

I think peer review comes with a set of positive and negatives. During my graduate school ventures I had the opportunity to do quite a few academic articles that were later published.

On the negative side, peer reviews often turns into a "whose smarter" match where reviewers try to impress upon you their supreme intelligence and take immense pleasure in berating your article. For some reviewers of prestigious journals, its almost like a ego trip - the reviewers have a standard of rejecting ANY article multiple times before accepting it, because its the only way to impress how "prestigious" the journal is. On the opposite end, it often turns into cliques where the "top scholars" get together and all review each others work, and they all accept each other. Scholars with past work of a high caliber can slide later in their career because they are well known and everyone assumes whatever they write must be publishable. I've seen both of these types of things numerous times.

On the positive side, peer review is a vetting process that almost always separates the wheat from the chaff. If you can't get your study published - by any first, second, or third tier journal - then very likely its not because you've got something that paradigm changing, but that you don't have enough evidence for whatever it is your writing.

[edit on 4-3-2008 by pacificwind]

reply to post by pacificwind


You've described the landscape very well, in my opinion. One other interesting outlier in the peer review process, is the paper or work of a brilliant leader in his/her field. About all the peer review process can do it such a case, is certify, that to the best of their knowledge, there are no errors, based on current thinking, which may be quite minuscule. In such an instance, a real peer review may take decades before any conclusions are reached.
A second instance of near-useless peer review, is in the area of time travel. If you look at the recent history of that genre of papers, there is so much that is debatable and not subject to meaningful empirical testing, that the entire field of peer reviewed papers is in complete disarray.

reply to post by ProfEmeritus


You are correct in that many of the articles that appear in scholarly journals are theoretical in nature, but this does not mean that "useful" ideas that are developed by industry are not subject to forms of peer review.

Let us take your example of a "cure for cancer." Drugs and other products are subjected to regulatory approval. While regulatory reviews have their problems, they do involve people withsome expertise examining a product. Just as a peer review checks an article for accuracy, a regulatory review examines a product to make sure it is safe and effective.

Ideas in industry are also often to the scrutiny of other scientists in the field. For example, investors may wish to have independent technical experts scrutinize a company's ideas before they lend money to a company to fund a venture. Scientists within a company may discuss ideas with eachother or have their ideas reviewed by a their bosses.

peer review being hailed as a marker of whether something is TRUE or not is a major obstacle to new new theories.
when people need to unlearn something before they can learn something, smoke starts pouring out of the ears, the eyes roll up, and the mind shuts off.

good thread.

"peer review" is a good theoretical approach to filtering the noise, but, unfortunately, in many cases, politics and ego are impassable brick walls that stop new truths from outing.

reply to post by Astyanax


Excellent post here. I read what Sokal had to say and his motives for exposing what he thought was a philosophical fallacy - the primacy of the text as a postmodern deconstructed reality and the idea that reality is a phenomenological structure. I may not agree with him but he had a point about peer review. However, if we go to the scientific world of evidence and reproducibility and hypothetico-deductive method - falsifiability of the experiment as an objective truth, we can then consider String Theory. There is no hard evidence for the existence of Strings that constitute matter. However there must be peer reviewed journals in that field which are every bit as fuzzy as the works of Derrida or his ilk. How come no-one attacks them? What they provide is a mathematical theory that fits in with observations - nothing more.

reply to post by Heronumber0

However, if we go to the scientific world of evidence and reproducibility and hypothetico-deductive method - falsifiability of the experiment as an objective truth, we can then consider String Theory. There is no hard evidence for the existence of Strings that constitute matter. However there must be peer reviewed journals in that field which are every bit as fuzzy as the works of Derrida or his ilk. How come no-one attacks them?

Yes, string theory is quite a hermetic field in which there seems to be no way to actually prove anything experimentally, at least none I've heard about so far. But you're wrong to imagine no-one attacks it. Many physicists are very strongly critical; they form quite a sizeable group within the community. A particularly vocal critic is the Harvard professor Lee Smolin, who is one of a school in physics that prefers theories of something called quantum gravity. Neon Haze, a senior ATS member who is also a physicist, champions QG and Smolin on this thread.

Science is full of controversies like this. Some stay unresolved for decades. Back and forth swings the pendulum of orthodoxy, as the evidence seems temporarily to favour one explanation or another. This is what the public sees. But for insiders, it's everybody struggling to push the pendulum in different directions based on what they think is the correct explanation. Eventually the matter will -- with luck -- be settled by the discovery of some new, apparently conclusive evidence.

reply to post by Astyanax


Thanks for that reply - I see that there is a 'resistance movement' against some quantum physics. However, I wonder how much empirical work has been done on Loop Quantum Gravity?

I also do not think that specialist Science journals help to build understanding for non-specialist scientist, let alone other members of the public. As an example where one would have to sit with a glossary to understand the text, this journal was famous for incomprehensible articles unless you were in the same field, working with the same clones and with the same organisms.

I will not mention its name due to copyright restrictions, but you can probably guess...

These phenotypes reveal specific functional potentials of meiotic and mitotic recA homolog genes. (1) JMs can occur if either DMC1 or RAD51/55/57 is present but not if both are absent. Thus, meiotic and mitotic recA homologs have an overlapping potential to promote (some aspect of) JM formation. (2) The level of IHJMs is extremely low when DMC1 is absent and JM formation is promoted only by RAD51/55/57 (i.e., in red1 dmc1 and dmc1 return-to-growth), and only ISJMs are seen. Thus, at least in these situations, when the mitotic recA homolog group RAD51/55/57 is promoting JM formation without DMC1, the mitotic group can only promote intersister interactions. In contrast, IHJMs form at significant levels whenever DMC1 is present (in red1 or red1 rad51).

reply to post by Astyanax


Thanks for that reply - I see that there is a 'resistance movement' against some quantum physics. However, I wonder how much empirical work has been done on Loop Quantum Gravity?

I also do not think that specialist Science journals help to build understanding for non-specialist scientist, let alone other members of the public. As an example where one would have to sit with a glossary to understand the text, this journal was famous for incomprehensible articles unless you were in the same field, working with the same clones and with the same organisms.

I will not mention its name due to copyright restrictions, but you can probably guess...

These phenotypes reveal specific functional potentials of meiotic and mitotic recA homolog genes. (1) JMs can occur if either DMC1 or RAD51/55/57 is present but not if both are absent. Thus, meiotic and mitotic recA homologs have an overlapping potential to promote (some aspect of) JM formation. (2) The level of IHJMs is extremely low when DMC1 is absent and JM formation is promoted only by RAD51/55/57 (i.e., in red1 dmc1 and dmc1 return-to-growth), and only ISJMs are seen. Thus, at least in these situations, when the mitotic recA homolog group RAD51/55/57 is promoting JM formation without DMC1, the mitotic group can only promote intersister interactions. In contrast, IHJMs form at significant levels whenever DMC1 is present (in red1 or red1 rad51).

reply to post by Astyanax


Thanks for that reply - I see that there is a 'resistance movement' against some quantum physics. However, I wonder how much empirical work has been done on Loop Quantum Gravity?

I also do not think that specialist Science journals help to build understanding for non-specialist scientist, let alone other members of the public. As an example where one would have to sit with a glossary to understand the text, this journal was famous for incomprehensible articles unless you were in the same field, working with the same clones and with the same organisms.

I will not mention its name due to copyright restrictions, but you can probably guess...

These phenotypes reveal specific functional potentials of meiotic and mitotic recA homolog genes. (1) JMs can occur if either DMC1 or RAD51/55/57 is present but not if both are absent. Thus, meiotic and mitotic recA homologs have an overlapping potential to promote (some aspect of) JM formation. (2) The level of IHJMs is extremely low when DMC1 is absent and JM formation is promoted only by RAD51/55/57 (i.e., in red1 dmc1 and dmc1 return-to-growth), and only ISJMs are seen. Thus, at least in these situations, when the mitotic recA homolog group RAD51/55/57 is promoting JM formation without DMC1, the mitotic group can only promote intersister interactions. In contrast, IHJMs form at significant levels whenever DMC1 is present (in red1 or red1 rad51).

Whooops! sorry for the triple post.


[edit on 8/3/2008 by Heronumber0]

reply to post by Heronumber0

I seem to catch the faintest whiff of an anti-science agenda...

I wonder how much empirical work has been done on Loop Quantum Gravity?

As far as I can tell, it is just as speculative and theoretical as string theory, though it is said to fit observed reality a little more elegantly than the latter.

I also do not think that specialist Science journals help to build understanding for non-specialist scientists

Of course not. That is not their purpose.

let alone (for) other members of the public.

Still less is this their purpose. In fact, any attempt to do so would compromise and ultimately defeat their purpose.

As an example where one would have to sit with a glossary to understand the text, this journal was famous for incomprehensible articles...

Doubtless the people who subscribe to the journal are able to understand what is in it, and that is all that matters.

Scientists are in no way obliged to dumb down their work for mass consumption.

Though what any of this has to do with peer review I really cannot see.