Tag Archives: critical thinking science

Nobel Prize Winning Biologist Criticizes Major Science Journals


Wow! Apparently even major science journals like Nature, Cell, and Science can be biased by the appeal of the popular:

[Nobel prize winner] Schekman criticises Nature, Cell and Science for artificially restricting the number of papers they accept, a policy he says stokes demand “like fashion designers who create limited-edition handbags.” He also attacks a widespread metric called an “impact factor”, used by many top-tier journals in their marketing.

A journal’s impact factor is a measure of how often its papers are cited, and is used as a proxy for quality. But Schekman said it was “toxic influence” on science that “introduced a distortion”. He writes: “A paper can become highly cited because it is good science – or because it is eye-catching, provocative, or wrong.”

via Walker Traylor and others on Facebook

You can read the whole article on the Guardian here:
Nobel winner declares boycott of top science journals

Good on Schekman for pushing back!

fake new scientist magazine

This “impact factor” strikes me as similar to Google PageRank and other metrics which have lead to the creation of linkbait (annoyingly catchy headlines and images), effectively turning the entire internet into a race-to-the-bottom popularity contest.

Consider this article from the Atlantic about the linkbait site “UpWorthy” on why these annoying headlines are now dominating social media.

…how sustainable is this? Can news organizations keep promising you that their content is not only wondtacular but also, actually, wondtacular-er? Does adjective creep in headlines exist—and, if so, how can it be fought?

Of course, in self-help and personal development, this kind of inflation in marketing claims of course the norm: “Get stronger, faster, make more money, easier and faster than you ever thought possible!”

It can be humbling to slowly, cautiously, yet persistently pursue personal improvements, or study something in depth. It is incredibly difficult to do science correctly and precisely, to actually find out what is happening without bias.

And indeed I struggle with how to market a business in an environment where everyone is promising impossibilities (in the personal development world) or no changes at all are possible (in much of the psychiatric and psychotherapy world).

I hope I can present a middle ground here at Scientific Goals that is sufficiently rigorous, but that will also not be ignored because it isn’t hype-y enough!

Want more stuff like this? Subscribe to our (at most) weekly newsletter:

Subscribe to our mailing list

(Obviously your email will never be shared —
I hate spam as much as you do.)

How can we evaluate scientific claims?

From Nature magazine, the excellent science publication:

…we suggest 20 concepts that should be part of the education of civil servants, politicians, policy advisers and journalists — and anyone else who may have to interact with science or scientists. Politicians with a healthy scepticism of scientific advocates might simply prefer to arm themselves with this critical set of knowledge.

You can read their suggestions here: Policy: Twenty tips for interpreting scientific claims
Article via Michael Perez on Facebook.

In addition to their tips, I suggest a few more. In personal development and self help, fake science claims are made all the time. In particular, pay attention to these things:

  • “Studies have shown” – Which studies specifically? What researchers? What were the titles of the papers and what publication were they published in? There have even been cases of complete fabrications of studies, such as the infamous “Yale Goals Study” which never existed, despite Tony Robbins and others continuing to “reference” it.
  • “Science shows us that…” – Again, what studies specifically?
  • Referring to a single study that is not a review of many studies. A single study may later be overturned or otherwise contradicted by other studies and so should be taken with a grain of salt. Usually people will refer to a study without again saying who the researcher is and where you can read the full text or at least the abstract of the study.
  • The only source is a popular report of the science. Very often there is significant distortion of the research findings when translating the original research paper to a press release or news article. One of the most common errors is to take a single study and draw very large conclusions from the study, framing the results as making a really big difference in our understanding of something. Almost all studies make a very minor difference in our total understanding, but over time our understanding can change quite a bit.

Good science is a combination of being intensely curious and willing to experiment while also being intensely critical and focused on rigorously eliminating bias wherever possible, whereas most personal development and self help authors and speakers are really just trying to sell you something and are willing to bend or distort the truth to do so.

Want more stuff like this? Subscribe to our (at most) weekly newsletter:

Subscribe to our mailing list

(Obviously your email will never be shared —
I hate spam as much as you do.)