How can we evaluate scientific claims?

From Nature magazine, the excellent science publication:

…we suggest 20 concepts that should be part of the education of civil servants, politicians, policy advisers and journalists — and anyone else who may have to interact with science or scientists. Politicians with a healthy scepticism of scientific advocates might simply prefer to arm themselves with this critical set of knowledge.

You can read their suggestions here: Policy: Twenty tips for interpreting scientific claims
Article via Michael Perez on Facebook.

In addition to their tips, I suggest a few more. In personal development and self help, fake science claims are made all the time. In particular, pay attention to these things:

  • “Studies have shown” – Which studies specifically? What researchers? What were the titles of the papers and what publication were they published in? There have even been cases of complete fabrications of studies, such as the infamous “Yale Goals Study” which never existed, despite Tony Robbins and others continuing to “reference” it.
  • “Science shows us that…” – Again, what studies specifically?
  • Referring to a single study that is not a review of many studies. A single study may later be overturned or otherwise contradicted by other studies and so should be taken with a grain of salt. Usually people will refer to a study without again saying who the researcher is and where you can read the full text or at least the abstract of the study.
  • The only source is a popular report of the science. Very often there is significant distortion of the research findings when translating the original research paper to a press release or news article. One of the most common errors is to take a single study and draw very large conclusions from the study, framing the results as making a really big difference in our understanding of something. Almost all studies make a very minor difference in our total understanding, but over time our understanding can change quite a bit.

Good science is a combination of being intensely curious and willing to experiment while also being intensely critical and focused on rigorously eliminating bias wherever possible, whereas most personal development and self help authors and speakers are really just trying to sell you something and are willing to bend or distort the truth to do so.

Want more stuff like this? Subscribe to our (at most) weekly newsletter:

Subscribe to our mailing list

(Obviously your email will never be shared —
I hate spam as much as you do.)