Warning signs of bogus science

Whether we want to choose a diet or a savings scheme, there are experts to help us along. Many of them dispense "scientifically proven" advice. We need to develop an early warning system to examine the claims of these experts. The basic radar system is skepticism: we keep our wits and do not get pushed into hasty choices; we question all the advice we get.

But sometimes good advice does go against common sense. On the other hand, not everything that goes against common sense is scientifically proven. Park's Rules are a few basic tests about whether the "science" you are confronted with could be bogus. I've rewritten them a little bit. This page is not copyright. Please feel free to comment or to reproduce and use them.

Evidence for the claim is anecdotal

Anecdotal evidence need not be untrue. It is simply not a discovery.

A double-blind test is a kind of test in which a tester administers an experimental treatment to one group of people and a control treatment to another. The speciality of the double-blind is that neither the subjects nor the tester knows who is a control and who is an experiment until the results are evaluated.

Reproducibility is the key to discovery. A sea route between Europe and the Americas which opened only once would have been an anecdote. A route which could be followed by ship after ship became a genuine discovery. Parks says in his article that "the most important discovery of modern medicine is not vaccines or antibiotics, it is the randomized double-blind test".

Large experimental facilities in high energy physics like the LHC try to have two experiments cross checking each other in order to ensure that results are reproducible.

The claim is believed widely, or has been believed for a long time

Belief may be either true or false. Belief is not discovery.

Popular belief, perhaps since the beginnings of humanity, held that the earth is still and the universe moves around it. Over the last few centuries evidence has piled up that this belief was false. The earth moves.

The discoverer has worked in isolation

A major advance in science does not come from a single person working alone, without contact with others. Einstein was not an isolated genius: he was in constant contact with many colleagues, often worked in collaboration with others, spent time discussing his thoughts with others. Nor was Einstein always correct; he often had to revise his ideas, and sometimes never managed to entirely correct his errors.

A possible modern counter-example is that of Andrew Wiles' proof of Fermat's conjecture. This is instructive, since it turns out not to be a classic counter-example at all. Wiles did work in relative secrecy for many years and developed a proof; but then he gave a seminar and submitted a paper to a prestigious journal of mathematics. Referees checked the proof and found a crucial error. Wiles managed to correct these errors after working in collaboration with Richard Taylor.

The claim is made in the mass media

Examples abound; the ``discovery'' of cold fusion by Pons and Fleischmann, the ``development'' of herbal diesel by Ramar Pillai. Why is this a warning sign?

One of the primary methods of science is to open one's ideas and techniques to criticism. Skepticism and tests of logical consistency can be applied by anyone at all, layman or expert. However, in any field of work, the people who are usually best equipped to examine and question methods, and even to try and reproduce the claims in their own labs, are those who have expertise in that field.

A claim made in the mass media which is not followed up by the publication of a detailed article in a professional journal is questionable because it means that it has not passed a detailed scrutiny by experts, and has not been reproduced by anyone else.

Science pages of newspapers contain a large number of news stories about medicine each day. How many of these report peer-reviewed papers? How many are pre-publication press releases? How small a fraction of the claims eventually go on to be proven and accepted treatments?

A powerful establishment is trying to suppress the discovery

The primary method of discovery in the sciences is to convince others through means of reason or reproducibility that the claim is justified. The first attitude of other scientists is always skepticism. This skepticism is a method: do not accept a statement until you have followed the argument or the experimental method and understood that there is no alternative possibility which has been left unexplored. A discovery is bound to be accepted if it turns out to be reproducible.

The politics of science, and of industry, which many self-proclaimed inventors confuse with science, does not consist of denying change, but in trying to monopolize credit or profit coming from the change.

The laws of naure must be revised to accomodate the discovery

The laws of science are not exactly untested. The laws of conservation of energy, the second law of thermodynamics, E=mc², etc, are all tested quantitatively an uncounted number of times every year. Less well-known laws are also laws because they are tested and found to work. A claim that a well-tested law needs revision is an extraordinary claim. It can only be accepted after extraordinarily thorough testing through experiments.

Recently UPI claimed that E=mc² was proven after 103 years. Is it likely that this would be taught as a basic part of physics courses if it had remained unproven for more than a century?

If the truth of a discovery hinges upon the incorrectness of an established law of nature, then it is more likely that the discovery is false than that the law is incorrect.