SAHSU: How to Lie with Statistics

Report from Radioactive Times Volume 3, Number 2, October 1999 (content not updated)

The Small Area Health Statistics Unit
was set up to ensure that problems like
the Seascale cluster were discovered by "professionals" -
-- not by journalists, with all the attendant embarrassing publicity.
But SAHSU seems to have set itself the task of developing
a new kind of epidemiology
-- one in which statistics is used
not to identify effects but to lose them.

After the discovery of the Sellafield 10-fold excess child leukaemia cluster Sir Douglas Black's report exonerated the radiation, but Black did not seem persuaded. Like any intelligent outsider to the issue he was clearly confused by a failure to link the largest source of radiation in Europe to a local cluster when radiation was the only proven cause of leukaemia. He advised the government to set up two independent bodies. The first, the Committee on Medical Aspects of Radiation in the Environment (COMARE) was to look at the science independent of NRPB, whom he clearly didn't trust.

He was also upset that the leukaemia cluster had first been signalled by the TV rather than by the local Public Health department. He advised setting up a specialist group to ensure that this would not be repeated. Thus the Small Area Health Statistics Unit (SAHSU) began. Its remit was to look for other cancer clusters near possible pollution sources in the UK.

Both ideas failed.

The failure of SAHSU is difficult to understand unless the assumption is that it was intended to fail. In a world that has been increasingly producing toxic and carcinogenic chemicals in addition to radiation there should be no shortage of clusters of people living near to point sources of pollution of every kind and suffering every type of disease imaginable. A friend at the London School of Hygiene and Tropical Medicine tells us that meetings of SAHSU were secretive and private and that it was generally accepted that they were there to lose evidence of health effects rather than to investigate them. They have since moved the unit to Imperial College.

SAHSU, from the publications and meteoric rise in importance of its director, Paul Elliott, appears to have set itself the task of developing a new type of epidemiology -- one in which statistics is used not to identify effects but to lose them.

One such technique is Bayesian Smoothing. It derives from misapplication of a statistical method developed by the Victorian mathematician, the Reverend Thomas Bayes. Bayes' technique allowed statistics to be used to analyse situations in which nothing was known but where new information could be added on to modify an initial guess or "prior estimate". Bayes' statistics is currently in vogue because of its application to teaching robots to learn from experience.

Instead of using the method to teach risk scientists to learn from experience of clusters around nuclear sites, SAHSU and the new epidemiology do something different. After the initial data is in they make a "post hoc" guess that a sharp rise in disease in a small area is unlikely to and so must be an artefact of chance. Thus an area of high incidence of a disease is doctored on the basis of their own prior estimate (or guess) that such a state of affairs is (subjectively) unlikely. Imagine the power this conveys to raise the dead and cure disease with a stroke of a pen.

And it is no good moaning that there is a nuclear site in the area of high cancer incidence rate. They will tell you that you can't draw inferences about the site because you are already biased, knowing that it is there. And if you find a nuclear site after you have found the high levels of cancer, well you can't do that either because now you are a "Texas Sharpshooter" and didn't have a prior hypothesis!

In the British Medical Journal ⁽¹⁾ Paul Elliott himself recently wrote off the Sellafield cluster by such an argument. Readers of Radioactive Times Vol. 2 No 4.⁽²⁾ will recall that the same arguments were used to write off the Northampton nuclear train cluster.

SAHSU has developed a small armoury of cluster-busting techniques. These people now even have a website and invite concerned public health specialists to contact them so that they can smooth away their fears, the clusters and any chance of action to protect the public.

As industrial expansion and economic growth increase pollution, so do point source cancer clusters occur more and more. Those who require growth to make money have thus to employ bogus scientists and mathematicians to provide a means to minimise and exclude the evidence. If a paper is written now, a cluster of the dead and dying has to run the gauntlet of all these new techniques before the news can be accepted for publication and become scientific fact.

We do not want to paint SAHSU as an entirely doubtful unit, however, since we can report that Dr Jarup, the Deputy Director, has made available to Green Audit a table of socio-economic indices for Welsh wards which will enable researchers to examine Welsh coastal cancer rates and more accurately define the effect of living with the radioactivity of the Irish Sea.

---

References

1 BMJ 1996;313:863-866 (5 October)

2 Link to Radioactive Times (Vol. 2 No 4.) Editorial - When miners' canaries die in vain

---

And here's something from the Laboratory RaT's collection of funny sayings:

For mapping rare cancers it is important that risks are standardised and smoothed in an appropriate manner in order to avoid conveying false information. Without such methodology spuriously elevated risks will be produced resulting in unjustified concern in the corresponding population.

M. Smans and J. Esteve: "Practical Approaches to Disease Mapping" in Paul Elliott et al. Geographical and Environmental Epidemiology: Methods for Small Area Studies
Oxford University Press (1992)

LLRC says: On the other hand, with such methodology, spuriously reduced risks will be produced resulting in unjustified unconcern in the corresponding government agencies.

---