Statistician's Comment on the Annex

To return to the subject of the statistical basis for the safeguarding recommendations in the HE Review, ie: so far, from the response to the FOI here:

"Annex:

Known to social care [1]

25 of the 90 LAs asked responded (28% response rate)

*Based on the data we have from the 25 LAs, the average (median) proportion of EHE children per LA known to social care is approximately 7%. We estimate there are approximately 3% of children (5-16 years) known to social care in maintained schools. [2]

*Within the 25 LAs for which we have data, there were 477 registered home educated children who were currently known to social care.

*On average (median) 7 children per LA were known to social care.

*Extrapolating to the national level (150 LAs, this means around 1350 home educated children are known to social care in some capacity (6.75%)

(1) Known to social care includes Sections 17, 37, 47 enquiries.
(2) Using 2005 data (the latest available), these are approximate figures and include disabled children."

Action for Home Education (AHEd) have sought expert advice on the above stats, with the following result:

From Mr. W. Wallace

(Mr. Wallace has worked in local government as a statistician and also as a university statistics lecturer and research fellow.)

===========================


"Dear (AHEd),

Quite frankly I can't believe that you received the Annex as an FOI request. I had been looking for a statistical / methodological appendix to the Badman report but had not found one. The methodology as shown does not stand up as plausible or acceptable statistically, apart from all the other issues concerning the precise information used i.e. abuse, disability services or known to SS for a variety of possibly unsubstantiated reasons.

The use of a sample median to gross up to a national value requires that all LA's have the same number of EHE children, which they do not.

It is not easily possible to estimate the statistical error introduced by doing this but suffice it to say that the standard error of such an estimate would be so large that it would not be worth using the statistic.

Also quoted is 477 registered EHE children known to the 25 LA's that responded out of the 90 asked to respond as part of the review. We have no way of knowing how representative the 25 are of the 150 LA's and this needs to be checked before any statistics can be quoted. Are we comparing like with like? If we were confident that these 25 reasonably reflected the total 150 then we might take the 477 and divide by the total number of EHE children in the 25 LA's. This would give us an estimate of the proportion of EHE children known to SS per LA. This is what they could have done but did not.

There are at least two main flaws to be noted:

1. The inappropriate use of one measurement instead of the target measure. Using 'known to SS' rather than recorded abuse (often termed an error of operationalisation).

2. Using a non-probability sample. No standard errors or confidence intervals can be computed. Only some qualitative value may be possibly obtained. This is error in sample selection.

I can say without any hesitation that the information on methodology casts grave doubt on any use of the results from the Badman Review.

What small amount of the Annex that is shown is enough to bring a case of maladministration. Refusing to show any further details for whatever reason is not going to support their case one bit. It is very serious that statistical methods can be misused to try to support a case that does not exist. "

W. Wallace BSc MSc MPhil FSS AFIMA