Audit Scotland’s Nursery provision report based on flawed methodology and naïve assumptions. Tories and Labour joyful as they feast on it.

audit-scotland

Here are the main points from the report which have cheered the Unionist parties so much:

‘There are significant risks that councils will not be able to expand funded ELC to 1,140 hours by 2020. In particular it will be difficult to increase the infrastructure and workforce to the levels required, in the limited time available. The Scottish Government should have started detailed planning with councils earlier, given the scale of the changes required. The Scottish Government expects the cost of delivering 1,140 hours of funded ELC will be about £840 million per year. Councils were required to prepare their initial plans for the expansion without some important information about how the system will operate from 2020. While councils’ plans will change over time, initial estimates of the cost of funded ELC following the expansion to 1,140 hours are about £1 billion per year, significantly higher than the Scottish Government’s figure.’ (p.5)

http://www.audit-scotland.gov.uk/uploads/docs/report/2018/nr_180215_early_learning.pdf

Completely ignoring any possibility of any fault in the councils, the Tory and Labour responses faithfully communicated by BBC Scotland, of course, are:

‘Conservative education spokeswoman Liz Smith said the report was “damning in terms of exposing the failures of SNP policy on child care”. For Labour, education spokesman Iain Gray added: “This independent report blows a £160m black hole in a flagship SNP government policy. Raising the amount of free childcare families are entitled too has been a cornerstone of the SNP’s offer to families – but this report reveals that, not only does the Scottish government not expect to fully fund it, but that it is also miles behind the levels of staff recruitment needed.”’

http://www.bbc.co.uk/news/uk-scotland-scotland-politics-43058121

Notice how Iain Gray is allowed a complete fib about government intentions but also seems confused as to who will recruit the staff?

However, back to my main concern, the methodology upon which Audit Scotland have based their report. It’s only been two years since I last supervised PhD projects and I’m going to assume that a funded national body with several researchers should be able to at least match that level. Here’s all the detail we get on it, on page 9:

Our findings are based on evidence from sources that include the following:

  • a survey of all 32 councils
  • interviews with senior staff in the Scottish Government, Care Inspectorate, Education Scotland, COSLA (COSLA chairman is Stephen McCabe, Labour, Inverclyde) and other national bodies
  • interviews with a range of staff in five fieldwork councils – City of Edinburgh, Glasgow City, Perth and Kinross, Renfrewshire and Scottish Borders
  • focus groups with partner-provider nurseries from the private sector
  • interviews with parents and carers, an online survey of parents and carers and a review of reports on the experiences of parents and carers
  • published information on ELC registrations and costs
  • a review of published evidence on the impact of ELC on child and parental outcomes
  • a review of councils’ expansion plans for 1,140 hours of funded ELC
  • a review of key documents including legislation and associated guidance, minutes of groups involved in planning for the expansions to both 600 hours and 1,140 hours of funded ELC, and council documentation about how ELC is delivered locally.

http://www.audit-scotland.gov.uk/uploads/docs/report/2018/nr_180215_early_learning.pdf

At first sight it looks impressive in terms of the access they’ve had but then you notice a lack of attempts to gather empirical, quantitative data with which you could triangulate the subjective, qualitative information they’ve gathered from a series of interviews (only 28 parents?), focus groups (only 5) and literature reviews. The key concerns they have about funding and time are not evidenced in any transparent statistical calculations or of the basis upon which the resulting estimates were produced by the councils, which the Audit Scotland researchers could then have analysed with a view to evaluating their actual worth. If all we have are the estimates unvalidated by any external evaluation, then the opinions of those interviewed, no matter how many nor how senior, are likely to be unreliable and, potentially, based on an agenda hostile to the government’s strategy. Is it at all likely that local government officers, often serving Labour and Tory administrations, would accept SNP central government costs and just get on with it? Is it possible that the Scottish Government’s accountants know what they’re doing and, also, know fine well that their local authority cousins expect to bid high first time?  As for the time required to expand the existing system, a full two years remain after a least one since the councils became aware of their task. This is not a revolution, merely an expansion of a pre-existing and well-established pre-school sector. Did Audit Scotland explore the possibility of some councils and the Labour-led COSLA being more concerned to obstruct than to implement? Indeed, why is it the Scottish Government’s role to ‘start detailed planning’? Shouldn’t that be a local task?

Surely, Audit Scotland have some sociologists familiar with such all-too-human behaviour.

Advertisements

13 thoughts on “Audit Scotland’s Nursery provision report based on flawed methodology and naïve assumptions. Tories and Labour joyful as they feast on it.

  1. Lollysmum February 15, 2018 / 2:36 pm

    Strikes me that there are many different agendas at play in this report & Audit Scotland should at the very least have considered this point & explained exactly how their judgements were made with no reference how they minimised the impact that those different agendas would have on the results of their own report. I’ve also worked with Audit England & my own view is that both organisations leave much to be desired. I also tutored students for 1st degrees & Masters & If my own social workers had turned in a report such as this, I would have torn it to shreds.

    It lacks any sort of merit & suggests that researchers were carefully chosen as were the interviewees for their personal agenda. Crucial facts are missing leaving the reader to leap to blaming government when clearly COSLA & councils have a huge part to play in implementing an updated system which in itself would be ground breaking. It is also a known fact that that uk gov has placemen in every organisation in Scotland & this actually reads as though an anti-scotgov agenda is in full operation. Why am I not surprised? I’m seeing huge gaps in information & ruddy great spanners being inserted into this report instead to cause as much trouble a possible.

    I would be rejecting this listing each & every error, misplaced assumption & highlighting missing information. Scotgov can reject such flawed reports if they point out its flaws & I’ve certainly rejected research reports from Audit England with less errors than this one displays.

    Audit England has been known to get reports completely wrong & it would now appear that Audit Scotland has joined that same ‘club.

    Liked by 2 people

    • johnrobertson834 February 15, 2018 / 3:49 pm

      Thanks Prof Lollysmum. Quite a put down and all the better for your inside knowledge. That’s it double-marked so we can forward the result to the Assessment Committee chair.

      Liked by 2 people

    • Aimsir February 17, 2018 / 12:13 pm

      Genuinely interested in detail of methodological errors. This would seem to me a fairly reasonable spread of data for an inspection, which is what it is, not a piece of academic research, though its findings do seem consistent with harder evidence eg Growing Up in Scotland. There is an assumption (not at all substantiated by evidence) that any kind of group care is good for children when in fact research (such as the EPPE study) shows that if it is not of good quality (there are some good examples here – mostly, interestingly, in the ‘family centre’ type setting, but if you are looking for high quality in terms of good and equitable outcomes for children the Nordic countries have far more consistently good outcomes in health, learning and development, have staff qualified to degree level in an appropriate discipline eg specialist early years teaching or social pedagogy, provide advice, learning and support for parents as well as a place to leave children while they work, and a curriculum based on supported self-directed play rather than formal education) it will serve to increase the gap between those who are most at risk of inequitable outcomes and those who start off already advantaged. A recent study by the GUS team shows that better off, better educated parents continue to access the highest quality care for their young children as they have more capacity to top up the ‘free offer’ (which of course includes nurseries attached to private schools) thus further reinforcing economic, social and educational disadvantage. Other than the number of hours provided, I would agree with the report writers that outcome objectives are ill-defined for children and parents but also question whether enabling more parents to enter low-paid, insecure and unsatisfying work is a particularly desirable result.

      Like

  2. Alasdair Macdonald. February 15, 2018 / 3:02 pm

    I heard the pieces on GMS about this matter. It began in a way which is now the norm. The Conservative spokesperson Lz Smith made an allegation and the subsequent aspects of the programme were predicated upon Ms Smith’s assertions. It was only towards the end of the programme that an SG spokesperson had the opportunity to respond. However, the parts leading up to this, plus the ‘hostile’ questioning put the spokesperson in the position of being perceived to be defending something which had gone wrong. As I do other things in the morning when the radio is on, I did not hear any contribution by Mr Iain Gray.

    Ms Smith clearly had a statement prepared which she delivered succinctly. She was not questioned in any incisive way.

    The SG spokesperson, first of all, indicated that Councils had been ‘fully funded’ for this initiative. She also indicated that, as is usual in the very long-standing practice of ‘national services, locally delivered’, we were in a period of dialogue/negotiation/problem-solving to enable the policy to begin to be implemented.

    I had not seen the Audit Scotland report, but, I suspected that what was being done was the selection of particular data out of context. What you have done is critique the actual methodology used by Audit Scotland and have indicated that there could also be party political ulterior motives in play.

    The provision of significant early years provision is very important for Scotland, not least in the aim to raise attainment and to ‘close the gap’ between children in areas of high SIMD and others. Most empirical evidence suggests that the earlier that interventions are made, the greater is the chance of having a significant impact. So, it is a topic eminently worth raising and of having an informed debate, not providing a platform for petty, very petty, points scoring. Again, GMS shows its bias.

    On the same programme there was an interview with a spokesperson for SCIAF, which had reported that it had dealt with two child sex abuse incidents in communities which SCIAF was assisting. The alleged perpetrators were not SCIAF staff or volunteers. As soon as SCIAF had become aware – and these incidents were a few years ago – it had reported the matter immediately to the local authorities and co-operated fully in the investigation. The local authorities were unable to find sufficient evidence to provide a case for prosecution. However, the interviewer went on to ask if SCIAF had informed other third sector bodies about the individuals concerned. The SCIAF spokesperson pointed out that no action had been taken by the relevant authorities, so SCIAF might have faced claims by the individuals. SCIAF reported that they had not received any requests for references, but said that had they been asked, they would have reported that incidents and allegations had been made (presumably in a way, which avoided legal action against them). The interviewer then went on to ask if SCIAF was prepared to apologise for these – unproven – incidents, for which no SCIAF staff and been involved and which SCIAF had drawn to the attention of the relevant authorities as soon as it had become aware of these suspicions. It was classic, “Are you still beating your wife? Yes or No?” stuff. Classic damned if you do damned if you don’t stuff.

    While, given the nature of the reports the case against Oxfam looks bad, one wonders if there is a further agenda here. Messrs Cameron and Osborne continued with the pledge that 0.7% of GDP should be devoted to international aid. This was strongly opposed by sections of their own party and by the Britain-first, Little Britain faction and their megaphones in the media. Ms Priti Patel, appointed as Secretary of State for the DfID, despite being an opponent of international aid, as well as a determined LEAVER, had to resign when she was in effect pursuing a foreign policy on behalf of herself and like-minded people, rather than the agreed policy of the UK Government. So, perhaps, the egregious conduct of some within Oxfam is being used to do ‘dog-whistling’ against other third sector bodies who are in receipt of DfID funds to carry out the UK’s policy on aid.

    Liked by 2 people

    • Aimsir February 16, 2018 / 9:28 am

      There are many approaches to provision of early years services, and not all of them are good. Many governments see the provision of ECEC as primarily a route to parental labour market participation rather than reflecting the rights of children to good support for them to be cared for within their families and a high quality (much research on what the features of this are) early learning and equalising development experience when this is appropriate. We need to be very careful that ‘free childcare’ does not end up being a subsidy for low wage employment that does not raise families out of poverty and indeed puts additional stress on many hard-pressed parents.

      Like

  3. William Henderson February 15, 2018 / 4:16 pm

    As I mentioned in an earlier reply – The propaganda war of attrition goes on.

    The media norm is “guilty until proved innocent” to be followed, if at all, by “but there’s nae smoke withoot fire”.

    Liked by 2 people

  4. broadbield February 15, 2018 / 6:32 pm

    Prof Robbo and Prof Lolly have nailed them. It’s pathetic that work like this (and GERS) gets published and respected. CommonWeal’s piece about needing better statistics gets to the heart of the matter.

    Liked by 1 person

  5. Alasdair Macdonald. February 16, 2018 / 10:18 am

    Interestingly, The Reporting Scotland piece in the evening was much more balanced and omitted any contribution by Ms Smith or Mr Gray. There were several vox pops by mothers whose children are participating in the pilot scheme and all were supportive.

    Perhaps sites like this are being perused for reactions and programmes amended in the light of this.

    Sadly, they cannot go by their own ‘Have Your Say’ forums, which are simply markets for the trading of trollish insults.

    The most recent one relating to North Lanarkshire Council’s proposal to provide meals for children 365 days per year has had a rash of the most inhuman nastiness, almost all from familiar ‘tags’ who hide behind anonymity.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s