The quality of statistical evidence on Scottish education is poor. Although useful summary information is published annually, that provides no more than a bare description of the system. The existing data provide no insights into how effective Scottish education is, where its strengths and weaknesses are, and how it might be improved. Few of the statistical sources allow valid comparison with other countries.
The purpose of the report published today by Reform Scotland’s Commission on School Reform is to propose a new statistical framework that would transform the range and quality of statistical data on Scottish education. The data proposed here would be objective and politically neutral. It would thus strengthen the basis of educational policy-making, and would offer the opportunity for sustained educational improvement.
The most notable current gaps in data are:
- There is no way of analysing pupils’ learning from the point at which they enter pre-school education at age 3 or 4 to the point at which they leave school. Individual children can be tracked, but no data set collates this into a form that might give insights into the performance of schools, local authorities or the system as a whole, nor into how pupils’ home circumstances affect their learning.
- There is no statistical evidence on effective teaching strategies: advice on this is wholly dependent on evidence from other countries. Learning from elsewhere is indeed very important, but that has to be translated into a local context, which cannot be done in any well-informed way here.
- There is no systematic information about young people’s subjective experiences in school, for example in relation to bullying, misbehaviour or absence from school altogether. There is no data set that would allow any understanding of how these experiences relate to pupils’ learning.
- The statistical evidence on the transition from school to work and post-school education is superficial.
In Scotland, schooling is thus a black hole out of which no information escapes that might be useful to immediate policy.
Nevertheless, amidst the general gloom, there are scattered examples of good statistical practice. Better data can be developed by building on these. The best-known is Scotland’s participation in the Programme for International Student Assessment – the PISA studies – which happen every three years in about half of the world’s countries, under the auspices of the Organisation for Economic Cooperation and Development. This has become the model for high-quality surveys of education systems. In other countries it is supplemented by surveys that are tailored to the specific characteristics of the national education system. Scotland provides nothing of that kind.
But the PISA studies are not enough. They catch pupils at only one moment in their progress – at age around 15. So they tell us nothing about how they got there, nor about the role of schools in helpng or hindering them.
There is one very high-quality Scottish survey – Growing Up in Scotland. This has been tracking a cohort of children who were born around 2005. It has built up a rich body of evidence on progress in the period since then. But it is only one cohort, and there are no plans to follow any new cohorts. This means, for example, that there is no way of using this survey to evaluate the impact of Curriculum for Excellence because the cohort started primary school in the year when the new curriculum was extended across all schools. There is no baseline evidence with which to compare these new experiences.
There is good-quality evidence on attainment from the Scottish National Standardised Assessments and from the Scottish Qualifications Authority, but these bodies of evidence exist in isolation from each other. The information which they produce is thus not collated in any form that would be useful for understanding the progress of pupils.
Drawing on the strengths of these and a few other examples of good practice, the Briefing proposes a new statistical framework for Scottish school education, based on two proposed new resources and a new context for the governance of statistical data:
Regular sample surveys of pupils
New sample surveys would gather evidence about pupils’ progress right through their time at school – starting in Primary 1, gathering further information in Primary 4, Primary 7, and Secondary 3, and having a final questionnaire shortly after leaving school. There woud be a new such longitudinal sample started every three years.
These sample surveys would collect information about curriculum, attainment, experience of school, and students’ social circumstances. At primary-school level, information would also be gathered from the class teacher about teaching methods, so as to contribute to understanding how these methods contribute to students’ progress. Data on attainment would come from the Scottish National Standardised Assessments and from the Scottish Qualifications Authority.
At the primary-school stages, the information on home circumstances would be asked from parents. At Secondary 3 and in the school-leaving stage it would be collected directly from the students. All participation would be voluntary by the students, and would also require active parental consent for all but the school-leaving stage: that is, the parents would have to agree explicitly to take part, not merely not opt out. Communication with the students and with their parents would be direct, not via the school. Only in these ways can the evidence be said to be wholly independent of the school system.
Annual tracking data on all students
Information about attainment would be collated annually. The purpose is to analyse pupil progress. As in the proposed surveys, the attainment in primary and early secondary would be measured by the Scottish National Standardised Assessments, and in later school stages by the results of SQA assessments. So this dataset would be a collation of information that is already held in fragmented ways. The model is the National Pupil Data Base in England, which tracks pupils from age 2 to 21. and has been running successfully and uncontroversially since 2002.
The universal coverage of the tracking would be a firm basis for assessing standards across Scotland. The number of participants would be large enough to provide reliable information about individual schools, and thus to hold schools accountable in public in a manner that would be more informative, and fairer to schools, than the current practice of reporting the results of SQA assessments and of some basic data on progression beyond school.
Office for Scottish Education Data
This new data environment would be overseen by a new Office for Scottish Education Data. As well as being responsible for these two surveys, the Office would thake charge of the full range of statistical data on Scottish school education, including, for example, all the statistical bulletins that are currently published by the Scottish government. Its responsibilities would extend to post-school data. The Office would be required to ensure comparability of Scottish statistical measures with those compiled in the rest of the UK and, where possible, the rest of Europe. The Office should be accountable directly to the Scottish Parliament, not to the government. It would be funded by a grant from the Parliament.
Lindsay Paterson is Professor Emeritus of Education Policy at the University of Edinburgh