Systematic Legal and Administrative Review. A New Methodological Approach to Analyzing Legal and Administrative Data in Educational Science

In modern democratic societies, education is highly regulated both legally and administratively: from the state’s legal educational mandate, to the constitutional rights of students (to education, inclusion, participation, etc.), to the legally defined roles and tasks of relevant actors (teachers, principals, superintendents) or cases in everyday school life (e.g. the neglect of supervisory duties by teachers, religious practice in the school building, approval of school magazines etc.). Despite the large number of legally relevant regulations, law has so far played a marginal role in educational science, empirical educational research, and the qualification of the aforementioned actors. Although there is some work on the legal literacy of specific actors (e.g., Decker, 2014), there is a general lack of interdisciplinary exchange on the legal dimensions of education. This gap coincides with a methodological gap regarding the analysis of legal and administrative data (e.g. school laws) – despite their advantages: they are usually freely accessible and not covered by any data protection regulations; moreover, they are published on government online platforms and are therefore non-reactive; finally, they are available in large quantities, which enables comprehensive data collections. To date, however, there is no systematic methodological approach to collecting and analyzing this type of data. Studies that refer to legal or administrative documents either use less transparent forms of document analysis or select relevant documents eclectically to suit the respective research question. Such approaches undermine current scientific standards as they are not transparent, therefore not verifiable, and do not meet established criteria of research quality. The method of the systematic legal and administrative review presented in this paper addresses this problem as an innovative approach to research legal and administrative data in educational science. The method combines elements of the systematic literature review (Booth et al., 2022) in terms of data collection with methods of qualitative content analysis (e.g., Krippendorff, 2019) in terms of data analysis. The paper provides an overview of relevant legal and administrative data (1), presents the reviewing steps of data collection and analysis exemplified by legislative data (constitution, laws, ordinances) (2), and discusses the advantages and disadvantages of this approach (3). Legal and administrative data can be defined as all official documents generated by the political authorities, i.e., legislation, administration, and jurisdiction, to regulate a social system – in this case, the education system. At the legislative level, this includes constitutions, laws (e.g. school laws) and ordinances (e.g. school regulations), which can be systematized into a hierarchy of norms (Kelsen, 1960/2020); at the judicial level, this includes all education-related jurisdiction, especially from constitutional and administrative courts; at the administrative level, it includes statutes, decrees, etc., which concretize legislative regulations. The steps used in systematic literature reviews are adapted for the collection of legal and administrative data (Booth et al., 2022). First, a search syntax and a search strategy must be developed to identify relevant regulations for the research question on the respective government platforms (identification phase). The subsequent data selection follows the well-established PRISMA standards (Moher et al., 2015): in the first selection step, the regulations are included or excluded based on their header and according to predefined inclusion and exclusion criteria (screening phase). The remaining regulations are then included or excluded based on their full text (eligibility phase). The content of the remaining regulations is then qualitatively analyzed (inclusion phase). Every step is carried out independently by two researchers to assess inter-rater reliability. Transparency, replicability, reliability, and comprehensiveness are the advantages of such an approach: it is transparent because it is possible to trace precisely the process that led to the included regulations. Thus, the selection procedure becomes replicable. The inter-rater reliability guarantees the research quality. Finally, in contrast to eclectic hand searches, it can be ensured that all relevant regulations are comprehensively identified. The first disadvantage is the amount of data: due to the large number of legal or administrative documents, the screening process takes a long time. However, this issue can be resolved by using computer-based techniques such as text mining. Second, as these are documents published by governments, the quality of the search results depends on the quality of each government database. Thus, the paper not only highlights the hitherto neglected potential of legal and administrative data but also provides an answer to how these data can be effectively researched using systematic legal and administrative review.