Research Projects

Research Profile

We examine (a) cognitive skills, (b) socio-emotional skills, (c) institutional learning enviroments (e.g., school), and (d) how the interplay of these factors affects educational outcomes and returns to education. To this end, we draw on representative data (e.g., from large-scale assessments) and capitalize on multivariate statistical models and integrative data analysis (e.g., meta-analysis). In doing so, we aim at contributing to evidence based educational practices and policies.

Dissertation and Post-Doc Projects

Current Research Projects (funded by Third-Parties)

BIG-GENDER: Big Data Meta-Analyses of Gender Differences in Students' Achievement and Achievement Motivation Based on Large-Scale Assessments

Project partner: Prof. Franzis Preckel

An essential requirement for any scientific and political discourse on gender differences in school (and beyond) is a reliable body of empirical knowledge on the nature, size, variability, and moderating factors of these differences. This knowledge is highly relevant for at least three reasons: It can be used (a) to learn about gender differences before university entry as plausible antecedents of still existing gender gaps in academic fields, (b) to provide scientific evidence that can help dispel the persistent stereotypes (e.g., that only boys can excel in mathematics) that may discourage girls from pursuing careers in science, technology, engineering, and mathematics (STEM), and (c) to identify target points for evidence-based decision making in educational policy (e.g., boys from families with low socioeconomic status [SES]). The main goal of this meta-analytic big data project is therefore to provide highly robust and widely generalizable knowledge on cross-national gender differences in students’ achievement and achievement motivation (concerning means and variances). To this end, we will meta-analyze individual student data from 999 representative student samples from 112 different countries/economic regions (total N > 4 million) participating in 24 cycles of international large-scale assessments covering the period from 1995 to 2015: the Trends in International Mathematics and Science Study (TIMSS; Grades 4, 8, and 12), the Progress in International Reading Literacy Study (PIRLS; Grade 4), and the Programme for International Student Assessment (PISA; 15-year-olds). This project will be the first to quantitatively synthesize this wealth of data with meta-analytic methods. Specifically, we will conduct three domain-specific meta-analyses to examine gender differences (concerning mean levels and variability) in achievement and achievement motivation in mathematics, science, and reading, respectively. We will study students’ age and SES, the selectivity of the sample (e.g., the bottom 10% or the top 5% of the achievement distributions), sociocultural indicators of gender equality, and historical changes as moderators of gender differences. Further, we will conduct one meta-analysis to examine gender differences in achievement and motivational profiles in multiple domains among three groups of top-performing students who belong to the top 5% in mathematics, science, or reading in their respective countries. To sum up, our project will provide novel insights into cross-national, temporal, and age-related trends concerning gender differences in the general student population and among top-performing students, as well as on the complex interactions between gender, the selectivity of the sample, SES, and sociocultural indicators of gender equality. BIG-GENDER  is funded by the German Research Foundation (DFG).

MULTI-DES: Multilevel Design Parameters and Effect Size Benchmarks for Students’ Competencies

Project partners: Prof. Cordula Artelt and Prof. Oliver Lüdtke

The advancement of educational research as well as the development of evidence-based educational practices and policies that aim to foster students' competencies hinges on the availability of rigorous research and its effective communication to a broad audience. Particularly,  educational intervention studies need to be conducted in ecological valid settings to probe whether the interventions actually work in the field. Such studies are often designed as so-called group-randomized or cluster-randomized trials where entire groups (e.g., entire schools) are randomly assigned to experimental conditions. To ensure sufficient statistical power several design parameters are essential to determine the sample size (i.e., number of students, classes, and schools) of group-randomized trials. Design parameters comprise estimates of between-school and between-class differences (in terms of intraclass correlations) as well as the amount of variance explained by covariates (e.g., pretest scores) at the individual, class or school level. Moreover, to effectively communicate effects of educational interventions on students´ competencies requires empirical effect size benchmarks that serve as vital aids to interpret empirical results. Crucially, both design parameters and effect size benchmarks should correspond to the target population. The present project is the first to rigorously examine design parameters and effect size benchmarks that are targeted to the multilevel framework of Germany´s school system. To this end, we will conduct four studies that capitalize on data from three German longitudinal large-scale assessments: the National Educational Panel Study (NEPS), the longitudinal extension of the German year 2003 cycle of PISA (PISA-I+), and the Assessment of Student Achievements in German and English as a Foreign Language (DESI). These large-scale assessments cover a wide range of students´ competencies and age groups (i.e., students from grade levels 1 to 12). More specifically, Study 1 will examine design parameters for two-level designs (students nested within schools) and three-level designs (students nested within classes, classes nested within schools) for the general student population as well as for different school types. Study 2 investigates how various covariates (i.e., pretest scores, sociodemographic characteristics, basic cognitive functions), their combination, and the time-lag between pre- and posttest  affect the precision/statistical power of group-randomized trials for two-level and three-level designs. Study 3 examines academic-growth as effect size benchmark for the general student population and different school types. Finally, Study 4 investigates performance gaps between weak and average schools as effect size benchmarks. MULTI-DES is funded by the German Research Foundation (DFG).

MULTI-DES 2: Multilevel Design Parameters for Sample Size Planning of Randomized Intervention Studies in Preschool, Elementary and Secondary School

Project partners: Prof. Cordula Artelt, Prof. Larry V. Hedges, and Prof. Oliver Lüdtke

Education in preschool, elementary and secondary school aims at fostering (cognitive) competencies and socio-emotional characteristics. Importantly, there is an increasing need for educational research to provide evidence which educational interventions and practices actually improve these outcomes. To provide this kind of evidence, large-scale randomized experimental studies are indispensable because they allow for strong causal inferences on the effectiveness of interventions in ecologically valid settings. Such studies are often conducted as (a) individually-randomized trials (IRTs) where individuals (e.g., students) or (b) cluster-randomized trials (CRTs) where entire groups (e.g., entire schools) are randomly assigned to experimental conditions. To ensure that IRTs and CRTs are sensitive to detect intervention effects (if they exist) requires to apply reliable design parameters to determine their sample size in an a priori power analysis. Given the multilevel organization of educational settings in Germany, design parameters involve information (e.g., in terms of intraclass correlations) on the amount of variance in outcomes attributable to various levels (i.e., children/students, groups/classes, daycare centers/schools) as well the amount of variance (R²) that can be explained by vital covariates at these levels, involving socio-demographics and baseline measures. Our precursor project MULTI-DES was the first project that provided such design parameters for students’ competencies in elementary and secondary school in Germany. Capitalizing on data from several German probability samples (e.g., the National Educational Panel Study [NEPS]), the present project now substantively expands the knowledge base of design parameters to plan the sample size of IRTs and CRTs. First, we analyze design parameters targeting competencies and socio-emotional characteristics of children in daycare centers—the institutional preschool setting which most children attend in Germany. Second, we analyze design parameters targeting students’ socio-emotional characteristics in elementary and secondary school. Third, it is impossible to provide design parameters with optimal fit to all potential outcomes or specific populations that may be targeted by IRTs or CRTs. We therefore now develop a tutorial (involving all necessary statistical procedures) that enables educational researchers to exploit large-scale data sets (e.g., PISA) to analyze their own design parameters for continuous (e.g., scores on a standardized competence test) and binary outcomes (e.g., whether students earn a high school diploma or not) and to conduct power analyses for IRTs and CRTs. The tutorial will be accompanied by a workshop to increase the national capacity of early career researchers to develop and analyze rigorous ICTs and CRTs that study the impact of education interventions. MULTI-DES 2 is funded by the German Research Foundation (DFG).

CoCoaPower—Center for Statistical Consulting & Coaching on the Design of Randomized Trials in Education

Project partner: Prof. Larry V. Hedges

Evidence-based policies and practices are key to successful education—and declared aim of governments worldwide, as in Germany. The overall goal of CoCoaPower is to forward this endeavor by supporting the accumulation of robust empirical evidence on the impacts of educational and psychological programs and innovations by means of randomized trials (RTs) in Germany. Specifically, we aim at advancing the methodological quality of (multisite) individually RTs ((MS)IRTs) and (multisite) cluster RTs ((MS)CRTs) by bundling and offering resources and expertise on design decisions in general, and on power analysis in particular. In doing so, CoCoaPower has two pillars: (1) A statistical consulting service will support researchers in planning rigorous (i.e., adequately powered) RTs by providing (a) concrete results obtained from power analysis specifically tailored to researchers’ target study designs, and (b) advice on general design decisions (e.g., with regard to the choice of a specific design or the selection of covariates). (2) A statistical coaching infrastructure will qualify researchers to develop well-designed RTs by providing (a) specialized workshops on power analysis for various RT designs, and (b) web-based tutorials on power analysis that guide researchers through the steps necessary to determine the relevant quantities of a strong study design. CoCoaPower builds on previous and current work in the context of the MULTI-DES project series and is funded by Larry V. Hedges.