An assessment tool grounded in students' perspective: Developing a quantitative measure for intellectual, personal, and civic learning gain using narrative interview data.

Introduction: Learning in higher education encompasses not only acquiring disciplinary knowledge and technical skills but also holistic development emphasizing intellectual, personal, and civic (IPC) learning gain. However, assessing holistic development is a challenging and ambitious endeavour;  current practices struggle to ensure whether students are ‘in development’ and have yet to go beyond grades and outcome-focused indicators. Moreover, educators face practical obstacles in research and implementation, such as large investments of time and effort, making an otherwise valuable qualitative assessment seldom feasible.

Method: This contribution describes the process and challenges of developing questionnaire items to systematically measure students' IPC learning gains from students’ narrative interview data. From three Dutch universities, thirty students (N = 30) were sampled from different disciplines and years of their bachelor studies. They participated in a semi-structured narrative interview in which they reflected on their IPC development during higher education. Using a mixed-method grounded theory (MM-GT) approach, we selected exemplary interview segments that indicated key aspects of IPC learning gain to form questionnaire items after multiple rounds of coding. ATLAS.ti was the qualitative software used for data analysis. Open coding, focused coding, axial coding, and selective coding were used as part of the grounded theory coding method. We used inductive and deductive thematic analysis. The draft questionnaire was then reviewed by an expert panel and pretested.

Conclusions: The resulting questionnaire contains 28 items for intellectual qualities, 24 items for personal qualities, and 28 items for civic qualities. Intellectual qualities include curiosity, open-mindedness, critical thinking, intellectual humility, and intellectual resourcefulness. Personal qualities include perseverance, resilience, self-discipline, and teamwork. Civic qualities include service/helpfulness, civic engagement, civic awareness, tolerance, civility.  Our contribution discusses this mixed-method approach and its added value, as well as how the research team navigated the challenges of constructing the measurement tool for IPC learning gain.