A quantitative methodology for analyzing the impact of the formulation of a mathematical item on students learning assessment

In this paper, we present a methodological approach to the investigation of the impacts of text formulation on students’ answers in mathematical problem solving-based assessment. After a review of the related literature in Mathematics education and a review of the methodologies used until now to investigate this research issue, we describe in depth our quantitative approach, providing motivations and examples of its statistical relevance and its potentiality in making interesting phenomena emerge, to be interpreted with further qualitative methods. We observed statistically significant evidences of different impacts of the variations on different categories of students (males/females; students with high and low performances in the whole test). The methodology and our preliminary results can inform researchers in mathematics education, teachers and experts in the agencies that are responsible for large-scale students learning assessment in several contexts (national and international).