Applying lexical statistics to rational cloze test to assess the relevant vocabulary in course readings

The study is part of the "adaptive-message learning" project (FIRB, 2009-2013), aimed at the individualization of learning messages in e-learning systems, which consists of adapting the wording of a study text to the ability of a reader cooperating with the text. Within the project, the objective of the assessment is therefore to measure indirectly this individual ability in relation to a precise context (linguistic and extra-linguistic), that is to know how effectively the students cope with course readings. In more detail, computer-generated, multiple-choice, rational-deletion cloze tests are used to measure the extent to which learners can handle relevant vocabulary in discipline-specific texts. Quantitative analysis of words in a corpus of LSP texts is used for identifying relevant vocabulary. The validation of this tool required specific attention. This is due to the conditions of use of the test within the project, which requires the definition of a procedure which must be largely automatic so as to satisfy the need for continuous assessment (the tests are administered at the beginning of each learning unit and periodically throughout). For this reason, it was necessary to compare the effects of different statistical criteria on classifying and selecting content words to delete in a passage. We consider the results of a set of tests conducted at the University of Modena and Reggio Emilia. Since tests were developed as part of a course, students who were actually taking the course were used as subjects. Self-contained excerpts from Kinesiology and Research Methodology in Education course readings were sampled, which are characterized by different levels of formalization of the language used for writing the texts: the latter written in an academic register in standard Italian while in the former a language specific to Rehabilitation Medicine is prevalent.