Critical Thinking Skills Through Work-Based Learning Activities: A Pilot Experience in the Educational Science Context

. The present paper aims to illustrate the learning results of a pilot experience focused on the development of critical thinking (CT) skills through work-based activities, as an example of continuous instructional improvement in higher education institution (HEI) contexts. Speciﬁcally, more than 100 university students of two master’s degree programs at University Roma Tre participated in two different online learning paths, carried out by the Centre for Museum Studies research group during the academic year 2020–2021, containing activities of analysis, interpretation, argumentation, and critical evaluation of work-based experiences in the educational ﬁelds. Speciﬁc online meetings with stakeholders were carried out within the paths, together with experiences based on the following teaching methodologies: problem solving, oral dissertation, digital storytelling, analysis, and critical reﬂection. Formative and summative evaluation activities were carried out in the pilot experience in order to collect and analyze data in relation to the promotion of professional and CT skills. Results from the pilot experience show a statistically signiﬁcant improvement in some CT indicators within students participating in the activity and an overall good evaluation of the learning courses, stakeholder meetings, and assigned work-based activities. In addition to CT, collaboration and creativity skills were also self-assessed by the students, as stimulated by the online activities.


Introduction
Nowadays, a debate regarding the role that higher education is supposed to play in the broader society is present at an international level. The debate refers to a dialectical conflict between two different stances: should university prepare students to fulfil the job market needs? Or is the university supposed to transfer knowledge without considering professional skill training? An education system that focuses on developing higher-order skills, especially critical thinking (CT), could overcome such a conflict.
As stated by Facione, CT is "purposeful, self-regulatory judgement which results in interpretation, analysis, evaluation, and inference, as well as explanation of the evidential, conceptual, methodological, criteriological, or contextual considerations upon which that judgement is based. CT is essential as a tool to inquiry" (p. 2) [1]. In order to define CT, it is necessary to focus on the dimensions that characterized it, and that are basically divided into cognitive skills (such as interpretation, analysis, assessment, inference, explanation, and self-regulation) and dispositions (such as curiosity, appreciation of individual differences, skepticism, and flexibility) -positive approaches that favor its development [1][2][3][4]. CT skills are more and more considered pivotal for human and social progress in terms of innovation, economic and knowledge growth by educational policy [5,6]. Also, CT provides students with tools to be autonomous thinkers, active citizens [7], and critical users of digital technologies [8]. As pointed out by several researchers, even during the COVID-19 pandemic, CT is defined as a necessary competence in order to support people in logical reasoning, thoughtful analysis, adaptability to new contexts, problem-solving, logical solutions, and social cohesion [9][10][11].
The complexity of the CT construct requires careful and rigorous pedagogical design and interpretation of data in the educational field in order to identify methodologies and tools that are truly effective in promoting CT in learners. Moreover, the promotion of CT learning and teaching methods should be considered as an urgent need in all the formal educational context, taking into consideration the different dispositions and cognitive skills to be promoted at a university level. University teaching activities aimed at promoting CT in students can be implemented through the use of real-world situations and/or workplace-based scenarios, as stated by Dominguez et al. [12]. In particular, active learning methodologies, such as authentic situations and problem-based activities, engage students in problem-solving and decision-making, thus promoting some of the six core CT skills set by the experts [13]: interpretation, analysis, evaluation, inference, and explanation. In particular, educational practices based on problem-based learning methodology can improve social science students' abilities to analyze, compare, and share experiences, as well as improve engagement with content and self-assessment [12]. In general, although research on the effectiveness of instructional interventions is inconsistent [12], if CT skills are solicited in disciplinary settings, outcomes appear improved [14].
In addition, great attention should be paid to the methods and tools for evaluating CT, which are considered an open challenge in the educational and professional field. As already defined in previous publications [15], CT assessment tests can be classified in different ways. Multiple-choice (MC) tests or questionnaires are those most widely used because -in general terms -they best meet the reliability criteria of an assessment test [16]. However, some authors point out that the MC measures are not suitable for higher-order skills assessment, such as CT, but they may be answered merely by lowlevel processing, such as factual recognition and selection [17]. Moreover, MC tests can never assess students' skills to synthesise ideas or written text [18]. Lastly, all the tests based on MC are not free, which limits their accessibility and their use in educational contexts. To address the limitations of MC tests, researchers and authors of the present paper have developed alternative assessment methods, which involve the adoption of open-ended tasks.
Starting from these assumptions, the Centre for Museum Studies (CDM) research group, coordinated by Antonella Poce, developed in the academic year 2020-2021 a series of higher education institution (HEI) learning paths aimed at promoting CT through specific activities carried out within workplace context. A specific evaluation rubric, already validated in previous publications [16], was used to assess CT skills solicited by the students during the activities.

Goals of the Research
According to the above-mentioned premises, the main goal of the herewithin described pilot experience is the promotion of CT skills in HEIs students through work-based learning activities. The research group also tried to answer the following research questions: • How do student CT indicator levels change in a university course designed to support students' CT levels? • What is the level of CT perceived by the students at the end of the activities?
• What other transversal competences are solicited by a university course foreseeing work-based activities? • Did the online activities, including assignments and meetings with educational professionals, improve the levels of CT in the participating students?
In order to answer the above Research Questions, the researchers designed different phases of monitoring and evaluation of the learning activities carried out by the students and two different evaluation tools were used.

Learning Activities
During the academic year 2020-2021, the research group developed a series of HEI learning paths aimed at promoting CT through specific activities carried out within workplace context. Specifically, 125 students from two master's degree programs at University Roma Tre participated in two different learning paths containing activities of analysis, interpretation, argumentation, and critical evaluation of work-based experiences in the educational field. The pilot experience can be considered an example of continuous instructional improvement, in which students are continuously solicitated in analyzing, interpreting, evaluating, and finding solutions through specific work-based activities.
The students participating in the activities are divided as follows: • Seven students from the master's degree course in pedagogical sciences attending the second-year course in "Experimentalism, Museum and Reading" from October to December 2020 (12 ECTS).
• A total of 118 students from the master's degree course in primary school education attending the first-year course in "Research Methodology in Education" from October to November 2020 (4 ECTS).
Within the above modules, specific online meetings with stakeholders were held, and experiences based on the following teaching methodologies were discussed: problem solving, oral dissertation, digital storytelling, analysis, and critical reflection.
In total, students were asked to participate in all meetings organized with stakeholders from the education sector and to accomplish at least three out of five activities assigned by the professionals themselves. The activities were organized throughout the courses and concerned contents in line with the courses' objectives.
For organization reasons and due to the different numbers of participating students, the activities of the course in "Research Methodology in Education" were carried out collaboratively by the students, who worked in 17 groups of 6-10 members each. For the "Experimentalism, Museum and Reading" course, students had the choice of working individually or in groups of up to three participants. Table 1 gives a summary of all the online activities' topics and meetings organized for each course. The stakeholders who took part in the online meetings are organizations, associations, and companies linked to University Roma Tre through partnership agreements and work in the field of education and research in education. The work-based activities related to the online meetings were designed by the stakeholders, with the support of the teachers and tutors of the university courses, in order to ensure a connection with the specific learning objectives of the courses themselves, as well as a continuous implementation of the learning path according to the monitoring results collected during the process.
In line with to the European Commission document Work Based Learning in Europe: Practices and Policy Pointers (2013) [19], work-based learning can be identified by three main models: -Model 1: alternance schemes or apprenticeships, also known as "dual system," in which students spend a significant amount of time training in companies and acquire general and occupation-related knowledge in VET schools or other education/training institutes. -Model 2: school-based VET, which includes on-the-job training periods in companies.
-Model 3: work-based activities are integrated in a school-based programme, through on-site labs, workshops, simulations or real business/industry project assignments.
The pilot experience here presented was designed taking into consideration the last model, in which the simulation of a professional and entrepreneurial environment was carried out with the cooperation of the participating stakeholders.

Evaluation Tools and Phases
During the two courses, formative and summative evaluation activities were carried out in the pilot experience in order to collect and analyze data in relation to the promotion of professional and CT skills. In particular, the evaluation phases were divided as follows: 1. Evaluation of the work-based activities carried out by the students or groups of students participating in the pilot experience, aimed at monitoring the solicitation of CT skills during the courses. 2. Final summative evaluation through a questionnaire, aimed at assessing the perception of the levels of transverse competence reached by the students at the end of the pilot experience and the quality of the activities and meetings organized.
Two different evaluation tools were used for the two phases of monitoring and evaluation of the pilot experience.
• CT evaluation rubric, already validated in previous publications [16], and composed by six macro indicators on a scale from 1 to 5: use of the language, argumentation, relevance, importance, critical evaluation, and novelty. • Questionnaire for self-assessment of transverse and professional skills and evaluation of the quality of the activities and online meetings carried out in the pilot phase. The questionnaire was created ad hoc for the experience, starting from indicators already validated and used in Poce, Agrusti, and Re [20].
In this analysis, 87 work-based products, realized by the student participating in activities, were evaluated in order to analyze the CT levels during the courses. Sixty-one products are part of the 4 work-based activities proposed for the "Research Methodology in Education" module and 26 for the "Experimentalism, Museum, Reading" module. Three evaluators blindly assessed each work-based activity using the CT evaluation rubric. The mean scores obtained were analyzed through basic statistical analysis. The questionnaire for self-assessment of transverse and professional skills and evaluation of the quality of the activities and online meetings was filled in by all the 125 students participating in the pilot experience. This evaluation tool is composed by different section and indicators, as summarized in the following Table 2. Data from the final evaluation questionnaire were analyzed through basic statistical analysis.

Analysis of the Data from the Final Evaluation Questionnaire
Generally, the data from the overall evaluation of the modules are positive. On average, 92% of the students agreed that the course promoted their professional skills. Most of the students strongly agree that they found new inspiration interacting with colleagues during meetings and work-based activities (63.2%), they perceived the support of the university team of professors and tutors during the course (64%), and they received feedback and useful answers during the online sessions (64.8%). Furthermore, more than 90% of the students agreed that the course will allow them to perform better in their studies (90.4%) and in their work activities (91.2%). Again, more than 90% of the students quite agree (62.4%) or strongly agree (29.6%) that their point of view was enhanced by the tutors and professors during the online meetings.
With regard to the quality of the content of the meetings, the data also show extremely positive results. A total of 56% of the participating students assign the maximum score of 5 to the didactic tools used as support during the online meetings, such as videos, PDFs, and slides. The language used by the stakeholders is defined as extremely clear for 45.6% of the students; 22.4% of the participants assigned a score of 5, and 46.4% assigned a score of 4 to the quality of the course in terms of "Exhaustiveness of content." However, only 8% of the students stated that the assigned work-based activities were not difficult at all, assigning the minimum score of 1 in terms of "Difficulties of the activities carried out." Actually, the activities had a difficulty level of 3 out of 5 for 32% of the participants and of 4 out of 5 for 20.8% (see Fig. 1). Concerning self-assessment of the professional competences promoted during the course, respondents give the highest mark to the competence of "Organising activities, workshops and educational paths," followed by "Independently carrying out research to create educational activities" and "Identifying students' needs, problems and barriers to learning" where thy consider themselves as very competent (33.6%, 32.8%, and 29.6%, respectively). The competence level of "Creating materials functional to educational interventions" is also self-assessed with high scores by the students: 23.2% of the participants perceive a maximum level (4 out of 4) of this professional competence at the end of the course and 58.4% perceive a level of 3 out of 4. The competence of "Team planning educational and research activities" is also self-assessed with high scores by the students (36% self-assessed the maximum level of competence): the reason is associated with the type of group activity set up for most of the participating students (118 out of 125), which proved to be particularly effective also in online education contexts. The competence related to "Planning educational interventions in partnership with other institutions" is the one which receives the lowest self-assessment scores from the students: 38.4% of the respondents score this competence 2 (out of 4).
Interesting results emerge from the analysis of the data concerning self-assessment of transverse competences. In the questionnaire, students had the possibility to select the 3 transversal competences that they felt had been most demanded during the course, either through online meetings with stakeholders or through the work-based activities. Collaboration, creativity, and CT were rated as the most supported skills, having been selected respectively by 91%, 67%, and 63% of students. These are 3 of the 4C skills, already defined by Trilling and Fadel [21] as the basic skills necessary for the promotion of more complex competences, skills, and attitudes. Moreover, the constructs of collaboration, creativity, and CT skills present similar characteristics and indicators: for example, the "Novelty" indicator, used in CT assessment, is closely related to the competence of creativity and divergent thinking. Furthermore, Collaboration is defined as an essential condition for the promotion of CT, considered as a competence that is realised living in society [22]. Research abilities and communication skills were also rated positively: 59% and 58% of students, respectively, selected them as the three most promoted skills during the course. The skills least selected by the students were problem solving (30%) and innovation (21%), even if some work-based activities were designed to foster such skills (see Fig. 2).

Evaluation of CT Skills
In total, 125 students participated to the learning experience. The first group of participants is composed of 7 students from the master's degree course in pedagogical sciences attending the second-year module in "Experimentalism, Museum and Reading." The averages of the assessments carried out in the double-blind mode by three evaluators were analyzed through basic statistical analyses in order to verify the level of solicitation of the following CT indicators: use of language, argumentation, relevance, importance, critical evaluation, and novelty. Analyses show that the average scores assigned to 4 out of 6 indicators increase from the first activity to the last one: the mean of the scores assigned to the first group of students increases respectively in the indicators of justification (from 3.6 to 3.8 points), relevance (from 3.5 to 3.9 points), importance (from 2.3 to 3.7 points), and critical evaluation (from 3.4 to 3.6 points). The average score obtained by the participants changes from 21 points in the first activity to 22 points in the last activity (first activity ds = 2.98; last activity ds = 2.76). The average score in outgoing activity is always higher than the average score in incoming one, except for the indicators use of language and novelty.
Students participating in the four activities slightly increase their ability into justify their opinions (argumentation) and also slightly improve the ability to evaluate sources, data, and background knowledge using a personal and critical elaboration (critical evaluation). The Wilcoxon signed rank test shows that the improvement in participants' importance levels from the first to the last activity is statistically significant (P = 0.048).The second group of participants is composed by 118 students from the master's degree course in primary school education attending the first year course in "Research Methodology in Education," who were divided in 17 groups. The average score obtained by the participants increased from 18.7 points (ds = 2.4) in the first activity to 22.5 points (ds = 2.7) in the last activity. The data analysis shows that the average score given by the participants increased in all the indicators: language use (pre-test = 3.14, ds = 0.6; post-test = 3.7, ds = 0.4), argumentation (pre-test = 2.9, ds = 0.5; post-test = 3.7, ds = 0.5), relevance (pre-test = 3.3, ds = 0.5; post-test = 3,8, ds = 0,4), importance (pre-test = 3, ds = 0,3; post-test = 3,8, ds = 0,4), critical evaluation (pre-test = 3,2, ds = 0,6; post-test = 3,9, ds = 0,5), novelty (pre-test = 3, ds = 0,7; post-test = 3,7, ds = 0,6). As shown in Fig. 3, students participating in the four activities increase their writing skills, demonstrating improvements in the use of the language indicator. They also show a higher capability of justifying their opinion (argumentation). They learned to be more relevant and consistent (relevance) in their expressions and to better link their previous knowledge with new situations (importance). Moreover, they show a higher ability into evaluating sources, data, and background knowledge using a personal and critical elaboration (critical evaluation), and using unusual terms to elaborate their ideas (novelty). In order to analyze whether the differences in the scores assigned to the indicators between the first and last activity were statistically significant, the Wilcoxon signed-rank test was used: the results show a significance P < 0.01 for all six indicators analyzed (see Table 3).

Discussion and Final Remarks
CT skills are increasingly considered as pivotal for human and social progress. For this reason, the creation of learning modules aimed at promoting CT should be enhanced at all levels of education. The present paper aims at illustrating some results of a pilot experience carried out by CDM research group based at University of Roma Tre for the purpose of increasing CT skills in HEI students through work-based activity and online meetings with stakeholders from the educational field. The pilot experience can be considered an example of continuous instructional improvement. Data analyses show a generally positive evaluation of the pilot experience carried out in the academic year 2020-2021 by 125 university students attending two different modules, "Experimentalism, Museum and Reading" and "Research Methodology in Education." The analysis of CT levels carried out using a specific evaluation rubric shows that the scores assigned by three evaluators to the outgoing activity is higher than the incoming activity for different CT indicators: as regards the students participating in "Experimentalism, Museum and Reading" module (N. 7), the improvement of importance levels from first to the last activity is statistically significant (P < 0.05). Regarding the students attending the "Research Methodology in Education" (N. 118) module, the improvement in CT levels is statistically significant for all the CT indicators under evaluation: use of the language, justification, importance, relevance, critical evaluation, and novelty. In general, the CT levels of students participating in the pilot experience improve during the module.
At the end of the activities, the students participating in the pilot experience selfassessed their level of CT as very good: CT is one of the three most selected transverse competences defined by the students as more solicited through the online meetings with stakeholders and work-based activities, together with collaboration and creativity. In addition, some professional competences were also positively self-assessed by the students, such as "Organising activities, workshops and educational paths," "Independently carrying out research to create educational activities," and "Identifying students' needs, problems and barriers to learning." Stakeholder meetings and work-based activities are also evaluated positively by the students, especially in terms of didactic tools, language, and exhaustiveness of content.
The pilot experience has large room for improvement, but it presents some important indications for the implementation of online learning paths aimed at enhancing CT skills with the support of external stakeholders and through continuous instructional learning design.