Beyond Employment Rate: A Multidimensional Indicator of Higher Education Effectiveness

This paper proposes a multidimensional indicator of higher education effectiveness that aims at going beyond the limits of measuring university effectiveness merely through employment rates. The units of analysis are the study programmes. Eleven indicators related to external effectiveness are selected, and their reliability for and relevance to the representation of the concept of effectiveness are empirically evaluated. The data are drawn from a longitudinal survey administered to graduates of the University of Padua, Italy, from 2008 to 2011. From our analyses, effectiveness appears to be a multidimensional concept composed by professional empowerment, employability and personal fulfilment. The right time for collecting relevant data on educational outcomes varies according to the types of indicators: indicators of professional empowerment assessed 1 year after graduation are most suitable, while for personal fulfilment measurement both short- and long-term evaluation are relevant, and, for employability, data collected 3 years after graduation cannot discriminate among study programmes.


Indicators of Education Effectiveness
When assessing the performance of a particular educational process, a popular approach is to evaluate a set of effectiveness indicators. Educational effectiveness connects students' learning outcomes to the educational processes they went through. In this paper, we point out the external effectiveness of higher education, namely the outcomes of graduates in relation to their professional success and in society at large (Hanushek 1979;Cowan 1985;Lockheed and Hanushek 1994).
Despite the proliferation of surveys of graduates, most scholars dealing with the external effectiveness of university education merely analyse the time elapsed between graduation and first employment (e.g. Biggeri et al. 2001;Nguyen and Taylor 2003) or the probability of employment at a certain point in time (Bratti et al. 2004). Although occupation is an important result of the educational investment, the employment rate only gives a partial image of educational effectiveness. There is a need to identify other dimensions and indicators of university effectiveness.
In the last few decades, there has been an upsurge of studies on indicators highlighting processes, outcomes, contexts and malleable input factors that represent elements of a never-ending improvement process of higher education (among others, UNESCO 1974;United Nations 1975;Cowan 1985;Stern 1986;Astin 1993;Scheerens and Bosker 1997;Draper and Gittoes 2004;Bird et al. 2005 However, it is worth noting the following points: • Indicators that can be used to measure the learning empowerment capacity of a single university may differ from those recommended by international organisations to compare between-country outcomes. In fact, an indicator is particularly meaningful if it is targeted to the social segment that it was built in. Symmetrically, indicators' values are conditional according to their framework (Land 1975;UNDP 2010;Australian Government 2011). • The focus on external effectiveness rather than effectiveness that is internal to a university results in the indicators being less indicative of what happened during the educational process (Gibbs 2010) as attention is on the end-use purpose of higher education rather than the educational process.
For measuring effectiveness, no simple measures are sufficiently valid in terms of relevance 1 and reliability. 2 Rather, a general framework is needed to encompass complex concepts such as educational quality, stakeholder satisfaction, student retention and learning outcomes (Finnie and Usher 2005;Chalmers 2008).
A system for evaluating educational effectiveness requires that outputs and outcomes of a process or a system be placed in relation with antecedent (academic inputs and educational processes) and concomitant (social, economical and physical systems) factors. The relationships that exist among higher education factors may be represented as shown in Fig. 1.
In this paper, we propose a multidimensional indicator of higher education effectiveness and evaluate the relationships among its dimensions and with the common concept of external effectiveness. Our perspective is based on the assumption that there is no unique 1 Relevant is an indicator of what we need to know. This property implies that the indicator is designed to match the research objectives (i.e. 'design validity', see Bockstaller and Girardin 2003); from the stakeholders' and decision-makers' viewpoints, an indicator's output is expected to be useful to inform their decisions. 2 Reliable is an indicator's measure that can be trusted. This property basically refers to the consistency of outputs in repeated trials under the same essential conditions ('output validity').
indicator of education effectiveness but rather that some indicators (amongst all the reliable ones) describe the target issue better than others.
The types of indicators that can be considered for assessing the external effectiveness of higher education are: (a) Employability rate The employability of a graduate varies with time, and the time reference has to be fixed to allow for a comparison of outputs in space and time and across categories. It can be computed as an employment rate with reference to a cohort of graduates who have all looked for a job during a certain time interval. Six months, 1 year or 3 years after graduation are typical intervals at the end of which an output can be estimated (with reference to Italy, see Istat 2009;Fabbris 2010;Cammelli and Gasperoni 2012;Lucarelli et al. 2012). Educational effectiveness may also be viewed as a long-term concept. It could be evaluated several years after graduation, although, as time goes by, the effects of inherited skills and those that stem from society may overcome those learnt at university (Dey et al. 1999). (b) Job-education matching rate and major-specific job achievement rate In principle, graduates with differing abilities should be sorted out in the job market, with the most competent obtaining university graduate-level jobs (e.g. jobs at their educational level) and the least competent obtaining jobs below this level. It is difficult to objectively match a graduate's skills with those required by his or her current job; thus, for this purpose, outer experts should be involved. Possible mismatches could generate either overqualification (or overeducation) or underqualification (or undereducation) if, respectively, there is a schooling surplus or the needs of a job exceed the graduate's skills (see the literature quoted in Boccuzzo and Paggiaro 2012). Some researchers report matching rates ranging between 45 and 60 % in western countries and mismatching rates of 10 to 25 % on both sides of the matching distribution (Daly et al. 2000;Hartog 2000;Groot and Maassen van den Brink 2000;Sloane 2003;Quintini 2011;Cainarca and Sgobbi 2012). Since mismatch is a concept close to unemployment (Sicherman and Galor 1990;McGuinness 2006;Verhaest and Omey 2009;McKee-Ryan and Harvey 2011), it represents a social concern. If we simply rely on graduates' judgement to match current and expected skills, matching can be detected if both a graduate's major is specifically required ('job-major matching') and the current job requires a higher education degree irrespective of the major ('job-education matching'). Both types of mismatching are university concerns, although they may be due to different causes. (c) Job refusal rate This indicator mirrors the number of graduates who, at a certain time after graduation, received job offers that they refused. It may measure higher education effectiveness inasmuch as a graduate refuses job offers that are less attractive than his or her current job. As Martini (2012) has shown, a refusal may occur because the profile of the offered job is so low that the graduate would rather stay unemployed. In addition, this indicator may measure a graduate's propensity to attract jobs of a certain quality and may then be used to assess his or her return to education at either the athenaeum or degree programme level. (d) Graduates' human capital The human capital of a graduate is his or her potential for rapid and successful professional integration and for a successful social life (see also Becker 1994;Mincer 1981;Heckman 1999;Blöndal et al. 2002;Lovaglio and Vittadini 2007;Hanushek and Woessmann 2008;Rindermann 2008). From an economic viewpoint, human capital is a body of knowledge, skills and attitudes that the economy requires to realise products and services and to enhance productive capability. Fabbris and Favaro (2012) have analysed three aspects of human capital that all graduates possess to varying degrees: (1) technical-specific competence (i.e., technical knowledge and know-how that are major specific), (2) cross-occupation competence (i.e., technical competencies that are needed to perform most university graduate-level jobs) and (3) psychological and ethical inclinations that are relevant to working. Information on the portfolio of competencies and values relevant to obtaining a job can be collected from graduates. When referring to those who are employed, it is possible to measure a correspondence between the skills achieved at university and those needed at work. Also, through surveys, the unemployed can provide information on the perceived match between education and work, but their judgements might lack understanding of the labour market needs. (e) Additional training rate While they are looking for a job, many university graduates may enrol in courses with various content. Typically, they enrol in language, computer science or other skill-improving courses. The training that graduates add to their already-achieved higher education can be determined by either their desire to acquire further professional skills or by their sense of insecurity about the market value of their degree. Hence, additional training could be due to a lack of basic education. The rate of postuniversity training can be measured by asking graduates to report their specific experiences after achieving their degree. According to study objectives, this may or may not include on-thejob training. (f) Adequacy of professional specialisation rate It may be that either underspecialisation or overspecialisation of achieved skills hinders a graduate's search for a job or affects the quality of obtainable jobs. The good dose of specialised competencies achievable from attending a study programme may aid graduates both in finding a job and in career development and, in turn, may lead to a successful social life. Hence, the search for equilibrium between overspecialisation and underspecialisation may be an objective of university decision makers. Therefore, it makes sense to investigate-both before and after gaining a job-graduates' feelings about the adequacy of their degree specialisation. (g) Satisfaction of graduates regarding their studies Customer satisfaction is a popular indicator of product or service effectiveness. In higher education, the satisfaction perceived by a graduate regarding his or her studies reflects an end-user evaluation of the outcomes generated by the educational services he or she received in relation to both the speed with which he or she obtained a job and the quality of that job. This indicator, which can be considered a global measure of graduate satisfaction for a particular study programme (IIEP-UNESCO 2011), is sometimes measured by surveying students who have finished their exams and are waiting to discuss their theses (see Consorzio Interuniversitario AlmaLaurea 2011; Iezzi and Mastrangelo 2012;contra: Berk 2005). However, it is more often determined for graduates before or after they obtain employment. (h) Willingness to repeat the educational experience A typical indicator of customer satisfaction is his or her stated intention to repeat the consumption experience. Asking graduates about the willingness to repeat an educational experience has a similar purpose. This question involves the willingness of graduates to attend either the same study programme, the same university or both. Their willingness to repeat an educational experience can be asked retrospectively at any time in their professional and social lives.
We can assume that all these outcome indicators are positively correlated to the objective of improving the effectiveness of higher education and thus have an impact on the economic system and the society as a whole. All the indicators describe aspects of effectiveness, but none describe the effectiveness concept in full. Therefore, an important objective is identification of the underlying concept of effectiveness and the estimation of how much each indicator represents the whole.
To shed light on the relationships among and the properties of a set of effectiveness indicators, we have processed data that the University of Padua collected on its own graduates via a longitudinal survey called Agorà. Restricting the analysis to graduates of a single university is a simplification that allows us to develop an indicator without the need for disentangling the differences in university effectiveness ascribable to each study programme from those resulting from different local contexts and labour markets. The survey was carried out from 2008 to 2011 and involved a stratified random sample of 4769 students who graduated from Padua University in 2007 and 2008. The graduates were surveyed at 6, 12 and 36 months after completing their university studies (for a total of 3 waves). The sample sizes for each survey were as follows: 4544 graduates responded to the first wave; 3392 responded to the second; and 2855 responded to the third (for more details, see Fabbris 2010).
The list of the indicators analysed in this study and their measurement peculiarities are presented in Sect. 2. In Sects. 3 and 4, the results of the analyses are presented, and Sect. 5 is the conclusion.

Data and Methods
The indicators that we analysed are presented in Sect. 2.1, and the model that we used to define the final set of external effectiveness indicators of higher education programmes, institutions or systems is described in Sect. 2.2. How these indicators can be aggregated to define a composite indicator of effectiveness is presented in Sect. 2.3.

Indicators
We examined the following 11 indicators: 1. Employability rate, computed as the number of graduates employed, irrespective of their job status at graduation, divided by the number of graduates who were no longer studying. 2. Job-education matching rate, computed as the proportion of employed graduates who stated that their job required skills that could only be learnt at a university. 3 3. Job-major matching rate, computed as the proportion of employed graduates who stated that the attended major is required for the job. 4 4. Overall job consistency rate, computed in reference to graduates who were working. 5 5. Job refusal rate, computed as the proportion of graduates who refused one or more jobs offered to them within a certain time period following graduation, regardless of whether or not they had a job. 6. Additional training rate, computed as the proportion of unemployed graduates attending one or more extra courses within a certain time period following graduation.
On-the-job courses were excluded from the numerator. 7. Adequacy of professional specialisation rate, computed as the proportion of graduates stating that the level of professional specialisation they achieved at university was adequate for the job they were searching for or had gained. 6 8. Mean satisfaction for achieved competence rate, computed in reference to all graduates, regardless of whether or not they had a job. 7 9. Willingness to repeat the educational experience rate, computed as the number of graduates willing to repeat their educational experience if it were possible to do so. 8 This calculation included both employed and unemployed graduates. 10. Mean job satisfaction, computed in reference to graduates who had jobs. 9 11. Overall assessment of educational experience, computed as the mean of the responses to this evaluation question on the questionnaire that was administered at graduation. 10 For analytical purposes, all indicators were coded so that 'high is good'.

Selection of a Subset of Suitable Indicators
The appropriate units of analysis to define and evaluate university effectiveness are not graduates but rather degree programmes. The reasons for this choice are manifold: First, from a theoretical point of view, the concept of educational effectiveness applies to the delivery of a service (i.e. university teaching) and is indiscriminate to its achievers (i.e. the students). Therefore, if we look at applications of effectiveness indices, they are typically used to evaluate and rank educational programmes, institutions and national or regional systems. Moreover, considering the indicators proposed in Sect. 2.1, some cannot even be computed for graduates; for instance, indicators of job-education matching, job-major matching, job satisfaction and job consistency cannot be applied to the unemployed. 11 We also had to solve the problem raised by asking questions in different ways to graduates who had a job and to those who did not. For example, the mean satisfaction of achieved competence and the adequacy of professional specialisation referred to the current job of the employed graduate and to an ideal job for the unemployed. This meant that the questions asked of these two subgroups were not comparable, so we only examined the data collected for these questions from the employed graduates.
The Agorà survey collected data from graduates of the 130 different study programmes offered at the University of Padua. In this study, 4 programmes were excluded because they had\6 responses, which resulted in a total of 126 analysed study programmes. 12 They are well balanced between bachelor's degrees and master's degrees (43.7 % of bachelor's degrees programmes) and between disciplines (51.6 % arts and humanities and social sciences, 48.4 % natural and formal sciences and engineering). The average percentage of male graduates is 44.8 %, and the average number of graduates' responses on which the effectiveness indicators for each study programme are based is 36, ranging from 6 to 181. Table 1 reports the mean and standard deviation for the proposed indicators measured for each survey wave; the situation depicted by the data at 12 months fell between those situations at 6 and 36 months, with a lower dispersion for mean job satisfaction and mean satisfaction for achieved competencies.
To select the most qualified indicators, it is important to examine their distributional dispersion and their proximity to the core of the effectiveness concept using the composite indicator as a benchmark (see also Nardo et al. 2005;Zhou and Ang 2009;Spiegelhalter et al. 2012). Thus, it is possible to determine the best time for an indicator to be measured by detecting the time at which it is least deviant from the composite indicator of effectiveness. We examined the reliability measures of the analyses conducted on the data 11 To analyse and summarise the indicators at the individual level, we initially forced the missing variables to 0 or to a minimum that was equivalent to the statement that people who do not work also do not have a consistent job or do not have a job requiring a university degree, etc. This procedure generated spurious correlations between the indicators that were either very low or artificially high. The artificially high correlations may have been due to the fact that people who do not work are forced to have analogous values on a number of other indicators. Another possibility to compute the composite indicator of effectiveness at the individual level could be to exclude from the analysis all graduates who do not work, but this shortcut would exclude employability as a dimension of higher education effectiveness. 12 In this analysis, each study programme is treated as a single observation, and then the possible presence of outlying cases is taken into account by the adopted models. However, a robustness control has been performed by adapting the final model (the one in Fig. 4) to the set of study programmes with at least 10 responses (n = 116) and to the set of study programmes with at least 20 responses (n = 98). The model fit is as good as the ones described in the paper, and the parameters are totally comparable. We report the analysis on study programmes with more than 5 responses to guarantee a better representation of all the majors. collected for the three survey waves (6 months, 1 year and 3 years after graduation). The results are summarised in Table 2.
Cronbach's alpha showed higher reliability for the data collected 1 year after graduation. The item-to-total correlations, which measure the correlation between each single indicator and the remaining set of matched indicators, showed that in all three datasets, the additional training rate 13 was barely correlated to the other indicators, with a value as low as 0.075 for the data collected 6 months after graduation. Additional training was almost completely unrelated to the other indicators, reflecting a mixture of personal choice and preference more than the general outcome of an educational path. The item-to-total correlation for the employability rate of the three observational occasions was also mild, but the employability rate is so relevant to the construction of a global effectiveness indicator that the variables were kept in the analysis even though the data suggested the existence of a separate dimension.

A Composite Indicator of Effectiveness
Separate factor analyses were conducted on the three datasets to determine the main dimensions underlying the proposed elementary indicators. Here we report the results from the data collected 1 year after graduation since the reliability of these data was higher and the dimensions were clearer than the data from the other time points. However, the primary indications are analogous regardless of which wave is analysed.
The analysis of the eigenvalues suggests the existence of three main dimensions, accounting for 65.9 % of the total variance. Table 3 reports the eigenvalues and the pattern The data on overall programme assessment were gathered only at graduation b For sample waves 1 and 2, the mean was the proportion of graduates who reported having a 'somewhat' or 'very' consistent job; for wave 3, it was the mean computed on the 10-point scale matrix (i.e. the partial correlations between simple indicators and factors) after an oblique ('Oblimin') rotation and a Kaiser normalisation were performed. 14 The first factor refers to the professional empowerment achievable by a higher education programme; it includes the need for a general university degree and/or a specific degree/major, the consistency between one's job and education and the adequacy of the achieved competencies and of the level of specialisation of the study programme. The second factor is the employability dimension, which incorporates the rate of job refusals. The third factor represents fulfilment and general satisfaction, including job satisfaction  and student retention (e.g. the will to enrol in the same study programme if one could go back in time). The overall assessment of the university experience expressed at graduation did not have a clear position in this architecture and was therefore excluded from further analyses.
We also excluded the job refusal rate indicator; when we looked at what happened to graduates within the first year after graduation (i.e. both in the first and second survey waves), the rate of job refusals was connected to the employability rate since most refusals came from graduates who had already gained a position (Martini 2012). However, the data collected in the third wave showed a lower correlation, indicating that the far-fromgraduation refusal events were affected by individual factors rather than employability. Thus, the short-term refusal rate showed merely a time-specific affinity with the employability dimension.

Weights for a Composite Indicator
In light of our preliminary analyses, it can be hypothesised that three correlated latent traits exist: professional empowerment, employability and personal fulfilment.
To account for the different dynamics of the graduates' entry into the labour force, and to avoid the employment rate indicator being the only indicator for the employability dimension, we considered the rates of employment at 6 months (emp06), 1 year (emp1) and 3 years (emp3) after graduation. Cronbach's alpha was 0.798 (standardised as 0.793).
Analogously, in the fulfilment dimension, we included the openness to repeat the educational experience at 6 months (rep06), 1 year (rep1) and 3 years (rep3) after graduation as well as job satisfaction at 1 (jsat1) and 3 years (jsat3) after graduation. We did not consider job satisfaction 6 months after graduation for two reasons: First, since the employment rate at 6 months post graduation is only 66 % (and as low as 9 % for law degree programmes), most data on the employed could be affected by personal situations, especially for some study programmes. This is probably why the reliability of measures referring to 6 months after graduation is low. Second, in the preliminary factor analysis conducted separately for each wave, job satisfaction at 6 months did not correlate with the openness to repeat the educational experience, but it did correlate with the professional empowerment dimension.
Cronbach's alpha for the fulfilment dimension was quite low (0.600), but the means and variances of job satisfaction and openness to repeat were totally different. Therefore, a standardised version of Cronbach's alpha (equalling 0.786) was considered to be more appropriate.
Professional empowerment was constructed with indicators pertaining to jobs that require higher qualifications (deg1) or a particular degree (hdeg1), job-major consistency (cons1) and satisfaction of the achieved competencies (comp1), 15 all at 1 year after graduation (Cronbach's alpha = 0.740; standardised alpha = 0.865).
The decision to include only data collected 1 year after graduation was based on theoretical and empirical considerations: the professional improvement indicators were more consistent and reliable in reference to data collected in this survey wave, thus suggesting that there is a 'right time' when graduates can most accurately evaluate their educational returns because they have sufficient knowledge of the labour market and a vivid enough memory of their university experience. Moreover, when we tried to introduce one or more work-related indicators surveyed at either 6 months or 3 years after graduation into the measurement model, all the fit indices worsened dramatically. Table 4 reports the correlation coefficients between the selected indicators and their significance levels. We observed that the indicators of professional empowerment were correlated with almost all the other indicators, while the employability and fulfilment indicators tended to correlate mainly, if not only, with the indicators that belonged to the same dimension.
We validated the measurement model described above using structural equation modelling (Jöreskog and Sörbom 1979); we obtained our parameter estimates using Lisrel 8.7 (Jöreskog and Sörbom 2004). Since the basic indicators were measured on a quantitative scale and their distribution was non-normal, we applied a robust maximum likelihood method (Browne 1987) and the Satorra-Bentler scaled Chi square (Satorra and Bentler 1988;Jöreskog et al. 2001).
Our conjectured measurement model is presented in Fig. 2 16 We found that the three dimensions were significantly intercorrelated. We assessed the discriminant validity by testing whether the 95 % confidence interval did not include the value 1 and by comparing the nested models with and without restrictions on single correlations (Anderson and Gerbing 1988). Both procedures confirmed discriminant validity. The most correlated constructs were fulfilment and professional empowerment, whereas employability was only moderately correlated to the other two dimensions. All the factor loadings were significant at the 95 % level; those on job satisfaction, however, were below the 0.5 threshold, which, as suggested by Nunally (1978), indicates that the fulfilment dimension is determined essentially by the 'willingness to repeat the experience' indicators.
In fact, we found that if the job satisfaction indicators were omitted, a model with equal or better fit was obtained. 17 The factor loadings and the correlations were the same (the larger difference a size of 0.01). Hence, job satisfaction cannot be considered a meaningful and adequate indicator of fulfilment because it does not add information to what is already covered by the willingness-to-repeat items.
The second-order model, represented in Fig. 3, was assessed on a reduced number of simple indicators (i.e. after excluding the job satisfaction items). The fit measures had a very good fit, 18 confirming the existence of an underlying common trait, namely university effectiveness. It should be noted that university effectiveness is mainly determined by the self-perception of professional empowerment and only marginally by employability. 16 An RMSEA below 0.06 and SRMR below 0.08, as well as an NFI and CFI above 0.95, indicate a good fit (Hu and Bentler 1999); for the GFI and AGFI, values of 0.90 or greater are recommended (Hooper et al. 2008); a critical N of 200 or larger indicates a satisfactory fit (Hoelter 1983  As previously described, we reduced the original 31 measures 19 of effectiveness to 10, though some redundancies remained. For the employability dimension, the indicator measured 3 years after graduation could be removed with no harm since the employment rate was quite high and homogeneous for all majors/courses of study. This reduction slightly increased the weight of the employability dimension on the global latent variable of effectiveness (see Fig. 4). Similarly, the three indicators of fulfilment were, to some extent, redundant; therefore, we removed the intermediate observation to take into account  Table 1. the short-and long-term evaluations. Finally, regarding the professional empowerment dimension, the loading for the average satisfaction with the achieved competencies was lower and therefore eliminated. The 'reduced' model is shown in Fig. 4, and its fit indices suggest a very good fit. 20 The contribution of the three dimensions is much more balanced than in the model in Fig. 3, and the loadings are homogeneous. However, the number of indicators for each subdimension is so limited that this solution cannot be generally recommended until its robustness can be further assessed.

Discussion
The main finding of our analyses presented in Sect. 3 is that the educational effectiveness construct is multidimensional. The underlying common dimension is external effectiveness, but there are distinct subdimensions-employability, professional empowerment and personal fulfilment-that cannot be ignored. Hence, the habit of only using employability as a measure of external effectiveness provides a limited picture of university outcomes. The relationship between university and the labour market plays a crucial role in the definition of subdimensions of effectiveness. In addition to employability, one should also take the potential for skill improvement and the marketability of degree programmes into account. Nonetheless, the labour market is not the only reference frame since graduates consider personal fulfilment to be a relevant dimension of their investments in higher education. It is worth noting that Mainardes et al. (2012) identified the labour market and personal fulfilment as the two main dimensions of expectations of students enrolled at 11 Portuguese universities. Although the third dimension identified by these authors-the level of academic effort-does not apply to graduates and cannot be related to external effectiveness, there is a clear parallelism between students' expectations and graduates' experiences.
Some of the examined indicators turned out to be inadequate as measures of university effectiveness. The additional training rate, for example, cannot be considered an adequate measure because postgraduation training course rates do not distinguish between graduates who attend courses because they feel their degrees are not enough marketable and those who have personal interests in learning new things.
In addition, the role of the job refusal rate is controversial since these refusals may be due to already having attained a position, but they may also be an effect of a wide and untargeted job search as well as the consequence of personality traits and subjective evaluations on the adequacy of the job offers (Martini 2012).
Other indicators have proved to be correlated to university effectiveness but did not fit adequately into the emerging dimensions, such as the overall assessment of the study programme expressed by students at graduation, or they were overly correlated with the other indicators, such as the rate of graduates who considered their specialisation level to be adequate. Average job satisfaction, which was initially included as an indicator of the personal fulfilment dimension, was also removed from the model because job satisfaction incorporates subjective evaluations and different expectations. Moreover, job satisfaction might be affected, at least to some extent, by personality traits (see, among others, Judge et al. 2002;Furnham et al. 2002).
The right time for collecting relevant data on educational outcomes is another important issue. Our analyses have shown that 1 year after graduation is the most suitable of the examined time points: The 1-year indicators were less dispersed than those constructed with data collected 6 months or 3 years after graduation. In fact, at 6 months, the graduates' views were still dominated by their university attitudes, and the rate of employed graduates varied substantially among majors and individual life stories, making the measures unreliable. On the other side, at 3 years, the graduates' work experiences, together with personal endowments and social life, were more dominant in determining the quality and quantity of their used skills and career acknowledgment.
However, the best time to survey data to build a composite indicator cannot be determined in absolute terms. For instance, any change in labour market dynamics could result in a change in the optimal time to survey data. Moreover, the right time might be different for different subgroups of graduates. For example, for law and psychology graduates, these data were collected at 18, 24 and 36 months after graduation to limit the effect of mandatory professional traineeships.
The optimal time for data collection may also differ according to the various indicators. For example, data collected 1 year after graduation are useful for measuring the skills empowerment dimension, while the employability and personal fulfilment dimensions can be enriched by using all the available survey waves.

Conclusions and Considerations
This work has focussed on a variety of indicators of the external effectiveness of higher education with the purpose of determining the most relevant among the reliable indicators. Relevance was based on university graduates' views, who were the higher education stakeholders easiest to survey to measure external effectiveness. The unit of analysis of the indicators was degree programmes.
The results would have been much more relevant if applied to universities located in different countries or, at least, in different geographical areas of the country, given the diversities encountered in different labour markets and local contexts. At this stage of the study, the main purpose was to define a number of subdimensions of the university effectiveness construct and a set of possible indicators for each dimension. Nevertheless, a future prospect of the study should involve more universities, assess the invariance of the main findings across different regions or countries and find a way to guarantee the comparability of the indicators (especially those based on the job experience) when measured in non-homogeneous labour markets.
After various trials, an essential model was estimated that allowed the indicators to be screened. Some indicators showed various types of inadequacies, which confined them to a second screening phase to measure their effectiveness. In particular, our analyses showed that there are inadequacies regarding the refusal rate for offered jobs, the rate of postgraduation training and the degree of technical specialisation gained from higher education programmes. Indeed, these indicators are only partially accepted in the literature because they measure effectiveness indirectly and are prone to individual overdispersion.
One indicator that was found to be inadequate was the mean rating of the educational experience assessed at graduation. It was unable to measure external effectiveness since this concept is perspective in nature while educational experience has to be assessed in retrospect.
Another indicator that generated problems in our model estimation was job satisfaction, which is the average satisfaction of one's job rated by the employed graduates. This indicator is often used as a measure of effectiveness. It is possible that job satisfaction did not fit our model because it may be only a partial consequence of the educational path as job opportunities are also a matter of chance.
The selectivity of the analytic process resulted in a few good indicators as opposed to a single best indicator. The procedure consisted of the selection of a few indicators for each of the three subdimensions emerging as relevant components of external effectiveness. The indicators were selected based on the whole group of study programmes, regardless of the different types of courses (bachelor's degrees or master's degrees) and the wide range of different majors. Although the final models showed a very good fit, a future advancement of this study should concern the invariance of the model between subgroups of study programmes with different characteristics in order to test whether the selected indicators and the relationships among the effectiveness dimensions are analogous in each segment.
The dimension of employability may be best represented by the employment rate of graduates at 6 months and 1 year after graduation. The dimension of personal fulfilment may be represented by the repeatability rate of the educational experience computed 6 months and 3 years after graduation. This latter indicator does not require graduates to be employed. On the other hand, the dimension of professional empowerment may be represented by the rates of job-major (perceived) consistency and graduates' average satisfaction of the competencies they gained at university. The optimum measurement time is about 1 year after graduation. The responses obtained to the question 'Is the university degree you achieved specifically required for your job, can your job be performed with similar results by other graduates or would a high school degree or even a title lower than high school suffice?' showed high importance for the definition of professional empowerment. Moreover, it can help determine if a gained job is a 'university graduate-level job'.
Among the dimensions of university effectiveness obtained through our analyses, personal fulfilment highlights the 'stakeholder sensitivity' for any definition of effectiveness, meaning that the definition of external effectiveness and, consequently, the relevance of each subdimension and simple indicator might vary according to the perspective of different categories of stakeholders.
This draws attention to the main limitation of our approach. All the indicators came from surveys administered to recent university graduates, who only expressed their experiences. However, graduates are only one possible informant for university effectiveness. It is easy to expect different results from different classes of stakeholders; that is, different subdimensions of effectiveness and different relationships among the indicators. Moreover, only data collected at 6, 12 and 36 months after graduation were available, preventing inclusion of the possible effect of long-term returns on the university effectiveness indicator.
These considerations raise the need for further surveys to collect data several years after graduation and for alternative categories of stakeholders (e.g. employers, university delegates, policymakers, students' parents or other social representatives) to look for new and/ or different dimensions of external effectiveness and to gather suggestions of alternate or additional simple indicators. Moreover, a comparison of different time perspectives and different points of view would highlight the common denominator and the specificities of the definition of effectiveness among various categories of stakeholders.