Evidence-based portfolios of innovation policy mixes: a cross-country analysis

In the past decade many papers have discussed innovation policies in terms of their design, assessment and impact. Drawing on those contributions, the EC-OECD STIP Compass suggests a detailed taxonomy of innovation policies with regard to their themes, target groups of beneficiaries and instruments. Implemented with a web platform, that taxonomy has promoted an extensive collection of information on innovation policy initiatives across OECD member countries. Using the STIP dataset, in this paper we employ that taxonomy to identify the innovation policy mixes by referring to all the dimensions encompassed in each policy initiative, i.e., themes, targets, instruments. The evidence-based multidimensional analyses yield a typology and set of topics of innovation policy mixes that we use to conduct a cross-country analysis of innovation policy portfolios. Although the results of the cross-country analysis are statistically significant, caution is necessary when using them because of three main fragilities of the STIP Compass: the absence of information on innovation policies at subnational level; the classification of the largest group of target beneficiaries, i.e. innovation intermediaries; the quality of some information essential for enhancing the analysis. The proposed navigation of the STIP Compass data is freely accessible online.


A comparative framework for innovation policies
Having been addressed in the past decade by the literature on innovation policies, the role of innovation policy mix is increasingly debated with regard to its multilevel implementation (Lanahan & Feldman, 2015;Magro & Wilson, 2013), the dimensions along which interactions occur (Flanagan et al., 2011) and the complex interactions with other policies (Magro et al., 2016;Magro & Wilson, 2019), such as regional, industrial or educational ones.In their survey of innovation policy mixes, Cunningham, Edler, Flanagan and Laredo (Cunningham et al., 2016) recognize the scarcity of evidence on innovation policy mix and present a survey based on the various contributions in the literature.Building on that survey, Edler & Fagerberg (2017) outline a taxonomy based on 15 policy instruments adopted to support innovation.Their relevance is related to the overall orientation (supply, demand) and goals of the policy (increase R&D; enhance skills; give access to expertise; improve systemic capabilities; enhance innovation; improve the framework; improve discourse).Using this taxonomy, Edler and Fagerberg review the empirical analyses that focus on the various instruments and conclude that evidence on policy impacts at national level shows the adoption of a variety of policy instruments across countries and "that a holistic -or systemic -perspective in policy is important (Fagerberg 2016), that sensitivity to context is essential (Flanagan and Uyarra, 2016), and that mechanical transfer of policy practice from one national system to another (without concern for contextual factors) is highly problematic" (p.14).
Besides a combination of policy instruments, Edler and Fagerberg highlight that governance is another dimension that characterizes innovation policies and that varies across countries, with different levels of co-ordination of specialized agencies, multiple governmental bodies (like the various ministries that address the many challenges within their domains of action by supporting innovation), subnational administrative bodies, and also an array of stakeholders (Lanahan & Feldman, 2015).
In this paper we argue that understanding innovation policies needs primarily a sound definition of "innovation policy mix" that encompasses all the policy dimensions.In fact, since the influential paper in which OECD used the term "innovation policy mix" (OECD, 2010), it has been increasingly adopted in the literature to refer to a mix of different policy instruments, implemented in various contexts that might refer to the same or to different themes or target groups of beneficiaries.We propose overcoming these blurred and overlapping definitions by using data from the STIP Compass platform1 , a new data source provided by the EC and OECD.In so doing, we build on the suggestion by Meissner and Kergroach (2019) that one should rely on the STIP Compass platform and the taxonomy to which it refers for an evidence-based mapping of what is an innovation policy mix.
In particular, we have two purposes in this paper.Firstly, we detect an evidence-based typology of innovation policy mixes by taking account of all the various dimensions characterising a policy and the topics characterizing each mix.Then, we estimate the relative importance of each type of policy mix in terms of the number and the monetary resources of the policy initiatives encompassed in that policy mix.Secondly, these evidence-based results are used to identify countries' portfolios of policy mixes.
The remainder of the paper is structured as follows.Section 2 describes the context of our analysis and provides an operational definition of innovation policy mix encompassing all the dimensions characterising innovation policy initiatives.Section 3 presents the STIP Compass database and describes the methods that we adopted to conduct the multidimensional analyses making it possible to define the typology of innovation policy mixes and to characterize their content in terms of topics.Section 4 presents the results.Section 5 concludes by focusing on the implications of our methodological contribution for the analysis of innovation policy mixes and crosscountry analysis of innovation policies portfolios, with suggestions on how to make the STIP Compass more effective for cross-country comparative analyses.The annexes present supplementary materials.A selection of tables and figures in the text and in the Annexes (marked with the symbol ) can be browsed online by using the Tableau Public navigation tool available at https://www.poliinnovazione.unimore.it/policy-mix/.

Innovation policy mixes and STIP Compass
Officially launched in April 2018, the STIP Compass platform is a joint initiative of the European Commission and the OECD.The portal, in English, hosts and provides access to quantitative and qualitative data on countries' STI policies.Its history can be traced back 20 years to the start of biennial OECD surveys of countries' STI policies in support of the OECD STI Outlook and with the first overall discussion on a taxonomy of innovation policy mix (OECD, 2010).Today, the STIP Compass supports the continuous monitoring and analysis of countries ' STI policies, and it aims to become a central platform for policy research and advice to assist government officials, analysts and scholars.2Data are freely accessible following the FAIR principles (Findable, Accessible, Interoperable, and Re-usable).At present, STIP Compass contains taxonomies of policy initiatives, databases, monitoring tools, and links among various data sources.A special webpage has been created to present the OECD Survey on science and innovation policy responses to the COVID-19 pandemic.The ambition is to become a reference platform for any discussion on innovation policy, and it is expected that both the OECD and the European Commission will refer to the STIP Compass for information on policies in support of innovation.
The STIP Compass taxonomy of policy initiatives has three dimensions: policy themes, target beneficiaries, policy instruments.The very detailed structure used to code these dimensions covers 50 policy themes (in five areas); 31 target beneficiaries (in eight groups); and 28 types of instruments (in five categories).Besides codes, the free text describing the objective and background of each policy initiative adds further information.
The possibility to access cross-country information in English and to refer to an exhaustive taxonomy of the dimensions describing innovation policy initiatives, makes the STIP Compass a powerful basis for furthering evidence-base knowledge about innovation policies, and about innovation policy mixes in particular.
Building on the taxonomy set out in the STIP Compass, in this paper we define an innovation policy mix with regard to three dimensions of policy initiatives: policy themes, target beneficiaries, policy instruments.Moreover, we rely on free text descriptions to single out the topics/domains of the policy initiatives, an information complementary to the codes adopted to enter the information.
In order to implement the evidence-based analytical framework, the paper reports multidimensional analyses of texts -both coded and free -characterising each policy initiative entered in the STIP Compass.The analyses referred to two corpora.A first corpus was created by structuring in a Boolean matrix the multiple codes that describe each of the dimensions -policy themes, the target groups and the type of policy instruments -entered in the STIP Compass to categorize the policy initiatives.The second corpus was created by lemmatising the descriptions, the objectives, and the background of each policy initiative.By means of correspondence analyses of the graphic forms of each corpus (respectively, of 109 codes and of about six thousand lemmas) and applying a hierarchical clustering technique to the results of the correspondence analyses, it was possible to identify the specific topics and dimensions characterizing the typology of policy mixes in terms of specific types of instruments adopted by countries in order to target particular groups of beneficiaries within the various policy themes.
By using the information on the yearly budget range of each innovation policy initiative, we estimated the shares of a yearly budget by type of policy mix, and by country, and we compared those shares with the number of instances of policy initiatives comprised in each of the policy mixes detected.Moreover, we implemented a categorization of organizations in charge of policy initiatives in order to identify what types of organization are involved in each policy mix.In order to define a cross-country pattern, the analysis identified the specific profile of each pattern observed in the current innovation policy scenario.

Data on innovation policy: from the STIP Compass to data under analysis
The STIP Compass database downloaded on 6 April 2020 comprised information on 5,685 Policy Initiatives, encompassing 6,850 Policy Instruments 3 implemented by 67 territorial entities (62 countries plus the European Union, Brussels Capital, Flanders, Wallonia, Wallonia-Brussels Federation).With the exception of Belgian regions, the STIP Compass had no information on policy initiatives at subnational level. 4  The elementary unit of our analysis was the Policy Initiative, with which we associated information on 50 policy themes (five areas); 31 target beneficiaries (eight groups); 28 types of instruments (five categories), a free text of description, organizations in charge of the policy, start and end year, yearly budget range.Not all the fields had the same degree of completeness, but coverage was overall large 5 .
Policy initiatives entered in the STIP data base have a multiple classification according to the three policy dimensions summarised in Table 1 with respect to the number of types of codes and their occurrences, and the categories/areas for each of the three dimensions.With regard to policy instruments, there is no information on 104 innovation policy initiatives.With an average of 1.2 instruments per policy initiative, the number of instruments ranges from 1 to 236 , with 84% of initiatives having only one instrument and less than 10% having two policy instruments7 .Free texts describing the objective and background of each policy initiative were analysed in a single free text field embedding all of them, and which is henceforth named "description".A general issue to be considered when analysing the subnational level of policies is the method of allocation of political power between the central State and the regional authorities that form it.In particular, for 17 out of 62 countries, information does not refer to the policies on research, innovation, science and technology, which are presumably undertaken at subnational level.The details concerning the classification of countries in terms of the administrative level of policy initiatives (national and subnational) on those dimensions are presented in Annex 3.  ing the various policy mixes across countries, its analysis is not possible with the current version of the STPI Compass.This is because the lack of information on subnational policies prevents identification of the related involvement of larger sets of organisations likely to give rise to differences in the relative importance of each category of organisations in charge for the policy initiative.
The last information that we considered in the STIP dataset was the yearly budget range, which we used to estimate the share of resources for each policy mix. 10 It varied greatly across countries, themes, target beneficiaries and policy instruments.Information was available for almost 70% of the policy initiatives, but missing values were concentrated in the USA.Estimated data were used only at aggregated level to weight the relative shares of resources associated with the policy mixes identified.

Methods
In our analysis, we defined the policy mix characterizing innovation policies with regard to policy themes, target beneficiaries, policy instruments.To address the first purpose of our research -i.e.detect the innovation policy mix -we applied two techniques: Correspondence Analysis and Cluster Analysis.Correspondence Analysis (Benzécri, 1973 and1992;Greenacre, 2016) was implemented to address the multidimensional characteristics of both the coding of the various dimensions of policy initiatives and the free texts describing them.A hierarchical Cluster Analysis 8 In order to use yearly budget range, which is expressed at current values, information on the end year of policy initiatives was used to exclude 62 instances of policy initiatives that ended before 2018 when aggregating the data on yearly average budgets.9 10 Yearly budget range was transformed by attributing the average value within each range (Annex 2) whereas OECD uses a transformation metric that adopts the minimum value of the range.
was conducted on the results of the Correspondence Analysis in order to group and classify the policy initiatives according to their similarities in terms of policy mix and topics in their descriptions.In both analyses -of multiple coding and free texts -we relied on quantitative analysis of textual data.Increasingly adopted in recent decades, automatic text mining combines linguistic concepts, computational methods, statistical techniques, and machine learning to analyse large text collections (Beaudouin, 2016;Berry & Kogan, 2010;Bolasco & De Mauro, 2013;Feldman & Sanger, 2007;Lebart et al., 1998;Sullivan, 2001;Wang et al., 2016;Zhang et al., 2016).
Codes associated with each policy initiative were used to create a <Policy Initiatives × the factorial map, the general system of relationships among row profiles, among column profiles, and among combinations of row and column profiles (Greenacre, 2016;Lebart et al., 1998;Murtagh, 2010Murtagh, , 2017)).Each factorial dimension represents a continuum along which the elements under study are arranged.The coordinates of the points along each factor are a measurement of the deviation from the barycentre.Therefore, the distance of each element from the barycentre can be interpreted in terms of the polarization of the phenomenon under analysis along the factorial axis.These characteristics make it possible to interpret the factorial axes according to their latent semantic meaning.When additional external information is available for the rows or columns, this can be displayed as supplementary points that do not play any role at all in determining the construction of the factorial axes.
The Cluster Analysis applied to the results of the Correspondence Analysis is an unsupervised and unambiguous classification of the policy initiatives reflecting the semantic similarity that exist in each cluster of policy initiatives (which were profiled according to the codes of themes, target groups, type of policy instrument, and according to the lexicon of their free text descriptions).A hierarchical clustering was implemented in both analyses -of coding and free texts -on the results of the first ten factors by applying the Ward method (Greenacre, 2016 pp.120;Murtagh & Legendre, 2014;Ward, 1963) and chi-square distance. 12  The conceptual homogeneity within each cluster of policy initiatives made explicit the semantic trait prevailing in the cluster.This information was used to label the clusters by relying on the characteristic elements highlighted through their test-value. 13The analogue of the value of a standardized normal variable, under the hypothesis of independence, the test-value lies between the values -1.96 and +1.96 (with a probability of 0.95) (Lebart et al., 1998, p. 98 and p. 137).We used it as a pragmatic criterion with which to highlight, respectively, a negative and a positive characteristic of the elements in each cluster, i.e. to identify the under-or over-represented elements in each cluster.With regard to the policy mixes detected by means of multiple coding, the semantic trait is statistically defined by the test-values of the codes encompassed in each cluster.
Analogously, when defining the topics of policy initiatives, we used the test-value of lemmas to single out the semantic trait of each cluster.In both cases, cluster labels were defined by referring to the characteristic elements, which were listed in decreasing order of their test-value.
These evidence-based results paved the way for cross-country investigation of the portfolios of policy mixes.In order to detect the characterization of each policy mix in terms of country, year of start, and yearly budget range, we computed -for each policy mix -the test-value of the various modalities of these supplementary categorical variables.

Results
In what follows we focus on four main results yielded by the analysis of the STIP Compass.
Firstly, Section 3.1 presents the policy mix detected by means of a cluster analysis of codes entered in the STIP Compass to classify policy initiatives with regard to their policy themes, target groups, and policy instruments.Secondly, we explore the topics of the policy initiatives that can be identified by applying text mining techniques to free text descriptions (Section 3.2).In Section 3.3, after cross tabulation of the two classifications, topics are used to enrich the interpretation of 12 "This distance is similar in formulation to the Euclidean (or Pythagorean) distance between points in physical space, except that each squared difference between coordinates is divided by the corresponding element of the average profile" (Greenacre, 2016, p. 32). 13Test-value for qualitative categorical variable is a statistical criterion associated with the comparison of two portions within the framework of a hypergeometric law.It "is approximately a standardized normal variable under the hypothesis of random distribution of the categories under consideration in the groups.However, since this calculation depends on a normal approximation of the hypergeometric distribution, it is only used when the counts within a grouping are not too small with respect to the total count" (Lebart et al., 1998, p.137).
the policy mixes detected.By using a statistical analysis of the variables that we considered to be characteristic categories of the policy initiatives (i.e., country, year of start, estimated yearly budget), Section 3.4 outlines the cross-countries portfolios of innovation policy mix.

Policy mixes emerging from cluster analysis on codes of policy initiatives
The  14 Labelling of each cluster is quite easy if the interactive query implemented in Tableau is used to order the 109 code in decreasing order of their test-value.Yearly budget (estimated average by class) Null 500,000 3,000,000 12,500,000 35,000,000 75,000,000 300,000,000 750,000,000 On factor 1 (to be read from left to right) there are policy initiatives to fund innovation targeted on private companies (clusters 12, 5, 6) vs. grants and research funds for public institutions (cl-1).On factor 2, the polarization encompasses two main distinct policy domains (to be read from bottom to top): governance of innovation policy and ethics (cluster 9, 8, 4 and 3) vs. training for innovation to foster research and to reduce the gender gap (clusters 11, 10 and 7).Almost barycentric in the factorial map is the policy mix cluster comprising intermediary organizations supporting the enhancement of innovation processes (cl-2).Put briefly, the factorial map highlights four main domains of innovation policy initiatives: the largest group consists of policies to fund innovation (42.4%); the other three domains range from 17.1% to 20.6%15 (percentage share by cluster of policy initiatives in Table 2).From a temporal perspective (Figure 2), in the domain of funding for innovation, policy initiatives on public research grants and research funds (cl-1) and on finance for innovation (cl-5) started to spread in the mid-2000s, while their importance declined in the last three years; in the domain of innovation governance, the shares of policy initiatives focused on the governance of strategic planning (cl-4) constantly increased in the policy initiatives started in the last ten years, and those dealing with ethics became more significant only in the last three years; policies supporting the activities of intermediary organisations (cl-2) exhibit the greatest continuous increase (over the last ten years).

Domains of policy mixes: themes, instruments and target beneficiaries
For each of the four main domains, different combinations of policy themes, instruments and target beneficiaries were identified for each cluster, thus characterizing the twelve policy mixes (details are available in Annex 4).The results of this multidimensional analysis of codes and the supplementary characteristic categories of each cluster (country, year of start, and budget range) are presented in what follows.
Funding innovation; finance, fund and grants In the domain of policies intended to support the funding of innovation, policy initiatives target a range of private and public beneficiaries.
With regard to private companies, policy themes refer to innovation in firms and innovative entrepreneurs, mainly SMEs and micro enterprises, supporting both the supply side -with access to finance for R&D innovation, collaborative R&I, and cluster -and the demand side, with initiatives to stimulate market creation.Cl-12 (finance for innovation_equity and loans) and cl-5 (finance for innovation_support) differ between the specific beneficiaries targeted (respectively, young-nascent private companies and firms of any age) and among the policy instruments, with aligned direct and indirect financial support.Direct financial support in cl-12 is mainly equity financing and loans and credits, while in cl-5 grants for business R&D and innovation, procurement programmes and innovation vouchers predominate.Indirect financial support consists mainly of debt guarantees and risk schemes in cl-12, while in cl-5 it concerns corporate tax relief for R&D and innovation.Policy initiatives in cl-5 started mostly in 2008 and are adopted by most countries, while those in cl-12 started in 2012 and are adopted mainly in Lithuania and Korea.The targets beneficiaries of policy initiatives on intellectual property rights (IPR) for science and industry knowledge transfer and sharing are both private and public beneficiaries.The main policy instrument is the guidance and regulation of IPR and related incentives.In this group of policies, which began in 2009, the UK is the main promoter of such policies.

Governance of innovation policy and ethics
The multifaceted domain of governance concerns both strategic agendas and planning (cl-4) and multilevel coordination (cl-9), which refer, respectively, to national STI plans and to hor-  (cl-11).Gender balance and inclusiveness policies are also addressed to public research systems, targeting women with the embedding of gender-specific analysis in research.

Intermediating for innovation
The policy mix supporting the activities of intermediary

An overview on policy mixes
The identification of policy mixes yields a further result that would make the process of querying the STIP Compass platform more effective.At present, the platform allows only querying at micro level by selecting a policy theme and visualizing aggregate results on the related target beneficiaries, policy instruments, and the details of the policy initiatives encompassed in the selection.By using a categorization of policy mixes like the one emerging from the multidimensional analysis presented in this section, the user can now read the 109 codes of the taxonomy elaborated by the EC-OECD with regard to their importance in innovation policy practices.In fact, only specific combinations of different policy themes, target groups of beneficiaries, policy instruments are actually enacted in policy initiatives 16 .This querying tool -at the meso level provided by the policy mix classification -has been implemented in the online navigation of our results presented with Tableau Public.
The twelve policy mixes have differing importance in terms of number of policy initiatives and relative budget shares (see the scatterplot in Figure 3). 17Although information on yearly budget ranges is not complete, estimated budget shares provide a first insight into the relative importance of the policy mixes.The largest share of resources for innovation policies, almost one third, is devoted to grants and research funds (cl-1).Another share of resources, ranging from 19% to 14%, consists of policies to finance innovation (cl-5), support intermediaries' activities 16 The complete list of codes and test-values by policy mix is available online at https://www.poliinnovazione.unimore.it/policy-mix. 17Figure A.4 in Annex4 shows the different estimates that would be obtained by using the metric to compute yearly budget ranges adopted by OECD.The greatest differences are in the groups of policy mix with the larger average budget shares, and in particular with regard to the policy mix grants and researcher funds (cl-1).
(cl-2), and implement strategic agendas and plans (cl-4).Finance for innovation (cl-12) and resources for PhD students (cl-7) are, respectively, almost 8% and about 6%.A very modest amount of resources is specifically devoted to the training of human resources in general (cl-11) and governance (for multilevel coordination, cl-9, and evaluation, cl-8).The smallest shares are devoted to reducing the gender gap in science and innovation (cl-10), to ethics (cl-3) and IPR (cl-6).These features should be considered also in light of the marked cross-country variability of budget shares and shares of policy initiatives by policy mix (boxplot in Figure 4a), a result that highlights the importance of a more complete data set from which to draw more robust conclusions concerning the actual composition of expenditure by innovation policy mix.

Policy topics emerging from cluster analysis of descriptions of policy initiatives
The policy mixes detected by clustering the policy initiatives over their multiple coding can be further explored by identifying the topics emerging from the analysis of the free text descriptions of policy initiatives.In particular, we applied a Cluster Analysis to the Correspondence Analysis of the Policy initiatives × Lexical units (5,575×5,374) matrix 18 to group the policy initiatives according to the similarity of their free texts describing the policy (which comprised also texts on the objectives and backgrounds of the innovation policy initiatives).The results are presented in the factorial map of Figure 5, where dots are the 5,575 policy initiatives for which free texts were available for the analysis.The position of dots represents, in the two dimensional factorial map, the relative distance among the policy initiatives with regard to the terms selected by the lexical analysis.As in Figure 1, the size of the dots is proportional to the estimated average of their yearly budget range.Colours in Figure 5 mark the 15 clusters identified with a hierarchical method.Cluster labels were assigned by reading the characteristic terms in decreasing order of their test-value, in each cluster (Annex 5, Table A.9).The dendrogram of the cluster analysis   18 In the dataset, within the 5,682 policy initiatives with at least one textual field, only 5,587 had a free text description; of these, 12 policy initiatives used less than 5 terms of content, so they were excluded from the analysis.Almost barycentric in the factorial map is the cluster of policy initiatives focusing on collaboration (cl-7).

An overview of topics
In terms of number of shares of instances and estimated yearly budget shares, the relative importance of the various topics/domains of policy initiatives is summarized in the scatterplot presented in Figure 6: the most important topics in terms of budget shares are those referring to collaboration (cl-7), funding of SMEs (cl-12), evaluation and implementation of innovation policies (cl-8), orientation of innovation across sectors (cl-11) and grants to universities (cl-6).As observed in the comment on policy mixes, also the topics of policy initiatives exhibit marked cross-country variability of average estimated yearly budget shares.

Topics of the twelve policy mixes
A cross-tabulation of the innovation policy mixes based on codes -of policy themes, target groups and policy instruments -and the topics based on free text descriptions provides significant insights regarding those policy mixes that embrace large groups of policy initiatives, and where the codes are not sufficient to single out the specificity of the innovation policies.Following the order of policy mixes listed in the rows of Table 3 and referring to the topics listed in the columns, the following features emerge.
Firstly, a large number of policy initiatives concerning grants and research funds (cl-1) have a modest focus on only researchers training & grants, while a large focus is on collaboration, on the specific sectors in which they have to be targeted; evaluation is significant for those policies.
As to be expected, the policy mix on finance for innovation (cl-5) focuses on SMEs, on start-ups and specific sectors, but also on collaboration.
The policy mix around intermediating innovation activities (cl-2) has a large set of specific focuses -sectors, public data and AI, SMEs -and it fosters collaboration.The evaluation and implementation of such policies is significant.
Ethics (cl-3) cross-cuts various topics, but the most important one is public data and AI.
Governance, strategies agendas and plans (cl-4) embraces both evaluation and implementation issues, but also specific sectors, and SMEs in particular.As to be expected, evaluation and implementation has high importance for the policy mix characterising multilevel governance.
Also the policy mix referring to training for innovation (clusters 7, 11, 10) has, as expected, a focus on corresponding topics, and significantly also on collaboration.

Cross-country portfolios of policy mixes
Building on the results on policy mixes and their topics, we can now focus on cross-country analysis.
In Section 3.1 we discussed the overall average composition of policy initiatives that can be identified in the STIP Compass dataset.As shown in Figure 4, several countries have a share of policy initiative in some policy mixes which is over/under the interquartile distribution.Given that almost 30% of policy initiatives have no information on yearly budget range (with an overall equal distribution of missing data in all the policy mixes 19 ), it seems more appropriate to present a cross-country analysis with regard to the relative shares in the number of policy initiatives and not with regard to relative shares of resources for each policy mix.In order to obtain an easily comparable configuration of countries in the top and in the lower quartile of the distribution of each policy mixes, we considered the test-values of the supplementary categorical variable "country" for each policy mix. 20The results are set out in Table 4: the figures refer to the set of 44 countries plus the European Union and a Belgian territorial entity, each of them having at least 1% (or at least 2%, for those marked with a star) of all codes cited in the Policy Initiatives × Codes (5,685 × 109) matrix and showing positive or negative characteristics for at least one of the detected policy mixes.In this way we can focus on the countries that in Figure 4b are plotted as outliers or are on the whiskers (i.e. points within 1.5 times the interquartile range) of the quartile distribution.For the 17 countries and four Belgian territorial entities not listed in Table 4, the variability of codes is too low for computation of the test-value.
With regard to countries listed in Table 4, we can compare countries' portfolios of innovation policies across the 12 policy mixes.These 44 countries plus the European Union and a Belgian territorial entity account for about 89% of all the policy initiatives entered in the STIP Compass.
The figures in Table 4 have been ordered to list countries in decreasing order of test-value with regard to the policy mix 'intermediaries' with the highest test-value for the larger group of countries.
The arrangement of figures in Table 4 highlights that the overall pattern is largely dominated by the over/under representation in the policy initiatives portfolio of policies supporting the innovation intermediaries' activities (cl-2).It is apparent that, even though -on average -such policies encompass a large share of policy initiatives, they are greatly above the average for a group of 11 countries -Columbia, Brazil, Thailand, United States, Portugal Malta, Germany, Italy, Canada, Korea, and Kazakhstan -plus the European Union.For some of these countries -Columbia, Brazil, Thailand -that policy mix crowds out almost all the others.For the United States, the composition of innovation policies includes also a significant share of policies on ethics (cl-3), on grants and research funds (cl-1), and on innovations intended to reduce the gender gap in science and technology (cl-10); while innovation policies to support PhD (cl-7), governance of innovation policies (cl-4, cl-9 and cl-8), finance for innovation (cl-5 and cl-12) and policies on IPR (cl-6) are well below the shares in the average composition of the innovation portfolio of policy mixes.Other countries show a more balanced composition (highlighted by the blanks in the table), as in the case of Germany.For countries with an over-presence of intermediaries' policies (cl-12) in the portfolio of innovation policies mixes, there is a focus on the same set of topics that we observed in Section 3.2: collaboration, evaluation and implementation of innovation, public data & AI, sectors towards which innovations must be oriented, SMEs.
Countries with a presence of policy initiatives in innovation intermediaries well below the average portfolio display a variety of different compositions of innovation policy mixes, mainly in regard to interventions on human resources training and granting and funding of innovation.
Although the results presented in this section are promising in terms of cross-country comparison, there are at least four reasons why we are very cautious about drawing conclusions from them.Firstly, figures on budget shares may define a different composition of the portfolios' policy mixes.Secondly, information on subnational policies may explain the low presence of some policy mixes in the portfolio of some countries in which innovation policies are undertaken not only at national level (as in the case of Austria, Brazil, Canada, Germany, Italy, Portugal, Thailand, United States, to mention only the ones listed in Table 4 21 ).Thirdly, the interplay of innovation policies and other policies -such as those intended to favour a stronger/more balanced structural composition of economic activities or those fostering the enhancement of education systems -is expected to affect the observed portfolio of policy mixes.In this regard, an analytical approach should consider national systems of innovation.Fourthly, over time, policy programmes structure the practices of the various agents involved in the innovation processes: countries that only recently started innovation policy programs may select policies that foster the emergence and consolidation of the preparedness to innovate, as far as they contribute more to shaping agents' attitude towards innovation.This may explain why Colombia has a very large share of innovation policies supporting innovation intermediaries, which can be a strategic tool to structure a national innovation system. 22Although analysis of all these dimension falls outside the scope of this paper, they represent a core set of complementary items of information that we would expect to see in the next generation of the STIP Compass.

Discussion and further developments
There is increasing consensus among scholars about the importance of the policy mix in supporting innovation processes (OECD, 2010;Cunningham, Edler, Flanagan, & Larédo, 2016;Edler & Fagerberg, 2017;Guimón & Paunov, 2019).A variety of tailored policy instruments are needed to target the diverse goals and beneficiaries of public intervention, since they are subject to different forms of governance.Their implementation may be straightforward (as in the case of tax incentives) or very complex (when holistic changes are made to enhance the creation of ecosystems).The impact of individual instruments and of their mix is difficult to assess because of the many interactions that occur in their enactment.Case studies have highlighted that these interactions do not always produce a reinforced effect, and that policy management may become a critical issue per se (Mazzucato & Semieniuk, 2017).When counterfactual analysis is used, significant findings highlight the potential of a more informed design of policy (Caloffi et al., 2018;Guerzoni & Raiteri, 2015), which to date has been largely determined by the path dependence of interventions in the countries considered.In general, comparative analysis is hampered by the need for information and by the need for a conceptual definition of what characterizes an innovation policy mix.To fill this gap, the essential support in building a cross-country comparative framework is provided by the STIP Compass.
Building on the detailed taxonomy provided by the STP Compass and on the wide-ranging and detailed information entered by 62 countries, this paper has conducted the first systematic analysis intended to define an evidence-based classification of innovation policy mixes.
The research results reported in this paper defined a policy mix in terms of the policy themes, the groups of beneficiaries targeted, and the policy instruments coded and associated with each policy initiative.We proposed two multidimensional analyses, respectively, of coded information on the three policy dimensions (109 codes) and of the free texts describing the objectives and backgrounds policies (over 6400 lemmas of nouns, adjectives and verbs).The results of the coded information provided the definitions of twelve policy mixes, while the analysis of free text descriptions singled out 15 topics that were used to explore the specific domains of the policy mixes detected.The results highlighted an evidence-based portfolio of innovation policy mixes adopted by countries.
The results of the analyses presented in the paper are available online for free consultation using Tableau Public to explore specific cross-country comparisons and obtain details on the features characterising each policy mix in terms of coded information and descriptions.Online queries allow cross-country comparisons both at the micro level on the codes of policy initiativesidentified by the taxonomy (defined in the STIP Compass by EC-OECD) of policy themes, policy instruments and target groups, thus reproducing what the STIP Compass already does -and at the meso level of categories of policy mixes and topics, identified in the analyses discussed in the paper.
Whilst we are aware of the incompleteness of the information available, the results of the multidimensional analyses make it possible to highlight some original results and to address some issues for further research.
Firstly, the twelve policy mixes detected single out specific combinations of themes, target groups of beneficiaries, and instruments.They highlight that not all the combinations of codes are enacted in policy practices; nor it would make sense, since the codes are tailored to refer to very specific modalities in each of the three dimensions.The multidimensional classification in terms of policy mix can become a meso level for querying the STIP Compass in a more effective crosscountry comparison.They are grouped into four main domains of innovation policy initiative.
The largest group consists of policy mixes intended to finance innovation with funds and grants.
These account for 42.4% of policy initiatives.The other three domains range from 17.1% to 20.6%: Governance of innovation policy and ethics for strategic plans and agendas and multilevel governance; Training for innovation to foster the attitude of human resources towards innovation (PhD researchers, general support, and reducing the gender gap); Intermediating for innovation.
The twelve policy mixes have differing importance in terms of number of policy initiatives.Relative budget shares are quite aligned to those proportions, but since information on yearly budgets is missing for almost 30% of policy initiatives, and significantly for the USA, results referring to budgets should be considered with caution.
The topics identified by the free text descriptions refer to four main areas: public interventions, private interventions, governance of innovation processes, and training for innovation processes.They add significant nuances to the various policy mixes, and in particular to the policies supporting the activities of innovation intermediaries, the largest group of policy initiatives (almost 20.6%), and which has a variety of focuses: collaboration, evaluation and implementation of innovation, public data & AI, sectors towards which innovations must be oriented, SMEs.The policy mix on ethics addresses various topics, but the most significant one concerns public data and AI.
The results on cross-country portfolios of policy mixes show that the majority of countries have an average portfolio of innovation policy mixes, while a group of countries has an overrepresentation of policy mixes focused on the activities of innovation intermediaries.In some countries, these latter crowd out all the other mixes, to some extent encompassing many policy initiatives intended to foster the country's preparedness for innovation.Countries with a presence of policy initiatives in innovation intermediaries well below the average display a variety of different compositions of innovation policy mixes, mainly in regard to interventions on human resources training and granting and funding of innovation.
Although the results emerging from the multidimensional analysis are promising, we are very cautious with regard to the cross-country comparison: figures on budget shares may define a different composition of the portfolios of innovation policy mixes; information on subnational policies may explain the low presence of some policy mixes in the portfolio of some countries in which innovation policies are undertaken not only at national level; the interplay of innovation policies and other industrial policies or education policies is expected to affect the these results; a longitudinal analysis would be needed to interpret them.
All these dimensions should be encompassed in the next generation of the STIP Compass and some critical fragilities should be addressed by the EC-OECD initiative.
The first fragility concerns the unit of analysis of policy instruments: choosing the national level as a reference for all EU and OECD countries simplifies the complexity of the policy instruments that in the different countries are developed on a subnational scale, as in the case of unitary states like Italy, or in the cases of federal states like Austria, Germany and the USA.
Beyond the methodological features suggested for the analysis of policy initiatives, one urgent message is conveyed by the findings reported in this paper: the need for a radical shift by the European Commission, which is now supporting the STIP Compass only for information on national policies.Innovation policy in Europe -as the Research and Innovation Smart Specialisation Strategy clearly shows (Isaksen & Trippl, 2017;Magro & Wilson, 2018;Russo, et al., 2019) -is built not only on national policies but also on subnational ones.Hence, in order to provide an effective analytical framework and tools for innovation policies in Europe, the European Commission cannot ignore the fact that subnational policies need to be entered in the STIP Compass.
This should be done without relying on the voluntary entry of information by almost 200 regions in Europe, but straightforwardly by populating the STIP Compass dataset automatically with information on regional innovation policies already collected at national level.This information should be delivered at EU level for the cross-country analysis of administrative data on regional policies supported by the European Structural and Investment Funds.The simplification adopted in conceptualizing the STIP Compass web platform only for national policy is unacceptable and disconcerting: there may be large and significant differences among the policy measures adopted by countries, precisely because of the interplay between different policy mixes at national and subnational levels, which is an issue already noted by OECD ( 2010), but later not incorporated in the STIP Compass.
A second cause of fragility relates to the classification of policy instruments and of target groups.For example, in the STIP Compass classification, "innovation intermediaries" are a type of target group, but in several cases they are not specified, while the ultimate target group of the innovation policy instruments is specified (such as SMEs).It is urgent to address the classification of the broad areas of intervention that are now covered by the label "innovation intermediaries" but are not classified as such in the conceptual framework of those who enter information in the database.A major effort to create a common language (codes) has been made through many tutorials produced by the STIP Compass staff, but some critical areas still require attention (and may be topics in the discussion on policy initiatives).
A more pervasive fragility concerns the quality of the data provided by countries.Essential for effective use of the STIP Compass is that the comparison refers to the policies that countries have implemented.Full information on the policy initiatives will shed light on the actual pattern characterizing the various policy mixes, thus allowing a cross-country interpretation of the composition of portfolios of policy initiatives in terms of resources that are invested for each component of the mix.Moreover, the temporal dimension deserves more accurate data: start-end dates could significantly improve our understanding of the development of innovation policies both in a cross-country perspective and a longitudinal one.et al. 2020), for the 62 countries in the STIP Compass, each Constitutional Law (translated into English) was searched according to the keywords: 'research', 'science', 'innovation', 'training' and 'education'.The queries were implemented in the sections "Municipal Government" and "Subsidiary Government" of Constitue.org 23.Tables 1 and 2 summarise the classification of the various domains of policy interventions at the different national or subnational levels of administration.In some countries, there is no specification of which administrative level is in charge of specific domains.

Annex 2 -Descriptive statistics on STIP Compass data
Three main methodological caveats should be considered when examining the results.A general problem stems from the use of a source not in the original language, with typical problems due to interpretation in translations.A second issue concerns the sub-constitutional sources that may regulate these policies, and that may not be mentioned in Constitutional law (for example, in case of administrative federalism, policies are defined by ordinary laws that devolve to lower levels).A third problematic aspect is the difference between existing law and living law, which might have produced an uncorrected attribution of the administrative level of intervention actually enforced in countries' practices. 4 Correspondence Analysis on the matrix Policy Initiatives × Codes (5,685×109) reduces the multidimensionality of 109 codes to the main factors of variability across the various policy initiatives.The hierarchical Cluster Analysis on the first ten factors yielded the results presented in the factorial map displayed in Figure 1, where we can interpret the characteristics of each cluster of policy mix.In fact, Figure 1 displays the 5,685 policy initiatives as dots, with size proportional to the estimated average of their yearly budget range.Colours mark the twelve clusters that were identified (the dendrogram is available in Annex 4, Figure A.3).The relative positions of dots represent, in the two-dimensional factorial map, the relative distance among the policy initiatives in terms of the multiple coding of each of the three dimensions under analysis, i.e., policy themes, target beneficiaries, and policy instruments.The two axes display the factors that summarise the greatest variability among the classifications of policy initiatives based on coding.Cluster labels were assigned by reading the mix of characteristic codes in each cluster 14 .The cloud of dots highlights a marked polarization of policy mixes along the two factors that can be interpreted according to the characteristic codes in each cluster, as will be discussed in what follows.
for innovation_equity and loansThe public research system is at the core of policy initiatives in cl-1, with policy themes focusing on competitive and non-competitive research, research infrastructure and large equipment, internationalisation of public research, with the relatively minor importance of interdisciplinary and high-risk research and themes of STI governance and collaborative research.Started in 2009 and 2010, these policies have a diversified range of yearly budgets.The targeted beneficiaries are all the various actors of public research systems, both researchers, students and teachers, and research and education organisations.The main policy instruments are direct financial support (with project grants and institutional funding for public research and centre of excellence grants), and support for collaborative infrastructures (mainly physical).
izontal policy coordination.Core policy themes of strategic agendas and plans are public research strategies, R&I for society and business policy strategies.Policy instruments are the national strategies and horizontal STI coordination bodies.A complementary set of policy initiatives in this domain focus on evaluation (cl-9), specifically on strategic policy intelligence and evaluation and impact assessment.Policy instruments are benchmarking and forecast, and they target the national government.Implemented only in recent years, and specifically in the USA, a new set of policy initiatives address ethics (cl-3): in emerging trends in STI in general, and in AI in particular.Policy instruments in this area are governance instruments (regulatory oversight and ethical advice bodies, and the consultation of stakeholders or experts), as well as guidance, regulation, and incentives for emerging technologies.Training for innovation to foster research and to reduce the gender gap Among the policy mixes on training, the cluster of policy initiatives on human re-sources_training (cl-11) is the longest established, while policies to reduce the gender gap in research (cl-10) are relatively recent.Together with policies to support PhD programmes (cl-7 human resources_phd), the three clusters exhibit an increasing trend, although it is marked by ups and downs.Inspection of the overall policy theme of human resources for research and innovation shows that international mobility doctoral and postdoctoral programmes, research careers, and inter-sectoral mobility to foster science-industry knowledge transfer and sharing characterize the cluster of policies targeting PhD students and postdocs and other early career researchers.Science, technology and innovation culture and STEM skills are at the core of human resources training

Figure 3 -Figure 4 -
Figure 3 -Scatterplot of budget shares vs. shares of instances, by policy mix

(
Annex 5, Figure A.6) shows three main groups of topics: funding for R&D; research and innovation (activities and policies); and researchers' training & granting, which encompass, respectively, 24.4%, 60.3% and 14.2% of policy initiatives.Instead of commenting on more detailed subgroups of topics along the hierarchical clustering, we extend interpretation of the correspondence analysis by browsing the polarization of clusters along the first two factors.

Figure 5 -
Figure 5 -Factorial map f1f2 -Distribution of Policy Initiatives -Matrix Policy Initiatives × Lexical units (5,575 × 5,374) Dots: size proportional to yearly budget (estimated average by range); colours by cluster a polarization (from left to the right) between private and public targets of policy initiatives.On moving from policies targeting innovation in private companies, mainly through the support of start-ups (cl-13), business toward the centre of the first factor, one finds policy initiatives targeting patenting (cl-15), an array of specific manufacturing and service sectors (cl-11), and artificial intelligence (cl-9).More to the right, the public target appears evident in policies targeting public beneficiaries, both individual researchers (with a focus on training and granting, cl-1) and universities (cl-6), public goods -open access and open science (cl-4) -and public issues like science and education (cl-3), ethics (cl-5) and gender (cl-2).On factor 2, the polarization (from bottom to top) encompasses two main distinct policy domains: governance of innovation vs. training for innovation.The first group of topics focuses on policy initiatives in sectors targeting innovation developments (cl-11) and ways to enhance innovation through open access-open science (cl-4), public data and artificial intelligence (AI) (cl-9), agreements, evaluation of innovation policies and ethics (clusters 10, 8, 5).The other main group embraces R&D (cl-14) and a variety of issues to be considered when training for innovation: enhancing women's engagement in science (cl-2), fostering universities (cl-6), and the educational approach to learning that uses Science, Technology, Engineering, the Arts and Mathematics (STEAM) with regard to children and young people (cluster 3); and specific support of researchers' training (cl-1).

Figure 6 -
Figure 6 -Topics/domains of innovation policy initiatives scatterplot of budget shares vs. shares of instances of policy initiatives, by topic

Figure A. 2 -
Figure A.2 -Share of policy initiatives by number of organisations in charge of the policy initiatives 8.9% of policy initiatives have no information on the organisations supporting them

Annex 4 -
Figure A.3 -Dendrogram of the hierarchical cluster of policy initiatives, classification based on multiple codingPercentage values refer to the relative weight of policy initiatives in each cluster

Figure A. 4 -
Figure A.4 -Scatterplot comparing estimates of yearly budget ranges, total values by policy mix: average value vs minimum value of the range

Table 1 -Summary of the number of codes and their occurrences, by policy dimension and category/area
Source: authors' elaboration of STIP Compass data[downloaded 06.04.2020] The 17 countries in the STIP Compass without information on subnational innovation policy initiatives concerning research, innovation, science and technology are Argentina, Austria, Brazil, Canada, China, Germany, India, Italy, Mexico, New Zealand, Peru, Portugal, Serbia, Slovakia, Spain, Switzerland, Thailand.With regard to the EU countries, the lack of information is quite significant, given that the STIP Compass is an OECD project funded by the European Commission and that ESI funds are specifically devoted to the Thematic Objective 1 Research and Innovation (see total budget by country, 2014-2020, https://cohesiondata.ec.europa.eu/themes/1) 5In particular, the following coverage of information is available: Policy Theme 100%; Target group 91%; Policy Instrument 98%; Description & Objectives & Background 100%; Start year 93%; organization in charge for policy initiatives 92%; Yearly budget range 69%.
The majority of the current policy initiatives entered in the STIP Compass started in the past two decades, with a wide variety of coverage by country (details in Annex 2 Figure A.1). Until 1983, the number of policy initiatives by year was at most 10 and then increased to 66 in 2000.Thereafter, the number of policy initiatives recorded in the database is growing (by three times in

Innovation policy initiatives by domain and policy mix, 1999-2019 Source
: authors' elaboration on STIP Compass data[downloaded 06.04.2020] organisations (cl-2) constitutes the largest group of policy initiatives.It encompasses various types of targeted beneficiaries classified as intermediaries (i.e.incubators, accelerators, science parks or techno parks; technology transfer offices; academic societies/academies; industry associations), but also private research and development labs.Policy instruments seek to create collaborative infrastructures (soft and physical) through networking and collaborative platforms and with information services and access to datasets.The governance is built on standards and certification for technology development and adoption.Colombia, Brazil, Thailand, and the United States are the countries with the most significant shares of such policies.

Table 3 -Cross-tabulation between 12 policy mixes based on the Codes (rows) and 15 topics based on the De- scriptions (columns): share of policy initiatives (percentage values)
Source: authors' elaboration on STIP Compass data[downloaded 06.04.2020]

Table 4 -Over-and under-presence of countries in each policy mix
Negative and positive significant test-values (respectively, below -2.326 and above+2.326) of the supplementary variable "country".List of countries that have at least 1% of all the codes (or at least 2%, when marked with *) in the <Policy Initiatives × Codes> matrix by each policy mix.Blue and red values indicate countries that are, respectively, over-and under-represented in a group.Blanks refer to test-values in the range above -2.326and below +2.326.Columns and rows have been ordered to list countries in decreasing order of test-value with regard to cl-2 19 Details are available in Annex 4, TableA.7.

Information available in the STIP Compass relative to Policy Initiatives, by year of start, by country, excerpt since 2000
Years are coloured by decade: years 2000s in a brown palette, years 2010s in a purple palette

Table A3 -The 1,589 organisations in charge of the policy initiatives (Os), by number of Os/per policy initiative and by number of occurrences per each organisation Number of organisations in charge per policy initiative Number of occurrences Number of organisations in charge for the policy initiatives
Source: authors' elaboration on STIP Compass data[downloaded 06.04.2020]

Table A .5 -Percentage of policy initiatives by: policy theme, target groups, policy instruments; categories and codes
Each policy dimension, each code belongs only to one category.Each policy initiative can have several codes within the same category.The percentage value of codes is calculated according to their presence in the 5,685 policy initiatives.The percentage value of categories is calculated by creating a Boolean matrix of presence/absence of categories in each of the 5,685 policy initiatives.] [

What Constitutional Laws say about administrative competences on research, science, innovation, training and education
By using the online platform Constitute.org (Zachary

Table A .6 -Summary of national and subnational levels of policy interventions regarding Research, Innovation, Technology, Science, Training, Education: EU and non-EU countries in the STIP Compass (62 countries)
Source: authors' elaboration using Constitute.org

Table A .7--National and subnational level of policy interventions concerning Research, Innovation, Technol- ogy, Science, Training, Education: details by country
23Detailed results can be browsed on request.

Table A .10 -Share of policy initiatives classified according to their characteristic terms, by Description class and group
Source: authors' elaboration on STIP Compass data[downloaded 06.04.2020]