Innovation Canada: A Call to Action

Program Effectivenessand processes. All of these contribute in varyingdegrees to business innovation, and thus tobusiness competitiveness and prosperity forCanadians, but the linkage to such ultimatelydesired outcomes is usually indirect and longterm. In the end, the linkage must be assessedbased on a combination of econometric analysis,anecdote, accumulated experience and intuitiveplausibility.The diversity of outcomes in the portfolio ofR&D programs in this review further complicatescomparison of their relative impact on businessinnovation and commercially relevant R&Dpartnerships. It is nevertheless possible toconceive of common intermediate outcomesfor similar program types, and potentially toevaluate the comparative effectiveness of thoseprograms in achieving the common outcomes.This is explained later in this chapter inRecommendation 1.6.International Practices inComparative ProgramAssessmentThe Panel met with government representativesfrom the United Kingdom (UK), Germany, theUnited States, Australia, Singapore, Finland andthe Organisation for Economic Co-operationand Development (OECD) to discuss approachesto performance measurement and thecomparative review of programs. It concludedthat all jurisdictions recognize the importance ofperformance measurement and regularlyevaluate individual programs within specificministerial accountabilities. However, whole-ofgovernmentassessments of innovationprograms for comparative effectiveness areeither still in development or not contemplatedat all. To the extent that comparativeassessments are undertaken, they look at thebroad performance of innovation systems andpolicy — for example, the recent study of theFinnish innovation system (Finland 2009) — andnot at the relative effectiveness of individualprograms.Other jurisdictions are thus confronted with thesame issues as Canada in assessing the value oftheir innovation support measures. In aforthcoming publication on business innovationpolicies, the OECD notes that, while growingattention is being paid to evaluationinternationally, the overall evaluation recordremains “patchy” (OECD forthcoming). ThePanel believes that Canada should encourage,through the OECD, focussed collaborationregarding analysis and best practices in theevaluation of innovation policy and associatedsuites of programs.Consultations withStakeholdersIn the absence of the data and methodologiesneeded for objective and consistent assessmentof relative program effectiveness, the opinions ofstakeholders provide some basis for judgment oncomparative effectiveness. For example,programs that have little uptake or awareness bytarget clients — even though they may be welldesigned — are not able to have a significantimpact on business innovation nationally.The Panel heard the opinions of domesticstakeholders through (i) consultations in theform of in-person group sessions, held in ninecities across Canada, and a call for writtensubmissions, which elicited 228 responses and(ii) a survey of firms conducted by EKOS ResearchAssociates Inc., which generated responses frommore than one thousand R&D-performingbusinesses of varying sizes, sectors and provinces.The full results of this survey will be madeavailable through the Panel’s website atwww.rd-review.ca.These activities shed valuable light on aspects ofthe comparative effectiveness of the programsunder review. It must of course be borne in5-3