Innovation Canada: A Call to Action

Program EffectivenessChapterProgram Effectiveness5This chapter addresses the first question inthe government’s charge to the Panel:“What federal initiatives are most effective inincreasing business research and development(R&D) and facilitating commercially relevantR&D partnerships?”While the Panel determined that Canada isconsidered to be among the leaders in programassessment, 1 it was concerned to learn, in thecourse of briefings with federal officials, thatthe tools are not in place to undertakecomparative assessments as contemplated inthis question. In the federal framework forprogram evaluation, “effectiveness” is definedby the Treasury Board of Canada Secretariat(2009) as “the extent to which a program isachieving expected outcomes.” There is nocommon evaluation framework in place todetermine relative program effectiveness acrossdepartmental lines. As a result, standardizedperformance and outcome indicators do notexist for the roughly $5 billion of businessinnovation programs in the review, and thesupporting information is not retained in acommon form or database.This changes the nature of the advice that thePanel is able to offer. Instead of assessing therelative effectiveness of the 60 programsdescribed in Chapter 3, the Panel is makingrecommendations that respond to stakeholderissues and concerns and that, if implemented,will establish the missing framework needed toshape a comprehensive and consistentevaluation of R&D program effectivenessgoing forward.To establish a context for the Panel’srecommendations, the following sectionssummarize (i) the relevant evaluation machineryalready in place in the federal government,(ii) some international experience in respect ofthe evaluation and comparative assessmentof programs that support business innovationand (iii) what the Panel heard from stakeholdersregarding the effectiveness (and shortcomings)of innovation support programs in Canada.Existing AssessmentProcedures forFederal ProgramsThere are several mechanisms in place forassessing federal program expenditures,including audits by the Auditor General,strategic reviews and ongoing programevaluations. Performance assessment has awell-defined role within the government’sexpenditure management system (EMS) —the overall framework for decision making onspending. In recent years, the EMS has evolvedto put greater focus on results. The 2006Federal Accountability Act requires departmentsand agencies to review the relevance andeffectiveness of their grants and contributions1 More specifically, the net public benefit evaluation of the SR&ED tax credit by the Department of Finance in 2007 (Parsonsand Phillips 2007) is considered to be state-of-the-art for program assessments of its type.5-1