Innovation Canada: A Call to Action

Innovation Canada: A Call to Actionevery five years. Budget 2007 announced a newEMS that includes a requirement for spendingproposals to clearly define expected results, andfor departments to manage against these resultsand formally evaluate program performance(Department of Finance 2007). Budget 2007also introduced strategic reviews ofdepartmental expenditures on a four-year cycleto determine whether they are achieving theirintended results and are aligned with thegovernment’s priorities. Because these strategicreviews are Cabinet documents, their resultswere not available to the Panel. In Budget 2011,the government announced a strategic andoperating review that will assess $80 billion indirect program spending across the federalgovernment in order to achieve at least$4 billion in ongoing annual savings by2014–15 (Department of Finance 2011).Although this program evaluation machinery isuseful for management purposes, it does not,for reasons discussed below, enable assessmentof which of the programs reviewed by the Panelare relatively more or less effective.Assessment of Program“Effectiveness”Effectiveness, as noted earlier, is defined by theTreasury Board Secretariat as simply “the extentto which a program is achieving expectedoutcomes.” Based on the Panel’s assessment ofprograms, it is clear that individual programs’“expected outcomes” are as varied as theprograms themselves, and to a large extent areincommensurable. For example, the ScientificResearch and Experimental Development(SR&ED) program, because it is deliveredthrough the tax system, does not have a resultsbasedaccountability framework and itsperformance is not evaluated on a regular basis.However, the Department of Finance hasoccasionally undertaken economic assessmentsof the program from a net benefit perspective— that is, estimating the economy-widebenefits and netting out the estimated costs ofadministration, compliance and the impositionof taxes (for the most recent assessment, seeParsons and Phillips 2007). Work undertaken forthis review applied the net public benefitmethodology to a sample of other programsbut, for reasons outlined in Chapter 6, the Panelhas concluded that the method is notsufficiently precise and well developed at thisstage for general use to assess the comparativeeffectiveness of programs.The more sector-focussed programs, such as theSD Tech Fund and FPInnovations, have intendedfinal outcomes that are relatively narrow in scope— for example, increased market growth, sectorcompetitiveness and specific environmentalbenefits. On the other hand, many programsinclude outcome objectives at the scale of theentire economy — for example, productivitygrowth or the overall prosperity of Canadians.Such ultimate impacts of individual programs areeffectively impossible to measure, since thespecific contribution of the program in questioncan rarely be isolated from the myriad factorsthat affect all macroeconomic outcomes.Intermediate outcomes, which occur closer to aprogram’s point of influence, are obviously easierto measure and attribution is stronger, althoughalmost never definitive. The Panel has observedthat intermediate outcomes are typicallyidentified according to each program’s specificobjectives. Internship programs, for example,have as desired outcomes increased jobopportunities in the business sector forgraduates. Other programs seek to increasepartnerships and collaboration, establishnetworks, foster an entrepreneurial culture,develop and retain researchers in Canada, oradvance the commercialization of new products5-2