accession program for agriculture and rural development in ... - MADR
accession program for agriculture and rural development in ... - MADR
accession program for agriculture and rural development in ... - MADR
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
Terms of Reference – SAPARD ProgramUPDATING OF MID - TERM EVALUATION OF THE SPECIAL PRE-ACCESSION PROGRAM FOR AGRICULTURE AND RURALDEVELOPMENT IN ROMANIAFOR THE PERIOD 2000 - 2005- February 2005 -
Index1. GENERAL INFORMATION1.1 Beneficiary...............................................................................................................................................31.2 Relevant data about beneficiary ...........................................................................................................31.3 Relevant data about beneficiary country.................................................................................................41.4 Bus<strong>in</strong>ess framework <strong>in</strong> the relevant sector .............................................................................................51.4.1 SAPARD Program <strong>in</strong> Romania............................................................................................51.4.2 Context of the Mid-Term Evaluation Updat<strong>in</strong>g of theSAPARD Program <strong>in</strong> Romania............................................................................................81.5 Correlated Programs <strong>and</strong> other donors activities .............................................................................102. CONTRACT OBJECTIVES2.1 General Objectives..................................................................................................................... 112.2 Specific Objectives..................................................................................................................... 113. CONSIDERATIONS AND RISKS3.1 Considerations emphasiz<strong>in</strong>g the necessity of the project ......................................................... 133.2 Risks .......................................................................................................................................... 144. FIELDS OF ACTIVITY4.1 Geographic scope <strong>and</strong> time frame ............................................................................................. 144.2 Regulatory Scope ....................................................................................................................... 144.3 General <strong>and</strong> Specific obligatory Activities .................................................................................. 144.3.1 Mid-Term Evaluation Updat<strong>in</strong>g Steps............................................................................ 154.3.2 Obligatory elements of the Mid-Term Evaluation Updat<strong>in</strong>g........................................... 164.3.2.1 Analysis of the ex-ante Evaluation <strong>and</strong> theMid-Term Evaluation 2000-2003 results ....................................................... 164.3.2.2 The validity of SWOT analysis carried out dur<strong>in</strong>g theProgram ex-ante Evaluation Program ........................................................... 174.3.2.3 Evaluation of adequacy <strong>and</strong> consistency of theDevelopment Strategy <strong>in</strong> NPARD.................................................................. 184.3.2.4 The Quantification of Goals – Outputs, Results <strong>and</strong> Impact ........................ 194.3.2.5 Evaluation of the effectiveness <strong>and</strong> efficiency of Program Implementation –Evaluation of policy <strong>and</strong> distribution of resources ......................................... 20Efficiency – <strong>in</strong>puts, compared with outputs <strong>and</strong> results................................. 20Effectiveness – outputs <strong>and</strong> results compared with specific<strong>and</strong> operational objectives ............................................................................. 21Impact - global objectives .............................................................................. 214.3.2.6 Quality of Program implementation <strong>and</strong> organization of Monitor<strong>in</strong>g ............. 224.3.2.7 Institutional impact ........................................................................................ 234.3.2.8 Conclusions <strong>and</strong> recommendations .............................................................. 234.3.2.9 Structure of Mid-Term Evaluation Updat<strong>in</strong>g Report ...................................... 254.3.2.10 Bibliography................................................................................................ 284.4 Project management, contractor’s tasks <strong>and</strong> responsibilities .................................................... 31Obligations of the Contract<strong>in</strong>g Authority <strong>in</strong> Romania.............................................................. 31Obligations of the Contractor.................................................................................................. 32Award<strong>in</strong>g criteria ..................................................................................................................... 334.5 Independence of the Evaluator................................................................................................... 36TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 1
5. LOGISTICS AND PROGRAMMING5.1 Place where the project is executed.......................................................................................... 375.2 Period of the Project .................................................................................................................. 376. REQUIREMENTS6.1 General Requirements ............................................................................................................... 386.2 Contractor Requirements ........................................................................................................... 386.3 Key Experts Requirements......................................................................................................... 396.3.1 Profile of International Experts.......................................................................... 396.3.2 Profile of Romanian Experts............................................................................. 417. REPORTS7.1 Inception Report........................................................................................................................ 457.2 Activity Reports ......................................................................................................................... 457.3 Mid-Term Evaluation Updat<strong>in</strong>g Report...................................................................................... 458. MONITORING AND EVALUATION8.1 Evaluation criteria ....................................................................................................................... 468.2 Special requirements.................................................................................................................. 48TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 2
In accordance with relevant EC regulations, at the bases of SAPARDProgram implementation <strong>in</strong> Romania, there is the National Plan <strong>for</strong>Agriculture <strong>and</strong> Rural Development (NPARD) elaborated by the M<strong>in</strong>istry ofAgriculture, Forests <strong>and</strong> Rural Development.The NPARD, preceded by a 2.5 million € test <strong>program</strong> f<strong>in</strong>anced through thePhare Project RO/98/IB/SPP/02, presents the strategy to be followed <strong>in</strong> theperiod 2000-2006 with the aim of improv<strong>in</strong>g the access to markets <strong>and</strong> thecompetitiveness of agricultural products, ameliorat<strong>in</strong>g the <strong>in</strong>frastructures,<strong>and</strong> develop<strong>in</strong>g <strong>rural</strong> economy as well as human resources.Giv<strong>in</strong>g the fact that needs <strong>for</strong> <strong>in</strong>vestments <strong>in</strong> <strong>agriculture</strong> are considerablethe Romanian Government has been develop<strong>in</strong>g complementary actions tothe SAPARD Program. Among them we could mention: the sett<strong>in</strong>g-up of apreferential credit system <strong>for</strong> agricultural producers; the coupon system; thesupport to young farmers <strong>and</strong> to entitled persons; the support to poorcommunities <strong>and</strong> productive groups <strong>in</strong> <strong>rural</strong> space by the Romanian Fund<strong>for</strong> Social Development (RFSD); the special <strong>program</strong>me <strong>for</strong> less-favouredareas “Assistance <strong>for</strong> agricultural activities <strong>in</strong> <strong>rural</strong> area”, grant<strong>in</strong>g f<strong>in</strong>ancialnon-reimbursable aid <strong>for</strong> the purchase of mach<strong>in</strong>es <strong>and</strong> equipment <strong>for</strong>agricultural productive activities or specific services; the negotiations with<strong>in</strong>ternational donors to get f<strong>in</strong>ancial support <strong>for</strong> <strong>agriculture</strong> <strong>and</strong> <strong>rural</strong><strong>development</strong>.The National Plan <strong>for</strong> Agriculture <strong>and</strong> Rural Development is a documentwhich may be improved <strong>and</strong> which will be updated on the basis of thenational strategy on economic <strong>and</strong> social cohesion; of the EuropeanCommission policies on <strong>agriculture</strong> <strong>and</strong> <strong>rural</strong> <strong>development</strong> fund<strong>in</strong>g; <strong>and</strong>also of the f<strong>in</strong>d<strong>in</strong>gs of monitor<strong>in</strong>g activities carried out <strong>in</strong> the framework ofthe present contract.1.4 Bus<strong>in</strong>ess framework <strong>in</strong> the relevant sector:1.4.1 SAPARD Program <strong>in</strong> RomaniaThe Special Pre-Accession Program <strong>for</strong> Agriculture <strong>and</strong> Rural Development– SAPARD – represents the f<strong>in</strong>ancial contribution of the Community,established by the Council Regulation (EC) 1268/1999 of 21 June 1999concern<strong>in</strong>g the support of the Community <strong>for</strong> the pre-<strong>accession</strong> measures<strong>in</strong> the field of <strong>agriculture</strong> <strong>in</strong> the c<strong>and</strong>idate countries from East <strong>and</strong> CentralEurope, dur<strong>in</strong>g the pre-<strong>accession</strong> period. The general objectives of thisTERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 5
Program are: 1) to contribute to the implementation of the acquiscommunautaire concern<strong>in</strong>g the CAP <strong>and</strong> related policies, <strong>and</strong> 2) to solvepriorities <strong>and</strong> specific problems <strong>for</strong> the susta<strong>in</strong>able adaptation of theagricultural sector <strong>and</strong> <strong>rural</strong> areas of c<strong>and</strong>idate Countries.On 21 st of June 2000, the European Commission decided on an annual<strong>in</strong>dicative allocation <strong>for</strong> each c<strong>and</strong>idate country with<strong>in</strong> a total global amountof about 520 million Euro/year, over the period 2000-2006.For Romania, the annual allocation amounts to about 153 million Eurowhich places our country the second on top, after Pol<strong>and</strong>, amongbeneficiary countries.The National Program <strong>for</strong> Agriculture <strong>and</strong> Rural Development (NPARD)presented by the Romanian Government was exam<strong>in</strong>ed by the STARCommittee of the European Union <strong>and</strong> approved officially by the EuropeanCommission by the Decision no. EC (2000) 3742 of 12 December 2000.NPARD is the <strong>program</strong>m<strong>in</strong>g document which covers the whole <strong>rural</strong> areas<strong>and</strong> ensures the implementation of the first operational Program - theSAPARD Program. In the follow<strong>in</strong>g years it has been modified four times,respectively by European Commission Decision No. H/2002/1936 on 12 ofJuly 2002, European Commission Decision on 27th of May 2003, EuropeanCommission Decision on 1st of August 2003 <strong>and</strong> European CommissionDecision on 17th September 2004.The strategic objectives of the Program <strong>in</strong> Romania aim at:• Priority 1: Improv<strong>in</strong>g the access to markets <strong>and</strong> of thecompetitiveness of agricultural processed products;• Priority 2: Improv<strong>in</strong>g <strong>in</strong>frastructures <strong>for</strong> <strong>rural</strong> <strong>development</strong> <strong>and</strong><strong>agriculture</strong>;• Priority 3: Development of <strong>rural</strong> economy;• Priority 4: Development of human resources.For the technical <strong>and</strong> f<strong>in</strong>ancial implementation of the SAPARD Program,the Romanian Government adopted on 14 th of September 2000 theGovernment Urgency Ord<strong>in</strong>ance no.142/2000, approved by Law no.309/2001 which establishes the sett<strong>in</strong>g up of the SAPARD Agency as<strong>in</strong>stitution subord<strong>in</strong>ated to the M<strong>in</strong>istry of Agriculture, Food <strong>and</strong> Forests,organized at central <strong>and</strong> regional level (8 regional offices correspond<strong>in</strong>g tothe 8 <strong>development</strong> regions of the country).TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 6
The Multi-Annual F<strong>in</strong>anc<strong>in</strong>g Agreement, which def<strong>in</strong>es the rules of theSAPARD Program implementation, was signed by the Commission ofEuropean Communities <strong>and</strong> the Government of Romania on 2 nd ofFebruary 2001 <strong>and</strong> ratified by Law no.316/2001.The Annual F<strong>in</strong>anc<strong>in</strong>g Agreements <strong>for</strong> 2000, 2001, 2002 <strong>and</strong> 2003 whichestablish the f<strong>in</strong>ancial commitment of the Community <strong>for</strong> Romania <strong>for</strong> eachof the above mentioned years as well as the period of validity were signedby the Commission of European Communities <strong>and</strong> the Government ofRomania on 27 th of February 2001, 30 th of January 2002, 1 st of April 2003<strong>and</strong> 31 st of July 2003 respectively <strong>and</strong> subsequently ratified by Law no.317/2001, Law no. 416/2002, Government Urgency Ord<strong>in</strong>ance n. 26/2003<strong>and</strong> Law no. 496/2003 respectively.On 11 th of July 2002, the Commission approved Romania’s proposals toamend the NPARD. The need to amend the NPARD resulted, ma<strong>in</strong>ly, <strong>in</strong>the process of f<strong>in</strong>aliz<strong>in</strong>g the specific procedures <strong>and</strong> operational structures<strong>in</strong> view of the SAPARD Agency accreditation <strong>and</strong> conferral of f<strong>in</strong>ancialassistance management to Romania.By the European Commission Decision (EC) 638/2002 of 31 st of July 2002,the SAPARD Agency was accredited <strong>and</strong> was conferred the decentralizedmanagement of the f<strong>in</strong>ancial assistance provided by the SAPARD Program<strong>for</strong> the implementation of three measures:-Measure 1.1 - “Improvement of process<strong>in</strong>g <strong>and</strong> market<strong>in</strong>g of theagricultural <strong>and</strong> fishery products”,-Measure 2.1 - “Development <strong>and</strong> improvement of <strong>rural</strong> <strong>in</strong>frastructure”,-Measure 4.2 - “Technical assistance”.On 5 th of December 2003, the European Commission conferred to theRomanian authorities the management <strong>for</strong> other three measures of theSAPARD Program:-Measure 3.1 - “Investments <strong>in</strong> agricultural hold<strong>in</strong>gs”,-Measure 3.4 - “Development <strong>and</strong> diversification of economicactivities, multiple activities, alternative <strong>in</strong>comes”,-Measure 4.1 - “Improvement of the vocational tra<strong>in</strong><strong>in</strong>g”.The other measures conta<strong>in</strong>ed <strong>in</strong> the National Program <strong>for</strong> Agriculture <strong>and</strong>Rural Development (NPARD) will be accredited <strong>in</strong> 2005, namely:TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 7
-Measure 1.2 - “Improv<strong>in</strong>g the structures <strong>for</strong> quality, veter<strong>in</strong>ary, <strong>and</strong>plant health controls, foodstuff quality <strong>and</strong> consumerprotection”,-Measure 2.2 - “Management of water resources”,-Measure 3.2 - “Sett<strong>in</strong>g up producers groups”,-Measure 3.3 - “Agricultural production methods designed to protectthe environment <strong>and</strong> ma<strong>in</strong>ta<strong>in</strong> the countryside”,-Measure 3.5 - “Forestry”.1.4.2 Context of the Mid-Term Evaluation Updat<strong>in</strong>g of the SAPARDProgram <strong>in</strong> RomaniaThe general framework which def<strong>in</strong>es the type of evaluations to beachieved <strong>in</strong> the different stages of the Program cycle – ex-ante evaluation,mid-term evaluation <strong>and</strong> ex-post evaluation – is ensured by the RegulationEC 1268/1999, Article 5 (1) <strong>and</strong> Regulation EC 1260/1999, Chapter IV.The actual modalities to achieve the mid-term evaluation, <strong>and</strong> its updat<strong>in</strong>g,are provided by the Multi-Annual F<strong>in</strong>anc<strong>in</strong>g Agreement, Section B, Article 5(5), Article 9 <strong>and</strong> Article 10, as follows:“The Manag<strong>in</strong>g Authority shall organize, <strong>in</strong> cooperation with theCommission <strong>and</strong> Romania, the mentioned mid-term evaluation.Mid-term <strong>and</strong> ex-post evaluation shall respond to common evaluationquestions def<strong>in</strong>ed by the Commission <strong>in</strong> consultation with Romania <strong>and</strong>shall, as general rule, be accompanied by achievement related criteria <strong>and</strong><strong>in</strong>dicators. In addition, evaluations may be required to answer specificquestions to the objectives of the Program.Romania shall ensure that a mid-term evaluation exam<strong>in</strong>es the <strong>in</strong>itialresults of the Program, their consistency with the ex-ante appraisal, therelevance of the targets <strong>and</strong> to what extent they have been atta<strong>in</strong>ed. It shallalso assess the quality of monitor<strong>in</strong>g <strong>and</strong> implementation.The mid-term evaluation shall be carried under the responsibility of theManag<strong>in</strong>g Authority <strong>in</strong> cooperation with the Commission <strong>and</strong> Romania.It shall be submitted to the Monitor<strong>in</strong>g Committee <strong>and</strong> shall be sent to theCommission three years after the date of approval of the Program but by31 December 2003 at the latest.TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 8
The Commission shall consider the implications of the evaluation with aview to review<strong>in</strong>g the Program.The mid-term evaluation shall, where appropriate, be updated <strong>and</strong>submitted to the Commission by 31 December 2005 at the latest.The Manag<strong>in</strong>g Authority shall <strong>in</strong><strong>for</strong>m the Commission about the follow-up tothe recommendations <strong>in</strong> the evaluation report, <strong>in</strong>clud<strong>in</strong>g any possibleupdates”.The updat<strong>in</strong>g of mid-term evaluation was proposed to the 6 th Monitor<strong>in</strong>gCommittee with<strong>in</strong> the Action Plan <strong>for</strong> technical assistance measure, <strong>in</strong>consideration of the implementation progress of the measures accredited <strong>in</strong>2002 <strong>and</strong> 2004 dur<strong>in</strong>g the period 2003-2005, not covered by the mid-termevaluation <strong>for</strong> 2000-2003, as well as of the fact that some measures’implementation had not yet started. The 7 th Monitor<strong>in</strong>g Committee, held <strong>in</strong>October 2004, approved the updat<strong>in</strong>g of mid-term evaluation <strong>and</strong> <strong>in</strong>structedthe Monitor<strong>in</strong>g Committee Secretariat to organize the relevantprocurement.In completion to the mentioned documents, the European Commissionissued the “Guidel<strong>in</strong>es <strong>for</strong> Evaluation of <strong>rural</strong> <strong>development</strong> Programs 2000-2006 supported from the European Agricultural Guidance <strong>and</strong> GuaranteeFund” (Doc. DG AGRI VI/8865/1999) <strong>and</strong> the “Guidel<strong>in</strong>es <strong>for</strong> the mid-termevaluation of <strong>rural</strong> <strong>development</strong> Programs 2000-2006 supported from theEuropean Agricultural Guidance <strong>and</strong> Guarantee Fund” (Doc. STAR VI/43517/2002) which <strong>in</strong>clude:• Def<strong>in</strong>ition, scope, concept <strong>and</strong> types of evaluation• Sett<strong>in</strong>g up, organisation <strong>and</strong> implementation of evaluation strategy• Methodology of evaluation• Carry<strong>in</strong>g out of the actual mid-term evaluation• Common evaluation questions with criteria <strong>and</strong> <strong>in</strong>dicators <strong>and</strong>Program specific questions with criteria <strong>and</strong> <strong>in</strong>dicators• Evaluation Report structure• Possible updat<strong>in</strong>g of the mid-term evaluationThe above mentioned papers as well as the “Guidel<strong>in</strong>es <strong>for</strong> the Evaluationof Rural Development Programmes Supported by SAPARD” (Doc. DG-TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 9
Agri, 4.2001) <strong>and</strong> the “Guidel<strong>in</strong>es <strong>for</strong> the mid-term evaluation of <strong>rural</strong><strong>development</strong> Programs funded by SAPARD 2000-2006” (Doc. DG-AGRI09/2002) should be considered as complementary to these ToR.1.5 Correlated Programs <strong>and</strong> other donors activities:1. The project “Rural Development <strong>in</strong> Romania” (PHARE RO 9505-04-03-005), aim<strong>in</strong>g to the def<strong>in</strong>ition of a complex <strong>rural</strong> <strong>development</strong> strategy<strong>and</strong> to the creation of an <strong>in</strong>tervention framework at <strong>in</strong>stitutional level <strong>for</strong> theimplementation of <strong>rural</strong> <strong>development</strong> policies. Project results aimed at theelaboration of a diagnosis study of Romanian <strong>rural</strong> space (with reference to2,685 municipalities), through which 45 <strong>in</strong>dicators have been analysed. Onthe basis of this study, areas characterised by different <strong>development</strong>potentials have been determ<strong>in</strong>ed. A Green Book has also been elaboratedconcern<strong>in</strong>g Rural Development <strong>in</strong> Romania <strong>and</strong> an Action Plan <strong>for</strong> RuralDevelopment has been adopted <strong>in</strong> the shape of an <strong>in</strong>ter-m<strong>in</strong>isterialdocument.2. The project «Institutional Build<strong>in</strong>g <strong>for</strong> M<strong>in</strong>istry of Agriculture <strong>and</strong> Food<strong>for</strong> SAPARD Program implementation» (PHARE RO 98/SPP/02), hav<strong>in</strong>g asma<strong>in</strong> objective the capacity strengthen<strong>in</strong>g of M<strong>in</strong>istry of Agriculture <strong>and</strong>Food <strong>in</strong> order to elaborate a structural policy <strong>and</strong> <strong>rural</strong> <strong>development</strong>strategy <strong>for</strong> the implementation of the SAPARD Program measures.Theprojects result consisted <strong>in</strong> the draw<strong>in</strong>g up of the National Plan <strong>for</strong>Agriculture <strong>and</strong> Rural Development (NPARD), <strong>in</strong>sur<strong>in</strong>g the legislative <strong>and</strong><strong>in</strong>stitutional framework <strong>for</strong> the SAPARD Agency sett<strong>in</strong>g up, its staff tra<strong>in</strong><strong>in</strong>gat central <strong>and</strong> local level <strong>for</strong> the technical implementation of the Program,<strong>and</strong> the preparation of the Special Preparatory Program (SPP)implementation.3. Special Preparatory Program SPP (PHARE RO 9807-01-03-05),which has represented a preparatory exercise <strong>for</strong> the SAPARD Programimplementation <strong>in</strong> Romania.4. Project EUROPEAID/114573/D/SV/RO – Mid-Term Evaluation of theSpecial Pre-Accession Program <strong>for</strong> Agriculture <strong>and</strong> Rural Development <strong>in</strong>Romania – SAPARD, f<strong>in</strong>anced by Measure 4.2 – Technical Assistancewith<strong>in</strong> SAPARD Program. The objective of the Project was the quality <strong>and</strong>efficiency evaluation of the SAPARD Program, <strong>in</strong> the period 2000–2003,with the aim of its improvement.The per<strong>for</strong>m<strong>in</strong>g of mid term evaluation waslimited to the analisys of <strong>in</strong>itial elements of SAPARD <strong>program</strong>m<strong>in</strong>g <strong>and</strong>TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 10
implemention <strong>in</strong> the previous period, <strong>in</strong>clud<strong>in</strong>g all k<strong>in</strong>d of outputs, results,impact <strong>in</strong>dicators <strong>for</strong>m the po<strong>in</strong>t of view of efectiveness, eficiency <strong>and</strong>correctness of the action carried out. A summary of Mid-Term EvaluationReport, <strong>in</strong>clud<strong>in</strong>g the observations <strong>and</strong> recomm<strong>and</strong>ations, was presented<strong>in</strong> the framework of the SAPARD Program Monitor<strong>in</strong>g Committee <strong>in</strong> July2004.2. CONTRACT OBJECTIVESThe Mid-Term Evaluation Update aim is the exam<strong>in</strong>ation <strong>and</strong> analysis ofresults produced by the SAPARD <strong>program</strong>, tak<strong>in</strong>g <strong>in</strong>to accountrecommendations <strong>and</strong> observations conta<strong>in</strong>ed <strong>in</strong> the already carried outMid-Term Evaluation. This would contribute to the improvement of thecurrent implementation stage of the Program <strong>and</strong> monitor<strong>in</strong>g system, <strong>and</strong>to the eventual re-orientation <strong>and</strong> adjustment of budget allocations amongthe Program Measures <strong>in</strong> its last year of implementation.2.1 General Objectives:Updat<strong>in</strong>g of Mid-Term Evaluation general objective is the exam<strong>in</strong>ation <strong>and</strong>analysis of SAPARD Program results <strong>in</strong> 2000-2005, also with reference tothe recommendations <strong>and</strong> observations conta<strong>in</strong>ed <strong>in</strong> the Mid-TermEvaluation Report concern<strong>in</strong>g the implementation period 2000-2003, withreference to the two major objective of the Program as def<strong>in</strong>ed <strong>in</strong> Article 1(2) of the EC Regulation 1268/1999, as well as the provisions of Regulation445/2002, Article 57(1).In particular the mid-term updated evaluation must supply <strong>in</strong><strong>for</strong>mation on theimplementation <strong>and</strong> impacts of SAPARD Program, with the aim of:1) <strong>in</strong>creas<strong>in</strong>g the accountability <strong>and</strong> transparency of the <strong>program</strong>implementation, with regard to the all actors <strong>in</strong>volved, <strong>and</strong> <strong>in</strong> particularthe adm<strong>in</strong>istrative <strong>and</strong> budget authorities <strong>and</strong> the public;2) improv<strong>in</strong>g the implementation of the <strong>program</strong> by contribut<strong>in</strong>g to <strong>in</strong><strong>for</strong>medplann<strong>in</strong>g <strong>and</strong> decision-tak<strong>in</strong>g concern<strong>in</strong>g needs, delivery mechanisms<strong>and</strong> resources allocations.2.2 Specific Objectives:The role of SAPARD Program Mid-Term Evaluation Updat<strong>in</strong>g is to improvethe previous evaluation, ensur<strong>in</strong>g necessary corrections <strong>and</strong> elim<strong>in</strong>at<strong>in</strong>g anyconfusions <strong>and</strong> <strong>in</strong><strong>for</strong>mation gaps, aim<strong>in</strong>g at display<strong>in</strong>g the actual state of theTERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 11
implementation <strong>and</strong> of its effects <strong>and</strong> at draw<strong>in</strong>g up functional proposals toput <strong>in</strong> practice past <strong>and</strong> future valuable recommendations.The specific objective of the update is to improve the SAPARD Programimplementation, to propose <strong>and</strong> discuss with the Contract<strong>in</strong>g Authorityreadjustments of the budget allocations <strong>for</strong> each Measure <strong>and</strong>, if the case,reorientations of them, <strong>in</strong> terms of strategic approach <strong>and</strong> hav<strong>in</strong>g <strong>in</strong> view thefuture <strong>program</strong>m<strong>in</strong>g objectives.This updat<strong>in</strong>g should be carried out <strong>in</strong> consideration of the outputs <strong>and</strong>results of Program implementation from the po<strong>in</strong>t of view of efficiency,effectiveness <strong>and</strong> quality of implementation as well as the adequacy <strong>and</strong>consistency of objectives established <strong>in</strong> NAPRD.In more details, the Update Mid Term Evaluation should assess keyaspects of the assistance as relevance, effectiveness, efficiency, utility <strong>and</strong>susta<strong>in</strong>ability, through:• The analysis of:- the correlation between the Program priorities <strong>and</strong> the proposedstrategy <strong>and</strong> objectives <strong>and</strong> the compatibility between the general, specific<strong>and</strong> operational objectives, as well as the utility, relevance, coherence,efficiency, effectiveness <strong>and</strong> durability of outcomes <strong>in</strong> relation to the realneeds of <strong>development</strong> of <strong>agriculture</strong> <strong>and</strong> <strong>rural</strong> areas tak<strong>in</strong>g <strong>in</strong>to account thepresent <strong>and</strong> future context, <strong>in</strong> view of proper adjustment of the Program.- the adequacy of the strategy conta<strong>in</strong>ed <strong>in</strong> the Program <strong>and</strong> theconsistency of its implementation tak<strong>in</strong>g <strong>in</strong>to consideration themodifications occurred <strong>in</strong> the current situation <strong>in</strong> comparison with theelaboration <strong>and</strong> <strong>in</strong>ception of the Program. This exam could highlight thenecessity of modifications concern<strong>in</strong>g strategies <strong>and</strong> budgetary allocationsamong different measures.- the efficiency <strong>and</strong> effectiveness of Program implementation tak<strong>in</strong>g<strong>in</strong>to account the objectives quantified at operational (outputs), specific(results) <strong>and</strong> general (impact) level.- the results of the previous evaluation (<strong>for</strong> example: the validity of theanalysis <strong>in</strong> relation to imbalances <strong>and</strong> gaps that the Program is designedto approach; the validity <strong>and</strong> relevance of objectives compared with theneeds, as well as the coherence between the objectives of the SAPARDassistance <strong>and</strong> the objectives of other <strong>for</strong>ms of national <strong>and</strong> external public<strong>in</strong>tervention which can <strong>in</strong>fluence the Program; the identification of newTERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 12
factors or other modifications which could have an effect on needs,strategies, efficiency <strong>and</strong> effectiveness of Program implementation).- the validity of assessment <strong>in</strong>dicators referred to <strong>program</strong> objectivesmonitor<strong>in</strong>g <strong>and</strong> evaluation.- the extension of horizontal issues (equal opportunities <strong>for</strong> men <strong>and</strong>women, conservation of natural patrimony, environmental protection, Stateaid) <strong>in</strong>tegration <strong>in</strong> the Program assistance.- the quality of Program implementation, mechanisms used to that end,the legal <strong>and</strong> adm<strong>in</strong>istrative structure created <strong>for</strong> its implementation (theclear def<strong>in</strong>ition of responsibilities of authorities <strong>and</strong> <strong>in</strong>stitutions <strong>in</strong>volved <strong>in</strong>the Program implementation <strong>and</strong> monitor<strong>in</strong>g, the quality of the monitor<strong>in</strong>gsystem, the quality, transmission method, transparency <strong>and</strong> frequency of<strong>in</strong><strong>for</strong>mation collection <strong>and</strong> availability <strong>for</strong> the monitor<strong>in</strong>g system, procedures<strong>and</strong> criteria of project selection).- the <strong>in</strong>tervention policy ability to produce results last<strong>in</strong>g <strong>in</strong> the longterm, <strong>and</strong> after the end of the <strong>program</strong> activities.• Answer to the applicable common evaluation questions as set <strong>in</strong>Annex 1, <strong>and</strong> elaborate substantial conclusions as regards: the overalleffects of SAPARD on the implementation of CAP <strong>and</strong> on solv<strong>in</strong>g thepriority <strong>and</strong> specific problems <strong>for</strong> the susta<strong>in</strong>able adaptation of agriculturalsector <strong>and</strong> <strong>rural</strong> areas <strong>in</strong> Romania.• Present the <strong>in</strong>termediate effective outputs (achieved <strong>in</strong>vestments,outcomes of the achieved <strong>in</strong>vestments compared with the values planned<strong>in</strong> the measures).3. CONSIDERATIONS AND RISKS3.1 Considerations emphasiz<strong>in</strong>g the necessity of the project:The evaluator shall be supported <strong>in</strong> the implementation of his tasks by theManag<strong>in</strong>g Authority <strong>and</strong> the SAPARD Agency. All available <strong>in</strong><strong>for</strong>mationrelevant to the evaluation shall be made accessible <strong>for</strong> the evaluator.The successful completion of the project is subject to the follow<strong>in</strong>g keyassumptions:• Sufficient level of cooperation with <strong>in</strong><strong>for</strong>mation sources outside theManag<strong>in</strong>g Authority <strong>and</strong> the SAPARD Agency (other PublicTERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 13
Adm<strong>in</strong>istrations, National Institute <strong>for</strong> Statistics, Specific Universities,etc…);• Good cooperation <strong>and</strong> support from social <strong>and</strong> economic partners.3.2 Risks:The major risks of the project can be attributed to the uniqueness ofSAPARD. Needed statistical data may not be available or may not bepresented <strong>in</strong> the appropriate manner (<strong>for</strong>mat) to allow the respectiveanalyses. This issue must be addressed <strong>in</strong> the Inception Report at theproject start up. This report must specify the data collection strategy <strong>and</strong>the analysis methodology to be followed.4. FIELDS OF ACTIVITY4.1. Geographic scope <strong>and</strong> time frameThe mid-term evaluation update should cover the period 2000-2005 <strong>and</strong>the “geographical area” of the NPARD.4.2 Regulatory ScopeThe mid-term evaluation update has as object the whole implementation ofthe National Program <strong>for</strong> Agriculture <strong>and</strong> Rural Development <strong>for</strong> 2000-2006set up with<strong>in</strong> the framework of Regulation (EC) 1268/1999 <strong>and</strong> Regulation(EC) 445/2002. The updat<strong>in</strong>g of mid-term evaluation should take <strong>in</strong>toconsideration the implementation progress of the measures accredited <strong>in</strong>2002 <strong>and</strong> 2004 dur<strong>in</strong>g the period 2003-2005, not covered by the mid-termevaluation <strong>for</strong> 2000-2003 (respectively Measures 1.1, 2.1, 4.2 <strong>and</strong>Measures 3.1, 3.4 <strong>and</strong> 4.1). The measures of the Program which are notyet operational must anyway be subject to evaluation <strong>in</strong> the context of thereview of the ex-ante evaluation follow<strong>in</strong>g the Commission’s guidel<strong>in</strong>es <strong>for</strong>the ex-ante evaluation: “The New Programm<strong>in</strong>g Period 2000-2006: the Exanteevaluation of the Structural Funds Interventions”. The present Mid-Term Evaluation Updat<strong>in</strong>g should give more details concern<strong>in</strong>g themeasures which did not constitute the object of the previous mid-termevaluation, respectively the measures 1.2, 2.2, 3.2, 3.3 <strong>and</strong> 3.5.4.3 General <strong>and</strong> Specific obligatory ActivitiesThe mid-term evaluation update should comprise:TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 14
• the exam of ex-ante evaluation validity;• the strategic evaluation (exam of effective implementation of thestrategy through the implementation of measures <strong>and</strong> submeasures);• utility, relevance <strong>and</strong> coherence of specific objectives <strong>in</strong> relation withthe general objectives (impact);• the evaluation of effectiveness, efficiency <strong>and</strong> durability of outputs<strong>and</strong> results.• The analysis of all the measures whose implementation has alreadystarted by the time of the present contract signature;• The analysis of the measures whose implementation has not yetstarted by the time of the present contract signature, but which are tobe implemented, <strong>in</strong> accordance with the requirements <strong>and</strong> guidel<strong>in</strong>esprovided <strong>for</strong> the ex-ante evaluation of the SAPARD Program.• A particular emphasis should be putted on the evaluation ofmeasures that have not been evaluated by the <strong>in</strong>itial Mid-TermEvaluation Report.• Analysis of acquis communautaire implementation through SAPARDProgram.The update of the mid-term evaluation shall <strong>in</strong>clude an analysis <strong>and</strong>appropriate comparison with the results achieved with the implementationof the pilot project under Special Preparatory Program (SPP) <strong>for</strong> thepreparation of SAPARD implementation.4.3.1 Mid-Term Evaluation Updat<strong>in</strong>g StepsStart<strong>in</strong>g from the Mid-Term Evaluation carried out with reference to theimplementation of the SAPARD Program <strong>in</strong> Romania, <strong>for</strong> the period 2000-2003, the evaluator should develop the mid-term evaluation updat<strong>in</strong>gthrough a process <strong>in</strong>clud<strong>in</strong>g the follow<strong>in</strong>g activities:• Structur<strong>in</strong>g – the set of common evaluation questions <strong>in</strong> Annex 1clarifies the effects to be evaluated, def<strong>in</strong><strong>in</strong>g the relevant criteria <strong>and</strong><strong>in</strong>dicators.• Data collection – The Contract<strong>in</strong>g Authority shall provide to theevaluator all available monitor<strong>in</strong>g data as well as access to thenecessary data from the SAPARD Agency. In case the above dataare not sufficient to answer properly the evaluation questions or toquantify the evaluation <strong>in</strong>dicators as set out <strong>in</strong> the SAPARDTERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 15
operational measures, the evaluator must collect additional relevantdata by appropriate means, <strong>in</strong>clud<strong>in</strong>g, where necessary, studies bysub-contract<strong>in</strong>g with relevant <strong>and</strong> competent research <strong>and</strong><strong>in</strong><strong>for</strong>mation providers.• Analysis – the evaluator must process <strong>and</strong> compare the data as wellas assess the Program effects <strong>and</strong> quality of its implementation <strong>and</strong>monitor<strong>in</strong>g.• Judgement – the evaluator must draw substantiated conclusionsbased on the above analysis. This judgement should concern theimpact, result <strong>and</strong> outputs of SAPARD Program implementation, aswell as the validity of <strong>in</strong>dicators <strong>and</strong> recommendations provided bythe <strong>in</strong>itial mid-term evaluation.• Report<strong>in</strong>g - the evaluator shall prepare both the versions of mid-termevaluation update report <strong>and</strong> f<strong>in</strong>al mid-term evaluation update report,as described at po<strong>in</strong>t 7.3. Also, the evaluator will respect thecommon structure described below <strong>and</strong> will take <strong>in</strong>to particularconsideration the specific measures that have not yet been the objectof the Mid-Term Evaluation carried out with reference to theimplementation of the SAPARD Program <strong>in</strong> Romania, <strong>for</strong> the period2000-2003.4.3.2 Obligatory elements of the Mid-Term Evaluation Updat<strong>in</strong>g4.3.2.1 Analysis of the ex-ante Evaluation <strong>and</strong> the Mid-TermEvaluation 2000-2003 resultsFirst of all, the mid-term evaluation updat<strong>in</strong>g should carry out the analysisof the ex-ante evaluation results, as well as of the conclusions <strong>and</strong>recommendations of Mid-Term Evaluation Report <strong>for</strong> 2000-2003.This analysis will answer to the follow<strong>in</strong>g questions:• Whether the strategy concern<strong>in</strong>g the support under the SAPARDProgram is adequate <strong>and</strong> consistent with the general objectives ofNPARD <strong>and</strong>, if not, the needed changes.• Whether the mechanisms <strong>for</strong> the implementation of the strategy areeffective <strong>and</strong> efficient.• Whether certa<strong>in</strong> factors or conditions which can <strong>in</strong>fluence negativelythe Program implementation or its efficiency <strong>and</strong> effectiveness doexist <strong>and</strong> which could be the possible solutions.TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 16
• Whether the Mid-Term Evaluation Report <strong>for</strong> 2000-2003 results areadequate to the present situation of <strong>program</strong> implementation <strong>and</strong>, ifnot, which are the reason <strong>and</strong> <strong>in</strong> which way the mid-term evaluationupdate will answer to the present situation needs.• Any other relevant <strong>in</strong><strong>for</strong>mation <strong>for</strong> the monitor<strong>in</strong>g <strong>and</strong> evaluation ofthe Program.4.3.2.2 The validity of SWOT analysis carried out dur<strong>in</strong>g the Program’sex-ante EvaluationA key element of the mid-term evaluation update is represented by theSWOT analysis hav<strong>in</strong>g <strong>in</strong> view that it has provided the base <strong>for</strong> theelaboration of the strategy conta<strong>in</strong>ed <strong>in</strong> NPARD. With<strong>in</strong> the ex-anteevaluation this analysis was checked <strong>in</strong>clud<strong>in</strong>g the adequacy of identifiedopportunities <strong>and</strong> risks.The mid-term evaluation update must verify if this analysis is still valid.Dur<strong>in</strong>g the mid-term evaluation update, the last evolutions <strong>and</strong> trends <strong>in</strong> theeconomic <strong>and</strong> social sector, <strong>and</strong> particularly <strong>in</strong> <strong>agriculture</strong> <strong>and</strong> <strong>rural</strong> areas,should be exam<strong>in</strong>ed <strong>in</strong> view of identify<strong>in</strong>g the changes occurred <strong>and</strong> theircauses.Also, both the priorities <strong>for</strong> the achievement of the general objectives <strong>in</strong>NPARD <strong>and</strong> their con<strong>for</strong>mity with the identified opportunities <strong>and</strong> needs aswell as the risk factors should be re-exam<strong>in</strong>ed.The updat<strong>in</strong>g of the mid-term evaluation should consider the past <strong>and</strong>current public <strong>in</strong>terventions, <strong>in</strong>clud<strong>in</strong>g Programs carried out by otherorganizations, <strong>in</strong> order to verify to what extent the Program priorities, asthey were identified by the SWOT analysis <strong>in</strong> the NPARD, correspond tothe present actual needs.With<strong>in</strong> this section of the mid-term evaluation update, the SWOT analysis<strong>and</strong> the contribution of the ex-ante evaluation should be re-viewed. Thevalidity of the ex-ante analysis should be checked upon <strong>and</strong>, whereappropriate, recommendations should be made <strong>for</strong> its updat<strong>in</strong>g.This section of the mid-term evaluation update should be relatively short,especially if no major changes of the <strong>in</strong>itial conditions have occurred or noTERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 17
elevant <strong>in</strong><strong>for</strong>mation has been identified, both lead<strong>in</strong>g to a significantmodification of the prelim<strong>in</strong>ary analysis.In the Conclusions of this section of the mid-term evaluation update thefollow<strong>in</strong>g elements should be presented:‣ Evaluation of the validity as regards the priority <strong>and</strong> major problemswhich must be solved as well as the necessity to adopt certa<strong>in</strong>changes <strong>in</strong> the conclusions of the analysis;‣ Evaluation of correlation between the strategic objectives <strong>and</strong> theidentified problems;‣ Any other <strong>in</strong><strong>for</strong>mation or factors which facilitate the economic <strong>and</strong>social cohesion, the protection of environment <strong>and</strong> equalopportunities <strong>for</strong> men <strong>and</strong> women as well as the implementation ofthe acquis communautaire.4.3.2.3 Evaluation of adequacy <strong>and</strong> consistency of the DevelopmentStrategy <strong>in</strong> NPARDOn the basis of the economic <strong>and</strong> social problems identified <strong>and</strong> theirrelations to the major needs, the mid-term evaluation update should carryout the exam<strong>in</strong>ation of the balance <strong>and</strong> appropriate comb<strong>in</strong>ation betweenthe priorities established <strong>in</strong> NPARD <strong>and</strong> the support measures provided bythe SAPARD Program.To that end, the mid-term evaluation update should verify the contributionof each specific priority objective to the achievement of the generalobjectives.In the end of this section, a revised concept of the Program should bepresented where appropriate. This can be determ<strong>in</strong>ed either by theimproper ex-ante/<strong>in</strong>itial mid-term evaluation or by omissions of itsrecommendations as well as by changes of the needs <strong>and</strong> major problemsdur<strong>in</strong>g the Program implementation which imply the re-evaluation of theconsistency of the <strong>development</strong> strategy, measures <strong>and</strong> priorities.The mid-term evaluation update must furnish to the Manag<strong>in</strong>g Authority <strong>for</strong>the SAPARD Program a clear motivation of the validity <strong>and</strong> weight of eachmeasure with<strong>in</strong> the general objectives.The mid-term evaluation update should analyse the <strong>in</strong>ter-connectionbetween general, specific <strong>and</strong> operational objectives.TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 18
4.3.2.4 The Quantification of Goals – Outputs, Results <strong>and</strong> ImpactThe mid-term evaluation update should consider the quantification of theoperational objectives <strong>for</strong> the <strong>program</strong> <strong>and</strong> <strong>for</strong> each measure, tak<strong>in</strong>g <strong>in</strong>toaccount that the achievement of the operational objectives contributes tothe achievement of the specific <strong>and</strong> general objectives of the Program.The mid-term evaluation update should exam<strong>in</strong>e the quantificationeffectiveness of measures operational objectives.The mid-term evaluation update should <strong>in</strong>clude an analysis of the logicalconnections between the quantified operational objectives of the measures<strong>and</strong> the general <strong>and</strong> specific objectives of the Program <strong>in</strong> view of ensur<strong>in</strong>gthe economic <strong>and</strong> social cohesion.Also, the mid-term evaluation update should analyse the implementation<strong>in</strong>dicators, the result <strong>and</strong> impact of each measure <strong>and</strong> sub-measure <strong>and</strong>should propose their improvement <strong>and</strong> adaptation to the present nationalcontext.The common evaluation questions with associated criteria <strong>and</strong> <strong>in</strong>dicators,as proposed <strong>in</strong> the Commission evaluation guidel<strong>in</strong>es <strong>for</strong> SAPARD <strong>and</strong>def<strong>in</strong>ed <strong>in</strong> Annex 1 of these Terms of Reference, must be adapted to thecontext of this evaluation update. It has to be established whether allquestions, criteria <strong>and</strong> <strong>in</strong>dicators are relevant <strong>and</strong> how will they be applied<strong>in</strong> each case. If this is not the case, a justification of their non-applicabilityshould be given.The <strong>in</strong>dicators must reflect the <strong>in</strong>ter-dependence between the outcomes ofthe proper implementation of the Program measures. Also, the mid-termevaluation update must identify every <strong>in</strong>dicator which is strongly <strong>in</strong>fluencedby external factors <strong>and</strong> propose improvement or replacement solutions.To that effect, the mid-term evaluation update should comprise:‣ Analysis of output, results <strong>and</strong> impact <strong>in</strong>dicators which quantify theobjectives <strong>and</strong> the major identified imbalances;‣ Verification of output, results <strong>and</strong> impact <strong>in</strong>dicators correlation withgeneral, specific <strong>and</strong> operational objectives;‣ Evaluation of output, results <strong>and</strong> impact <strong>in</strong>dicators <strong>in</strong> the context ofmonitor<strong>in</strong>g of equal opportunities <strong>for</strong> men <strong>and</strong> women, environmentprotection <strong>and</strong> other horizontal issues;TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 19
‣ Analysis of output, results <strong>and</strong> impact <strong>in</strong>dicators applicability basedon their capacity to furnish a precise image of the implementation,monitor<strong>in</strong>g <strong>and</strong> evaluation of the Program.The mid-term evaluation up-date should supply <strong>in</strong><strong>for</strong>mation concern<strong>in</strong>g thecurrent stage of the acquis communautaire implementation, as well as the<strong>agriculture</strong> <strong>and</strong> <strong>rural</strong> <strong>development</strong> improvements through SAPARDProgram <strong>in</strong> Romania.The mid-term evaluation update must also furnish <strong>in</strong><strong>for</strong>mation concern<strong>in</strong>gthe contribution of measures objectives to equal opportunities <strong>for</strong> men <strong>and</strong>women <strong>and</strong> environment protection, as well as to the implementation of theacquis communautaire.4.3.2.5 Evaluation of the effectiveness <strong>and</strong> efficiency of ProgramImplementation – Evaluation of policy <strong>and</strong> distribution ofresourcesThe updat<strong>in</strong>g of mid-term evaluation has the role to <strong>for</strong>mulate conclusionsconcern<strong>in</strong>g the effectiveness <strong>and</strong> efficiency of the Program implementation,as a whole <strong>and</strong> by measure, tak<strong>in</strong>g <strong>in</strong>to account the <strong>in</strong>termediate outputs<strong>and</strong> results <strong>and</strong> their impact with reference to the activities, specific <strong>and</strong>general objectives.Based on the <strong>for</strong>mulated conclusions the mid-term evaluation updateshould make results <strong>for</strong>ecasts. The operational measures should be usedas base unit of the analysis, which means that the analysis should answerthe correspond<strong>in</strong>g general <strong>and</strong> specific evaluation questions.Efficiency – <strong>in</strong>puts, compared with outputs <strong>and</strong> resultsThe updat<strong>in</strong>g of mid-term evaluation must exam<strong>in</strong>e the efficiency of theProgram implementation from the po<strong>in</strong>t of view of exist<strong>in</strong>g relationsbetween relevant outputs <strong>and</strong> <strong>in</strong>puts needed to produce them.Firstly, the analysis should identify the average costs per unit (outputscompared with <strong>in</strong>puts). In<strong>for</strong>mation regard<strong>in</strong>g costs per unit should becompared with regional, national <strong>and</strong> European values <strong>in</strong> the samecategory.Evaluation of efficiency should be made also with regard to the “deadweight”effects (modifications <strong>in</strong> the situation of beneficiary which wouldTERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 20
have occurred even without public f<strong>in</strong>anc<strong>in</strong>g), “replacement” effect (effectobta<strong>in</strong>ed <strong>in</strong> a certa<strong>in</strong> geographic area to the detriment of another one), <strong>and</strong>“leverage” (the fact that public f<strong>in</strong>anc<strong>in</strong>g implies also beneficiary’s ownexpenditure).Effectiveness – outputs <strong>and</strong> results compared with specific <strong>and</strong>operational objectivesThe mid-term evaluation update should exam<strong>in</strong>e the implementationeffectiveness start<strong>in</strong>g from the monitor<strong>in</strong>g <strong>in</strong>dicators established <strong>in</strong> theProgram.The analysis of the extent of specific <strong>and</strong> operational objectivesachievement should be made as follows:o Analysis of progresses registered <strong>in</strong> achiev<strong>in</strong>g the operational objectivesshould be based on the effective outputs of the Program <strong>for</strong> eachmeasure, compared with the value of prelim<strong>in</strong>ary <strong>in</strong>dicators established<strong>in</strong> the respective measure.o Analysis of progresses <strong>in</strong> achiev<strong>in</strong>g the specific objectives should bebased on the results perta<strong>in</strong><strong>in</strong>g to each priority axis established <strong>in</strong> theProgram.o Analysis of progresses <strong>in</strong> achiev<strong>in</strong>g the operational <strong>and</strong> specificobjectives should be based on current application stage of theobservations <strong>and</strong> recommendation of the Mid-Term Evaluation Report<strong>for</strong> the period 2000-2003.Follow<strong>in</strong>g the analysis execution, the mid-term evaluation update shouldpresent the conclusions concern<strong>in</strong>g the effectiveness of the Program.Impact - global objectivesWhen evaluat<strong>in</strong>g the effectiveness <strong>and</strong> the impact of the measures<strong>in</strong>tended <strong>for</strong> <strong>rural</strong> <strong>development</strong>, the relevant common evaluation questionsshould be utilized together with the relevant criteria <strong>and</strong> <strong>in</strong>dicators toestablish the actual contribution (added value) of the SAPARD Program <strong>in</strong>Romania.The mid-term evaluation update must present conclusions concern<strong>in</strong>g theadequacy of the <strong>development</strong> strategy provided <strong>in</strong> NPARD <strong>and</strong> thedistribution of f<strong>in</strong>ancial resources among priorities <strong>in</strong> order to establish theTERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 21
impact of the Program <strong>in</strong> relation to the identified needs. Conclusionsshould be based on the analysis of the effectiveness <strong>and</strong> efficiencyregistered <strong>in</strong> the achievement of Program objectives. Conclusionsregard<strong>in</strong>g the <strong>in</strong>adequate weight of priorities or measures <strong>in</strong> the Programshould constitute the basis <strong>for</strong> recommendations concern<strong>in</strong>g the changesneeded <strong>in</strong> the structure of the f<strong>in</strong>ancial support.4.3.2.6 Quality of Program implementation <strong>and</strong> organization ofMonitor<strong>in</strong>gThe impact of the policy is directly determ<strong>in</strong>ed by the management <strong>and</strong>function<strong>in</strong>g capacity of the implement<strong>in</strong>g bodies.The mid-term evaluation update must exam<strong>in</strong>e the quality <strong>and</strong> efficiency ofimplementation, management <strong>and</strong> monitor<strong>in</strong>g mechanisms tak<strong>in</strong>g <strong>in</strong>toconsideration that any weakness can negatively <strong>in</strong>fluence the impact off<strong>in</strong>ancial assistance provided by the Program.The quality of Program implementation <strong>and</strong> monitor<strong>in</strong>g should be exam<strong>in</strong>edby:o Evaluation of transparency <strong>and</strong> clear separation of responsibilities <strong>in</strong>the management <strong>and</strong> monitor<strong>in</strong>g of the Program implementation;o Evaluation of promotion activities <strong>and</strong> level of Program knowledge bythe implementation bodies;o Evaluation of control mechanisms based on the audit reports <strong>and</strong>irregularities ascerta<strong>in</strong>ed;o Analysis of projects eligibility <strong>and</strong> selection criteria <strong>in</strong> view of ensur<strong>in</strong>gthe quality of their implementation which should be reflected <strong>in</strong> theProgram objectives;o Verification of existence of flexible, transparent procedures <strong>and</strong>criteria <strong>for</strong> the selection of projects so that the achievement ofProgram objectives <strong>and</strong> utilization of available f<strong>in</strong>ancial resourcesshould be efficiently ensured;o Evaluation of the manner <strong>in</strong> which the economic <strong>and</strong> social partnerscontribute to the quality of Program implementation;o Evaluation of the harmonization level of national legislation to theacquis communautaire as well as its implementation degree,follow<strong>in</strong>g the execution of the SAPARD Program <strong>in</strong> Romania;o Evaluation of the current adm<strong>in</strong>istrative system capacity to implementthe SAPARD Program <strong>and</strong> the acquis communautaire related toTERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 22
Common Agricultural Policy, tak<strong>in</strong>g <strong>in</strong>to consideration the newmeasures which follow to be accredited <strong>in</strong> 2005.This analysis must be based, where appropriate, on the answers to therelevant cross-cutt<strong>in</strong>g evaluation questions as set out <strong>in</strong> annex 1;4.3.2.7 Institutional impact• Exam<strong>in</strong><strong>in</strong>g the progress achieved <strong>in</strong> build<strong>in</strong>g the adm<strong>in</strong>istrativecapacity needed <strong>for</strong> the implementation of the acquis communautaireconcern<strong>in</strong>g the CAP, as well as changes <strong>in</strong> the management <strong>and</strong>organization of relevant public policies <strong>in</strong> Romania;• Assess<strong>in</strong>g the degree to which the national legislation support<strong>in</strong>g theimplementation of the Program is developed <strong>and</strong> harmonized with theacquis communautaire.The analysis must be based, where appropriate, on the responses to therelevant cross-cutt<strong>in</strong>g evaluation questions as set out <strong>in</strong> annex 1.4.3.2.8 Conclusions <strong>and</strong> recommendationsEach section of the mid-term evaluation update should present conclusionsconcern<strong>in</strong>g the weaknesses of SAPARD Program implementation <strong>in</strong>Romania <strong>and</strong> recommendations <strong>for</strong> its improvement. This, start<strong>in</strong>g from theanalysis of the Mid-Term Evaluation Report <strong>for</strong> the period 2000-2003 <strong>and</strong>from the exam<strong>in</strong>ation of its results with reference to <strong>in</strong>dicators used <strong>and</strong>recommendations proposed.Conclusions <strong>and</strong> recommendations should be <strong>in</strong>cluded <strong>in</strong> the mid-termevaluation update <strong>and</strong> should refer to the follow<strong>in</strong>g aspects:1. Exam<strong>in</strong>ation of the <strong>in</strong>itial mid-term evaluation• applicability of recommendations by Public Adm<strong>in</strong>istration <strong>and</strong> otherrelevant organisms;• validity (completeness, adequateness <strong>and</strong> suitability) of output,results <strong>and</strong> impact <strong>in</strong>dicators proposed by the <strong>in</strong>itial Mid-TermEvaluation Report.2. Analysis of the ex-ante evaluation• Adequacy of the exist<strong>in</strong>g strategy <strong>and</strong> the necessity to complete orchange it;TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 23
• Effectiveness of <strong>in</strong>struments <strong>for</strong> the implementation of the CommonAgricultural Policy;• Risk factors which <strong>in</strong>fluence the effectiveness <strong>and</strong> efficiency ofimplementation;• Relevant <strong>in</strong><strong>for</strong>mation concern<strong>in</strong>g possibilities of evaluation <strong>and</strong>monitor<strong>in</strong>g.3. SWOT validity• Validity of priorities <strong>and</strong> major imbalances which have to beovercome <strong>and</strong>/or <strong>in</strong>troduction of necessary changes;• Correlation between the objectives <strong>and</strong> identified needs;• Factors which facilitate the economic <strong>and</strong> social cohesion,environment protection <strong>and</strong> equal opportunities <strong>for</strong> men <strong>and</strong> women.4. Evaluation of adequacy <strong>and</strong> consistency of the <strong>development</strong> strategy<strong>in</strong> NPARD• Present validity of the global strategy concept <strong>and</strong> structure;• Justification of the share <strong>and</strong> weight of each priority axis;• Program coherence – correspondence between Program objectives,NPARD <strong>and</strong> the National Development Plan objectives, <strong>in</strong> view ofensur<strong>in</strong>g the achievement of economic <strong>and</strong> social cohesion as well asthe correspondence between national policies <strong>and</strong> priorities <strong>and</strong> thecommunity ones;• Correspondence between operational, specific <strong>and</strong> generalobjectives.5. Quantification of goals – outputs, results <strong>and</strong> impact• Adequacy of <strong>in</strong>dicators used to quantify the objectives <strong>and</strong>imbalances;• Adequacy of <strong>in</strong>dicators concern<strong>in</strong>g the general, specific <strong>and</strong>operational objectives;• Adequacy of <strong>in</strong>dicators which monitor the impact on the equalopportunities <strong>for</strong> men <strong>and</strong> women, environment protection <strong>and</strong> otherhorizontal issues;• Adequacy <strong>and</strong> timetable of data collect<strong>in</strong>g procedures;• Applicability <strong>and</strong> utility of <strong>in</strong>dicators used to obta<strong>in</strong> a correct <strong>and</strong>timely image of the efficiency of Program implementation, monitor<strong>in</strong>g<strong>and</strong> evaluation.6. Evaluation of effectiveness <strong>and</strong> efficiency of Program implementationTERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 24
• Results <strong>and</strong> progresses registered <strong>in</strong> achiev<strong>in</strong>g the objectives;• Efficient <strong>and</strong> sufficient <strong>in</strong><strong>for</strong>mation concern<strong>in</strong>g beneficiaries.• F<strong>in</strong>ancial share of each priority based on the first results obta<strong>in</strong>ed <strong>and</strong>the resulted impact compared with that <strong>for</strong>eseen <strong>and</strong>, <strong>in</strong> case ofimproper results, the needed changes;• Efficiency of outputs <strong>and</strong> results compared with the utilization off<strong>in</strong>ancial resources;• Effectiveness of horizontal measures implementation with regard tothe equal opportunities <strong>for</strong> men <strong>and</strong> women <strong>and</strong> the environmentprotection;• Progresses registered <strong>in</strong> achiev<strong>in</strong>g the general, specific <strong>and</strong>operational objectives;• Correspondence between the <strong>for</strong>eseen <strong>and</strong> effective impact ofProgram implementation <strong>for</strong> each priority axis <strong>and</strong> measure;• Correspondence between the first results obta<strong>in</strong>ed <strong>and</strong> f<strong>in</strong>ancialallocations <strong>for</strong> each priority <strong>and</strong> measure with<strong>in</strong> the Program;• Correspondence between allocated resources <strong>and</strong> obta<strong>in</strong>ed outputs<strong>and</strong> results.7. Quality of implementation <strong>and</strong> monitor<strong>in</strong>g management• Effectiveness <strong>and</strong> efficiency of Program implementation <strong>and</strong>management;• Transparent <strong>and</strong> competitive procedures <strong>for</strong> project selection;• Actual accountability accord<strong>in</strong>g to the requirements of the relevantnational <strong>and</strong> community legislation;• Contribution of economic <strong>and</strong> social partners.4.3.2.9 Structure of Mid-Term Evaluation Updat<strong>in</strong>g ReportThe mid-term evaluation updat<strong>in</strong>g Report must describe comprehensivelythe evaluated Program, <strong>in</strong>clud<strong>in</strong>g its context <strong>and</strong> scope, procedures <strong>and</strong>results of evaluation as well as the conclusions <strong>and</strong> recommendations.The structure of the mid-term evaluation update Report to be presented tothe European Commission (Regulation EC 1750/1999) is the follow<strong>in</strong>g:TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 25
a) Executive summaryThe executive summary should conta<strong>in</strong> the ma<strong>in</strong> results <strong>and</strong> conclusions ofthe mid-term evaluation update. It is preferably that the executive summarydoes not exceed 5 pages.b) IntroductionThe <strong>in</strong>troduction should present general <strong>and</strong> contextual <strong>in</strong><strong>for</strong>mationconcern<strong>in</strong>g the Program: specific national policies, economic <strong>and</strong> socialneeds justify<strong>in</strong>g the assistance, def<strong>in</strong>ition of beneficiaries or target groups.Also, the Introduction should furnish <strong>in</strong><strong>for</strong>mation on actions previouslyimplemented. To that effect, it should <strong>in</strong>clude key elements (updated) oftheir implementation - parties <strong>in</strong>volved, <strong>in</strong>stitutional framework, period,general f<strong>in</strong>ancial <strong>in</strong><strong>for</strong>mation, a short description of priorities <strong>and</strong> measurestaken <strong>in</strong>to account <strong>for</strong> evaluation.At the same time, the Introduction should furnish <strong>in</strong><strong>for</strong>mation on theevaluation process: presentation of the Terms of Reference, scope <strong>and</strong>objectives of the evaluation, common <strong>and</strong> Program specific evaluationquestions.It should present a succ<strong>in</strong>ct description of previous evaluations relevant tothe Program.c) Methodological aspectsIn this section the logic of the methodological framework <strong>and</strong> itsconsequences should be presented. Also, it should be presented thegeneral evaluation system <strong>and</strong> methods used dur<strong>in</strong>g the evaluationprocess:- data sources, data collect<strong>in</strong>g methodology (questionnaires,<strong>in</strong>terviews, size <strong>and</strong> selection criteria <strong>for</strong> samples), <strong>in</strong><strong>for</strong>mationregard<strong>in</strong>g the calculation method <strong>for</strong> the evaluation of data quality <strong>and</strong>correctness <strong>and</strong> the identification of possible <strong>in</strong>accuracies;- methodology applied to answer to the evaluation questions <strong>and</strong> toissue the conclusions.The Report should clarify any problem or limitations <strong>in</strong> relation to theapplication of this methodology.d) Presentation <strong>and</strong> analysis of collected <strong>in</strong><strong>for</strong>mationTERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 26
applied by the <strong>in</strong>stitutions <strong>in</strong>volved <strong>in</strong> the implementation <strong>and</strong> managementof the Program <strong>and</strong> <strong>in</strong> the future <strong>rural</strong> <strong>development</strong> <strong>program</strong>m<strong>in</strong>g period.This section should clearly answer to the evaluation questions dulyreasoned on the basis of the evaluation work, to all the evaluationquestions proposed <strong>in</strong> the specifications as well as any other questiondef<strong>in</strong>ed dur<strong>in</strong>g the structur<strong>in</strong>g phase of the evaluation. The conclusion partof each question must follow on directly from the analysis <strong>and</strong> <strong>in</strong>clude areference to the judgement criterion.f) AnnexesThe annexes should conta<strong>in</strong> detailed <strong>in</strong><strong>for</strong>mation <strong>in</strong> relation to the Terms ofReference, complete sets of data, analytical details, detailed monographs<strong>and</strong> the structure of questionnaires.An additional but not compulsory section which can contribute to theevaluation credibility is that dedicated to op<strong>in</strong>ions from the economic <strong>and</strong>social partners as regards the quality of achieved <strong>and</strong> presentedevaluation.4.3.2.10 BibliographyIn the preparation of the mid-term evaluation update, the follow<strong>in</strong>gdocuments must be used:• “Guidel<strong>in</strong>es <strong>for</strong> the Evaluation of Rural Development ProgrammesSupported by SAPARD” (Doc. DG-Agri, 04.2001);• “Guidel<strong>in</strong>es <strong>for</strong> the mid-term evaluation of <strong>rural</strong> <strong>development</strong>Programs funded by SAPARD 2000-2006” (Doc. DG-AGRI 09/2002);• “The National Program <strong>for</strong> Agriculture <strong>and</strong> Rural Development”approved by the European Commission on 12 December 2000, asmodified by European Commission Decision No H/2002/1936 on 12of July 2002, European Commission Decision on 27 th of May 2003,European Commission Decision on 1 st of August 2003 <strong>and</strong> EuropeanCommission Decision on 17th September 2004.• “Multi-Annual F<strong>in</strong>anc<strong>in</strong>g Agreement” <strong>for</strong> the Special AccessionProgramme <strong>for</strong> Agriculture <strong>and</strong> Rural Development <strong>in</strong> Romania,between the Commission of the European Communities on behalf ofthe European Community <strong>and</strong> Romania, signed <strong>in</strong> Brussels on 2 nd ofFebruary 2001 <strong>and</strong> ratified by Romanian Law no. 316/2001 (OfficialJournal of Romania no. 362/2001);TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 28
• “Annual F<strong>in</strong>anc<strong>in</strong>g Agreement 2000” <strong>for</strong> the Special AccessionProgramme <strong>for</strong> Agriculture <strong>and</strong> Rural Development <strong>in</strong> Romania,between the Commission of the European Communities on behalf ofthe European Community <strong>and</strong> Romania, signed <strong>in</strong> Brussels on 27 th ofFebruary 2001 <strong>and</strong> ratified by Romanian Law no. 317/2001 (OfficialJournal of Romania no. 338/2001);• “Annual F<strong>in</strong>anc<strong>in</strong>g Agreement 2001” <strong>for</strong> the Special AccessionProgramme <strong>for</strong> Agriculture <strong>and</strong> Rural Development <strong>in</strong> Romania,between the Commission of the European Communities on behalf ofthe European Community <strong>and</strong> Romania, signed <strong>in</strong> Brussels on 30 th ofJanuary 2002 <strong>and</strong> ratified by Romanian Law no. 416/2002 (OfficialJournal of Romania no. 508/2002);• “Annual F<strong>in</strong>anc<strong>in</strong>g Agreement 2002” <strong>for</strong> the Special AccessionProgramme <strong>for</strong> Agriculture <strong>and</strong> Rural Development <strong>in</strong> Romania,between the Commission of the European Communities on behalf ofthe European Community <strong>and</strong> Romania, signed <strong>in</strong> Bucharest <strong>and</strong>Brussels on 1 st of April 2003 <strong>and</strong> ratified by Romanian GovernmentUrgency Ord<strong>in</strong>ance no. 26/2003 (Official Journal of Romania no.283/2003);• “Annual F<strong>in</strong>anc<strong>in</strong>g Agreement 2003” <strong>for</strong> the Special AccessionProgramme <strong>for</strong> Agriculture <strong>and</strong> Rural Development <strong>in</strong> Romania,between the Commission of the European Communities on behalf ofthe European Community <strong>and</strong> Romania, signed <strong>in</strong> Brussels <strong>and</strong>Bucharest on 31 st of July 2003 <strong>and</strong> ratified by Romanian Law no.496/2003 (Official Journal of Romania no. 843/2003);• Mid-Term Evaluation of SAPARD <strong>in</strong> Romania <strong>for</strong> the period 2000-2003;• Annual SAPARD Implementation Reports.The follow<strong>in</strong>g documents are recommended:• Work<strong>in</strong>g documents of SAPARD Agency, elaborated on the basis ofthe implementation procedures;• “The evaluation of socio-economic Programs; <strong>in</strong>dicators <strong>for</strong> themonitor<strong>in</strong>g <strong>and</strong> the evaluation of Programs (MEANS, Vol.2). Thiswork approaches methodological aspects referr<strong>in</strong>g to the def<strong>in</strong>ition ofTERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 29
<strong>in</strong>dicators <strong>and</strong> furnishes the list of <strong>in</strong>dicators <strong>for</strong> the ma<strong>in</strong> fields of<strong>in</strong>tervention of the Structural Funds;• “Guidel<strong>in</strong>es <strong>for</strong> Evaluation of <strong>rural</strong> <strong>development</strong> Programs 2000-2006supported from the European Agricultural Guidance <strong>and</strong> GuaranteeFund” (Doc. DG AGRI VI/8865/1999);• “Guidel<strong>in</strong>es <strong>for</strong> the mid term evaluation of <strong>rural</strong> <strong>development</strong>Programs 2000-2006 supported by the European AgriculturalGuidance <strong>and</strong> Guarantee Fund” (DOC STAR VI/43517/2002);• Council Regulation (EC) No.1268/1999 of 21 June 1999 onCommunity support <strong>for</strong> pre-<strong>accession</strong> measures <strong>for</strong> <strong>agriculture</strong> <strong>and</strong><strong>rural</strong> <strong>development</strong> <strong>in</strong> the applicant countries of central <strong>and</strong> easternEurope <strong>in</strong> the pre-<strong>accession</strong> period;• Council Regulation (EC) No 1260/1999 of 21 June 1999 lay<strong>in</strong>g downgeneral provisions on the Structural Funds;• Commission Regulation (EC) No. 2759/1999 of 22 December 1999lay<strong>in</strong>g down rules <strong>for</strong> the application of the Council Regulation (EC)No.1268/1999 on Community support <strong>for</strong> pre-<strong>accession</strong> measures <strong>for</strong><strong>agriculture</strong> <strong>and</strong> <strong>rural</strong> <strong>development</strong> <strong>in</strong> the applicant countries of central<strong>and</strong> eastern Europe <strong>in</strong> the pre-<strong>accession</strong> period;• Commission Regulation (EC) No. 2222/2000 lay<strong>in</strong>g down f<strong>in</strong>ancialrules <strong>for</strong> the application of Council Regulation (EC) No.1268/1999 onCommunity support <strong>for</strong> pre-<strong>accession</strong> measures <strong>for</strong> <strong>agriculture</strong> <strong>and</strong><strong>rural</strong> <strong>development</strong> <strong>in</strong> the applicant countries of central <strong>and</strong> easternEurope <strong>in</strong> the pre-<strong>accession</strong> period;• Council Regulation (EC) No 2008/2004 of 16 November 2004amend<strong>in</strong>g Regulation (EC) No 1268/1999 on Community support <strong>for</strong>pre-<strong>accession</strong> measures <strong>for</strong> <strong>agriculture</strong> <strong>and</strong> <strong>rural</strong> <strong>development</strong> <strong>in</strong> theapplicant countries of central <strong>and</strong> eastern Europe <strong>in</strong> the pre<strong>accession</strong>period;• Council Regulation (EC) No 1257/1999 of 17 May 1999 on support<strong>for</strong> <strong>rural</strong> <strong>development</strong> from the European Agricultural Guidance <strong>and</strong>Guarantee Fund (EAGGF) <strong>and</strong> amend<strong>in</strong>g <strong>and</strong> repeal<strong>in</strong>g certa<strong>in</strong>Regulations;• Commission Regulation (Ec) No 1750/1999 of 23 July 1999 lay<strong>in</strong>gdown detailed rules <strong>for</strong> the application of Council Regulation (EC) No1257/1999 on support <strong>for</strong> <strong>rural</strong> <strong>development</strong> from the EuropeanAgricultural Guidance <strong>and</strong> Guarantee Fund (EAGGF);• Commission Regulation (EC) No 445/2002 of 26 February 2002lay<strong>in</strong>g down detailed rules <strong>for</strong> the application of Council RegulationTERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 30
(EC) No 1257/1999 on support <strong>for</strong> <strong>rural</strong> <strong>development</strong> from theEuropean Agricultural Guidance <strong>and</strong> Guarantee Fund (EAGGF);• COM(2004)490, proposal <strong>for</strong> a Council Regulation on support <strong>for</strong><strong>rural</strong> <strong>development</strong> by the European Agricultural Fund <strong>for</strong> RuralDevelopment (EAFRD) <strong>and</strong> the subsequent regulation whenapproved.4.4 Project management, contractor’s tasks <strong>and</strong> responsibilities:The direct beneficiary of the project <strong>and</strong> Contract<strong>in</strong>g Authority responsible<strong>for</strong> the management of the project is the Manag<strong>in</strong>g Authority with<strong>in</strong> theM<strong>in</strong>istry of Agriculture, Forests <strong>and</strong> Rural Development (MAFRD).SAPARD Agency have to approve the documents concern<strong>in</strong>g the project,as mentioned <strong>in</strong> the Manual of Procedure <strong>for</strong> Measure 4.2 „TechnicalAssistance” <strong>and</strong> is responsible of the execution of the payments afterauthorization of payment requests, <strong>in</strong> compliance with all major proceduresapplied <strong>for</strong> SAPARD.Obligations of the Contract<strong>in</strong>g Authority <strong>in</strong> RomaniaThe Contract<strong>in</strong>g Authority has the follow<strong>in</strong>g obligations:• To ensure a good cooperation <strong>and</strong> communication of the Contract<strong>in</strong>gAuthority relevant personnel with the Contractor staff dur<strong>in</strong>g thewhole duration of the Evaluator activity <strong>in</strong> relation to the mid-termevaluation update;• To support the Evaluator <strong>in</strong> the technical <strong>and</strong> logistic organization ofthe evaluation process;• To grant the necessary support to the evaluation team experts;• To ensure <strong>and</strong> to facilitate the access to available data <strong>and</strong> nationallegislation <strong>for</strong> the correct carry<strong>in</strong>g out of the evaluation;• To be fully available to promptly answer the evaluator requests <strong>for</strong>help <strong>in</strong> the solution of critical situations that limit or embed the correctfulfilment of its functions, especially with reference to the collection<strong>and</strong> process<strong>in</strong>g of <strong>in</strong><strong>for</strong>mation provided <strong>in</strong> the territory;• To follow up the correct utilization of the funds <strong>for</strong> the carry<strong>in</strong>g out ofevaluation;• To submit the f<strong>in</strong>al mid-term evaluation update Report to theEuropean Commission after hav<strong>in</strong>g submitted its synthesis <strong>and</strong>conclusions to the monitor<strong>in</strong>g committee.TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 31
Obligations of the ContractorThe Contractor has the responsibility:• To ensure all the equipment, endowments, material goods <strong>and</strong>services, as well as logistic support needed to per<strong>for</strong>m the mid-termevaluation update;• To ensure the f<strong>in</strong>ancial <strong>and</strong> adm<strong>in</strong>istrative management necessary<strong>for</strong> the carry<strong>in</strong>g out of the mid-term evaluation update;• To provide the qualified key experts, as specified at po<strong>in</strong>t 6.3 of theseToR, as well as other needed support<strong>in</strong>g staff, be<strong>in</strong>g responsible <strong>for</strong>fees, lodg<strong>in</strong>g, <strong>in</strong>ternational <strong>and</strong> local travel of is <strong>in</strong>ternational experts;• To organize the visits of Romanian <strong>and</strong> <strong>for</strong>eign experts <strong>in</strong> the country<strong>and</strong> outside the Country;• To ensure the correct utilization of the funds <strong>for</strong> the carry<strong>in</strong>g out of themid-term evaluation update accord<strong>in</strong>g to the contract signed with theContract<strong>in</strong>g Authority;• To start the project activities on the first day after the contractsignature, as provided by po<strong>in</strong>t 5.2 of these ToR;• To submit to the Contract<strong>in</strong>g Authority detailed Reports on activitiesdeployed to that effect, as provided at po<strong>in</strong>t 7 of these ToR;• To report to the Contract<strong>in</strong>g Authority on the activities deployed <strong>and</strong>eventual obstacles met <strong>in</strong> view of adoption of operative remedialmeasures;• To identify the relevant quantitative <strong>and</strong> qualitative <strong>in</strong>dicators ofoutput, results <strong>and</strong> impact, tak<strong>in</strong>g <strong>in</strong>to account the <strong>in</strong>dicators<strong>for</strong>mulated <strong>in</strong> the Mid-Term Evaluation Report <strong>for</strong> 2000-2003, <strong>in</strong>NPARD <strong>and</strong> <strong>in</strong> the Guidel<strong>in</strong>es of the Commission.• To collect the necessary available data <strong>and</strong> to elaborate <strong>and</strong> collectother relevant data (<strong>in</strong>terviews, questionnaires, etc…) necessary tothe evaluation of identified relevant <strong>in</strong>dicators;• To process the collected data <strong>and</strong> <strong>in</strong>dicators, <strong>in</strong>clud<strong>in</strong>g thecorrespond<strong>in</strong>g results <strong>in</strong> the mid-term report update.• To ensure the application of the ToR, with reference to the content<strong>and</strong> structure of the mid-term evaluation update report.• To provide the Contract<strong>in</strong>g Authority with one Inception Report, oneInterim Activity Report <strong>and</strong> one F<strong>in</strong>al Activity Report, as described atpo<strong>in</strong>t 7.1 <strong>and</strong> 7.2;• To provide the Contract<strong>in</strong>g Authority with the first version of the Mid-Term Evaluation Updat<strong>in</strong>g Report, the draft f<strong>in</strong>al version of the Mid-TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 32
Term Evaluation Updat<strong>in</strong>g Report <strong>and</strong> the F<strong>in</strong>al Mid-Term EvaluationUpdat<strong>in</strong>g Report, as described at po<strong>in</strong>t 7.3 of these Terms ofReference. These reports must be characterised by pert<strong>in</strong>ent <strong>and</strong>credible results, judgements <strong>and</strong> recommendations. Each one ofthese reports must respect the structure described at po<strong>in</strong>t 4.3.2.9 ofthese ToR;• To respect <strong>in</strong>dications <strong>for</strong>mulated, <strong>in</strong> maximum 15 days s<strong>in</strong>ce thesubmission of the Inception Report <strong>and</strong> the activities Reports, by theContract<strong>in</strong>g Authority;• To consult with the Contract<strong>in</strong>g Authority regard<strong>in</strong>g the first version ofthe Mid-Term Evaluation Report, tak<strong>in</strong>g <strong>in</strong>to account its observations<strong>and</strong> recommendations;• To consult with the Contract<strong>in</strong>g Authority, dur<strong>in</strong>g the elaboration ofthe Mid-Term Evaluation Updat<strong>in</strong>g, with reference to the EuropeanCommission remarks concern<strong>in</strong>g the Mid-Term Evaluation Report <strong>for</strong>SAPARD Program dur<strong>in</strong>g the period 2000-2003;• To <strong>in</strong>corporate modifications recommended by the Manag<strong>in</strong>gAuthority, the Monitor<strong>in</strong>g Committee <strong>and</strong> the European Commission<strong>in</strong> the mid-term evaluation update Report, as described at po<strong>in</strong>t 5.2 ofthese Terms of Reference;• To ensure the f<strong>in</strong>alisation, <strong>in</strong> the time <strong>and</strong> conditions provided <strong>for</strong> bythese ToR, of the reports mentioned at po<strong>in</strong>t 7 (on paper <strong>and</strong> onelectronic <strong>for</strong>mat, CD/floppy, <strong>in</strong> Romanian <strong>and</strong> English);• To ensure his own <strong>in</strong>dependence <strong>and</strong> the <strong>in</strong>dependence of all hiswork<strong>in</strong>g staff, as described at po<strong>in</strong>t 4.4 of these ToR.In the fulfilment of their responsibilities both the Contract<strong>in</strong>g Authority <strong>and</strong>the Contractor shall respect the “ethics clauses" as set <strong>in</strong> the Manual <strong>for</strong>the implementation of Measure 4.2 “Technical Assistance”.Award<strong>in</strong>g criteriaOn receiv<strong>in</strong>g tenders, the representatives of the contract<strong>in</strong>g authority willrecord them <strong>and</strong> provide a registration number <strong>for</strong> those delivered by h<strong>and</strong>.Only tenders conta<strong>in</strong>ed <strong>in</strong> envelopes received by the date <strong>and</strong> time<strong>in</strong>dicated <strong>in</strong> the tender dossier will be considered <strong>for</strong> evaluation.TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 33
Initially, only the technical offers will be opened. The sealed envelopesconta<strong>in</strong><strong>in</strong>g the f<strong>in</strong>ancial offers will be reta<strong>in</strong>ed by the Contract<strong>in</strong>g Authorityonce signed by the President <strong>and</strong> the Secretary of the EvaluationCommittee.The Evaluation Committee checks the adm<strong>in</strong>istrative compliance of tenderswith the <strong>in</strong>structions given <strong>in</strong> the tender dossier. Any <strong>for</strong>mal errors or majorrestrictions affect<strong>in</strong>g per<strong>for</strong>mance of the contract or distort<strong>in</strong>g competitionwill result <strong>in</strong> the rejection of the tender concerned.The quality of each technical offer shall be evaluated <strong>in</strong> accordance withthe award criteria <strong>and</strong> the associated weight<strong>in</strong>g as detailed <strong>in</strong> theevaluation grid <strong>in</strong>cluded <strong>in</strong> this tender dossier. No other award criteria willbe used. The award criteria will be exam<strong>in</strong>ed <strong>in</strong> accordance with therequirements as <strong>in</strong>dicated <strong>in</strong> the Terms of Reference.When evaluat<strong>in</strong>g technical offers, each member awards each offer a scoreout of a maximum 100 po<strong>in</strong>ts <strong>in</strong> accordance with the technical evaluationgrid (sett<strong>in</strong>g out the technical criteria, sub-criteria <strong>and</strong> weight<strong>in</strong>gs) laid down<strong>in</strong> the tender dossier.Where the content of a tender is <strong>in</strong>complete or deviates substantially fromone or more of the technical award criteria laid down <strong>in</strong> the tender dossier,the tender is automatically rejected <strong>and</strong> no po<strong>in</strong>ts are awarded.Besides the awarded numerical score, each member of the EvaluationCommittee must expla<strong>in</strong> the reasons <strong>for</strong> his choice <strong>and</strong> defend his scoresbe<strong>for</strong>e the Evaluation Committee. The Committee discusses each technicaloffer <strong>and</strong> each member awards it a f<strong>in</strong>al score. The aggregate f<strong>in</strong>al score isthe arithmetic average of the <strong>in</strong>dividual scores. Once the EvaluationCommittee has established the f<strong>in</strong>al score of each technical offer (thearithmetic average of the scores awarded by each member), any tenderfall<strong>in</strong>g under the 80-po<strong>in</strong>t threshold is automatically rejected. If no tenderachieves 80 po<strong>in</strong>ts or more, the tender procedure is cancelled. Thecommittee considers only tenders that have obta<strong>in</strong>ed at least 80 po<strong>in</strong>ts. Ofthese tenders, the best technical offer is then awarded 100 po<strong>in</strong>ts. Theothers receive po<strong>in</strong>ts calculated us<strong>in</strong>g the follow<strong>in</strong>g <strong>for</strong>mula:Po<strong>in</strong>ts = (<strong>in</strong>itial score of the tender <strong>in</strong> question/<strong>in</strong>itial score of the besttechnical offer) x 100.TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 34
Upon completion of the technical evaluation, the envelopes conta<strong>in</strong><strong>in</strong>g thef<strong>in</strong>ancial offers <strong>for</strong> tenders which were not elim<strong>in</strong>ated dur<strong>in</strong>g the technicalevaluation are opened <strong>and</strong> signed by the President <strong>and</strong> the Secretary ofthe Evaluation Committee at the session. At the session, the Committeechecks that the f<strong>in</strong>ancial offers conta<strong>in</strong> no arithmetic errors. Any arithmeticerrors which do not affect the offer content are corrected without prejudiceto the c<strong>and</strong>idate.Comparison of the f<strong>in</strong>ancial offers takes account of the total contract value.F<strong>in</strong>ancial offers exceed<strong>in</strong>g the maximum budget allocated <strong>for</strong> the contractare elim<strong>in</strong>ated.The lowest f<strong>in</strong>ancial offer receives 100 po<strong>in</strong>ts. The others are awardedpo<strong>in</strong>ts by means of the follow<strong>in</strong>g <strong>for</strong>mula:Po<strong>in</strong>ts = (lowest f<strong>in</strong>ancial offer/f<strong>in</strong>ancial offer be<strong>in</strong>g considered) x 100.The most economically advantageous tender is established by weigh<strong>in</strong>gtechnical quality aga<strong>in</strong>st price on an 80/20 basis. This is done bymultiply<strong>in</strong>g:• the scores awarded to the technical offers by 0.80• the scores awarded to the f<strong>in</strong>ancial offers by 0.20.The result<strong>in</strong>g technical <strong>and</strong> f<strong>in</strong>ancial scores are then added together, <strong>and</strong>the contract is awarded to the tender achiev<strong>in</strong>g the highest score.The entire procedure (technical <strong>and</strong> f<strong>in</strong>ancial evaluation) is recorded <strong>in</strong>m<strong>in</strong>utes to be signed by all members of the committee <strong>and</strong> approved by theContract<strong>in</strong>g Authority.Evaluation of offers will be per<strong>for</strong>med accord<strong>in</strong>g to the procedure manual<strong>for</strong> the implementation of Measure 4.2 „Technical Assistance”. The scor<strong>in</strong>gof offers submitted by c<strong>and</strong>idates will be per<strong>for</strong>med on the basis of aevaluation grid <strong>in</strong> which the score of technical offer will represent 80%, <strong>and</strong>that of f<strong>in</strong>ancial offer 20% of the total score. The evaluation grid is shown <strong>in</strong>annex no. 2.Technical evaluation grid will conta<strong>in</strong> the scor<strong>in</strong>g <strong>for</strong> organisation <strong>and</strong>methodology of technical offer, <strong>and</strong> the evaluation of the experts.TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 35
Organisation <strong>and</strong> Methodology will be evaluated on the basis of therationale, strategy <strong>and</strong> <strong>program</strong>m<strong>in</strong>g proposed by the c<strong>and</strong>idate. Thissection will represent the 30% of the technical evaluation f<strong>in</strong>al score.CVs of key experts, which are compulsory, will be evaluated on the basis ofqualifications <strong>and</strong> skills, general <strong>and</strong> specific professional experience. Thissection will represent the 70% of the technical evaluation f<strong>in</strong>al score.F<strong>in</strong>ancial offers will be evaluated on the basis of the total contract value,compared to the maximum budget available <strong>for</strong> the contract.The contract is f<strong>in</strong>anced through Measure 4.2 „Technical Assistance” with<strong>in</strong>SAPARD Program <strong>and</strong> is a global price contract, mean<strong>in</strong>g that thecontractor will be paid only if the specified outcome is achieved <strong>in</strong> itscompleteness.With<strong>in</strong> the framework of the service contract, the f<strong>in</strong>anc<strong>in</strong>g could beeffectuated <strong>in</strong> two <strong>in</strong>stalments, but only after the presentation of the Firstversion of the Mid-Term Evaluation Updat<strong>in</strong>g Report <strong>and</strong> the F<strong>in</strong>al Mid-Term Evaluation Updat<strong>in</strong>g Report by the Contractor, <strong>and</strong> approval by theContract<strong>in</strong>g Authority of the Reports mentioned at po<strong>in</strong>t 7 of these ToR. Noadvance payments are <strong>for</strong>eseen.Subcontract<strong>in</strong>g with<strong>in</strong> the framework of the contract is allowed. The valueof the subcontracted services must not exceed 30% of the total contractvalue.4.5 Independence of the EvaluatorThe evaluation shall be carried out by an <strong>in</strong>dependent Contractor (experts)<strong>in</strong> the sense of no direct <strong>in</strong>volvement <strong>in</strong> the management, implementation<strong>and</strong> f<strong>in</strong>anc<strong>in</strong>g of the Program.The evaluator shall refra<strong>in</strong> from any relationship which would compromiseits <strong>in</strong>dependence or that of its personnel.The evaluator shall limit its role <strong>in</strong> connection with the project to theprovision of services.TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 36
The evaluator <strong>and</strong> anyone work<strong>in</strong>g under its authority <strong>and</strong> control <strong>in</strong> theper<strong>for</strong>mance of mid-term evaluation update shall be excluded from accessto EC f<strong>in</strong>anc<strong>in</strong>g available under the SAPARD Program.The experts must not have been direct beneficiary of the SAPARDProgram, nor should they have been <strong>in</strong>volved <strong>in</strong> its management or <strong>in</strong> thedraft<strong>in</strong>g of these terms of reference.Payments to the Evaluator under the project shall constitute the only<strong>in</strong>come or benefit it may derive <strong>in</strong> connection with the project <strong>and</strong> neitherhe/she nor his/her personnel shall accept any commission, discount,allowance, <strong>in</strong>direct payment or other consideration <strong>in</strong> connection with,relation to or <strong>in</strong> discharge of his/her obligation under the project.5. LOGISTICS AND PROGRAMMINGThe evaluation should <strong>in</strong>volve the active participation of the relevantInstitutions (M<strong>in</strong>istry of Agriculture, Forests <strong>and</strong> Rural Development,Agriculture <strong>and</strong> Rural Development Counties’ Directorates, SAPARDAgencies with its BRIPS <strong>and</strong> other Institutions).5.1 Place where the project is executedProject activities will be executed <strong>in</strong> Bucharest <strong>and</strong> <strong>in</strong> the Counties, <strong>in</strong>function of planned activities.The mid-term evaluation update must cover the geographic area of theNAPARD.The evaluator will participate to meet<strong>in</strong>g <strong>and</strong> discussion not only on theRomanian territory but also outside Romania (consultations with theEuropean Commission <strong>in</strong> Brussels), on request.5.2 Period of the ProjectUpdat<strong>in</strong>g of mid-term evaluation should be completed <strong>in</strong> six months.The project activities must start on the first day after the contract signature.The updat<strong>in</strong>g should be carried out <strong>in</strong> three stages:1. the collection by the Evaluator of the necessary data (consult<strong>in</strong>gdocuments, <strong>in</strong>terviews, questionnaires, etc…), process<strong>in</strong>g of theTERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 37
collected data, analysis of the results <strong>and</strong> preparation of the first versionof Mid-Term Evaluation updat<strong>in</strong>g Report, which is to be discussed withthe Contract<strong>in</strong>g Authority coord<strong>in</strong>at<strong>in</strong>g the mid-term evaluation updateactivities, no later than 10 th of October 2005;2. the completion of the mid-term evaluation updat<strong>in</strong>g <strong>and</strong> submission ofthe draft f<strong>in</strong>al version of Mid-Term Evaluation Updat<strong>in</strong>g Report to theContract<strong>in</strong>g Authority, no later than 10 th of November 2005, so thatenough time should be allocated <strong>for</strong> exam<strong>in</strong>ation by the Monitor<strong>in</strong>gCommittee <strong>and</strong> possible adjustment or completion of the Report;3. The F<strong>in</strong>al Mid-Term Evaluation Updat<strong>in</strong>g Report must be submitted tothe Contract<strong>in</strong>g Authority no later than 10 th of December 2005.In case the Manag<strong>in</strong>g Authority recommends modifications to the reportsmentioned above at po<strong>in</strong>ts 1, 2 <strong>and</strong> 3, the evaluator must <strong>in</strong>corporate them<strong>and</strong> resubmit the respective reports <strong>in</strong> no more than 5 days s<strong>in</strong>ce therequest.The Manag<strong>in</strong>g Authority will present the F<strong>in</strong>al Mid-Term EvaluationUpdat<strong>in</strong>g Report to the European Commission no later than 31 st ofDecember 2005.In case the European Commission recommends modifications <strong>in</strong> the report,the contract will automatically be extended to <strong>in</strong>clude the evaluatorobligation of <strong>in</strong>corporat<strong>in</strong>g the recommended modification <strong>and</strong> resubmitt<strong>in</strong>gthe Update Mid-Term Evaluation Report <strong>in</strong> no more than 7 days s<strong>in</strong>ce therequest by the Manag<strong>in</strong>g Authority.6. REQUIREMENTS6.1 General RequirementsSpecialised staff <strong>and</strong> staff-related expenses <strong>for</strong> missions <strong>in</strong>side <strong>and</strong> outsideRomania, equipments, endowments, goods <strong>and</strong> services concern<strong>in</strong>g theimplementation of contract activities, <strong>and</strong> logistic support <strong>for</strong> the successfulaccomplishment of undertaken tasks are under complete responsibility ofthe services provider-contractor.6.2 Contractor Requirements- Companies with legal status, Romanian, <strong>for</strong>eign or mixed, which has asobject the activities requested through these Terms of Reference;TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 38
- The company has to dispose of staff resources – at least 3 humanresources of the permanent staff, currently work<strong>in</strong>g <strong>in</strong> the relevantsectors <strong>for</strong> this contract, <strong>and</strong> should have suitable work<strong>in</strong>g space,suitable endowments, technical equipment <strong>and</strong> <strong>in</strong>struments necessaryto carry out the updat<strong>in</strong>g of the mid term evaluation report.- The company should present an activities portfolio show<strong>in</strong>g experience<strong>in</strong> projects <strong>and</strong> <strong>program</strong>s evaluation, <strong>in</strong> <strong>rural</strong> <strong>development</strong> sector;- The company (<strong>in</strong>dividual or consortium) annual turnover average, <strong>in</strong> thelast two operat<strong>in</strong>g years (2002, 2003) must exceed 1.000.000 EURO;- The Company balance sheet should demonstrate a profit <strong>for</strong> each oneof the last two years (2002, 2003);- The Company (<strong>in</strong>dividual or consortium) must have taken part to theimplementation of an <strong>in</strong>ternational project of a nature <strong>and</strong> complexitysimilar to those of the present contract, <strong>in</strong> the last five years;- Credentials from at least one client who benefited of the Contractormonitor<strong>in</strong>g <strong>and</strong> evaluation services.6.3 Key Experts RequirementsWork<strong>in</strong>g staff <strong>for</strong> the contract should be composed of specialists <strong>in</strong>monitor<strong>in</strong>g, evaluation, <strong>agriculture</strong>, <strong>rural</strong> <strong>development</strong> <strong>and</strong> economics(m<strong>in</strong>imum 8), autochthones or <strong>in</strong>ternational, which could satisfy thefollow<strong>in</strong>g requirements:6.3.1 Profile of International ExpertsKey expert 1 – Project Team Leader <strong>and</strong> Coord<strong>in</strong>ator, expert I gradeQualifications <strong>and</strong> skills:- Economical or technical superior studies <strong>in</strong> the field of agri-food or <strong>rural</strong><strong>development</strong>;- Specialization studies <strong>in</strong> evaluation management <strong>and</strong> <strong>in</strong> the <strong>rural</strong><strong>development</strong> sector;- High proficiency <strong>in</strong> the English language, <strong>and</strong> very good knowledge oftechnical English concern<strong>in</strong>g the specific sector.General professional experience- Adequate knowledge of the EU <strong>rural</strong> <strong>development</strong> policy, <strong>program</strong>ssupported by Structural Funds as well as pre-<strong>accession</strong> <strong>in</strong>struments,especially SAPARD Program, <strong>for</strong> the countries <strong>in</strong> Central <strong>and</strong> EasternEurope.TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 39
Specific professional experience- Experience of m<strong>in</strong>imum 10 years as Team Leader <strong>in</strong> evaluation <strong>and</strong>management of technical assistance projects or <strong>program</strong>mes;- M<strong>in</strong>imum 3 years experience as Team Leader <strong>in</strong> monitor<strong>in</strong>g <strong>and</strong>evaluation of SAPARD Program <strong>and</strong> Structural Funds Programs, <strong>in</strong> thefield of <strong>agriculture</strong> <strong>and</strong> <strong>rural</strong> <strong>development</strong>;- Coord<strong>in</strong>ation experience <strong>in</strong> monitor<strong>in</strong>g <strong>and</strong> evaluation – m<strong>in</strong>imum 5years;- Number of projects on monitor<strong>in</strong>g <strong>and</strong> evaluation of SAPARD Program<strong>and</strong> other <strong>program</strong>s f<strong>in</strong>anced by Structural Funds, <strong>in</strong> the field of<strong>agriculture</strong> <strong>and</strong> <strong>rural</strong> <strong>development</strong>, <strong>in</strong> which the expert has worked asTeam Leader (m<strong>in</strong>imum 2);- Adequate knowledge of <strong>rural</strong> <strong>development</strong> issues <strong>and</strong> of the socioeconomicsituation <strong>in</strong> the <strong>rural</strong> areas of Central <strong>and</strong> Eastern Europe.Key expert 2 – Expert <strong>in</strong> the Evaluation of Rural Development Programs,expert II gradeQualifications <strong>and</strong> skills- Economical or technical superior studies <strong>in</strong> the field of agri-food or <strong>rural</strong><strong>development</strong>;- Specialization studies <strong>in</strong> monitor<strong>in</strong>g <strong>and</strong> evaluation of <strong>rural</strong> <strong>development</strong><strong>program</strong>s;- High proficiency <strong>in</strong> the English language, <strong>and</strong> very good knowledge oftechnical English concern<strong>in</strong>g the specific sector.General professional experience- Adequate knowledge of the EU <strong>rural</strong> <strong>development</strong> policy, <strong>program</strong>ssupported by Structural Funds as well as pre-<strong>accession</strong> <strong>in</strong>struments,especially SAPARD Program, <strong>for</strong> the countries <strong>in</strong> Central <strong>and</strong> EasternEurope.Specific professional experience- M<strong>in</strong>imum 3 years experience <strong>in</strong> the analysis <strong>and</strong> evaluation of <strong>rural</strong><strong>development</strong> <strong>program</strong>s <strong>in</strong> Central <strong>and</strong> Eastern Europe Countries;- Number of projects on <strong>rural</strong> <strong>development</strong> monitor<strong>in</strong>g <strong>and</strong> evaluation <strong>in</strong>which the expert was <strong>in</strong>volved (m<strong>in</strong>imum 2);- Adequate <strong>and</strong> substantial knowledge of SAPARD Program function<strong>in</strong>g<strong>and</strong> socio-economic situation of <strong>rural</strong> areas <strong>in</strong> Central <strong>and</strong> EasternEuropean Countries, as well as <strong>rural</strong> <strong>development</strong> major issues.TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 40
Key expert 3 – Environment Impact Assessment, expert II gradeQualifications <strong>and</strong> skills:- Superior studies <strong>in</strong> the field of agrarian economics <strong>and</strong> environment;- Specialization studies <strong>in</strong> environment management;- High proficiency <strong>in</strong> the English language, <strong>and</strong> very good knowledge oftechnical English concern<strong>in</strong>g the specific sector.General professional experience- Adequate knowledge of the EU environmental policies;- Adequate knowledge of the EU pre-<strong>accession</strong> f<strong>in</strong>ancial <strong>in</strong>struments <strong>for</strong>the countries <strong>in</strong> Central <strong>and</strong> Eastern Europe.Specific professional experience- M<strong>in</strong>imum 3 years experience <strong>in</strong> the implementation of EnvironmentImpact Assessments (EIA) of agricultural <strong>and</strong> <strong>rural</strong> <strong>development</strong><strong>program</strong>s;- Environmental Impact assessment experience <strong>in</strong> <strong>rural</strong> <strong>development</strong>projects or <strong>program</strong>s (m<strong>in</strong>imum 2 years);- Number of projects on Environmental Impact Assessment (EIA) of <strong>rural</strong><strong>development</strong> plans <strong>in</strong> which the expert was <strong>in</strong>volved (m<strong>in</strong>imum 2);- Experience <strong>in</strong> the EIA <strong>for</strong> <strong>program</strong>s f<strong>in</strong>anced by EU Structural Funds<strong>and</strong>/or <strong>for</strong> SAPARD Program <strong>in</strong> Central <strong>and</strong> Eastern EuropeanCountries.6.3.2 Profile of Romanian ExpertsKey expert 4 – Assistant Project Team Leader, expert II gradeQualifications <strong>and</strong> skills- Economical or technical superior studies <strong>in</strong> the field of agri-food or <strong>rural</strong><strong>development</strong>;- Specialization studies <strong>in</strong> management <strong>and</strong>/or monitor<strong>in</strong>g <strong>and</strong> evaluation;- High proficiency <strong>in</strong> the English language.General professional experience- Adequate knowledge of the EU pre-<strong>accession</strong> f<strong>in</strong>ancial <strong>in</strong>struments <strong>for</strong>Central <strong>and</strong> Eastern European Countries.Specific professional experienceTERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 41
- M<strong>in</strong>imum 2 years experience <strong>in</strong> projects or <strong>program</strong>s coord<strong>in</strong>ation;- M<strong>in</strong>imum 3 year experience <strong>in</strong> the analysis <strong>and</strong> evaluation of <strong>rural</strong><strong>development</strong> <strong>program</strong>s <strong>in</strong> Romania <strong>and</strong> <strong>in</strong> the implementation oftechnical assistance projects <strong>in</strong> this field;- Number of projects of monitor<strong>in</strong>g <strong>and</strong> evaluation <strong>for</strong> <strong>rural</strong> <strong>development</strong><strong>program</strong>s <strong>in</strong> which the expert was <strong>in</strong>volved (m<strong>in</strong>imum 2);- Adequate <strong>and</strong> substantial knowledge of Romanian National Program <strong>for</strong>Agriculture <strong>and</strong> Rural Development 2000-2006.Key expert 5 – Expert <strong>in</strong> the Evaluation of Rural Development Programs,expert II gradeQualifications <strong>and</strong> skills- Economical or technical superior studies <strong>in</strong> the field of agri-food or <strong>rural</strong><strong>development</strong>;- Specialization studies <strong>in</strong> monitor<strong>in</strong>g <strong>and</strong> evaluation of <strong>rural</strong> <strong>development</strong><strong>program</strong>s;- High proficiency <strong>in</strong> the English language.General professional experience- Adequate knowledge of <strong>rural</strong> <strong>development</strong> policy <strong>and</strong> EU pre-<strong>accession</strong>f<strong>in</strong>ancial <strong>in</strong>struments <strong>for</strong> Central <strong>and</strong> Eastern European Countries;- Adequate knowledge of the social <strong>and</strong> economic situation <strong>in</strong> Romanian<strong>rural</strong> areas <strong>and</strong> of major <strong>rural</strong> <strong>development</strong> issues.Specific professional experience- M<strong>in</strong>imum 3 years experience <strong>in</strong> the analysis <strong>and</strong> evaluation of <strong>rural</strong><strong>development</strong> <strong>program</strong>s <strong>in</strong> Romania;- Number of projects on <strong>rural</strong> <strong>development</strong> monitor<strong>in</strong>g <strong>and</strong> evaluation<strong>program</strong>s <strong>in</strong> which the expert was <strong>in</strong>volved (m<strong>in</strong>imum 2);- Adequate <strong>and</strong> substantial knowledge of Romanian Program <strong>for</strong>Agriculture <strong>and</strong> Rural Development 2000-2006.TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 42
Key expert 6 – Agrarian/Rural <strong>development</strong> economist, expert II gradeQualifications <strong>and</strong> skills- Economical or technical superior studies <strong>in</strong> the field of agri-food or <strong>rural</strong><strong>development</strong>;- Specialization studies <strong>in</strong> <strong>agriculture</strong> <strong>and</strong> <strong>rural</strong> <strong>development</strong> economics;- High proficiency <strong>in</strong> the English language.General professional experience- Adequate knowledge of EU agricultural <strong>and</strong> <strong>rural</strong> <strong>development</strong> policies<strong>and</strong> of pre-<strong>accession</strong> f<strong>in</strong>ancial <strong>in</strong>struments.Specific professional experience- M<strong>in</strong>imum 3 years of professional experience <strong>and</strong> adequate knowledge ofagricultural <strong>and</strong> <strong>rural</strong> economy of Romania;- Number of projects on monitor<strong>in</strong>g <strong>and</strong> evaluation of <strong>agriculture</strong> <strong>and</strong> <strong>rural</strong><strong>development</strong> <strong>in</strong> which the expert was <strong>in</strong>volved (m<strong>in</strong>imum 2).Key expert 7 – Environment Impact Assessment, expert II gradeQualifications <strong>and</strong> skills:- Superior studies <strong>in</strong> the field of agrarian economics <strong>and</strong> environment;- Specialization studies <strong>in</strong> environment management;- High proficiency <strong>in</strong> English language.General professional experience- Adequate knowledge of the EU environmental policy <strong>and</strong> pre-<strong>accession</strong>f<strong>in</strong>ancial <strong>in</strong>struments <strong>for</strong> Central <strong>and</strong> Eastern European Countries;- Adequate knowledge of SAPARD Program <strong>in</strong> Romania.Specific professional experience- M<strong>in</strong>imum 3 years experience <strong>in</strong> the implementation of EnvironmentImpact Assessments (EIA) of agricultural <strong>and</strong> <strong>rural</strong> <strong>development</strong><strong>program</strong>s;TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 43
- Impact assessment experience <strong>in</strong> <strong>rural</strong> <strong>development</strong> project or <strong>program</strong>s(m<strong>in</strong>imum 2 years);- Number of projects on Environmental Impact Assessment (EIA) of <strong>rural</strong><strong>development</strong> <strong>program</strong>s, <strong>in</strong> which the expert was <strong>in</strong>volved (m<strong>in</strong>imum 2).Key expert 8 – Technical translator, expert III gradeQualifications <strong>and</strong> skills- Degree <strong>in</strong> <strong>for</strong>eign languages, specialis<strong>in</strong>g <strong>in</strong> English;- High proficiency <strong>in</strong> technical English.General professional experience- Adequate knowledge of project cycle management term<strong>in</strong>ology, as wellas European Union policies.Specific professional experience- M<strong>in</strong>imum 3 years of professional experience, preferably <strong>in</strong> <strong>in</strong>ternationalagricultural <strong>and</strong> <strong>rural</strong> economy projects (<strong>in</strong>clud<strong>in</strong>g <strong>in</strong>vestments, ecology,etc.);- Number of projects <strong>in</strong> which the expert was <strong>in</strong>volved (m<strong>in</strong>imum 2);The Contractor could select <strong>and</strong> employ other experts accord<strong>in</strong>g to thespecific contract activities. He will ensure the effective participation, wherepossible, of autochthonous specialised staff <strong>and</strong> suitable autochthonous or<strong>in</strong>ternational staff with<strong>in</strong> the project teams. Experts should not be <strong>in</strong> <strong>in</strong>terestconflict as regards their responsibilities under the contract.7. REPORTSDur<strong>in</strong>g the activities aimed at produc<strong>in</strong>g the first version, the draft f<strong>in</strong>alversion <strong>and</strong> the F<strong>in</strong>al Mid-Term Evaluation Updat<strong>in</strong>g Report (see po<strong>in</strong>t 7.3),the evaluation Team will also prepare three reports to <strong>in</strong><strong>for</strong>m theContract<strong>in</strong>g Authority of the work progress, the results achieved <strong>and</strong> theproblems eventually encountered (see po<strong>in</strong>ts 7.1 <strong>and</strong> 7.2).TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 44
7.1 Inception ReportAn Inception Report shall be submitted to the Contract<strong>in</strong>g Authority <strong>for</strong>approval <strong>and</strong> discussion with<strong>in</strong> 15 days after the contract start-up. TheInception Report will set out a detailed work plan, analys<strong>in</strong>g the dataavailability situation <strong>and</strong> present<strong>in</strong>g the data collection <strong>and</strong> analysismethodology.The contractor should carry out a need assessment <strong>and</strong>, together with thebeneficiary, prepare the technical specifications of the equipmentnecessary to be supplied.7.2 Activity ReportsThe Evaluator shall present to the Contract<strong>in</strong>g Authority one <strong>in</strong>terim activityreport <strong>and</strong> one f<strong>in</strong>al activity report. These reports shall describe theundertaken activities, results achieved <strong>and</strong> problems faced <strong>in</strong> the collectionof reliable data so that due measures can be adopted to solve theproblems. The reports shall also conta<strong>in</strong> detailed <strong>in</strong><strong>for</strong>mation on theorganized meet<strong>in</strong>gs, visits <strong>and</strong> trips.The <strong>in</strong>terim <strong>and</strong> the f<strong>in</strong>al reports shall be presented to the Contract<strong>in</strong>gAuthority, respectively at the end of the third <strong>and</strong> sixth month of activity.All reports, <strong>in</strong> Romanian <strong>and</strong> English languages, signed by Contractor,should be submitted to MAFRD-Manag<strong>in</strong>g Authority <strong>in</strong> four paper copies,two <strong>in</strong> English <strong>and</strong> two <strong>in</strong> Romanian, <strong>and</strong> two on electronic support (CD-ROM, 3½ Floppy Disk), one <strong>in</strong> English <strong>and</strong> one <strong>in</strong> Romanian.7.3 Mid-Term Evaluation Updat<strong>in</strong>g ReportThe first version of the Mid-Term Evaluation Update Report of the SAPARDProgram implementation <strong>in</strong> Romania should be submitted by the Evaluatorto the Manag<strong>in</strong>g Authority no later than 10 th of October 2005, <strong>for</strong> the firstdiscussion with the Contract<strong>in</strong>g Authority.The draft f<strong>in</strong>al version of the Mid-Term Evaluation Update Report should besubmitted by the Evaluator to the Contract<strong>in</strong>g Authority no later than 10 th ofNovember 2005, so that enough time should be allocated <strong>for</strong> exam<strong>in</strong>ationby the Monitor<strong>in</strong>g Committee <strong>and</strong> possible adjustment or completion of theReport.TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 45
The F<strong>in</strong>al Mid-Term Evaluation Updat<strong>in</strong>g Report must be submitted to theContract<strong>in</strong>g Authority no later than 10 th of December 2005.In case the Manag<strong>in</strong>g Authority or the Monitor<strong>in</strong>g Committee require orrecommend modifications to the reports mentioned <strong>in</strong> the previousparagraphs, the evaluator must <strong>in</strong>corporate them <strong>and</strong> resubmit therespective reports <strong>in</strong> no more than 5 days s<strong>in</strong>ce the request.The first version, the draft f<strong>in</strong>al version <strong>and</strong> the F<strong>in</strong>al Mid-Term EvaluationUpdate Report of the SAPARD Program implementation <strong>in</strong> Romania mustrespect the structure described at po<strong>in</strong>t 4.3.2.9 of these ToR <strong>and</strong> should besubmitted <strong>in</strong> 6 hard copies, 3 <strong>in</strong> English <strong>and</strong> 3 <strong>in</strong> Romanian, <strong>and</strong> four onelectronic support (CD-ROM, 3½ Floppy Disk), 2 <strong>in</strong> English <strong>and</strong> 2 <strong>in</strong>Romanian.The Management Authority, with<strong>in</strong> MAFRD, is responsible <strong>for</strong> theimplementation of the project <strong>and</strong> should ensure the supervision <strong>and</strong> thenecessary monitor<strong>in</strong>g.Telephone: 004-021-3072442 or 004-021-3078565Fax: 004-021-3078606E-mail: valent<strong>in</strong>.toma@maa.roContact person: Mr. Valent<strong>in</strong> Toma, Counselor8. MONITORING AND EVALUATION8.1 Evaluation criteriaThe assessment of the mid-term evaluation update report shall be basedon the ability of the evaluator to meet the requirements <strong>for</strong> evaluation reportcontents <strong>and</strong> evaluation process set <strong>in</strong> the Commission guidel<strong>in</strong>es:"Guidel<strong>in</strong>es <strong>for</strong> the evaluation of Rural Development Programs supportedby SAPARD" <strong>and</strong> "Guidel<strong>in</strong>es <strong>for</strong> the mid-term evaluation of <strong>rural</strong><strong>development</strong> Programs funded by SAPARD".The quality of the evaluation shall depend on the ability of the evaluator toprovide pert<strong>in</strong>ent <strong>and</strong> credible feedback from f<strong>in</strong>al beneficiaries, regionaldirectorates <strong>and</strong> <strong>in</strong>ternal audit unit of the SAPARD Agency, professionalassociations <strong>and</strong> non governmental organizations, local authorities <strong>and</strong><strong>development</strong> agencies.TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 46
The follow<strong>in</strong>g criteria <strong>for</strong> assess<strong>in</strong>g its quality should be applied based onthe Guidel<strong>in</strong>es of the European Commission (MEANS, Vol.1, page 179):-Meet<strong>in</strong>g the needs: Does the evaluation correspond to <strong>in</strong><strong>for</strong>mation needsexpressed by the Terms of Reference requirements <strong>and</strong> does it answer <strong>in</strong>an appropriate way to the common evaluation questions?-Methodology adequacy: is the analysis method consistent <strong>and</strong> sufficientlystrong to provide requested answer to the evaluation questions?-Relevant scope: Have the relevant element been fully exam<strong>in</strong>ed(pr<strong>in</strong>ciples <strong>and</strong> actions with<strong>in</strong> the Program, outputs, results <strong>and</strong> their impact<strong>in</strong>clud<strong>in</strong>g the <strong>in</strong>ter-action between them as well as the consequences of thepolicies established or unpredicted)?-Defensible design: Is the approach of evaluation adequate <strong>and</strong> appropriateso that the entire set of <strong>in</strong><strong>for</strong>mation <strong>in</strong>clud<strong>in</strong>g methodological limitationscould answer the evaluation questions?-Reliable data: To what extent the collected, elaborated <strong>and</strong> selectedprimary <strong>and</strong> secondary data are appropriate <strong>and</strong> offer a high degree ofreliability?-Sound analysis: Have the quantitative <strong>and</strong> qualitative <strong>in</strong><strong>for</strong>mation beenstudied properly <strong>and</strong> systematically so that correct answers should beprovided to the evaluation questions?-Credible results: Are the results logical <strong>and</strong> justified by the analysis basedon data collection <strong>and</strong> are they properly presented?-Impartial conclusions: Are the recommendations correct, un<strong>in</strong>fluenced byop<strong>in</strong>ions from personnel or <strong>in</strong>terests holders <strong>and</strong> are they sufficientlydetailed to be operational?-Clear Report: Is the Report describ<strong>in</strong>g clearly the evaluated Program<strong>in</strong>clud<strong>in</strong>g its framework <strong>and</strong> scope together with the procedures <strong>and</strong> resultsof evaluation so that the <strong>in</strong><strong>for</strong>mation can be easily understood?- Usefulness of the recommendations: do recommendations logically arisefrom the evaluations’ f<strong>in</strong>d<strong>in</strong>gs <strong>and</strong> conclusions? Are recommendationsuseful to the improvement of the f<strong>in</strong>al phase of the SAPARD Programimplementation <strong>and</strong> to the follow<strong>in</strong>g <strong>rural</strong> <strong>development</strong> <strong>program</strong>m<strong>in</strong>gperiod?TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 47
8.2 Special requirementsIn<strong>for</strong>mation <strong>and</strong> documents regard<strong>in</strong>g the results <strong>and</strong> other additionaldocuments, used, received or elaborated by the Contractor dur<strong>in</strong>g theproject implementation, l<strong>in</strong>ked to the SAPARD Program <strong>and</strong> actiondeveloped, should be classified with a high degree of confidentiality <strong>and</strong>should not be used by Contractor without the express agreement of theMAFRD-Manag<strong>in</strong>g Authority <strong>and</strong> the SAPARD Agency.The contractor should not issue any public declaration, or issue or presentany document concern<strong>in</strong>g MAFRD-Manag<strong>in</strong>g Authority <strong>and</strong> SAPARDAgency, on theirs own name or without previous agreement.TERMS OF REFERENCE - Updat<strong>in</strong>g of Mid-Term Evaluation of the SAPARD Program 48