The World Bank's Country Policy and Institutional Assessment (CPIA)

The World Bank’sCountry Policy andInstitutional AssessmentAn Evaluation

THE WORLD BANK GROUPWORKING FOR A WORLD FREE OF POVERTYThe World Bank Group consists of five institutions—the International Bank for Reconstruction and Development(IBRD), the International Finance Corporation (IFC), the International Development Association (IDA), theMultilateral Investment Guarantee Agency (MIGA), and the International Centre for the Settlement of InvestmentDisputes (ICSID). Its mission is to fight poverty for lasting results and to help people help themselves and theirenvironment by providing resources, sharing knowledge, building capacity, and forging partnerships in the publicand private sectors.THE INDEPENDENT EVALUATION GROUPIMPROVING DEVELOPMENT RESULTS THROUGH EXCELLENCE IN EVALUATIONThe Independent Evaluation Group (IEG) is an independent, three-part unit within the World Bank Group.IEG-World Bank is charged with evaluating the activities of the IBRD (The World Bank) and IDA, IEG-IFC focuses onassessment of IFC’s work toward private sector development, and IEG-MIGA evaluates the contributions of MIGAguarantee projects and services. IEG reports directly to the Bank’s Board of Directors through the Director-General,Evaluation.The goals of evaluation are to learn from experience, to provide an objective basis for assessing the results of theBank Group’s work, and to provide accountability in the achievement of its objectives. It also improves Bank Groupwork by identifying and disseminating the lessons learned from experience and by framing recommendations drawnfrom evaluation findings.

The World Bank’sCountry Policy andInstitutional AssessmentAn Evaluationhttp://www.worldbank.org/ieg2010The World BankWashington, D.C.

© The International Bank for Reconstruction and Development / The World Bank1818 H Street NWWashington, DC 20433Telephone: 202-473-1000Internet: www.worldbank.orgE-mail: feedback@worldbank.orgAll rights reserved1 2 3 4 13 12 11 10This volume is a product of the staff of the International Bank for Reconstruction and Development / The World Bank Group. The findings, interpretations,and conclusions expressed in this volume do not necessarily reflect the views of the Executive Directors of The World Bank or the governmentsthey represent.The World Bank does not guarantee the accuracy of the data in this work. The boundaries, colors, denominations, and other information shownon any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsementor acceptance of such boundaries.Rights and PermissionsThe material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation ofapplicable law. The International Bank for Reconstruction and Development / The World Bank encourages a dissemination of its work and willnormally grant permission to reproduce portions of the work promptly.For permission to photocopy or reprint any part of this work, please send a request with complete information to the Copyright ClearanceCenter Inc., 222 Rosewood Drive, Danvers, MA 01923, USA; telephone: 978-750-8400; fax: 978-750-4470; Internet: www.copyrights.com.All other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, The World Bank, 1818 HStreet, NW, Washington, DC 20433, USA: fax:202-522-2422; email:pubrights@worldbank.org.Cover image: Unisphere at Flushing Meadows – Corona Park. New York, New York. © Rudy Sulgan/Corbis. The Unisphere was built as the themesymbol for the 1964–65 World’s Fair.ISBN: 978-0-8213-8427-5e-ISBN: 978-0-8213-8429-9DOI: 1596/978-0-8213-84275Library of Congress Cataloging-in-Publication data have been applied for.World Bank InfoShopE-mail: pic@worldbank.orgTelephone: 202-458-5454Facsimile: 202-522-1500Printed on Recycled PaperIndependent Evaluation GroupCommunications, Strategy, and LearningE-mail: ieg@worldbank.orgTelephone: 202-458-4497Facsimile: 202-522-3125

ContentsviiixxixiiixixxxvxxviiAbbreviationsAcknowledgmentsForewordExecutive SummaryManagement Responsexxii Management Action RecordChairman’s Summary: Committee on Development Effectiveness (CODE)Advisory Panel Statement1 1 Introduction and Evolution of the CPIA4 Evolution in the Content of the CPIA8 Other Changes in the CPIA9 Role of the CPIA in IDA Allocation13 2 Relevance of the CPIA for Growth, Poverty Reduction, and EffectiveUse of Development Assistance15 The CPIA and Determinants of Sustained Growth25 The CPIA and Determinants of Poverty Reduction29 The CPIA and the Effective Use of Development Assistance31 Findings and Recommendations39 3 Reliability of the CPIA Ratings41 Comparability with Other Indicators48 CPIA Ratings Generation Process57 4 Findings and Recommendations59 Overview59 Main Findings62 Recommendations65 Appendixes67 A: 2008 Country Policy and Institutional Assessment Criteria71 B: Public Sector Literature Review81 C: 2007 CPIA Criteria on Economic Management, Structural Policies, andPolicies for Social Inclusion/Equity and Evidence in the Literature91 D: Literature Review on Aid Effectiveness93 E: Examples of Positive Impacts of Aid Projects from RandomizedEvaluations in Education, Health, Infrastructure, and Agriculture95 F: Review of Financial Sector Criterion99 G: Country Policy and Institutional Assessment (CPIA) andLoan Performancei i i

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n t103 H: Comparing Country Policy and Institutional Assessment Ratings bythe World Bank, African Development Bank, and Asian DevelopmentBank105 I: Comparator Indices of the CPIA109 J: Number of IDA and IBRD Countries for Which External Data are Available111 K: Comments by Network on Regional CPIA Rating Proposals, 2007113 Endnotes119 ReferencesBoxes4 1.1 Changes to the CPIA Criteria in 1998 That Reflect the Emphasis onInstitutions11 1.2 The Governance Adjustment in IDA’s Country Performance Ratings,FY1998–200819 2.1 Channels through Which Inequality Affects Growth44 3.1 Comparator Indicators50 3.2 The Process of Preparing CPIA Ratings51 3.3 The Network Reviews of CPIA RatingsFigure25 2.1 Relationship between Changes in PBA and the Ratio of Cluster D Ratingsto Ratings on Other Clusters for “Core IDA” CountriesTables5 1.1 CPIA Criteria 1998–200810 1.2 Evolution of IDA’s Performance-Based Allocation Formula and theAdjustment for Governance16 2.1 Mapping of the “Consensus” Determinants for Sustained Growth and theCPIA Criteria22 2.2 Correlations between Ratings of CPIA Clusters, 200722 2.3 Correlations between CPIA with Different Cluster Weights and CPIAwith Equal Cluster Weights, 200723 2.4 Simulation Results: Effects on Performance-Based Allocations for “CoreIDA” Countries Arising from a Larger Weight on the “Governance”Cluster Compared to Equal Weights on All Clusters33 2.5 Simulation Results: Changes in PBA from Dropping q834 2.6 Simulation Results: Changes in Trade Ratings Arising from Changesin Weights of Trade Subcriteria by Numbers and Shares of IBRDand IDA Countries35 2.7 Simulation Results: IBRD and IDA Countries That Would ExperienceChanges in Trade Ratings due to Changes in Weights for TradeSubcriteria37 2.8 Simulation Results: Changes in PBA Arising from Changes in Weights forTrade Subcriteria42 3.1 Rank Correlation Coefficients between CPIA Ratings and ComparatorRatings for 200745 3.2 Other Indicators—Expert Judgment or Hard Data?46 3.3 Are Other Comparator Indicators Closer to the Bank or to AfDB and ADB?47 3.4 Comparison of Changes in CPIA Ratings 2006–07 between the Bank,AfDB, and ADBi v

C O N T E N T S49 3.5 Rank Correlations between the CPIA and Other Indicators: IBRD versusIDA Countries52 3.6 Numbers and Shares of Initial Regional Rating Proposals on Which theNetworks Disagreed with the Regions, by Criteria, for 200753 3.7 Number and Share of Initial Regional Rating Proposals on Which theNetworks Disagreed with the Regions, by Region and IBRD and IDACountries for 200753 3.8 Network Disagreements with Initial Regional Rating Proposals54 3.9 Share of Instances Where Networks Prevailed When Networks Disagreedwith Regions over Proposed Increases in Ratings from 200654 3.10 Share of Instances Where Networks Prevailed When NetworksDisagreed with Regions, 200755 3.11 Shares of Countries on Which Networks Commented in 2007 WhenRegions Proposed the Same Ratings as in 2006v

AbbreviationsADBAfDBAML-CFTARPPBCPCGAPCPIACPRDACDBESIETIGCIGDPGNIpcGNPpcHIV-AIDSIBRDICPICRGIDAIEGIMFIPISRITMDGsNGOsNPLsNTBOECDOEDOLSOPCSPBAPRMEDQAGSMEsUNDPVATWGIAsian Development BankAfrican Development BankAnti-money laundering and combating the financing of terroristsAnnual review of project performanceBasel core principlesConsultative Group to Assist the PoorCountry policy and institutional assessmentCountry performance ratingsDevelopment Assistance Committee of the Organisation for Economic Co-operationand DevelopmentDoing BusinessEnvironmental Sustainability IndexEnabling Trade IndexGlobal Competitiveness IndexGross domestic productGross national income per capitaGross national product per capitaHuman immunodeficiency virus acquired immunodeficiency syndromeInternational Bank for Reconstruction and DevelopmentIDA country performanceInternational Country Risk GuideInternational Development AssociationIndependent Evaluation GroupInternational Monetary FundImplementation progressImplementation Status and Results ReportInformation technologyMillennium Development GoalsNon-governmental organizationsNon-performing loansNon-tariff barrierOrganisation for Economic Co-operation and DevelopmentOperations Evaluation Department (the previous name of IEG)Ordinary least squaresOperations Policy and Country ServicesPerformance-based allocationEconomic Policy and Debt DepartmentQuality Assurance GroupSmall and medium enterprisesUnited Nations Development ProgrammeValue-added taxWorldwide Governance Indicatorsv i i

Harvesting rice fields, Vietnam. Photo by Tran Thi Hoa/World Bank

AcknowledgmentsThis evaluation of the World Bank’s CountryPolicy and Institutional Assessment wasprepared by the Country Evaluation andRegional Relations division of the IndependentEvaluation Group.The evaluation was led by Helena Tang, theauthor of the report. Saubhik Deb carried outthe statistical and econometric analyses for thereport. Background papers were prepared byJulia Cage, Shailaja Fennel, Patrick Honohan, andBiaoyun Qiao. Inputs were provided by CarlaPazce, Rupa Ranganathan, and Dusan Vujovic.Production and logistical support was providedby Agnes Santos. Barbara Balaj copy edited thestudy for publication.The peer reviewers were Victoria Elliott, AartKraay, and Steven Webb. The team receivedguidance and support from the IEG managementteam, and also benefited from the commentsreceived from the various IEG units.The evaluation team extends its thanks to thestaff from all six Regions, the various networks,Operations Policy and Country Services, andResource Mobilization who were interviewed forthe report.Director-General, Evaluation: Vinod ThomasDirector, Independent Evaluation Group – World Bank: Cheryl GraySenior Manager, IEGCR: Ali M. KhadrTask Manager: Helena Tangi x

Woman in India. Photo by Curt Carnemark/World Bank

ForewordThe Bank’s Country Policy and InstitutionalAssessment (CPIA) assesses the conducivenessof a country’s policy and institutional frameworkto poverty reduction, sustainable growth, andthe effective use of development assistance. TheCPIA enters the calculation of country performanceratings that, since 1980, have been usedto allocate International Development Association(IDA) resources to eligible client countries.This evaluation was undertaken at the request ofBoard members to assess the appropriatenessof the CPIA as a broad indicator of developmenteffectiveness and as a determinant of the allocationof IDA funds. As indicated in the ApproachPaper, this evaluation reviews the effects of theCPIA ratings on IDA allocations but does notreview the IDA allocation formula itself.The evaluation finds that the CPIA contentbroadly reflects the determinants of economicgrowth and poverty reduction identified in theeconomics literature, but some criteria needto be revised (such as the trade criterion thatplaces much greater emphasis on imports thanexports) and streamlined, and one criterion(assessment of disadvantaged socioeconomiccategories other than gender) added. The CPIAratings also correlate well with ratings of similarindicators, and more so for International Bankfor Reconstruction and Development than forIDA countries. In part, this could be caused bythe CPIA exercise’s practice over the past severalyears of taking into account a country’s stageof development, which also means that theCPIA is no longer an index in the true sense ofthe word. It is difficult to establish an empiricallink between the CPIA and economic growthoutcomes, although CPIA ratings are found to bepositively associated with aid effectiveness in thenarrower sense—specifically, the performance ofBank loans.The CPIA’s 16 criteria are grouped into fourclusters—economic management, structuralpolicies, policies for social inclusion and equity,and public sector management and institutions.These clusters are weighted equally to derive theoverall CPIA rating. In contrast, the IDA allocationformula weights the clusters unevenly—the firstthree clusters are each given a weight of 8 percent,and the last cluster (the governance cluster) aweight of 68 percent and portfolio performancethe remaining weight of 8 percent. The literatureoffers no evidence to justify any particular set ofweights on the four clusters, whether in derivingthe overall CPIA rating or in calculating the IDAallocation. Neither is there justification for whythe clustering is as it is—having all social sectorscombined with the environment in one cluster,for example. There is also insufficient evidenceto conclude that the governance cluster associatesbetter with Bank loan performance than theother three clusters.The report lays out four recommendations:disclose International Bank for Reconstructionand Development ratings; discontinue the “stageof development” adjustment to the ratings;review and revise the content and clustering ofthe criteria; and discontinue the current aggregationof the criteria into an overall index.Vinod ThomasDirector-General, Evaluationx i

Girl in classroom, Mexico. Photo by Curt Carnemark/World Bank

Executive SummaryThe World Bank’s Country Policy and Institutional Assessment (CPIA)assesses the conduciveness of a country’s policy and institutionalframework to poverty reduction, sustainable growth, and the effectiveuse of development assistance. It plays an important role in the countryperformance ratings that have been used for allocating resources from theInternational Development Association (IDA) to eligible countries since 1980.The CPIA consists of 16 criteria grouped into fourclusters—economic management, structuralpolicies, policies for social inclusion and equity,and public sector management and institutions—weighted equally to derive the overall CPIArating. Since the beginning of fiscal 2009, IDAhas made transparent the weights of the clustersused in the IDA allocation formula—24 percenton the first three CPIA clusters combined and 68percent on the fourth (governance) cluster, withthe remaining 8 percent weighted on portfolioperformance. In other words, the governancecluster has eight and a half times the weight ofeach of the other three clusters in the formula.This has also made transparent the weak linkbetween the overall CPIA index and IDA allocations,with a country’s governance performance(particularly relative to its performance in theother clusters) being more important in thelatter.The content of the CPIA broadly reflects thedeterminants of growth and poverty reductionidentified in the economics literature. However,some criteria need to be revised and streamlinedand one criterion added. The literature offers noevidence to justify any particular set of weightson the four clusters used for IDA allocation orthe way the criteria are clustered (for example,having social sectors and environment in onecluster). The literature offers only mixed evidenceregarding the relevance of the content of theCPIA for aid effectiveness broadly defined—thatis, that it represents the policies and institutionsimportant for aid to lead to growth. However, theCPIA is associated with aid effectiveness definedmore narrowly as the better performance ofBank loans. But there is insufficient evidence toconclude that the most heavily weighted CPIAcluster associates better with loan performancethan the other three clusters.The CPIA ratings are generally reliable andcorrelate well with similar indicators, but it isdifficult to establish an empirical link between theCPIA and growth outcomes. Network reviewers’validation of ratings helps guard against potentialbiases in having Bank staff rate countries onwhich their work programs depend. The CPIAratings correlate better with similar indicatorsfor International Bank for Reconstruction andDevelopment (IBRD) than for IDA countries.This correlation could in part be because moreinformation is available on IBRD countries,and in part because the CPIA ratings are meantto take into account the stage of development(which is more pertinent for IDA countries andwhich means ratings for these countries aremore subject to judgment than those for IBRDcountries). This tendency is exacerbated by thedifferent practices with respect to accounting forthe stage of development, as none of the regionalx i i i

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n treviewers of the CPIA do this, whereas networkreviewers vary in their practices.The International Evaluation Group (IEG) makesfour recommendations. First, disclose the ratingsfor IBRD countries in the interest of accountabilityand transparency. Second, remove accountingfor the stage of development in the ratingexercise to reduce subjectivity. Third, undertakea thorough review of the adequacy of eachcriterion, including a review of experience andthe literature, and revise as necessary, based on,among other things, the findings of this evaluation.Fourth, consider not producing an overallCPIA index although continue to produce andpublish the separate CPIA components.OverviewThis evaluation takes the premise that beyondinforming IDA allocation, the CPIA is useful asa broad indicator of development effectiveness.It reviews the appropriateness of the CPIA asan indicator that assesses the conduciveness ofa country’s policies and institutions to fosteringpoverty reduction, sustainable growth, andthe effective use of development assistance. Itassesses the relevance of the content of the CPIAthrough a review of the economics literature. Italso assesses the reliability of CPIA ratings in twoways— through comparing CPIA ratings withsimilar indicators, and through reviewing theCPIA ratings generation process. Based on theseassessments, the evaluation derives recommendationsfor enhancing the CPIA.Relevance of CPIAThe contents of the CPIA are largelyrelevant for growth and poverty reduction.The CPIA criteria map well with the determinants—policiesand institutions—of growth andpoverty reduction identified in the literature,although some criteria can usefully be revisedand streamlined and one can be added (seerecommendations).The evidence is mixed regarding therelevance of the content of the CPIA foraid effectiveness as broadly defined in theliterature. Indeed, the review of the literatureindicates there is little consensus on the impactof aid on growth itself and on the conditionsunder which aid can have a positive impact ongrowth.However, the CPIA is associated with aideffectiveness in a narrower sense—that is,with respect to the performance of WorldBank loans. Empirical analysis finds that theoverall CPIA ratings are negatively associatedwith the share of problem loans that in turn iscorrelated with loan outcomes.Empirical analysis indicates that thereis insufficient evidence to conclude thatthe governance cluster associates betterwith loan performance than with theother clusters. Based on this finding, as wellas the lack of consensus in the literature on theconditions under which aid has an impact ongrowth, it can be surmised that the way in whichthe CPIA enters the formula for the allocationof IDA funds is driven much more by fiduciaryand possibly other concerns of donors than bythe objectives of achieving sustained growth andpoverty reduction.The CPIA strives to allow for countryspecificity, that is, that different sets ofpolicies and institutions can achievesimilar outcomes. However, there aresome pitfalls. The CPIA instructions to staffindicate that outcomes need to be taken intoaccount when assessing policies and institutions,which helps to account for countryspecificity. Indeed, outcome indicators areincluded in the assessment of some criteria.They could also be added to other criteria, inparticular trade.The trade criterion does not adequatelyallow for country specificity. The specificationof particular tariff rates for different ratingsreflects a one-size-fits-all approach to tradeliberalization that is not supported by countryexperience. Export performance (an outcomeindicator) needs to be included in the assessmentand would help to allow for countryspecificity.x i v

e x e c u t i v e s u m m a r yThe trade criterion also does not reflectthe importance of complementary institutionsfor successful liberalization. Thetwo-thirds weight on trade restrictiveness andone-third weight on trade facilitation is notsupported by country experience that showsthat at moderate tariff levels (which practicallyall countries currently have), complementaryfactors (macroeconomic stability and trade facilitation)are more important than further tariffreduction to promote integration into the globaleconomy.The CPIA is missing an assessment ofdisadvantaged socioeconomic groupsother than gender. Currently, only genderis being assessed with respect to equality, yetcountry evidence indicates that social exclusionof other marginalized groups could have severepoverty and growth implications.Important linkages among certain criteriaare not reflected in the CPIA. Except for thethree economic management criteria, all of theCPIA criteria are assessed independently. Thiscould be problematic in two instances. First, theassessment of trade liberalization needs to takeinto account the extent of intersectoral labormobility because liberalization in the absence oflabor mobility could exacerbate poverty. Second,fiscal policy needs to be assessed in conjunctionwith the quality of budgetary and financialmanagement to ensure that the fiscal condition ofthe country in its entirety is realistically captured.Reliability of CPIA RatingsThe Bank has made efforts over timeto improve the definition of the CPIArating scale to enhance the reliability ofthe ratings. These efforts have been aimed atreducing staff discretion in providing ratings.The CPIA ratings correlate well with similarindicators in terms of relative rankings ofcountries and direction of change. For eachof the 16 criteria, the rank correlation coefficientsof CPIA ratings with similar indicators averagebetween 0.7 and 0.8. Other indicators correlatebetter with the Bank’s CPIA ratings than withthose of the African Development Bank and theAsian Development Bank, the closest comparatorsto the Bank, as they use almost exactly thesame CPIA guidelines.CPIA ratings correlate better with similarindicators for IBRD than for IDA countries.This could be due in part to the greater amountof information available on IBRD than on IDAcountries, which increases the likelihood ofdifferent institutions having similar assessmentson IBRD countries. It could also be due partly tothe fact that the CPIA rating exercise takes intoaccount the stage of development (introducedsince 2004). This is more pertinent for IDAcountries and hence would subject ratings ofthose countries to more judgment in an exercisethat is already centered on staff judgment.Accounting for the stage of developmentin the CPIA ratings is problematic. Inaddition to the judgment involved, accountingfor the stage of development is also problematicbecause of the different practices adopted acrossthe Bank. Regional reviewers do not take thisinto account, whereas network reviewers vary intheir practices. Further, accounting for the stageof development means that the CPIA is no longeran index in the true sense of the word.The review process for the CPIA, which givesthe networks responsibility for validatingthe ratings, helps to guard againstpotential biases in ratings, although thereare exceptions. A major advantage of the CPIAexercise is having well-informed professionaljudgment of staff as the central determinant ofthe ratings. At the same time, however, havingstaff rate the countries on which their workprograms depend could lead to rating biases.Analysis of the 2007 review process indicates thatfor instances where the networks challengedthe regions’ initial proposals of a rating increasefrom 2006, the networks prevailed 73 percentof the time for IDA countries. They prevailedmore often—86 percent of the time—for IBRDcountries. However, these instances made uponly 6 percent of the ratings for IDA countriesx v

e x e c u t i v e s u m m a r ycluster, as they are currently assessed in morethan one criterion in this cluster.• Strengthening the assessment of the environmentcriterion and making the process moreefficient. Currently, staff need to answer 85questions for only one rating.• Reporting only one consolidated rating forthe three economic management criteria toavoid confusion.Fourth, consider not producing an overallCPIA index, and continue to produce andpublish the separate CPIA components.The overall CPIA index is not used as such forthe allocation of IDA funds. With respect to thebroader use of the CPIA as an index of policies andinstitutions, country specificity implies that theappropriate weights of the different clusters coulddiffer depending on a country’s initial conditionsand stage of development. Producing the differentcomponents of the CPIA without assigningweights to them to arrive at an aggregate indexwould allow for different weights to be appliedaccording to country contexts and use.x v i i

Students building houses in Kahyelitsha township, South Africa. Photo by Trevor Samson/World Bank

Management ResponseManagement welcomes the Independent Evaluation Group (IEG)report on the World Bank’s Country Policy and Institutional Assessment(CPIA). In management’s view the findings of the reviewinclude several useful insights that will contribute to further strengthenthe CPIA.The report makes four recommendations:• Disclose the ratings for International Bankfor Reconstruction and Development (IBRD)countries.• Remove accounting for the stage of developmentfrom the CPIA exercise, or, if this cannotbe done, clarify and justify in the guidelineswhich criteria should be subject to the adjustmentsand how the adjustments shouldbe made.• Undertake a thorough review of the CPIA andrevise the criteria as needed (the evaluationcontains recommendations regarding a fewspecific criteria, such as trade and financialsector).• Consider not producing an overall CPIA indexalthough continuing to produce and publishthe separate CPIA components. Except forthe recommendation on disclosing the CPIAratings for the IBRD countries, managementbroadly concurs with the recommendationsemanating from this evaluation.Relevance of the CPIA. The evaluation findsthat the contents of the CPIA are largely relevantfor growth and poverty reduction and that theymap well with the policies and institutions that areidentified in the literature as relevant for growthand poverty reduction. On the basis of a reviewof the literature, the IEG evaluation concludesthat there is little consensus on the impact ofaid on growth and poverty reduction and onthe conditions, including the role of policiesand institutions, under which aid can influencegrowth. The IEG evaluation finds, however, thatthe CPIA is associated with aid effectiveness in thenarrower context of the performance of WorldBank loans. Poor CPIA scores are correlatedwith the share of problem loans, which in turn iscorrelated with loan outcomes.CPIA criteria. The evaluation also containsrecommendations on a few CPIA criteria, suchas the criteria covering trade, the financial sector,and the equity of public resource use, whichIEG finds could be streamlined and revisited.Management considers these recommendationsuseful and intends to use them to informthe next review of the CPIA. After assessinggaps in coverage, the IEG evaluation notes thatthe CPIA is missing an assessment of disadvantagedsocioeconomic groups other than gender.Management intends to address this issue in thecontext of the CPIA review.Reliability. The IEG evaluation notes the effortsthe Bank has made over time to strengthen theCPIA and enhance the reliability of the scores. Itfinds that in terms of relative ranking and directionsof change, the CPIA scores are correlated wellwith existing indicators, but it notes that thecorrelations are higher for IBRD than for InternationalDevelopment Association (IDA) countries.The report also analyzes the process used bythe World Bank to generate the CPIA scores—ax i x

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tprocess in which the regions put forward a set ofproposals for country scores that are then subjectto review by the networks and central departments.IEG finds that this internal review processgives the networks an important role in validatingthe scores, helping to prevent potential biasin the scores and to address possible conflicts ofinterest. The review concluded that there is nostrong evidence of upward bias for either the IDAor IBRD country scores. Management welcomesthese findings and views them as useful inputs forfurther strengthening the CPIA process.General CommentsDisclosure of IBRD scores. The objective ofthe CPIA exercise is first and foremost to providean assessment of country performance that will beused to determine IDA allocations. To underscorethis point, by the suggestion of the Board, thesescores are disclosed as the IDA Resource AllocationIndex. IEG argues that disclosure of theIBRD scores is important from an accountabilityand transparency standpoint and will strengthenthe ratings. The report neither elaborates on theargument nor discusses trade-offs. Accountabilityand transparency are important, but there areother issues to consider.A major reason not to disclose the IBRD scoresis the possible effect on market perceptionsand credit ratings and associated financialconsequences for the countries concerned.Moreover, the Bank would not want to be seen asa credit rating agency. Unlike the scores for IDAcountries, the scores are not discussed or sharedby Bank staff with their IBRD counterparts; theIBRD country scores do not play a role in lendingdecisions, and their confidentiality limits theiruse. They have been used internally in analyticwork and by the Quality Assurance Group andIEG on portfolio-related issues. When the 2004external panel reviewed the CPIA and discussedthese issues, it leaned toward dropping the IBRDcountries from the exercise. IEG notes (chapter4) that the report recommendations are aimed atenhancing the CPIA beyond its use for IDA allocations,and that if the CPIA is viewed only in anIDA-allocation context, the need to include IBRDcountries can be questioned.Management disagrees with the recommendationto disclose the IBRD scores and prefersrestricting the coverage of the CPIA exercise tothe IDA-only countries. In the context of theforthcoming CPIA review, management willanalyze in more depth the value added and thecosts of preparing CPIA scores for IBRD countriesfor internal Bank uses, as well as other relevantaspects. The conclusions of this work will informmanagement’s decision on how to go forward,namely regarding the coverage of the CPIA, and,if warranted, the consideration of alternativeapproaches to disclosure. In the meantime theCPIA exercise will continue to cover the IBRDcountries.Accounting for development stage. TheCPIA guidelines state that staff may need to takeinto account the size of the economy and itsdegree of sophistication in their assessments.The criteria were developed so that higherscores could be attained by a country that,given its stage of development, has a policy andinstitutional framework that fosters growth andpoverty reduction. This approach recognizesthat in many areas, countries cannot be judgedby the same yardstick if they are at very differentstages of development. Some of the policyobjectives may be considered to be invariant toincome—for example, the desirability of havinga well-managed budget. But others depend, forexample, on the sophistication of the financialsystem (expectations regarding regulatorycapacity would be different for a high-incomecountry than for a low-income country) or onthe degree of urbanization. Social protection in alargely urban, formal economy (unemploymentinsurance, pensions, and so on) is fundamentallydifferent from the problem of protecting a poorrural subsistence economy from weather-relatedharvest shocks.The report raises a number of concerns regardingthe CPIA treatment of the stage of development.At the same time, the evaluation and therecommendations (including those concerningthe revision of the financial sector criterion)recognize that stage of development considerationsare important (appendix box F.1).x x

M a n a g e m e n t R e s p o n s eUnless this dimension is considered, some ofthe criteria scores may be linearly correlatedwith income—which is not the objective of theexercise. Controlling for a country’s stage ofdevelopment seems necessary, as what constitutesgood policy in many of the areas coveredby the CPIA is linked to stage of developmentas well as to country-specific characteristics. Thereport points out that accounting for the stageof development in the CPIA exercise may notalways have been uniformly applied. It suggeststhat, if the approach continues to be used,the guidelines should clarify and provide therationale for its use in specific criteria, showinghow the adjustment should be carried out indetermining the final scores. Managementagrees with this recommendation and in thecontext of the review of the CPIA will revise theguidelines accordingly.Review of the CPIA criteria. Periodic reviewsof the CPIA to update and refine the content ofthe criteria and the conduct of the exercise havebeen a mainstay of the CPIA’s history, and theyshould continue to remain so going forward. Butthese reviews should also be done at sufficientintervals so that the CPIA scores have somevalidity over time. Consensus on developmentthinking moves slowly. As the IEG report notes,these periodic reviews resulted in several breaksin the CPIA series, as some criteria were dropped,some were added, and some were revised. As thereport notes, the last major revision took placein 2004, informed by the recommendations of anexternal panel that undertook an in-depth reviewof the CPIA.The IEG report suggests that perhaps the timehas come for Bank management to undertake athorough review and revision of the CPIA. Managementgenerally concurs with this suggestion andplans a revision of the CPIA, to be completed bythe time IDA 16 is launched. Management wishesto point out, however, several important considerationsto take into account in planning thetiming of the review. First, in revising the CPIA,it is important to balance making the instrumentflexible enough to reflect new developments withmaintaining some stability in the criteria thatwill allow for comparisons of scores over time.Revisions will create another break in the CPIAseries, and, as in 2004, there will be substantial costin reworking the country scores and in explainingto the governments and external audiencesthe new criteria, the differences in relation tothe previous criteria, and the rationale for thechanges. Second, following the introduction ofa new set of criteria, changes in some scores donot necessarily reflect a deterioration or improvementin performance, but result from the changesin the criteria. Because the scores are used forIDA allocations, the revisions of the criteria couldresult in aid volatility. And third, the CPIA criteriaare used by other multilateral development banks,and management also intends to consult themthroughout the process of revising the instrument.Management would add, however, withrespect to the IEG report’s detailed recommendationson how some criteria could be revised, that itfinds these suggestions useful and intends to usethem to inform the next revision.Caveats. Although management broadly agreeswith the thrust of the findings of the IEG evaluation,it would like to point out that the reportcontains a few examples of statements—specifically,regarding the interpretation of some ofthe findings—that would have benefited fromfurther elaboration or qualification. Overall,management agrees with most of the IEGfindings and, with the exception noted above,accepts its recommendations. Management’sspecific responses to the IEG recommendationsare given in the Management Action Record.x x i

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tManagement Action RecordIndependentEvaluation Group (IEG)RecommendationsRequiring a ResponseDisclose ratings forInternational Bank forReconstruction andDevelopment (IBRD)countries.Remove accounting for thestage of development fromthe CPIA rating exercise.Undertake a thoroughreview of the CPIA andrevise the criteria asnecessary. This needs toentail a detailed review of theliterature for each criterionand needs to reflect the latestthinking on developmentand lessons learned. It alsoneeds to take into account therecommendations of IEG onspecific changes to the criteriathat were derived from theevaluation.Management ResponseDisagree. The objective of the Country Policy and Institutional Assessment (CPIA) exerciseis first and foremost to provide an assessment of country performance that will be usedin determining International Development Association (IDA) allocations. IEG argueswithout elaboration that disclosure of the IBRD ratings is important for accountability andtransparency and would further enhance the quality of the ratings. Whether “disclosure”will further the quality of the ratings is not self-evident. Accountability and transparencyare important in their own right, but there are other issues to consider. A major reasonnot to disclose the ratings is the possible effect on market perceptions and credit ratingsand the associated financial consequences for the countries concerned. IEG notes (chapter4) that the report recommendations are aimed at enhancing the CPIA beyond its use forIDA allocations. It suggests that if the CPIA is viewed only in an IDA-allocation context,the need to rate IBRD countries can be questioned. Management disagrees with therecommendation to disclose the IBRD scores and prefers to limit the coverage of theCPIA to the IDA-eligible countries only. Given that the IBRD scores are used internally bythe Bank, the forthcoming CPIA review will include a more in-depth analysis of the valueadded and the costs of preparing for internal uses CPIA scores for IBRD countries. Theconclusions of this work will inform management’s decision on next steps. In the meantimethe CPIA exercise will continue to cover the IBRD countries.Partially agreed. As the report notes (for example, chapter 2), there is relative consensusin the literature that there is no single recipe for growth and that country specificities,including the stage of development, need to be taken into account. Some of the policyobjectives may be invariant to income (for example, desirability of well-managed budgets),but others are not (for example, expectations regarding regulatory capacity in lowincomecountries versus middle-income countries; social protection in a largely urbanformal economy versus a poor rural subsistence economy). The IEG report suggests (therecommendations in the executive summary and chapter 4) that if accounting for thestage of development stage cannot be removed, then it is important to clarify in theguidelines which criteria should take into account the stage of development, what therationale is for doing so, and how the adjustments should be made. Management agreeswith this suggestion. Therefore, as part of the broad review of the CPIA (see below), theguidelines will be revised to clarify which criteria should be adjusted to account for stageof development and how the adjustment should be made.Agreed. Periodic reviews of the content and methodology have been a fixture of theevolution of the CPIA, and going forward they should continue to be. As the IEG evaluationrecognizes, these reviews create discontinuities, as some criteria are added, dropped, orrevised. The last major revision took place in 2004, informed by the recommendations of anexternal panel that undertook an in-depth review of the CPIA. Consensus on developmentthinking moves slowly, and revisions should be undertaken with sufficient intervals sothat the CPIA scores have some consistency over time. From the standpoint of countryrelations and aid volatility, it is also important to avoid situations where changes in scoresresult from modifications in the criteria rather than from a deterioration or improvementin country performance. The CPIA is used by other multilateral development banks and anextensive consultation process would be necessary. The IEG evaluation found that “perhapsthe time has come… for a thorough review of the CPIA” (chapter 2). Management broadlyagrees but underscores that such a review needs to be carefully planned and done in thecontext of IDA 16. The specific suggestions provided in the IEG evaluation will inform thisreview, to be completed by the time IDA 16 is launched.(continued on next page)x x i i

M a n a g e m e n t R e s p o n s eManagement Action RecordIndependentEvaluation Group (IEG)RecommendationsRequiring a ResponseConsider not producing anoverall CPIA index althoughcontinue to produce andpublish the separate CPIAcomponents.Management ResponseAgreed. Management will take this IEG recommendation into consideration in the contextof the review of the CPIA mentioned above. IEG’s rationale for this recommendation is thatproducing the different components of the CPIA without assigning weights to them in orderto arrive at an aggregate index would allow different weights to be applied according tocountry context and uses. In management’s view, in the absence of robust evidence asto what these weights should be, there is value in applying a uniform weighting schemeacross all countries and producing an overall index that summarizes the informationcontained in the different criteria and provides a clear reference point. Moreover, becausethe scores for all the criteria are disclosed, nothing prevents the users from creating analternative index based on their preferred set of weights. As part of the review of theCPIA, management will consider whether or not to produce an overall index.x x i i i

x x i vYoung boy sitting by centuries-old cistern, Hababa, Yemen. Photo by Bill Lyons/World Bank

Chairman’s Summary:Committee on DevelopmentEffectiveness (CODE)The Committee on Development Effectiveness (CODE) considered thereport The World Bank’s Country Policy and Institutional Assessment(CPIA) – An Evaluation, prepared by the Independent EvaluationGroup (IEG), and the draft Management Response. A statement by the externaladvisory panel on the IEG report was distributed as background documentfor the meeting.SummaryThe Committee welcomed the timely discussionof the IEG report, which confirms the usefulnessof CPIA as a broad indicator of developmenteffectiveness. The Committee noted thatthe CPIA is not only being used for allocation ofInternational Development Association (IDA)resources, but also for other purposes such asthe debt sustainability framework, for which anassessment on the impact of the CPIA review wasrequested. In this vein, there was an agreementthat the purpose of the CODE discussion was notto address the use of CPIA in the performancebasedallocation formula for IDA.The discussion focused on the four recommendationsin the evaluation. Members and managementbroadly agreed with IEG’s findings on thecontent of the CPIA and the recommendationto review the individual CPIA criteria. There wasextensive discussion about IEG recommendationto disclose International Bank for Reconstructionand Development (IBRD) ratings. Somemembers questioned the value added of disclosingCPIA for IBRD and stressed the importanceof consultations with the countries being ratedbefore further considering the matter. Othersendorsed the recommendation and the benefitsof disclosure for accountability and transparency,although they recognized the complexity of thisissue. There was general consensus that furtherreview and consultations would be neededwith a view to consider improving transparencyover time. Regarding “accounting for the stageof development” in the CPIA exercise, somemembers believed that this dimension shouldstill be incorporated in CPIA and supportedmanagement’s proposal to clarify the relevantstaff guidelines.Recommendations and Next StepsThe Committee recommended to managementthe following:The review of the CPIA should take intoaccount the comments and suggestions raisedat the meeting to enhance its quality. Thiswould include reviewing the CPIA criteriaas called for in the evaluation—for example,with respect to trade and finance, social andenvironmental components, and incorporationof criteria on disadvantaged groupsx x v

Advisory Panel StatementComments by Jürgen Zattler on some ofthe recommendationsDeputy Director General, Federal Ministryof Economic Cooperation and Development,GermanyThe Independent Evaluation Group (IEG)suggests removing accounting for the stagesof development from the Country Policy andInstitutional Assessment (CPIA) exercise:• It is much more difficult for a small fragile stateto account for all standards that the CPIA demandsthan for India. Hence, there would bean unfair treatment for less-developed countriesto receive a fair allocation.• Alternatively to accounting for stages of developmentby regional and network reviewers,there could be a more differentiated weightingof the various criteria. The most importantcriteria to fulfill for a least-developed countryin fragility should be weighed higher. Hence,fragile states can achieve a higher ratingquickly if they concentrate on the most urgentcriteria first. This measure also providesan incentive system to sequence measures fordevelopment.IEG recommends that it should be considerednot to produce an overall CPIA index althoughcontinue producing and publishing the separateCPIA components.• If the separate clusters should be weighed individuallyaccording to the individual countrysituation, then this would be in line with myproposal above to weigh criteria according totheir importance for development.IEG recommends a thorough review of CPIA andrevise criteria if necessary. This I can fully support.• Criterion 8 can be dropped or reformulatedpossibly measuring policies aimed at povertyreduction such as agriculture (as proposed inthe review) or even infrastructure.• Assessment of other marginalized socioeconomicgroups besides gender should bedefinitely integrated. In general, participationand minority protection could be integrated(possibly in the governance cluster).• There should be a separation of social sectorsand environment, possibly creating a separateenvironmental cluster with more differentiatedcriteria, but with a reduced number ofquestions for the reviewers.IEG suggests that there is no proof that the highweight of the governance cluster increases loanperformance.• Establishing good governance is one of thecore and most difficult tasks for a fragile stateor a least-developed country and managing todo so could be especially rewarded by weighingthe governance cluster higher.IEG suggests the disclosure of CPIA ratingsfor International Bank for Reconstruction andDevelopment (IBRD) countries to increasetransparency. I support this.• However, there might be many further issueswhere transparency and accountability canbe better addressed (such as publishing themargin of error, and increased use of externalsources for double-checking). The reviewcould have touched upon more issues.We would like to emphasize our support forthe contents of the articles in chapter 2 regardingrevising the trade and financial sectorcriteria.x x v i i

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tComments by K.Y.AmoakoExecutive Secretary,Economic Commission for AfricaUnited Nations Under-Secretary GeneralThe IEG report should provide a sound basis forstreamlining the structure, criteria and indicatorsof the CPIA to enhance its alignment to thegoals of economic growth, poverty reductionand development effectiveness. The report alsoprovides the basis to discuss where to positionthe CPIA’s process and results in the WorldBank’s toolkit for improving the effectivenessof its support for economic growth and povertyreduction.The recommendations for changes in the criteriafor trade to include exports and reduce the weightgiven to trade protection, and for the inclusion ofagriculture as a criteria in the CPIA, are welcome.These are particularly germane for growth andpoverty in Sub-Saharan Africa. On trade, indicatorsof export diversification and compliance withregional integration obligations would be useful.Indicators for the agriculture criterion shouldnot only focus on public expenditures on agriculture,but should also seek to reflect progress inresearch and extension services, adoption ofnew technologies, strengthening land tenure,provision of credit to farmers, as well as marketing,distribution and pricing issues.Expanding microcredit and developing microcreditinstitutions can help to enhance financial intermediationand to develop financial services andcontribute to the deepening of the financial sectorin general. Thus, the inconclusive evidence on thegrowth impact of microfinance notwithstanding,its place in the CPIA should be retained.The overarching nature of governance wouldjustify the large overweighting. Besides, forthose countries with long periods of poorgovernance, the potential impact of improvementsin governance may be large comparedto other clusters. However, the indicators in thegovernance cluster, particularly in q15 and q16,may not be the most relevant indicators to assessprogress in governance in low-income countries,particularly since governance challenges tendto be country specific. Furthermore, reliableinformation may not exist to make objectiveassessments, and staff of the World Bank may notpossess the required skills/competencies to makethe right calls on these issues that require deepappreciation of the political economies. There isa need for more work on the governance criteriato strengthen the relevance of governance indicators,identify gaps in information and take stepsto close those gaps. Work in this area would gainfrom the use on national and regional governanceexperts that are close to the scene.The recommendation that the Bank not producean overall CPIA index, although continuingto produce and publish the separate CPIAcomponent, is a good one. An aggregate index isnot likely to be a basis for informed policy discussionsand probably takes away the focus on thecomponent ratings, where debate and analysiswould be most useful.The report notes that “the strength of the CPIAratings is Bank staff professional judgment.” Thusthe process through which the Bank harvests itsconsiderable expertise for the CPIA is important.The evaluation report assumes that the processis fine. Nonetheless one may question whetherthe existing process, which could be viewedas overly bureaucratic, is best for tapping theexpertise in the World Bank. Other related issuesinclude the nature of consultations with governmentsand other informed stakeholders, supportfor economic and sector work and the qualityof the statistical information base. For lowincomecountries, the Bank is the main sourceof economic and sector analysis and support forstatistics development invariably depends onexternal assistance. Countries with a combinationof relevant World Bank staff with limitedexperience, limited recent economic and sectorwork and lack of good statistics, may end up withunreliable CPIA ratings.The CPIA is carried out every year. This couldbe too frequent as the policies, institutions andperformance do not change that rapidly. Furthermore,the annual revisions of the Internationalx x v i i i

A d v i s o r y Pa n e l S tat e m e n tDevelopment Association (IDA) allocationscannot be helpful to country programming bythe World Bank and budget planning by thegovernments. Although the CPIA does stimulatethinking about a range of development issues, itis not a substitute for detailed policy and institutionalanalysis that would help the countriesmake policy and build institutions. Is the CPIAcrowding essential country work in the environmentof constrained administrative budgets? Inparticular, there is the question of value additionof the CPIA for non-IDA countries and thus theneed for CPIA for non-IDA countries.Comments by Ravi KanburT.H. Lee Professor of World Affairs andEconomicsCornell UniversityI welcome this report on the CPIA. It is athorough assessment and it raises a number ofimportant issues that Bank management needsto address. Moreover, given the key role playedby the CPIA in the IDA allocation process, andin many analytical contributions to the developmentliterature, the report’s assessments are ofinterest to the broader development communityas well. By and large, I support the analysis andthe recommendations of the report. However, inmy comments I will highlight where I think theconclusions could have been much sharper.I will structure my comments around the fourprincipal recommendations of the report.First, disclose the ratings for IBRDcountries.I agree. But the report could call for moretransparency all around.One suggestion is that all previous ratings, IBRDand IDA, in all previous years, should be madepublic. There is no reason why this cannot bedone. This will allow analysts in general, and notjust Bank researchers, to analyze the relationshipsbetween the different components ofthe CPIA and development performance. Thedebate will serve to strengthen the review ofCPIA components that this report calls for (seebelow). I think the report should make this anexplicit recommendation (or subrecommendation)to get management’s response to it.A second suggestion is that when the ratings aredisclosed each year, the Bank should engage ina debate and discussion with local scholars andanalysts on a country’s ratings. A group of us didthis a couple of years ago in Ghana, with somesurprising results—some local scholars thoughtthe Bank was being too soft on some scores.A more radical option is to bring in local expertiseat the time of rating—perhaps in the form of astanding panel of distinguished country expertswho can provide their inputs to the Bank countryteam.Second, remove accounting for the stage ofdevelopment from the CPIA exercise.The central issue here is country specificity (seealso my comments on the third recommendationbelow). The conceptual foundation of theCPIA is a cross-country econometric regressionof a development outcome (usually growth butit could be a social indicator as well) against anumber of “right hand side” (RHS) variables.It is these RHS variables that the CPIA clustersand categories are meant to capture. But in anyregression there are points above and belowthe line. The question is, do these deviationscontain information, or are the deviations purelyrandom, with no information content whatsoever?The difficulty for a CPIA type exercise arisesbecause we think that there is indeed informationcontent in the deviations—that the “Bangladeshparadox” (why does a country with suchpoor governance ratings does so well on socialindicators?) is indeed a paradox.As noted in the report, the “stage of development”accounting is a way of trying to put backcountry specificity. The intention is good but, asdocumented by the report, the way it is done isnot. I support the recommendation to removeaccounting for the stage of development as itis currently done, but this still leaves open thex x i x

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tquestion of how country specificity is to bebrought in to the assessment (see below).Third, undertake a thorough review of theCPIA and revise the criteria as necessary.I support this recommendation strongly. Indeed,after this major review I would suggest somethinglike a cycle of three-year reviews. An alternative isto have a standing committee of external expertskeep a watch on the CPIA process, with a majorreview every three-to-five years to incorporatenew knowledge of the development process.By and large I support the specific subrecommendationsunder this category. However,I would like to highlight a point which, althoughit is present in the report, is not emphasizedenough. This is the importance of bringing inactual outcome variables in the CPIA. I haveargued elsewhere (Kanbur 2005) —that bringingin the evolution of outcome variables is oneway of factoring in country specificity that, forwhatever reason, is not easily captured by theCPIA variables (think again of the Bangladeshparadox). As noted in the report, some outcomevariables are already brought in to the CPIAassessment. The report itself argues for somemore outcome variables, for example when itrecommends “Revision of the trade criterionto include a subcomponent on exports thatevaluates performance as well as policies andinstitutions.”My basic point is that the major review of theCPIA that is recommended in the report mustexplicitly address the question of systematicinclusion of outcome variables in the assessmentas part of an overall investigation of how countryspecificity is to be brought into the assessment,which itself is part of the fundamental questionwhich the review must start with—”What observablevariables are good predictors of developmentperformance along the dimensions we areinterested in?”Fourth, consider not producing an overallCPIA index although continuing to produceand publish the separate CPIA components.I support this recommendation. It will thenrender transparent how different uses, forexample the IDA allocation process, weight thedifferent components. It will allow researchers totry out different weights for different purposesand advance the development debate in that way.But (see my comments on the first recommendation),in order for the research and the debate tobe comprehensive, the Bank should release allprevious ratings, component by component, forall previous years.To conclude, let me say again that I welcome thisreport and I trust Bank management will respondto it positively.x x x

Chapter 1Evaluation Highlights• This evaluation assesses the relevanceof the CPIA criteria and thereliability of the ratings.• The CPIA has evolved since its inceptionto cover 16 criteria in fourclusters.• Since IDA 12 the CPIA has beenused to allocate IDA funds with alarger weight on the governancecriteria—specifically, the governancecluster has 8.5 times theweight of each of the other threeclusters.

Introduction and Evolutionof the CPIAThe Country Policy and Institutional Assessment (CPIA) assesses thequality of a country’s present policy and institutional framework, with“quality” referring to the conduciveness of the framework to fosteringpoverty reduction, sustainable growth, and the effective use of developmentassistance (World Bank 2008b, p.1). It plays an important role in the countryperformance ratings (CPRs) that have been established annually since 1980as a basis for the allocation of resources from the International DevelopmentAssociation (IDA) to eligible countries.Although CPIA ratings were initiated and used forIDA allocation purposes, they can and are beingused for wider purposes. For example, the Bankuses CPIA ratings for other corporate activitiesincluding the Global Monitoring Report. Thisevaluation takes the premise that beyond informingIDA allocations, the CPIA is useful as a broadindicator of development effectiveness.Currently the CPIA consists of 16 criteria groupedinto four clusters, with each cluster having equalweight in the overall CPIA rating. The four clustersare: economic management (cluster A); structuralpolicies (cluster B); policies for social inclusionand equity (cluster C); and public sector managementand institutions (cluster D) (see appendix Afor a summary of the contents of each criterion).This is the first self-standing evaluation of theCPIA by the Independent Evaluation Group(IEG). Prior to this, IEG had undertaken a reviewof the CPIA in the context of a “Review of thePerformance-Based Allocation System” for itsIDA 10–12 Review in 2001. 1Since the 2001 IEG review, there have beenseveral developments and changes pertainingto the CPIA. These include two restructurings ofthe CPIA: in 2001 following the IEG 2 review, andin 2004 following an external panel review. 3 Theexternal panel review of CPIA ratings and methodologywas instituted by Bank management in thecontext of the discussions about broadeningthe disclosure of CPIA ratings for IDA-eligiblecountries. Other developments pertaining to theCPIA include IDA negotiations and the resultingchanges in the country performance ratingsused in the performance-based allocation (PBA)system for allocating IDA resources.This evaluation will address the followingquestions:3

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tThe evaluation assessesthe relevance of CPIAcriteria and the reliabilityof the CPIA ratings.1. What is the relevance of the CPIA criteriawith respect to the policies and institutionsthat are important for sustained growth,poverty reduction, and the effective use ofdevelopment assistance?2. How reliable are the CPIAratings (focusing on the mostrecently available ratings at the timeof this writing, the 2007 ratings, inreflecting such policies and institutionsin the countries concerned?Chapter 2 of this report will evaluate therelevance of the CPIA criteria with respect togrowth, poverty reduction, and the effective useof development assistance based on a review ofthe economics literature.Changes to the CPIA in Chapter 3 will address the second1998 increased emphasis question—the reliability of the CPIAon institutions. ratings in two ways. First, it willcompare the ratings of the variousCPIA criteria with those of other indicators thatmeasure similar criteria. Second, it will review theCPIA ratings generation process within the Bank.Chapter 4 will summarize the findings presentedin the previous chapters. Recommendationswill be drawn aimed at strengthening the CPIAas an indicator that represents the factors in thecountry important for sustaining growth, fosteringpoverty reduction, and the effectiveness ofdevelopment assistance.The remainder of this chapter presents a briefdiscussion on the following: (i) the evolution inthe content of the CPIA, including its relationshipwith the underlying development paradigm andIDA negotiations outcomes; (ii) other changes inthe CPIA; and (iii) the role of the CPIA in the IDAallocation formula.Evolution in the Content of the CPIAThe CPIA has been evolving since its introductiondue either to changes in the Bank’s thinkingwith respect to the development paradigm orto IDA negotiations, or to both. As stated by theBank, “The [CPIA] methodology has evolvedover time, reflecting lessons learned andmirroring the development paradigm”(WorldBank 2004d).Among the most prominent changes made to theCPIA were those introduced in 1998, the spiritof which has remained in place to date. Themost important of the changes was the greateremphasis placed on institutions. Criteria wereadded to the CPIA on the capacity to manageand implement policies, and existing criteriawere revised to include/emphasize institutionalaspects (box 1.1). Greater weight was given tothe public sector management cluster, which wasraised from 14 percent of the CPIA in 1997 to 20percent in 1998 (table 1.1). 4In addition, the Bank started emphasizingthat the CPIA assess countries’ policies andBox 1.1: Changes to the CPIA Criteria in 1998 That Reflect the Emphasis on InstitutionsTwo criteria were added to the macroeconomic cluster, “macroeconomicmanagement capacity” and “sustainability ofstructural reforms.” The latter evaluates the commitment of theauthorities to reforms and the support of such reforms from thesociety at large.The criterion legal and regulatory framework was renamedproperty rights and rule-based governance, and specific referenceswere added on contract enforcement, impartial judicialdecisions, time spent by businessmen negotiating with bureaucrats,and theft and crime that raise the cost of doing business.A specific reference to environmental regulations was addedto the environment criterion.The civil administration criterion was replaced by the criterionon accountability of the public service, with specific referencesadded regarding accountability mechanisms, and the voice andparticipation of the general public in public activities.Source: IEG, based on World Bank documents.4

I n t r o d u c t i o n a n d E v o l u t i o n o f t h e C P I ATable 1.1: CPIA Criteria 1998–20081998 2000 2004–08General MacroeconomicPerformance (0.05)Management of Inflation andCurrent Account (0.05)MacroeconomicManagement (0.08) (q1)MacroeconomicManagement andSustainability of Reforms(0.25)Fiscal Policy (0.05) Fiscal Policy (0.05) Fiscal Policy (0.08) (q2)Management of External Debt(0.05)Macroeconomic ManagementCapacity (0.05)Sustainability of StructuralReforms (0.05)Economic Management(0.20)Management of External Debt(0.05)Management andSustainability of theDevelopment Program (0.05)Economic Management(0.25)Debt Policy (0.08) (q3)Trade Policy (0.05)Foreign Exchange Regime(0.05)Trade Policy and ForeignExchange Regime (0.05)Trade (0.08) (q4)Financial Stability and Depth(0.05)Financial Stability and Depth(0.05)Financial Sector (0.08) (q5)Policies for Sustainableand Equitable Growth(0.40)Banking Sector Efficiency andResource Mobilization (0.05)Competitive Environment forthe Private Sector (0.05)Property Rights and Rule-Based Governance (0.05)Structural Policies(0.30)Banking Sector Efficiency andResource Mobilization (0.05)Competitive Environment forthe Private Sector (0.05)Structural Policies(0.25)Business RegulatoryEnvironment(0.08) (q6)Factor and Product Markets(0.05)Factor and Product Markets(0.05)Environmental Policies andRegulations (0.05)Policies and Institutions forEnvironmental Sustainability(0.05)(continued on next page)5

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tTable 1.1: CPIA Criteria 1998–20081998 2000 2004–08Pro-Poor Targeting ofPrograms (0.05)Gender (0.05)Gender Equality (0.05) (q7)Safety Nets (0.05) Equity of Public Resource Use(0.05)Equity of Public ResourceUse (0.05) (q8)Policies for ReducingInequalities(0.15)Policies for SocialInclusion/ Equity(0.25)Building Human Resources(0.05)Social Protection and Labor(0.05)Policies for SocialInclusion/ Equity(0.25)Building Human Resources(0.05) (q9)Social Protection and Labor(0.05) (q10)Poverty Monitoring andAnalysis (0.05)Poverty Monitoring andAnalysis (0.05)Policies and Institutionsfor EnvironmentalSustainability (0.05) (q11)Quality of Budget and PublicInvestment Process(0.05)Property Rights and Rule-Based Governance (0.05)Property Rights and Rule-Based Governance (0.05)(q12)Public SectorManagement(0.20)Efficiency and Equity ofRevenue Mobilization (0.05)Efficiency and Equity of PublicExpenditures (0.05)Public Sector Managementand Institutions(0.25)Quality of Budgetary andFinancial Management (0.05)Efficiency of RevenueMobilization (0.05)Efficiency of PublicExpenditures (0.05)Public SectorManagement andInstitutions(0.25)Quality of Budgetary andFinancial Management(0.05) (q13)Efficiency of RevenueMobilization (0.05) (q14)Quality of PublicAdministration (0.05) (q15)Accountability of the PublicService (0.05)Transparency, Accountability,and Corruption in the PublicSector (0.05)Transparency,Accountability, andCorruption in the PublicSector (0.05) (q16)Source: IEG, based on World Bank documents.Note: Weight of the criterion in parenthesis.6

I n t r o d u c t i o n a n d E v o l u t i o n o f t h e C P I Athe institutions that implement these policies,rather than development outcomes. Finally, theratings were to be given based on the “level”of the countries’ policies and institutions at thetime, rather than the changes in these policiesand institutions compared to the previous year,as had been done in the past. This in turn waspredicated on the assumption that the levelsof such policies and institutions were the maindeterminants of aid effectiveness.The focus on the public sector continued in 1999.This reflected the interest of IDA deputies duringthe IDA 12 replenishment exercise. They notedthat “accountability, transparency, the rule of lawand participation represent four major pillars ofgovernance that are critical to the developmentprocess and the effective use of IDA resources.”This was also indicative of the new thinking by theBank on public sector effectiveness. The weightof the public sector cluster was raised by another5 percent to 25 percent with the transferenceof the criterion property rights and rule-basedgovernance from the policies for sustainableand equitable growth cluster to the public sectorcluster.There was also a large increase in emphasis onsocial policies in 1999, reflecting the interest ofIDA deputies during the IDA 12 Replenishmentexercise. Two criteria were added to the socialpolicy cluster: equality of economic opportunityand building human resources. The addition ofthese two criteria increased the weight of thesocial policy cluster from 15 percent in 1998 to25 percent in 1999.In sum, the changes in 1998 and 1999 resultedin much greater emphasis on social policies andon the public sector in the CPIA. The weights forthese two clusters rose from 15 and 14 percent,respectively, to 25 percent each, and the weightfor the economic management cluster fell from 25to 20 percent, and that for the structural policiescluster fell similarly, from 40 to 30 percent.Adjustments continued to be made to the CPIAafter 1999 through 2003, although these wereless extensive than those made in 1998 and 1999,and the weights for the four clusters remainedunchanged. The few prominent changes duringthis period included the replacement of thecriterion on equality of economic opportunityby a criterion on gender in 2000. This effectivelyexcluded discriminatory effects by socioeconomicgroup (for example, by race, caste, andethnic group) from the assessment. Anotherchange was the replacement of the criterion onsocial safety net by a criterion on social protectionand labor. This broadened theassessment of the protection of thepoor beyond safety nets, and reflectedthe new social protection strategy thatwas launched by the Bank at the time(World Bank 2000b).The changes in 2001 included the addition ofan explicit reference to economic growth in theassessment of fiscal policy. In 2002, domesticdebt was included in the public debt criterion(formerly only external debt was covered),and other communicable diseases (in additionto human immunodeficiency virus/acquiredimmunodeficiency syndrome [HIV/AIDS], whichwas there already), was added to the buildinghuman resources criterion.Finally, the most recent and major restructuringof the CPIA occurred in 2004. It was based on therecommendations of an external panel reviewof the CPIA noted above. Several changes wereintroduced. The number of criteria was reducedfrom 20 to 16. This entailed collapsing fourcriteria into two 5 and dropping two that werecovered by other CPIA criteria. 6In 1999, emphasis onsocial policies wasincreased in the CPIA.The content of virtually all the criteria All the CPIA criteriawas revised. For example, more detailed were revised in 2004specification was provided on what and equal weights werewas being assessed under each of the given to each cluster.three criteria of the economic managementcluster. Greater emphasis was also placedon customs and trade facilitation for the tradecriterion. Assessment of gender disparities in politicalparticipation was added to the gender criterion.Tuberculosis and malaria were added to HIV/AIDSin the assessment of building human resourcescriterion, among other changes.7

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tIn 2004, the weight on theeconomic managementcluster was raisedfrom 20 to 25 percent,whereas the weight onthe social policy cluster(excluding environment)was reduced from25 to 20 percent.Although the CPIA isintended to assess policiesand institutions, outcomesalso affect ratings.Equal weights for each criterion were replacedby equal weights for each cluster. This raised theweight of the economic management cluster from20 to 25 percent (bringing it back to the 1997–98weight). The additional 5 percent weight given tothe economic management cluster was effectedby dropping the monitoring and analysis ofpoverty outcomes and impacts criterion fromthe social polices cluster. Although, ostensibly,the weight of the social policy clusterremained at 25 percent, it was onlybecause the environment criterionwas transferred to this cluster (fromthe structural policy cluster). In effect,therefore, the weight is lower forthe non-environment social criteria.Between 2004 and 2008, the CPIAcriteria and their weights remainedunchanged. 7In sum, over the past decade adjustmentshave been made to the CPIA with respect tothe content, the number of criteria, and theweights of the criteria. Notwithstanding theseadjustments, the coverage of the CPIA hasremained largely unchanged since the changesin 1998–2000, which introducedthe emphasis on the public sectorand social policies (with the latterincluding the gender criterion)(table 1.1).Other Changes in the CPIAMany changes have been made over time in thepreparation process of the CPIA ratings, someresulting from the findings of the IEG review of2001 and the external panel review of 2004. Thechanges are as follows.Benchmarking. Beginning in 1998, abenchmarking step has been introduced to theratings process to strengthen the comparabilityof country scores. This entails the introduction ofan initial phase in the CPIA preparation processof selecting and rating benchmark countries(IDA and IBRD), against which ratings for othercountries would be compared during the CPIApreparation process.The Bank’s six regions, the networks, and thecentral departments are all involved in theselection of a representative sample of countriesthat cover all the regions and include IBRD- andIDA-eligible borrowers. Both good and poorperformers are included in the sample, and theratings of these countries have a similar distributionas the overall distribution of CPIA ratings.The set of benchmark countries is reviewedevery year, taking into account the need to bothmaintain some continuity in the sample and torefresh it. The number of benchmark countrieshas increased from 11 in 1998 to 19 in 2008, withthe inclusion of additional IDA countries comprisingall of this increase. Over the same period,the share of IDA countries in the benchmarkinggroup has risen from 45 to 68 percent.Policies/institutions versus outcomes. Asmentioned earlier regarding the evolution in thecontent of the CPIA, as part of the 1998 restructuringof the CPIA, the Bank emphasized thatcountries’ policies and institutions are beingassessed rather than outcomes. Nonetheless,in the instructions to staff (the CPIA questionnaire)with respect to preparing the ratings, itwas clearly stated that Bank staff need to takeinto account country outcomes when assigningratings, a statement that has remained in eachof the CPIA questionnaires since then. Further,although most of the metrics and indicatorsspecified for the assessment of various CPIAcriteria are policies and institutions, for a fewcriteria outcome indicators were also included,in particular for the financial sector and forgender.Rating scale and definition. In 1998, the ratingscale was changed from five points to six points.In 2001, explicit definitions were provided forthe rating levels of 2, 3, 4, and 5 for each of theCPIA criteria (previously only the 2 and 5 ratinglevels were defined). In 2004, the definition ofrating levels was extended to rating levels 1 and6. Prior to 2004, a “6” rating was given for criteriathat had received a “5” rating for three or moreyears, and a “1” rating was given for criteria thathad received a “2” rating for three or more years(World Bank 2003a).8

I n t r o d u c t i o n a n d E v o l u t i o n o f t h e C P I AGuideposts. These were introduced in 2001and by 2005 were provided for each criterion.There have been both additions to and removalof guideposts since their introduction.Country context. Since 2004, following therecommendations of the external panel review,a specific instruction has been added in the CPIAquestionnaire that staff may need to take intoaccount “the size of the economy and its degree ofsophistication in implementing the guidelines.”Specific references regarding this point havebeen added to the criteria on the financialsector and social protection and labor.Written record. This requirement wasintroduced in 2001 for staff to provide writtenjustification to accompany their rating proposals.The practice has been maintained since then.Disclosure. At the start of IDA 12 in fiscal 2000, IDAinitiated the disclosure, in quintile format, of theCPIA and IDA Country Performance (ICP) relativeratings for IDA eligible-countries. Managementinstructed country teams of IDA-eligible countriesto discuss with each country’s authorities theircountry's CPIA and ICP ratings and the resultingcountry IDA allocation. The quintile-based ratingresults for the CPIA, its four clusters, the countryportfolio, and the quintile-based ICP rating werethen posted on the external World Bank Web site.On September 7, 2005, following the recommendationof the external review panel, the Boardapproved the disclosure of CPIA ratings for IDAeligible countries. For the first time in June 2006,IDA disclosed the numerical scores for all CPIAcriteria and the overall score for all IDA-eligiblecountries as the “IDA Resource AllocationIndex.” This index is a misnomer, though, as IDAresources are actually not allocated according tothis index, as will be discussed in the next section.Role of the CPIA in IDA AllocationThe role of the CPIA in country performanceassessments, and in turn in the allocation of IDAfunds through the PBA formula, has evolved overtime (see table 1.2). In particular, certain CPIAcriteria—specifically those in the cluster on publicsector management and institutions (referredto as the governance criteria)—have played agreater role in IDA allocation than others, andtheir role in IDA allocation has received increasedattention over the last decade.Beginning in 1998 with IDA 12, and in response toIDA Deputies’ suggestions, IDA allocations havebeen adjusted by the country’s performancein the governance-related CPIA Since 2004, the ratingcriteria and in procurement practices. exercise is meant toAccording to IDA, “The stress on take into account thegovernance has evolved over the past size of the countrydecade and was put in place by donors and sophisticationbecause of its importance for improvingthe development performance ofof its economy.partner countries and for mitigating fiduciaryrisks to aid funds” (World Bank 2006b, p. i.).The governance adjustment was first introducedin 1998 for IDA allocations in the form of agovernance discount. This was replaced bythe governance factor in 2001 to addressthe problem of discontinuity in allocationsunder the governance discount. For both thegovernance discount and the governance factor,the adjustment took into account the ratings ofthe CPIA governance criteria and thatof the procurement criterion of theAnnual Review of Project Performance(ARPP).In 2004, the number of CPIAgovernance criteria included in the governancefactor fell from six to five, as the 2004 restructuringof CPIA had removed one of the governancecriteria. This reduced the effective weight of thegovernance criteria in the country performancerating from 68 to 66 percent. (See box 1.2 for amore detailed discussion of the evolution of thegovernance adjustment.)Adjusting the country performance rating bythe governance factor rendered the allocationformula more complex. The CPIA governancecluster and the procurement flag from theARPP were double counted (see table 1.2). Theexponential multiplier (of 1.5) on the governancerating (to arrive at the governance factor) madeSince IDA 12, greaterweight has been given tothe governance criteriafor allocating IDA funds.9

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tTable 1.2: Evolution of IDA’s Performance-Based Allocation Formula and the Adjustment forGovernancePeriodElements in the performance-basedallocation formula(per capita)Country Performance RatingCPIAPortfolioperformanceratingGovernance adjustments in IDAallocation formula1991–96 GNPpc –0.25 , CPR 1.8 100% a 0% n.a.1997 GNPpc –0.125 , CPR 0.5 CPR < 2 100% b — n.a.GNPpc –0.125 , CPR 1.6 2 < CPR 2.91998–2000 GNPpc –0.125 , (CPR/3) 1.75GNPpc –0.125 , CPR 2 CPR < 3CPR > 380% 20% Governance discount (introduced fiscal2000):For countries with 3 or more ratingsof 2 or below out of the 6 CPIAgovernance criteria, and over 30% ofprojects with deficient procurementpractices (according to the ARPPrating), the CPR is cut by one-third.2001–08 GNPpc –0.125 , CPR 2 CPR = (0.8*CPIA + 0.2*ARPP) *governance factorFrom fiscal2009GNPpc –0.125 , CPR 5where governance factor = (governancerating/3.5) 1.52001–03: governance rating = averageof 7 governance criteria (6 CPIA criteriaplus procurement criterion of the ARPPportfolio rating).2004–07: governance rating = averageof 6 governance criteria (5 CPIA criteriaplus a 3-year moving average of theprocurement flag of the ARPP portfoliorating).Fiscal 2008: governance rating =average of 5 governance criteriaCPR = 0.24*CPIA A-C+ 0.68*CPIA DGovernance rating = average of 5+ 0.08*Portfolio performance rating c governance criteriaSource: IEG, based on various IDA reports.Note: ARPP = Annual Review of Project Performance; CPR = Country Performance Rating; GNPpc = Gross National Product Per Capita.a. From 1998 through fiscal 2007, this was represented by the projects at risk rating in the ARPP. Projects at risk consist of actual and potential problem projects. Ratings of actual problemprojects are done by task managers and reported in the implementation supervision reports. Ratings of potential problem projects are done by the Quality Assurance Group which looks ata number of criteria including the country’s history of failure rate, defined as over 50 percent unsatisfactory outcome ratings by IEG. Beginning in 2001, the measurement of procurementenhancement was improved to capture not only the timeliness of the procurement process but also its quality. Beginning in fiscal 2008, only actual problem projects were included in theportfolio performance rating for the CPR. Hence, from then, the procurement flag has been dropped from the governance factor.b. The CPIA included a portfolio performance element which made up 20 percent of the weight in 1994, 10 percent in 1995–96, and 7 percent in 1997.c. CPIAA-C refers to CPIA clusters A (economic management), B (structural policies), and C (policies for social inclusion/equity), and CPIAD refers to CPIA cluster D (public sector managementand institutions).The formula usedwas very complex.the calculation and interpretation ofthe performance rating more complex.The complexity of the allocation formula wasespecially problematic as IDA was taking stepsto be transparent about how its resources wereallocated, which led to the decision to discloseCPIA and country performance ratings beginningin June 2006 (see chapter 1, section on disclosure).In this light, at the Mid-Term Review of1 0

I n t r o d u c t i o n a n d E v o l u t i o n o f t h e C P I ABox 1.2: The Governance Adjustment in IDA’s Country Performance Ratings, FY1998–2008IDA introduced adjustments to the country performance ratingsused in the performance-based allocation system in 1998,under IDA 12. These adjustments initially took the form of agovernance discount. Specifically, IDA reduced the countryperformance rating in the allocation formula by one-third forthose countries with three or more highly unsatisfactory ratingsout of seven governance factors. Effectively, this reducedIDA allocations for those countries affected by the discounton average by half.The 7 governance factors were 6 CPIA criteria plus the country’sperformance on procurement practices, according to theARPP. The 6 CPIA criteria were: (i) management and sustainabilityof structural reforms; (ii) property rights and rule-based governance;(iii) quality of budget and public investment process; (iv)efficiency and equity of revenue mobilization; (v) efficiency andequity of public expenditures; and (vi) accountability of the publicservice. Ratings were considered to be highly unsatisfactory if theywere “2” or below for the CPIA criteria and in the case of ARPPprocurement criterion, if over 30 percent of projects had deficientprocurement practices.The governance discount produced a discontinuity effect at thepoint where the discount was triggered, with allocations droppingby one-half when only one criterion dropped from 2.5 to 2.0 (WorldBank 2001b, p. 4). Perhaps because of this, there were upwardpressures on the ratings at the cut-off point (World Bank 2002b,p. 4). IDA deputies were also concerned about the punitive biasof the governance discount, and the fact that it was not affectingall countries with weak governance (World Bank, 2002b, p. 4).To address these drawbacks, the governance discount wasreplaced by a governance factor in 2001 (World Bank 2002b, p. 4),which is equivalent to (governance rating/3.5). 1.5 The governancerating is derived from the country’s average rating for the sevengovernance criteria mentioned above, with 3.5 being the mid-pointof the rating. This governance factor is applied to the overallcountry performance rating. Under this new design, governanceperformance at all levels is taken into account in IDA allocations:countries that score above the mid-point on governance-relatedcriteria receive a premium, and those that score below receivea discount.There was still a discontinuity in allocation despite the replacementof the governance discount with the governance factor.Specifically, a one point drop in just one of the seven governancecriteria results in a 7.5 percent drop in the overall IDA rating, andin turn a 15 percent drop in the country’s allocation (World Bank2003b, p. 2).The seven governance criteria together had an effective weightof 68 percent in the IDA CPR. The effective weight fell slightly to66 percent (World Bank 2004a, p. 3) in 2004 with the restructuringof the CPIA criteria that removed one of the governance criteria(the one on management and sustainability of the developmentprogram).Adjusting the country performance ratings by the governancecriteria raised the dispersion of these ratings, as was intended byIDA to better differentiate the allocated resources depending onthe country’s quality of governance. The governance adjustmentalso raised the volatility of the country performance ratings. Theprocurement ratings, in particular, were more volatile than ratingsof the CPIA governance criteria. To address this issue, in 2004, IDA14 introduced a three-year moving average for the procurementratings (World Bank 2006a, p. 13).Source: IEG, based on IDA documents.IDA 14, IDA Deputies requested Bank managementto “…simplify the allocation formula andreduce unwarranted ratings volatility” (WorldBank 2006a).IDA deputies decided that, beginning with IDA 15(fiscal 2009), the country performance rating will besimplified to make the weights of the componentsmore explicit. Specifically, the country performancerating (CPR) will be changed to:CPR = (0.24 * CPIA A–C+ 0.68 * CPIA D+ 0.08*portfolio performance),where CPIA A–Crefers to the average of the ratingsof CPIA clusters A (economic management), B(structural policies) and C (policies for socialinclusion/equity), and CPIA Drefers to ratings ofCPIA cluster D (public sector management andpolicies). Correspondingly, the PBA formula waschanged from:1 1

The formula has beensimplified for IDA 15,making transparentthe relative weightsof the different CPIAclusters in the allocationformula—that is, that thegovernance cluster has8.5 times the weight ofeach of the other clusters.PBA = f(CPR 2 , population,GNIpc –0.125 )to PBA = f(CPR 5 , population,GNIpc –0.125 ),where GNIpc is gross national incomeper capita. The exponent on the CPRwas changed from 2 to 5 to maintainthe same dispersion of ratings andtherefore of allocations as before. 8These changes in the CPR (and associatedchange in the PBA) have made the IDA allocationformula more transparent—specifically that CPIAcluster D has 8.5 times the weight of each of theclusters A–C in the CPR. Yet, at the same time, bybreaking up the CPIA into the different constituentparts that are used in the CPR, the changeshave also made transparent the weakness of thelink between the overall CPIA index and IDAallocations.1 2

Chapter 2Evaluation Highlights• The CPIA covers the main determinantsof sustained growth andpoverty reduction, although somecriteria can be usefully revised andstreamlined and one added.• The evidence is less clear regardingthe relevance of the content of theCPIA for aid effectiveness broadly,that is, that it represents the policiesand institutions important for aid tolead to growth.• CPIA ratings are associated with anarrow definition of aid effectiveness,specifically the better performanceof Bank loans.• There is insufficient evidence toconclude that the governance clusterassociates better with loan performancethan the other clusters.• The effects of a larger weight ongovernance in the IDA allocationformula (compared to equal weightson each cluster) are not due just tothe governance rating but to howdifferent that rating is compared toratings on the other clusters.

Hong Kong SAR, China

Relevance of the CPIA forGrowth, Poverty Reduction,and Effective Use ofDevelopment AssistanceAccording to the Bank, the CPIA assesses the quality of a country’s presentpolicy and institutional framework, where quality refers to howconducive that framework is to fostering poverty reduction, sustainablegrowth, and the effective use of development assistance (World Bank 2008b).The review of economic literature (theoretical/conceptual as well as empirical) indicates that,by and large, the CPIA criteria pertain to policiesand institutions that are found to be importantfor sustained growth and poverty reduction (andwelfare more generally). The evidence is moremixed as to the criteria’s importance for aideffectiveness.The CPIA and Determinants ofSustained GrowthThe literature on the determinants of sustainedgrowth has undergone a significant evolutionduring the last 50 years. In the 1950s and 1960s,it was widely argued that long-run economicperformance depends on capital investmentand that raising savings through a “big push”(Rosenstein-Rodan 1943) would launch countriesinto self-sustaining growth or “take-off” (Rostow1960). In the 1980s, the literature begins toemphasize the importance of a good economicpolicy environment (Williamson 1990; WorldBank 1993) characterized by reduced tariffs,appropriate foreign exchange rates, and lowinflation. 1 Then, in the 1990s, the literatureemphasizes that these policies would have onlylimited impact in the absence of more fundamentalinstitutional reforms (World Bank 1998).Today, there is relative consensus in the literaturearound the idea that there is no single recipe forgrowth and that country specificities—includingthe country’s stage of development—needto be taken into account. 2 Of course, countriescan learn from each other, but no simple recipecan be pulled off the shelf to stimulate growth.Each country needs to learn through trial anderror what works for it (World Bank 2004e).This does not mean, however, that there are nogrowth determinants. What it does point to isthe need “…to identify the exact set of policiesand institutional changes needed to addressbinding constraints on growth, based on firstprinciples in each instance” (World Bank 2005).By and large, this evaluation finds that the CPIAcovers those growth determinants over which thereis relative consensus in the literature.These are: institutions and governance;education; productivity and technologicalinnovation; and equity and equalityof opportunity (table 2.1).The CPIA coversdeterminants of growthfor which there isrelative consensus.1 5

R e l e va n c e o f t h e C P I Aone exception is corruption, where some earlierliterature (from the mid-1960s to the mid-1990s)posits that corruption can have a positiveimpact on growth in instances where there arepre-existing policy distortions such as pervasiveand cumbersome regulations, in which casecorruption can help efficiency and growth. Butall of the literature from the mid-1990s onwardhas found that corruption has a negative impacton growth (appendix B).The institution and governance indicators identifiedin this literature are covered in three of thefive governance indicators under the publicsector management and institutions cluster ofthe CPIA. These are the criteria on propertyrights and rule-based governance (q12), thequality of the bureaucracy (q15), and transparency,accountability, and corruption in thepublic sector (q16).The importance of institutions goes beyondthese indicators. In particular, the institutionalcontext within which policies are formulatedis also important. For example, regardingmacroeconomic policies, it is not just low andstable inflation that is important, but the convictionof the private sector that low and stableinflation is a permanent feature of the economicenvironment. The latter requires an appropriateinstitutional underpinning for price stability(World Bank 2005; Montiel and Servén 2006).In the fiscal arena, an appropriate institutionalsetting needs to also ensure transparency,sustainable solvency, flexibility, and a pro-growthstructure of the budget. The institutional aspectsof macroeconomic (and fiscal) policy are coveredin the macroeconomic management criterion(q1), the fiscal policy criterion (q2), the qualityof budgetary and financial managementcriterion (q13), and the efficiency of revenuemobilization criterion (q14) of the CPIA.Although there is adequate coverage of the policy(q2) and institutional (q13) aspects of fiscalmanagement, an issue arises over the coordinationof the assessment of these two criteria (inthe Bank they are assessed by different groups).This is particularly pertinent for low The evidence onincome countries which may perform the impact of mostwell on the macro/fiscal stability front, institutional indicatorsyet have weak fiscal management on growth is positive.capacity. In such cases, a good ratingfor q2 needs to be tempered by anappropriate rating for q13 in order that the fiscalaspect of the country in its entirety is realisticallycaptured.Similarly, finance depends on institutions, includinginformational and regulatory institutions,institutions that strengthen creditor rights,contract enforcement, and accounting practicesand the legal and judicial framework (Levine,Loayza, and Beck 2000; World Bank 2005). Theseinstitutions are covered in the CPIA criteria onthe financial sector (q5) and on property rightsand rule-based governance (q12).Human capitalHuman capital—and in particular education—is one of the main determinants of sustainedgrowth around which there is consensus inthe literature. In particular, the link betweenprimary enrollment and subsequent growth iswell established in the literature (see appendixC). In addition, improvements in secondary andtertiary education systems are also important,depending on the stage of development of thecountry. 3 Education is adequately covered in thebuilding human resources criterion (q9) of theCPIA, which includes assessment ofboth basic and post-basic education.Although the importance of educationon growth is clearly and stronglysupported by evidence, the evidenceon the impact of health on both the level ofeconomic development (per capita incomes) andeconomic growth is less conclusive,mainly because population increasesthat result from better health have anegative effect on per capita income(see appendix C). Nonetheless, itis clear that health is important forwelfare (the non-income dimensionof poverty—see discussion later in thischapter).The institutional contextin which economicpolicies are formulatedis important.Although education isclearly important foreconomic growth, theevidence of health oneconomic growth ismore mixed— althoughhealth is clearlyimportant for welfare.1 7

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tInvestment, productivity, and technologicalinnovationIt is widely acknowledged among economiststhat strong, enduring growth requires high ratesof investment (World Bank 2008a; Aghion andHowitt 2009). All of the different growth theorieshave investment of one type or another drivinggrowth. 4 Both private and public investments areimportant. Further, savings is equally important;indeed, the evidence shows that there is no caseof a sustained high investment (and high growth)path that is not backed up by high savings (withthe latter aided by fiscal prudence) (World Bank2008a).The CPIA covers the Private investmentmultiple elements Fostering private investment requiresnecessary to foster reducing risks for private investors,private investment. through stable fiscal and monetarypolicy, stable investment regimes,sound financial systems, and a clear and transparentbusiness environment including flexibility ofthe labor market. It also requires ensuring therule of law, and measures to fight corruption(World Bank 2001a).The CPIA covers all these elements importantfor private investment in several of its criteria,including those on macroeconomic management(q1), fiscal (and debt) policies (q2 andq3), the financial sector (q5), business regulatoryenvironment (q6), property rights andrule-based governance (q12), and transparency,accountability, and corruption in thepublic sector (q16).Public investment, All of the above criteria are conceptuallydistinct except for macroeco-including forinfrastructure, is nomic management, fiscal, and debtcovered in the criterion policies. Debt policies are clearly anfor fiscal policy. intrinsic part of fiscal policies, andfiscal policies in turn are clearly alsoan intrinsic part of macroeconomic management.Indeed, the macroeconomic managementcriterion refers to public spending. It alsorefers to monetary/exchange rate policies aimedat price stability, which cannot be achievedwithout taking into account fiscal policies at thesame time. Thus, it appears that the existingthree macroeconomic criteria can be usefullyassessed as one.Public investmentPrivate investment needs to be complemented bypublic investment to enhance competitiveness andcreate new market opportunities. Complementarypublic investment in expanding infrastructure andcommunications and in upgrading the skills of thelabor force is particularly important (Easterly andRebelo 1993; World Bank 1994; Sachs 2005, 2008;Collier 2007; World Bank 2008a).In fast-growing Asia, for example, public investmentin infrastructure accounts for 5–7 percentof gross domestic product (GDP) or more. InChina, Thailand and Vietnam, total infrastructureinvestment exceeds 7 percent of GDP.History suggests that this is the correct orderof magnitude for high and sustained growth,although it is difficult to be precise (World Bank2008a). Finally, public investment can be used asa tool to increase equality of opportunity, whichis another determinant of sustained growth,discussed next. The CPIA covers public investmentin it criterion on fiscal policy (q2), wherethere is an explicit reference to “the provisionof public goods, including infrastructure….consistent with medium-term growth.”Equity and equality of opportunityDevelopment economics has seen a major shift inview regarding the role of inequality on growth,from one that initially saw increases in equalityas a natural accompaniment to development(Kuznets 1955) or actually facilitating development(through the incentives it provides)(Lewis 1954) to the current view that inequalityis detrimental to growth (Todaro 1997; Aghion,Caroli, and García-Peñalosa 1999; Bardhan 2000;Hoff and Stiglitz 2001). This view is supportedby the results of several empirical (Alesina andRodrick 1994; Perotti 1992, 1993, 1996; Perssonand Tabellini 1994) and other studies (box 2.1).Equality of opportunity is an important element,and indeed the starting point, of equity. 5 Systematicdenial of opportunities to a group becauseof its ethnicity, religion, caste, or gender could1 8

R e l e va n c e o f t h e C P I ABox 2.1: Channels Through Which Inequality Affects GrowthInequality can have adverse consequences on efficiency, andhence growth, through various channels. Inequality of wealthaffects investment in physical and human capital. A better distributionof wealth reduces credit constraints; broader availabilityof credit has a significant and positive effect on growth (Perotti1992; Bardhan 2000; World Bank 2005). Micropanel studies showthat households with few physical and human assets are oftencaught in a poverty trap that sharply reduces their chance ofeconomic advancement and thus harms the overall economicperformance of the economy (Christiaensen, Demery, and Paternostro2002; Woolard and Klasen 2005).Inequality often induces more political instability, as well ascrime and insecurity of property rights, all of which depress investmentand productivity growth (Alesina and Perotti 1996; Bardhan2000). Inequality (in the form of unequal access to investment opportunities)can lead to macroeconomic volatility (Aghion, Caroli,and Garcia-Penalosa 1999), which in turn has been found to reducegrowth (Hausmann and Gavin 1996; Breen and Garcia-Penalosa2005). Finally, too much inequality may also lead to social tensionexpressed through violent redistribution, which has a negativeimpact on growth (Bourguignon 2004).Source: Cage (2009a)undermine social peace and spark politicalunrest (World Bank 2008a). There is evidencethat gender inequality—particularly in access toeducation—reduces economic growth as it failsto make adequate use of female resources (WorldBank 2001a; Klasen 2002; Knowles, Lorgelly, andOwen 2002; Klasen and Lamanna 2003).The literature has provided some measures thatcan improve equality and equality of opportunity.These include strengthening property rightsover land (Besley and Burgess 2003), expandingaccess to education (Bardhan 2000; Dreze andSen 2002; Chhibber and Nayyar 2007), and meanstestedincome transfers. 6 Strengthening propertyrights over rural land has resulted in higheragricultural productivity and output in China (Lin1992) and India (Banerjee, Gertler, and Ghatak2002); and strengthening land rights in urbanareas can help poor households gain access tocredit (De Soto 2000; Field 2002). Redistributionof land has played an important role in fosteringeconomic growth; 7 today, such reforms wouldtake the form of subsidized transactions in theland market (Bardhan 2000; Bourguignon 2004).Access to credit is also important for reducinginequality (Aghion, Caroli, and Garcia-Penalosa1999; Bardhan 2000; Besley and Burgess 2003;Beck, Demirguc-Kunt, and Levine 2004; Bourgui-gnon 2004). There is a debate, however, as to howimportant microfinance is compared with overallfinancial development (as measured by privatesector credit as a share of GDP intermediatedthrough the formal banking Equality of opportunitysector). There are individual success is an important startingstories of microfinance, including from point for equity.impact assessments that show microfinancein general helps the poor, although all participantsmay not benefit equally. 8 However, otherstudies find that overall financial development(measured by financial depth) has had a larger andmore certain impact on growth andpoverty reduction than the expansionof microfinance (Honohan 2004b; Beck,Demirguc-Kunt, and Levine 2004).The CPIA covers the measures for equity (redistribution)and for equality of opportunity identifiedin the literature. These include property rights(q12); access to credit (q5); access to education(q9); gender equality (q7); and income transfers(q10) (table 2.1). The financial sector criterioncovers both financial depth and microfinance.Thus, it covers all relevant ground irrespective ofwhether microfinance is important.There are two issues related to the equity andequality aspect of growth that are important forthe CPIA. The first is that only gender issues areAlthough access to creditis clearly important,the importance ofmicrofinance is less clear.1 9

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tincluded in the assessment. Concerns related toother socioeconomic groups that are discriminatedagainst (due to race, caste, ethnic group) arenot included. Yet evidence indicates that povertycan have a strong ethnic dimension in somecountries (Bodewig and Sethi 2005). This impliesthat tackling social exclusion of such groups isimportant not only for reducing poverty, but alsofor raising growth for the country as a whole.Although equity is The second issue is whether aimportant, it is already criterion on equity of public resourcecovered in other CPIA use is needed. The criterion has twocriteria and in ways subcomponents: public expendituresmore amenable to (66.6 percent weight) and revenueassessment than the collection (33.3 percent weight) thatcriterion on equity of affect the poor. According to thepublic resource use. relevant network’s review team, theassessment of public expendituresfocuses on spending on education,health, rural infrastructure, and safety nets. Ofthese, education and safety nets have been identifiedin the literature as being important and,as mentioned in the previous paragraph, theyare already covered by two other CPIA criteria,q9 (education) and q10 (safety nets). As forthe other two items, spending on health is alsocovered in q9, and spending on rural infrastructurecould be explicitly mentioned in the fiscalpolicy criterion (which already mentions publicspending on infrastructure).Integration into the global According also to the network revieweconomy is important team, the public revenue subcomponent,which focuses on whetherfor growth, but the CPIAtrade criterion does not taxes are progressive or regressive, isallow for flexibility in the very difficult to assess. Such an assessmentneeds to be based on incidenceapproach or give adequateattention to exports. analyses, which are typically notundertaken for many countries (or atbest undertaken sporadically), so that results aregenerally outdated even if existent. In other words,Bank staff does not have enough information tomeaningfully rate this subcomponent. The lack ofinformation is even more acute for IBRD countries,as the issue is not as important for them.In sum, the public expenditure subcomponent ofthe equity of resource use criterion is capturedby other CPIA criteria, although there is notenough information to meaningfully rate thepublic revenue subcomponent. At the same time,some socioeconomic groups that are discriminatedagainst are not included in the assessmenton equity and equality of opportunity.The recommendation is to replace the criterionon equity of resource use with a criterion onequity and equality of opportunity for othersocioeconomic groups. Alternatively, there canbe a reformulation of the criterion on equity ofresource use by, among other things, incorporatingan assessment of other socioeconomic groups.Integration into the global economyIntegration into the global economy is awidely accepted determinant of growthalthough there is considerable debate in theliterature on how this can be achieved. Theexperience of the 1990s demonstrates thatthere are many possible ways to integrateglobally (World Bank 2005). The challenge isfor policy makers to identify which best suitstheir country’s political economy, institutionalconstraints, and initial conditions. Some analystsare in favor of granting temporary modest levelsof import protection to emerging industrieswhere there is a demonstrated need (Williamson2004). Others have focused on choosing the rightform of protection, advocating subsidies to initialentrants rather than the use of import duties.Indeed, when tariffs (the reduction of which isthe most common policy prescription for tradeopenness) are tried as an explanatory variablefor growth, they are not found to be statisticallysignificant (Rodrik 2000).The experience of the 1990s also indicates thattrade reforms need to be part of a comprehensivegrowth strategy in order to be successful.Efforts to promote exports would need to bepart of such a growth strategy, as the experiencealso shows that the successful liberalizers eitherexplicitly or implicitly promote export growth(World Bank 2005). Many complementaryfactors are needed for export growth, the mostimportant of which are macroeconomic stabilityand the building of trade-related infrastructureand institutions.2 0

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tinstitutional or policy priorities. Taking this intoaccount would require larger weights to be appliedto those criteria/clusters that are more important(or are the “binding constraints”) toThe four CPIA clusters growth. The Global Competitivenessare highly correlated, Index has, as of 2009, taken steps inso the weighting scheme this direction by applying differentdoes not matter much weights (derived based on econometricfor representing the analysis) to different components of theoverall policies and index for countries at different stages ofinstitutions of a country, development (World Economic Forumbut they do matter for the 2008). Currently, the application ofallocation of IDA funds. equal weights to each of the four CPIAclusters does not allow for this aspect ofcountry specificity. Chapter 3 will discuss the waysin which the different aspects of country specificityare addressed in the CPIA rating exercise.Weighting scheme of the CPIACountry specificity aside, the question of how toweight the various criteria in the CPIA has alsodrawn a lot of attention because of the muchgreater weight given to cluster D (the governancecluster) in the formula for IDA allocation (seechapter 1). There are three main observationspertaining to weighting.First, the CPIA ratings for the four clusters arerelatively highly correlated (table 2.2), withthe correlation coefficients ranging from 0.65between clusters A and B to 0.88 between clustersC and D. This implies that countries that performwell on one cluster generally perform well on theother clusters.Second, it follows from the relatively highcorrelations that the weighting scheme used forthe CPIA does not matter very much in termsof representing the overall policies and institutionsof a country. This can be seen from therelatively high correlations between an unequallyweighted CPIA (no matter which cluster gets thegreater weight) and a CPIA with equal weights onthe four clusters (table 2.3).Third, different weights on the CPIA clusters do,however, and very importantly, matter for theallocation of IDA funds, as discussed below.As an illustration, this evaluation undertooksimulations to compare the PBA under a weightingscheme of equal weights for each cluster(as is done for the CPIA overall country score)and a scheme of greater weight on governance(as is done in the PBA formula). The simulationreplaced the country performance ratings (CPR)in the PBA formula with equal weights on thefour clusters, holding all the other factors thataffect the PBA constant. 12 The simulation wasperformed on “core” IDA countries—that is,those that are not subject to exceptions to thePBA due to post-conflict or re-engaging status orto caps on allocations. 13The simulation results indicate quite substantialchanges to the PBA of countries (table 2.4).Although the simulations are based on data onTable 2.2: Correlations between Ratings of CPIAClusters, 2007CPIA ACPIA BCPIA CCPIA overallCPIA A1.00 0.65 0.71 0.87CPIA B0.65 1.00 0.77 0.89CPIA C0.71 0.77 1.00 0.91CPIA D0.73 0.84 0.88 0.94Source: IEG.Table 2.3: Correlations between CPIAwith Different Cluster Weights andCPIA with Equal Cluster Weights, 2007Cluster that has thegreater weight (as inPBA formula)Correlation with overallCPIA (equal-weightedclusters)Cluster A 0.93Cluster B 0.95Cluster C 0.96Cluster D 0.97Source: IEG.2 2

R e l e va n c e o f t h e C P I ATable 2.4: Simulation Results: Effects on Performance-Based Allocations for “Core IDA”Countries Arising from a Larger Weight on the “Governance” Cluster Compared toEqual Weights on All ClustersChange in performancebasedallocation (%)Rating of governancecluster (cluster D)Average ratingof clusters A to CRating of cluster D as ashare of average rating ofclusters A to C (%)Country 1 31.4 3.4 3.4 99.7Country 2 31.4 3.4 3.4 99.7Country 3 27.4 3.9 4.0 98.3Country 4 20.5 3.2 3.3 96.0Country 5 19.5 3.3 3.5 95.5Country 6 16.8 3.5 3.7 94.3Country 7 15.3 3.7 3.9 94.1Country 8 15.0 3.3 3.5 94.0Country 9 14.3 3.5 3.7 93.7Country 10 13.1 3.5 3.8 93.2Country 11 11.4 3.5 3.8 92.6Country 12 9.5 3.5 3.8 92.1Country 13 6.2 3.2 3.5 90.9Country 14 5.9 3.0 3.3 90.6Country 15 4.6 3.3 3.7 90.3Country 16 4.1 3.5 3.9 90.0Country 17 2.8 3.2 3.6 89.4Country 18 1.0 3.3 3.7 88.9Country 19 0.3 3.3 3.7 88.4Country 20 –0.2 3.5 4.0 88.5Country 21 –1.5 2.8 3.2 87.8Country 22 –1.6 3.4 3.9 87.9Country 23 –1.9 3.4 3.9 86.9Country 24 –2.1 2.9 3.3 87.0Country 25 –4.1 2.7 3.1 86.8Country 26 –4.2 3.3 3.8 86.6Country 27 –6.2 3.0 3.5 85.7Country 28 –8.3 2.9 3.5 83.9Country 29 –8.7 3.3 3.9 85.1Country 30 –12.7 3.7 4.4 83.3Country 31 –13.0 2.7 3.3 82.1Country 32 –15.2 3.0 3.6 82.3(continued on next page)2 3

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tTable 2.4: Simulation Results: Effects on Performance-Based Allocations for “Core IDA”Countries Arising from a Larger Weight on the “Governance” Cluster Compared toEqual Weights on All Clusters (continued)Change in performancebasedallocation (%)Rating of governancecluster (cluster D)Average ratingof clusters A to CRating of cluster D as ashare of average rating ofclusters A to C (%)Country 33 –16.0 2.9 3.6 81.3Country 34 –16.3 2.2 2.7 81.5Country 35 –17.1 3.7 4.6 81.0Country 36 –18.2 3.3 4.1 80.9Country 37 –18.6 3.2 4.0 80.9Country 38 –20.4 2.7 3.4 79.9Country 39 –27.3 2.5 3.3 74.8Country 40 –29.4 2.6 3.5 75.2Country 41 –32.5 2.9 3.9 73.9Source: IEG.Note: “Core IDA” countries refer to those IDA countries that are not subject to exceptions to the PBA due to post-conflict or re-engaging status or to caps on allocations. Small states arealso excluded because their base allocations exceed the PBA.The effects of a largerweight on governance(compared to equalweights for each cluster)on the PBA are not duejust to the governancerating, but also to howdifferent that rating iscompared to ratingson other clusters.actual countries, the names of the countriesare not presented in the table. It should benoted that the PBA constitutes only part of theoverall IDA allocation, which also includes abase allocation of special drawing rights (SDR)1.5 million per country per year. The simulationresults demonstrate that, as intended by the PBAformula, a country that has a higher rating on thegovernance cluster but the same average ratingson the other three clusters compared to anothercountry would gain (from the larger weight ongovernance compared to equal weights for allclusters), whereas the other country would lose.This can be seen from the simulation results forcountry 5 and country 27, both ofwhich have the same average ratingof 3.5 for clusters A–C, but country 5has a higher rating of 3.3 for clusterD compared to country 27’s rating of3.0. Under the current PBA formula,country 5 would have a PBA nearly20 percent higher than a formula forwhich all clusters have equal weights,whereas country 27 would have a PBAof 6 percent lower.At the same time, however, the much largerweight on the governance cluster (comparedto equal weights on each cluster) has also ledto perhaps unexpected results. Specifically, thesimulation results indicate that the effects of themuch larger weight on governance on the PBAare not due just to the governance rating, butto how different the governance rating is fromratings on other clusters.Two countries, country 4 and country 30, can beused as examples to illustrate this point. Country30 has a better governance rating (3.7) thancountry 4 (3.2). Country 30 also performs betteron all the other clusters compared to country 4,with the ratings for clusters A to C averaging 4.4compared to country 4’s average rating of 3.3.Yet, country 30 suffers a loss in PBA of 13 percentunder the current PBA formula (compared to aformula with equal weights on all four clusters),whereas country 4 actually gains 20.5 percent.These results are attributable to the fact thatcountry 30’s governance rating is much worsethan its ratings on other clusters, whereas country4’s ratings on governance are only slightly worse2 4

R e l e va n c e o f t h e C P I Athan its ratings on the other clusters. In otherwords, country 30 suffers a loss under the currentPBA formula not because it has poor governance,but because its governance performance relativeto performance on other fronts is worse thanthat of country 4, even though it performs betterthan country 4 on all fronts.More generally, table 2.4 shows that all core IDAcountries (excluding small states) have worsegovernance ratings than ratings on other clusters,yet some countries gain and other countries losefrom the larger weight on governance. Whetherthey gain or lose depends on how much worsethe ratio of their governance ratings to ratings onother clusters is than other countries (figure 2.1).The CPIA and Determinants ofPoverty ReductionIt is a straightforward supposition that growthwill lead to poverty reduction— if it does notlead to greater inequality at the same time. Alarge empirical literature on the relationshipbetween growth and changes in inequality findsno statistical correlation between the two. 14 Thismeans that, on average, inequality does notchange with changes in per capita income, whichin turn means that, on average, growth leadsto reduction in poverty. Indeed, it iswell-established in the literature that,again on average, economic growth isassociated with a reduction in poverty(Ames and others 2001; Besleyand Burgess 2000; Ravallion 2001;White and Anderson 2002; Christiaensen,Demergy, and Paternostro2002; Dollar and Kraay 2002; Besleyand Burgess 2003; Klasen 2002).Hence, the determinants of growthdiscussed in the preceding sectionare as important as the determinantsof poverty reduction.Although, on average, growth leads to povertyreduction, this is by no means the case for allcountries or for everyone in a country. Actualdata show considerable variation—there arecases where inequality goes up with growth, andcases where inequality goes down with growth(Kanbur 2004). One paper finds a huge rangein the gains to the poor from a given rate ofgrowth. 15 The reasons behind this wide range ofFor countries that haveworse governance ratingsthan ratings on otherclusters, some gain andothers lose from the largerweight on governancein the PBA formuladepending on how muchworse the ratio of theirgovernance ratings toratings on the otherclusters is comparedto other countries.Figure 2.1: Relationship between Changes in PBA and the Ratio of Cluster D Ratings to Ratingson Other Clusters for “Core IDA” Countries100%Ratio of cluster D rating to rating onother clusters95%90%85%80%75%70%–40% –30% –20% –10% 0% 10% 20% 30% 40%Change in performance-based allocationsSource: IEG.Note: “Core IDA” countries refer to those IDA countries that are not subject to exceptions to the PBA due to post-conflict or re-engaging status or to caps on allocations. Small states arealso excluded because their base allocations exceed the PBA.2 5

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n teffects include differences in initial inequalitiesbetween countries and between regions withincountries that create differences in how muchthe poor share in aggregate growth (or contraction).Another paper finds that the incomes ofthe poor do not grow one-for-one with increasesin average income (Foster and Svékely 2008).Growth that reduces This evidence implies that growth thatinequality will have a reduces inequality will have a largerlarger impact on poverty. impact on poverty. Therefore, policiesneed to take into account the distributionalimpact of economic growth (the so-called“pro-poor growth” policies) (Ghura, Leite, andTsangarides 2002; Besley and Burgess 2003). Inaddition, there are also policies that can havea direct impact on poverty independent of thegrowth channel (the so-called “super pro-poor”policies). 16Financial depth, humancapital, equality,and institutions andgovernance directly affectpoverty reduction.The rest of this section will discuss the determinantsof pro-poor growth and super pro-poorpolicies. This will be followed bya discussion on two controversialdeterminants of poverty reduction—trade and the environment. Finally,other non-income dimensions ofpoverty will be addressed.Pro-poor growth and super pro-poor policiesThere is a significant overlap between the determinantsof pro-poor growth and of super pro-poorpolicies identified in the literature. Financialdepth, human capital, equality, and institutionsand governance are found to be important forpoverty reduction directly as well as throughthe growth channel. They are also important forreducing inequality and hence enhancing thepoverty reduction effects of economic growth.Quite apart from reducing poverty throughgrowth, financial sector development maybenefit the poor directly by facilitating access tocredit and improving risk sharing. 17 Empirically,there is evidence that financial (banking) depthis negatively associated with headcount poverty,even after taking into account mean income andinequality. 18 Further, financial development hasalso been found to reduce income inequality bydisproportionately boosting the incomes of thepoor (Beck, Demirguc-Kunt, and Levine 2004).Financial depth is addressed in the CPIA criterionfor the financial sector.As discussed earlier, education improves theequality of opportunity for the poor and otherdisadvantaged groups (including women). Anempirical study on Brazil finds that investmentsin human capital are very important to makegrowth more pro-poor (Menezes-Filho andVasconcellos 2004). Education is covered in theCPIA criterion on building human resources.Not only does inequality have a negative impacton growth, but higher initial levels of inequalityalso lower the poverty reduction impact ofgrowth (Ravallion 2001; Bourguignon 2004;Chhibber and Nayyar 2007). Hence, changingthe initial level of inequality enhances pro-poorgrowth. This requires measures to redistributewealth as well as to improve equality of opportunity.(Such measures and how they are addressedin the CPIA are discussed earlier in this chapter.)Regarding institutions and governance,establishing property rights can help the pooraccess credit (Fleisig 1995; de Soto 2000; Field2002) and enhance their ability to utilize andinvest in land they cultivate (Deolalikar andothers 2002). It has been empirically foundthat increased protection of property rights hasstrong effects in reducing poverty (Acemoglu,Johnson, and Robinson 2001). One estimatefinds that increasing the protection of propertyrights across the globe by half of one standarddeviation would halve global poverty (Besley andBurgess 2003).Protection of property rights requires thepresence of rule of law, specifically controls overcrime and violence. Studies also find that thevictims of crimes are more likely to come fromthe poorer part of the population (Bourguignon1999; Deolalikar and others 2002; Heinemann andVerner 2006). One study finds that police corruption,especially in slum areas of poorer countries,may increase the uncertainty of property rightsof the very poor (Andvig and Fjeldstad 2008).2 6

R e l e va n c e o f t h e C P I ACorruption also directly affects poverty by increasingincome inequality. One possible reason couldbe that the benefits of corruption are likely toaccrue to the better-connected individuals inthe society, who belong mostly to high-incomegroups (Gupta, Davoodi, and Terme 1998). It isalso possible that corruption distorts governmentallocations of goods and services (Tanzi 1998).One study finds that an increase in the corruptionindex of a country by one standard deviation(2.52 points on a scale of 1–10) increases theGini coefficient by 5.4 points (Gupta, Davoodi,and Terme 1998, 2002). Another study finds thatcorruption decreases the share of governmentexpenditures on health and education (Gupta,Davoodi, and Terme 2002).Finally, government accountability can have adirect impact on poverty. It has been argued thatno country with a free press has ever had a majorfamine (Dreze and Sen 1989), and that a free flowof information pressures (even non-democratic)governments into public action (Dreze and Sen1995).All of these different elements of institutionsand governance that have been identified in theliterature as being important determinants ofpro-poor growth or super pro-poor policies arecovered in various criteria under the CPIA clusteron public sector management and institutions.Aside from these four determinants of pro-poorgrowth, which are also determinants of growth,agriculture has been identified in the literatureas being important for pro-poor growth. Pro-poorgrowth needs to take place in sectors where thepoor are active and draw on the factors of productionthat the poor possess. The vast majority ofthe poor live in rural areas, and a majority ofthem depend directly or indirectly on agriculturefor their livelihood. The factor of productionthat the poor possess and use most is labor, andsometimes land as well (World Bank 2000c; Amesand others 2000; World Bank 2000a; Ravallionand Datt 2002; Eastwood and Lipton 2001).Therefore, pro-poor growth must focus on ruralareas, improve incomes in agriculture, and makeintensive use of labor (Klasen 2002).Indeed, a review of the East Asian Agriculture is importantexperience indicates that “the for pro-poor growth;countries that have been most successfulin attacking poverty have achieved add a reference on theit would be useful torapid agricultural growth and broader provision of publiceconomic growth that makes efficient goods in agricultureuse of labor and have invested in the to the fiscal policyhuman capital of the poor”(Rosegrant criterion of the CPIA.and Hazell 2000). Another paper findsthat rural growth reduced poverty in both ruraland urban areas, although urban growth had onlysome impact on urban poverty (Ravallion andDatt 2002).Improvements in labor productivity in agricultureare found (in both cross-country analyses andcountry case studies) to have been more pro-poorthan such improvements in non-agriculturalsectors (Eastwood and Lipton 2001). Increasesin agricultural yields by 20 percent are found toreduce the numbers of the poor by 18 percentin a cross-country empirical investigation (Irz andothers 2001). Agricultural research, in particular, isimportant in this respect, as it has led to crop yieldgains in the past. Some researchers conclude fromthis that “it is unlikely that there are many otherdevelopment interventions capable of reducingthe numbers in poverty so effectively” (Irz andothers 2001; Hazell and Haddad 2001). The CPIAdoes not explicitly cover agriculture—nor doesit cover any other economic sectors. It does,however, allow for the provision of public goodsin the fiscal policy criterion. This is pertinent foragriculture, in view of its importance identified inthe literature. Hence, the CPIA instructions mayusefully include a specific reference to publicgoods in agriculture (in addition to the currentmention of infrastructure) for the fiscal policycriterion.Trade and povertyFrom an analytical point of view, the relationshipbetween trade and poverty is ambiguous—thatis, trade can have a positive or a negative impacton poverty, as it does on growth (Agenor 2004;World Bank 2005). Indeed, there are severalchannels through which trade can affect poverty,and they can have opposite effects. Thesechannels are household production, household2 7

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tAssessment of the trade consumption, participation in thecriterion needs to labor markets, government revenues,take into account the and social expenditures. 19 Given thatextent of intersectoral theory is ambiguous with respect tolabor mobility. the impact of trade on poverty, theissue becomes an empirical one. Alas,the empirical literature—both cross-country andcase studies—is equally inconclusive (Winters,McCulloch, and McKay 2004; Ravallion 2004;Harrison 2006).What seems to be clear, though, is that impedimentsto exports exacerbate poverty (although itis less definitive as to what the effects of importliberalization are). One study finds that informalexport barriers to trade (such as transport costs,cumbersome customs practices, and costlyregulations and bribes) have significant adverseeffects on poverty in Moldova. 20 Another study(Balat, Brambilla, and Porto 2007) finds thatlower export marketing costs encourageagricultural exports and lower theLinkages betweenthe environment and poverty levels of those engaged inpoverty are complex. export cropping compared to othersin the rural areas. Export marketingcosts could be reduced by investmentsin infrastructure such as roads, provision ofmarketing information, provision of credit andtechnical assistance to farmers, and promotionof out-grower schemes among others (Balat,Brambilla, and Porto 2007; Otsuka 2002;Anderson 2003; Harrison 2006).Labor mobility is another important complementaryfactor. The negative impact of trade reformson poverty in India is found to be related to theextremely limited mobility of labor across regionsand industries in the country (Topalova 2004,2005). Similarly, labor market reforms are foundto be important for minimizing the adverseeffects of trade reform on the poor in Colombia(Goldberg and Pavcnik 2005).The CPIA does not adequately take into account theimportance of complementary factors in the tradecriterion to avert the potential negative impactsof trade liberalization on poverty. Specifically,this relates to the complementary factor of labormobility. The CPIA does address labor mobility(specifically flexibility in hiring and firing) in itscriterion on business regulatory environment(q6). However, it is important that such mobilitybe ensured before trade liberalization proceeds,lest liberalization exacerbate poverty. Therefore,assessment of the trade criterion needs to takeinto account the extent of labor mobility.Environment and povertyAs in the case of linkages between the environmentand growth, the linkages between theenvironment and poverty are similarly complex.On the one hand, there is evidence that environmentalregulations have a negative impact onpoverty. One paper finds that although, in general,low-income households appear to bear a disproportionateshare of existing environmental risks,policies that reduce environmental risks are notnecessarily progressive (Parry and others 2005).Another paper studying the distributional effectsof environmental policy finds that many effectsof such policies are likely regressive (Fullerton2008). On the other hand, there is also evidencethat resource degradation has a negative impacton the poor (World Bank 2008f).Nonetheless, the impact of the environmenton welfare is clear—environmental pollutionis clearly detrimental to health (World Bank2008d). Given that poverty needs to be viewedas a multidimensional concept that includeswelfare, environmental sustainability shouldclearly be taken into account as an importantfactor in poverty reduction.Multidimensional povertyOver the last decade or so, there has beenincreasing recognition that the notion ofpoverty encompasses more than just incomepoverty. Indeed, poverty includes a host ofother dimensions that are central to the MillenniumDevelopment Goals (MDGs). These otherdimensions are education, health, genderequality, and environmental sustainability, all ofwhich are covered by the CPIA.The notion has also been advanced that povertygoes beyond these income and non-incomemeasures of physiological deprivation (inability to2 8

R e l e va n c e o f t h e C P I Ameet basic material needs) to incorporate measuresof social deprivation (for example, access to thecomponents of power such as decision making). 21The World Bank has indicated that “poverty ismore than inadequate income or human development—itis also vulnerability and a lack of voice,power, and representation” (World Bank 2000c).It is in this context that the concept of empowermentof the poor has emerged. Specifically, itis thought that because the poor are the mainactors in the fight against poverty, they must bebrought to center stage in designing, implementing,and monitoring anti-poverty strategies.This requires, among other things, empowering“pro-poor” coalitions, which can involve parts ofgovernments, non-governmental organizations,donors, and civil society (World Bank 2000a).Such coalitions can be helped by a free press,democratic institutions, and accountable governments—particularly in countries where the poorare the majority. The CPIA covers media freedomand accountability in its criterion on transparency,accountability, and corruption in thepublic sector.The CPIA and the Effective Use ofDevelopment AssistanceThe notion of using the CPIA as an indicatorin the allocation of IDA resources is basedon two premises. The first premise is that IDAresources are important for supporting “…theworld’s poorest countries in their efforts to boosteconomic growth, lower poverty and improve theliving conditions of people” (World Bank 2008e).The second premise is that such resources couldonly be used effectively in the presence of soundpolicies and institutions that are assessed underthe CPIA. This section reviews the theoreticaland empirical bases for these two notions.The theoretical foundation of the effect of aidon growth 22 is the neoclassical growth model. 23Under this model, aid fills the gap in domesticsavings or foreign savings to finance investment,leading to growth. This theoretical foundationunderpins a sizeable empirical literature (of wellover 100 papers) on the impact of aid on growthfrom the 1960s through the mid-1990s.This theoretical foundation has been The notion of povertyquestioned by several researchers, encompasses bothdating back to the 1960s. Their main income and non-incomecriticism is the key assumption of the dimensions as well astheory that foreign aid finances investmentinstead of financing consump-social deprivation.tion. Perhaps partly reflecting this less thanrobust theoretical foundation, no consensus inthe empirical literature can be found through themid-1990s of the impact of aid on growth. Thevarious reviewers of this literature come to differingconclusions. One review (Hansen and Tarp2000) concludes that a majority of the literatureup to the mid-1990s finds that aid has a positiveimpact on growth. Two other reviews (Clemens,Radelet, and Bhavnani 2004; McGillivray andothers 2005) find no consensus.A watershed in the empirical literature wasreached in the mid-1990s with the publication ofa seminal paper in 1994 that empirically testedthe assumption that aid financed investment(Boone 1994). 24 The paper finds that aid didnot finance investment but financed public andprivate consumption instead. Furthermore, thehigher consumption did not benefit the poor, asreflected in the absence of a significant impact ofaid on improvements in infant mortality, primaryschooling ratios, and life expectancy. 25 This paperdemarcates the earlier generation of aid impactliterature based on the financing gap models fromthe latest generation, that is, underpinned by thenew growth theory. The latter generation of literaturespecifically takes into account the effect ofeconomic policies and institutions on growth. 26For many researchers, the Boone 1994 paperconfirms the “macro-micro paradox”: that manyaid-funded projects report positive microleveleconomic returns which are somehow undetectableat the macrolevel. The literature that emergesafter this paper can be classified into three strands:those that deny the existence of the macro-microparadox; those that try to explain it; and thosethat, like the 1994 paper, do not find any impactof aid at all (see appendix D for the list of papers).Those that belong to the first strand find thataid works on average, without conditions (the2 9

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tAmong those researcherswho find that aid hasan impact on growth,one group finds that itworks unconditionallyand the other group findsthat it only works underspecific conditions.Different analyses findthat different conditionsmatter for aid to havean impact on growth.Some analyses findthat aid has noimpact on growth.“unconditional” strand). Then there are thosethat accept the contention that aid does not workon average but seek to identify conditions underwhich it could be effective (the “conditional”strand). Foremost among the conditional strandis the 1997 World Bank working paper (Burnsideand Dollar 2000) that spearheads this strand ofliterature with the finding that aid is effective (thatis, has a positive impact on growth) only in thepresence of good policies (specificallyfiscal, monetary, and trade policies 27 ),which are themselves important forgrowth. Further, the paper finds thataid does not lead to good policies,but that having the right policies inplace matters for aid to be effective interms of higher growth.A number of other empirical papers thatfollowed find other conditions to be importantfor aid effectiveness. These conditions rangefrom countries emerging from civil war and havegood policies, to countries proneto external shocks such as climaticand trade shocks (or terms of tradeshocks), to countries outside of thetropics.The third strand of literature finds that aidhas no impact on growth at all. In addition tothe seminal paper of 1994 mentioned above,two others find that by and large aid did notincrease investment and that investment didnot raise growth (Easterly 2001, 2003). A recentpaper that takes into account the motivations ofdonors in granting aid 28 also finds it“… difficult to discern any systematiceffect of aid on growth” (Rajan andSubramaniam 2008).Not only are the findings of this latest generationof literature diverse, they are also not robust. Apaper that tested many of these studies (belongingto both the “conditional” and the “unconditional”strands) for robustness finds that none ofthe results of these studies withstood the tests. 29Perhaps the most striking outcome of the tests isthat modification of the sample period affects theregression results the most, thereby highlight-ing the fragility of the empirical results of thesepapers.Yet none of the researchers who find no impactof aid on growth, or who overturn the findingsof aid having an impact on growth, concludedefinitively that aid is not effective, or thatpolicies, governance, or exogenous conditionsdo not matter for aid to be effective. What they doconclude is that cross-country empirics may notbe very useful for analyzing whether and when aidworks, fraught as they are with data problems. 30In the midst of this ongoing controversy, someresearchers have forged new ground by analyzingthe impact of different types of aid on growth.One type of disaggregation is by donor objectives,based on the rationale that donors have strategicas well as developmental objectives for givingaid; hence, not all aid will lead to higher growth.In general, bilateral aid has strategic/geopoliticalobjectives (although bilateral aid from somecountries has developmental goals), whereasmultilateral aid has developmental objectives. Byand large, the emerging literature on this findsthat multilateral aid leads to higher growth, butnot bilateral aid. 31Another type of disaggregation focuses on aidthat could have an impact in the short run.One analysis finds that such “short-impact aid”(that includes budget and balance of paymentssupport, investments in infrastructure, and aidfor productive sectors such as agriculture andindustry) causes growth, on average, regardlessof the recipient’s quality of institutions andpolicies. 32Hence, a decade after the publication of the WorldBank working paper that put the conditional aideffectiveness literature on the map, there is noconsensus in the cross-country literature on theimpact of aid on growth.Findings of this literature range from (i) aidhaving no impact on growth to (ii) aid having apositive impact on growth conditional on policiesor exogenous factors (but with no consensus onwhich policies and institutions matter) to (iii) aid3 0

R e l e va n c e o f t h e C P I Ahaving a positive impact on growth regardless ofpolicies and institutions.Yet, despite the ambiguity of the impact of aidfrom the cross-country empirical literature, thereare many specific examples of aid being effective.These examples range from the eradicationof certain diseases (for example, smallpoxglobally and polio in the western hemisphere)or the Green Revolution in India in the 1960s,to improvements in school attendance andhealth indicators resulting from conditional cashtransfer programs more recently.Yet another strand of aid effectiveness literaturehas emerged, whereby efforts are focused onnarrower evaluations of the impact of specificaid project interventions. Such evaluations havebeen conducted in the context of so-calledimpact evaluations—or randomized evaluations—thatevaluate the impact of specificinterventions by comparing the effects on thosewho received the intervention with a comparablegroup who did not. 33Randomized evaluations over the last 10 yearsor so have found positive benefits of aid projectsin education, health, physical infrastructure,and agriculture, among others (see appendixE). This has led some researchers to proposethat development assistance should be mainlydevoted to such project-specific efforts.Empirical analysis of CPIA and loanperformanceThe preceding discussion indicates that theevidence is mixed regarding the relevanceof the content of the CPIA on aid effectivenessin the broad sense—that is, whether theCPIA represents the policies and institutionsimportant for aid to lead to growth. An empiricalanalysis of the association between CPIA ratingsand aid effectiveness is fraught with data difficulties(see chapter 2, section on findings andrecommendations). Therefore, this evaluationtakes a different approach and examines therelevance of the CPIA in a narrower sense—thatis, whether it is associated with the performanceof Bank loans.Econometric analysis finds that thepolicies and institutions that areassessed by the CPIA matter for loanperformance (see appendix G). Specifically,overall CPIA ratings are foundto be negatively associated with theshare of problem projects as assessedby Bank staff in loan implementationstatus reports. (The share of problemprojects is also found to be positivelyassociated with loan outcomes with acorrelation coefficient of 0.63).Further, ratings of each of the four CPIA clustersare also found to be negatively associated with loanperformance. It is not possible, however, to discernthe relative importance of the four CPIA clusterson loan performance because the ratings of thefour clusters are highly correlated with each other.Hence, there is not sufficient evidence to concludethat the governance cluster associates better withloan performance than the other clusters.Findings and Recommendations The CPIA covers theBy and large, the CPIA criteria cover main determinants ofthe main determinants of sustained sustained growth andgrowth and poverty reduction identifiedin the literature. The CPIA coverspoverty reduction.important determinants for both income andnon-income poverty, with the latter includingmany of the MDGs (specifically, education,health, gender equality, and environmentalsustainability). It also covers some of the keyaspects of another important notion of poverty—empowerment.It would have been useful to analyze empiricallythe impact of CPIA ratings including whenpresent together with IDA assistance, on theactual growth performances of the countriesrated. However, this was not possible becauseof the major restructuring of the CPIA contentas well as the 2004 rating scale. The discontinuityin the CPIA ratings implied by therestructuring would invalidate anyanalysis using data that spans 2004.Using only data from 2004 onwardwould not allow for a long enoughtime period for such analysis.Empirical analysisfinds that policies andinstitutions assessed bythe CPIA matter for loanperformance, althoughthere is insufficientevidence to concludethat the governancecluster matters morethan the others.Data constraints limitthe use of econometricanalysis to establish alink between CPIA ratingsand growth outcomes.3 1

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tAlthough data limitations make it difficult toestablish an empirical link between the CPIA andgrowth outcomes and hence aid effectivenessbroadly, the CPIA is found to be associated withaid effectiveness in a narrower sense—specificallyin the performance of Bank loans. However,there is insufficient evidence to conclude that anyone of the four CPIA clusters is more importantfor loan performance than the others.Based on the findings in this chapter, the evaluationderives recommendations regarding thefollowing broad issues.Tax policy could Weighting of the CPIAreasonably be The findings indicate that when thecombined with the CPIA is considered broadly as an indexfiscal policy criterion. of a country’s policies and institutions,the weighting scheme is not soimportant because the various CPIA clusters arehighly correlated. However, the weighting of thedifferent clusters in the PBA formula does matterfor a country’s allocations of IDA funds.The utility of aggregating These findings raise the questionthe CPIA clusters into of the usefulness of aggregating thean overall index is various CPIA clusters into an overallquestionable. index according to any predeterminedweighting scheme. In the caseof the broad use of the CPIA, it does not allowfor country specificity which could imply differentweights on the different clusters, dependingon the initial conditions and stage of developmentof the country. In the case of IDA allocation,the overall index is already not used as such(see chapter 1). The recommendation, then, isfor Bank management to consider not producingan overall CPIA index, although continuing toproduce and publish the separate componentsof the CPIA.Equity of public resourceuse is covered in othercriteria and could bedropped or reformulated.Streamlining CPIA criteriaThe CPIA is quite exhaustive in its coverage ofthe main determinants of growth and povertyreduction. Indeed, considerationneeds to be given to streamlining it.First, the criterion regarding theequity of public resource use (q8) islargely covered by other criteria; considerationneeds to be given to dropping or reformulatingit. This criterion has two subcomponents.The first subcomponent on public expendituresis by and large covered by other CPIA criteria,though Bank staff do not have enough informationto meaningfully rate the second one on taxrevenues. Dropping q8 would lead to only minorchanges in the relative rankings of countries—the rank correlation between the CPIA with q8and one without is 0.999—as well as only smallchanges to the PBA (table 2.5). A few morecountries would gain than would lose. However,the changes on both the upside and downsidewould be rather small, with the largest loserexperiencing a 1.7 percent drop in PBA and thelargest winner a 2.5 percent gain.Second, currently tax policy is assessed in thecriterion on efficiency of revenue mobilization(q14a). Yet tax policy is an intrinsic partof fiscal policy, and it would be reasonable tocombine the assessment of the two in the fiscalpolicy (q2) criterion. Further, the part of the taxpolicy subcriterion that deals with trade—specificallyimport taxes and export rebate or dutydrawback—really belongs to the trade criterion(q4). In fact, q4 already deals with import taxes,so there is an overlap that needs to be removed.Export rebate or duty drawback needs to beincorporated into the trade criterion, given theimportance of promoting exports for integrationinto the global economy.Third, there are some overlaps in contentbetween various criteria in the public sectormanagement and institutions cluster that couldbe usefully streamlined. Judicial independenceis covered in both the criterion on propertyrights and governance (q12) and the criterionon transparency, accountability, and corruptionin the public sector (q16). Administrativecorruption is assessed in the criteria onefficiency of revenue mobilization (q14) (in thesubcomponent on tax administration), qualityof public administration (q15) (in the subcomponenton merit and ethics), and transparency,accountability, and corruption in the publicsector (q16).3 2

R e l e va n c e o f t h e C P I AFourth, interviews with Bank staff conductedfor this evaluation suggest that it is onerous forcountry teams to have to answer 85 questionsto arrive at one rating for the environmentcriterion. 34 This is particularly the case in lightof the mixed evidence of the environment ongrowth, as well as the mixed evidence of theenvironment on poverty. IEG recommends thatBank management drastically simplify the assessmentof this criterion.Fifth, the three economic management criteriaare not conceptually distinct from each other,unlike the rest of the CPIA criteria. Discussionswith the relevant network reviewer indicatethat the three criteria are indeed assessed as anintegral whole. Yet separate scores are preparedand reported for each of the three criteria,which could lead to double (or triple) countingor confusion. For example, a country that hassuffered deterioration in fiscal policy wouldexperience a reduction in both the fiscal policyand the macroeconomic management ratings,which means that such deterioration would bedouble counted. Yet if Bank staff tries to avoiddouble counting by downgrading only the fiscalpolicy but not the macroeconomic managementrating, the resulting ratings would appear contradictory.Although there may be merit in preparingthree separate scores, Bank management needsto consider publishing only the consolidatedeconomic management rating to avoid theimpression of contradictory scores when staff areavoiding double (or triple) counting.Omissions to the CPIA criteriaNotwithstanding the exhaustiveness of the CPIAcoverage, there are a few omissions. First, thereis the exclusion of socioeconomic groups otherthan gender (such as by race, caste, and ethnicity).IEG recommends that assessment of thetreatment of such socioeconomic groups beincluded in the CPIA.Second is the absence of any reference at all toagriculture. As mentioned, this can be remediedby adding a reference to agriculture in thecriterion on fiscal policy with reference to publicgoods.Table 2.5: Simulation Results: Changes in PBA fromDropping q8Countries that would lose PBAPercentageCountrylostCoordinating assessment of CPIAOverlaps could be reducedcriteriabetween the criteria in theThe reviews of a few criteria need topublic sector managementbe coordinated. This has emergedand institutions cluster.in the context of the trade criterion,which needs to be evaluated in connectionwith the assessment of the labor criterion.Similarly, fiscal policy (q2) needs tobe evaluated in conjunction with thequality of budgetary and financialmanagement (q13).Countries that would gain PBAPercentageCountrygainedArmenia –0.04 Mozambique 0.12Honduras –0.13 Bolivia 0.13Madagascar –0.13 Tanzania 0.18Azerbaijan –0.18 Sierra Leone 0.37Nigeria –0.23 Nepal 0.41Lao, PDR –0.27 Zambia 0.42Tajikistan –0.32 Senegal 0.43Burkina Faso –0.39 Malawi 0.44Nicaragua –0.44 Mongolia 0.44Georgia –0.57 Ghana 0.46Vietnam –0.69 Mali 0.72Niger –0.85 Mauritania 0.72Yemen, Rep. of –0.88 Guinea, Rep. of 0.72Chad –0.98 Kyrgyz Republic 1.04Uganda –1.01 Bangladesh 1.05Rwanda –1.14 Cameroon 1.05Ethiopia –1.63 Kenya 1.31Papua New Guinea –1.71 Sri Lanka 1.33Uzbekistan 1.40Benin 1.64Cambodia 1.79Moldova 1.81Bosnia andHerzegovina2.49Source: IEG.Note: PBA=performance-based allocation.Assessment of theenvironment criterionneeds to be simplified.3 3

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tThe absence of agriculture Content of the CPIA criteriaand socioeconomic The 2004 restructuring of the CPIA wasgroups other than gender the last time its criteria were reviewedneeds to be addressed. and revised. Bank managementalso agreed with the external panelrecommendation at the time that, for the sakeof continuity and comparability, the CPIA criteriawill not be revised too frequently (World Bank2004d). Bank management further indicated thatperiodic reviews (for example, every three years)of the CPIA will be undertaken by an externaltechnical advisory committee charged withreviewing the CPIA methodology, procedure,and ratings quality.Add a subcomponenton exports to the tradecriterion; assign equalweights to exports,trade restrictiveness,and trade facilitation.This evaluation has found that the time has comefor Bank management to undertake a thoroughreview and revision of the CPIA. Thegeneral growth literature reviewconducted for this evaluation derivesfindings and recommendations forrestructuring the trade criterion;likewise, an in-depth literature reviewundertaken for the financial sectorTable 2.6: Simulation Results: Changes in TradeRatings Arising from Changes in Weights of TradeSubcriteria by Numbers and Shares of IBRD and IDACountriesNochange inratings ontradeRise inratings ontradeFall inratings ontradeNumber of countriesTotalIBRD 34 16 15 65IDA 36 12 27 75Total 70 28 42 140Share of countries (%)IBRD 52 25 23 100IDA 48 16 36 100Total 50 20 30 100Source: IEG.Note: IBRD= International Bank for Reconstruction and Development; IDA= International DevelopmentAssociation.criterion derives findings and recommendationsfor revising that criterion.Revise trade criterionThe importance of complementary institutionsfor global integration is not adequately reflectedin the weighting scheme for the trade criterion,nor is adequate attention given to exports. IEGrecommends that trade restrictiveness and tradefacilitation be given equal weights (replacing thecurrent weighting scheme of 75 and 25 percent,respectively, on these two subcomponents).In addition, a subcomponent on exports needsto be added (with the same weight as the othertwo) to assess export performance and policiesand institutions to reduce anti-export bias. Thesubcomponent on exports needs to include theassessment of export rebate and duty drawbackthat should be transferred from the criterionon efficiency of revenue mobilization (q14) assuggested in chapter 2 (in the section pertainingto streamlining CPIA criteria). All three tradesubcomponents—trade restrictiveness, tradefacilitation, and exports—need to be given equalweights.Simulations of equal weights for the tradesubcomponents on restrictiveness and facilitationindicate that there will be no change inratings on trade for half of the countries (70 of140 countries), with slightly more than half of theIBRD countries (52 percent) and slightly less thanhalf of the IDA countries (48 percent) experiencingno rating changes (table 2.6).About equal numbers of IBRD countries will gainand lose ratings on trade (16 and 15 countries,respectively), although many more IDA countrieswould lose (27 countries) than would gain (12countries). This implies that IDA countries haveworse ratings for trade facilitation than for traderestrictiveness compared with IBRD countries.For all but one country, the change in ratings(whether up or down) is 0.5 points; the exceptionis Tunisia, which would gain 1 point (table 2.7).The proposed change would increase thecomparability of the CPIA trade rating with that3 4

R e l e va n c e o f t h e C P I ATable 2.7: Simulation Results: IBRD and IDA Countries That Would Experience Changes in TradeRatings Due to Changes in Weights for Trade SubcriteriaFall in ratings on tradeRise in ratings on tradeIDA IBRD IDA IBRDMalawi Trinidad and Tobago Cape Verde MoroccoBurkina Faso Colombia Uganda Iran, Islamic Rep. ofSão Tomé and Principe Montenegro Senegal MalaysiaNiger Micronesia, Fed. Sts. Nigeria GuatemalaTajikistan Thailand Ethiopia UruguayGuyana Ukraine Rwanda SwazilandAzerbaijan Chile Bosnia and Herzegovina AlgeriaLao, PDR Albania Bangladesh NamibiaMoldova Croatia Lesotho St. Kitts and NevisMauritania Bulgaria Sri Lanka EstoniaYemen, Rep. of Lebanon Eritrea BotswanaMongolia Kazakhstan Pakistan South AfricaArmenia Dominican Republic Korea, Rep. ofPapua New Guinea Paraguay BelizeChad Costa Rica LatviaComorosBoliviaHondurasKyrgyz RepublicGeorgiaCongo, Dem. Rep. ofTogoTimor-LesteCôte d’IvoireHaitiAngolaCongo, Rep. ofSource: IEG.Note: All changes are 0.5 points, with the exception of Tunisia which would experience a 1-point gain.IBRD= International Bank for Reconstruction and Development; IDA= International Development Association.Tunisiaof a comparator—the Enabling Trade Index(ETI). Specifically, the rank correlation coefficientbetween the proposed CPIA trade ratingand one based on the ETI 35 will rise from 0.70 to0.75. 36 This in turn reflects the greater comparabilityof the ratings on customs/border administrationthan on trade restrictiveness (tariffs andnontariff barriers) between the CPIA and the ETI.3 5

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tThe financial sectorcriterion focuses more onintermediate outcomes.It would be useful toinclude in the assessmentthose policies andinstitutions that fosteran enabling environmentfor the financial sector.Indeed, the rank correlation for customs/borderadministration between those two indexes is0.77, compared with a rank correlationof 0.59 for trade restrictiveness.The proposed change would alterthe PBA, although not very significantly.37 Many more countries wouldlose than would gain, reflecting thelarger number of countries that haveworse ratings on trade facilitationthan on trade restrictiveness. Themagnitudes of the changes would besmall, however. Bangladesh would gain the most(by 1.6 percent), whereas Chad would lose themost (3.4 percent) (table 2.8).Financial sector criterionA review of the literature indicates that thefinancial sector criterion does cover thedimensions along which finance is currentlythought to be important: stability; depth andefficiency; and access. However, the way in whichsome of the dimensions are currently beingassessed can be strengthened.First, the application of equal weights to the threefinancial sector dimensions can be revisited,particularly in light of the ongoing global financialcrisis, as well as the considerable evidence of alarge impact of banking crises on output losses(Hoggarth, Reis, and Saporta 2002) 38 and on thenational budget (Laeven and Valenciana 2008;Honohan 2008b). 39 Further, it is alsoSome of the indicators widely accepted that financial stabilityis a prerequisite for the effectiveof the financial sectorcriterion need to be deployment of many types of developmentassistance (although there isstrengthened.less systematic evidence on this front(Honohan 2009). Hence, it would be useful forBank management to consider giving financialstability a larger weight than the other twodimensions in the CPIA criterion.Second, compared to almost every other CPIAcriteria, there is a greater focus in this criterionon assessing intermediate outcomes rather thanpolicies and institutions. This is particularlythe case in the assessment of financial depth,which focuses almost entirely on intermediateoutcomes, such as size of financial markets,interest rate spreads, and so on. It would beuseful to include in the assessment policies andinstitutions that foster an enabling environmentfor the financial sector such as the legal, contractual,informational, and governance framework.Third, some of the indicators used in the assessmentof the financial sector criterion can bestrengthened. One example is the indicator onbanking system soundness, which specifies twoalternative intermediate outcome measuresrelated to non-performing loans (NPLs). 40Although NPLs may predict crises to some extent,they are typically a lagging indicator, with highvalues suggestive of a problem that has alreadycrystallized. Hence these NPL measures are crudeand inadequate even as indicators of current orimminent problems. This is clearly reflected inthe fact that NPLs for residential mortgages didnot provide a sensitive early warning system inthe recent crisis in advanced economies. Thereare other good or even better flags for systemicrisk, for example, indicators of foreign exchangerisk in finance such as the dollarization of bankingdeposits or assets (De Nicoló, Honohan, andIze 2005; Cashin and Duttagupta 2008). Rapidgrowth of credit also needs to be monitored asa possible warning sign. The assessment of thisdimension would be strengthened by taking intoaccount such indicators.Appendix F presents a more detailed discussionof the indicators in the CPIA financial sectorcriterion based on a review of the literature. Italso offers recommendations for restructuringand strengthening the criterion.3 6

R e l e va n c e o f t h e C P I ATable 2.8: Simulation Results: Changes in PBA Arising from Changes in Weights for Trade SubcriteriaCountries that will lose PBA (%) Countries that will gain PBA (%)Country Percentage lost Country Percentage gainedMali –0.6 Rwanda 1.2Zambia –0.6 Senegal 1.3Benin –0.6 Uganda 1.4Cambodia –0.6 Bosnia and Herzegovina 1.4Cameroon –0.6 Ethiopia 1.4Ghana –0.6 Sri Lanka 1.4Guinea –0.6 Nigeria 1.5Kenya –0.6 Bangladesh 1.6Madagascar –0.6Mozambique –0.6Nepal –0.6Nicaragua –0.6Sierra Leone –0.6Tanzania –0.6Uzbekistan –0.6Vietnam –0.6Armenia –2.2Georgia –2.3Moldova –2.3Burkina Faso –2.4Malawi –2.4Mongolia –2.4Honduras –2.4Bolivia –2.5Azerbaijan –2.6Papua New Guinea –2.6Kyrgyz Republic –2.6Niger –2.6Mauritania –2.7Yemen, Rep. of –2.7Lao, PDR –2.8Tajikistan –2.9Chad –3.4Source: IEG.Note: PBA= Performance-based allocation. Ranking changes based on equal weights for all CPIA criteria.3 7

3 8Women weaving on loom, China. Photo by Curt Carnemark/World Bank

Chapter 3Evaluation Highlights• The CPIA correlates well with similarindicators in terms of both therelative rankings of countries andthe direction of change.• CPIA ratings correlate better forIBRD than for IDA countries withratings of similar indicators—thiscould be because ratings for IDAcountries take into account thestage of development.• Accounting for the stage of developmentis problematic because ofthe judgment involved and becauseof uneven practice across the Bank.• Having network reviewers validateratings helps minimize potentialbias in the ratings.

Morocco. Photo by Curt Carnemark/World Bank.

Reliability of the CPIA RatingsAssessing the reliability of the CPIA ratings is an intrinsically difficult ifnot impossible task, given that a benchmark (that is, the “true” rating)does not exist. Recognizing this major limitation, this evaluationassesses reliability in two ways: it compares the CPIA ratings with those ofsimilar indicators, and it reviews the Bank’s CPIA ratings generation process.With respect to the former, CPIA ratings are foundto correlate well with similar indicators in terms ofthe rankings of countries and in terms of directionof change. This means that the CPIA ratings arenot out of line with other indicators that measuresimilar policies and institutions. With respect tothe latter, the Bank’s review processes are foundto guard against potential biases in ratings.CPIA ratings are found to correlate better forIBRD than for IDA countries. One reason for thiscould be that more information is available onIBRD than on IDA countries, which increases thelikelihood of different institutions having similarassessments on IBRD countries. Another reasonfor this could be the need to take into accountthe stage of development in the CPIA ratings. Thismeans that more judgment is involved in ratingIDA countries, because accounting for the stageof development is more important for IDA thanfor IBRD countries. This introduces additionalsubjectivity into the rating exercise, which isalready centered on the expert judgment of staff.The issue is further complicated by the fact thatdifferent networks treat the issue of the stage ofdevelopment differently, which means that evenmore unevenness is introduced into the CPIAratings exercise.The quality of CPIA ratings could be enhanced byminimizing the amount of subjectivity involvedin the rating exercise. This could be done byexcluding accounting for the stage of developmentfrom the ratings exercise.Comparability with Other IndicatorsFourteen indicators are identified that could becompared with the CPIA (table 3.1). Two of the14 are strictly comparable with the CPIA—theseare the CPIA ratings by the Asian DevelopmentBank (ADB) and the African Development Bank(AfDB) (appendix H). The ADB uses exactly thesame questionnaire as the Bank. For the AfDB, allthe questions are either exactly the same or veryclose to those of the Bank with the exception ofthe question on trade (q4), for which the AfDBincludes an assessment on economic cooperationand regional integration. 1 The rest of theindicators are selected to match as closely aspossible with relevant CPIA criteria, some of4 1

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tTable 3.1: Rank Correlation Coefficients between CPIA Ratings and Comparator Ratings for 2007CPIA ADB AfDB ICRG GCI ETI DBBertelsmannTransformationIndexIndex ofEconomicFreedom WGICorruptionPerceptionIndexIbrahim Indexof AfricanGovernance ESIGenderGapIndexGenderEmpowermentMeasure Averageq1 0.74 0.82 0.78(26) (50)q2 0.61 0.82 0.72(26) (50)q3 0.67 0.91 0.79(26) (50)q4 0.67 0.72 0.71 0.70(26) (91) (85)q5 0.62 0.75 0.71 0.67 0.69(26) (50) (91) (139)q6 0.80 0.84 0.62 0.55 0.57 0.79 0.67 0.89 0.72(26) (50) (90) (85) (139) (111) (114) (140)q7 0.78 0.66 0.67 0.61 0.68(26) (50) (93) (60)q8 0.58 0.80 0.69(26) (50)q9 0.80 0.68 0.69 0.72(26) (50) (91)q10 0.72 0.77 0.72 0.74(26) (50) (111)(continued on next page)4 2

R e l i a b i l i t y o f t h e C P I A R at i n g sTable 3.1: Rank Correlation Coefficients between CPIA Ratings and Comparator Ratings for 2007(continued)CPIA ADB AfDB ICRG GCI ETI DBBertelsmannTransformationIndexIndex ofEconomicFreedom WGICorruptionPerceptionIndexIbrahim Indexof AfricanGovernance ESIGenderGapIndexGenderEmpowermentMeasure Averageq11 0.71 0.77 0.59 0.69(26) (50) (110)q12 0.75 0.82 0.73 0.69 0.74 0.69 0.88 0.76(26) (50) (95) (92) (111) (114) (140)q13 0.54 0.89 0.72(26) (50)q14 0.61 0.72 0.67(26) (50)q15 0.68 0.80 0.66 0.83 0.74(26) (50) (95) (140)q16 0.72 0.79 0.51 0.82 0.75 0.86 0.86 0.77 0.76(26) (50) (95) (111) (114) (140) (135) (46)Average 0.69 0.79 0.63 0.69 0.63 0.62 0.77 0.70 0.87 0.86 0.77 0.59 0.67 0.61 0.72Source: IEG.Note: The number of observations is presented in parenthesis below the correlation coefficients. All correlations are for 2007 ratings except for the Ibrahim Index of African Governance (2006 ratings) and Environmental Sustainability Index (ESI)(2005 ratings). ADB = Asian Development Bank; AfDB = African Development Bank; DB = Doing Business; ETI = Enabling Trade Index; GCI = Global Competitiveness Index; ICRG = International Country Risk Guide; WGI = Worldwide GovernanceIndicators. All coefficients are significant at the 1 percent level.4 3

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tBox 3.1: Comparator IndicatorsThis evaluation selected 12 other indicators, in addition to theCPIA ratings produced by the AfDB and the ADB, to comparewith the Bank’s CPIA:• Three are produced by the Bank—the Logistics PerformanceIndex, Doing Business, and the Worldwide Governance Indicators.• Four are produced by the World Economic Forum—the EnablingTrade Index, the Environmental Sustainability Index,the Global Gender Gap Index, and the Global CompetitivenessIndex.• The rest are produced by the Bertelsmann Foundation (BertelsmannTransformation Index), Heritage Foundation (Indexof Economic Freedom), the Political Risk Services Group (theInternational Country Risk Guide), Transparency International(Corruption Perception Index), the United Nations DevelopmentProgramme (the Gender Empowerment Measure),and the Mo Ibrahim Foundation (Ibrahim Index of AfricanGovernance).Appendix I provides details on the specific subindicators usedfor comparison.Source: IEG.which are guideposts for certain CPIA criteria(box 3.1).For certain CPIA criteria, very few comparatorscan be identified. In particular, it is difficult toidentify comparators other than those by theADB and AfDB for the three economic managementcriteria (q1, q2, and q3), 2 and the criteria onequity of public resource use (q8), the quality ofbudgetary and financial management (q13), 3and efficiency of revenue mobilization (q14).Hence, for those criteria, only comparisons withratings by ADB and AfDB are made.In contrast, quite a few more comparators arefound for the criteria on business regulatoryenvironment (q6) and transparency, accountability,and corruption in the public sector(q16). For both these criteria, six other indicatorsin addition to the ones from ADB and AfDB arefound.Rank correlationsInterpretation of the correlation coefficients 4 ofthe CPIA ratings with other indicators is complicatedby the fact that the content of the indicators,other than those of the other developmentbanks, is not exactly the same as the CPIA. Theoverlap between these other indicators andthe CPIA criteria varies. Some overlap muchbetter with the content of certain CPIA criteriathan others. Nonetheless, for each criterion thecorrelation coefficients with both the developmentand nondevelopment bank comparatorsare similar and relatively high, averaging between0.7 and 0.8 (see last column of table 3.1). Thisprovides some assurance that the CPIA ratingsare not out of line with those of other indicatorsthat assess similar policies and institutions. Thecorrelation coefficients presented in table 3.1also indicate that there is not much differencebetween all 16 CPIA criteria in terms of theircomparability with other indicators.Although on average the CPIA ratings correlatewell with other indicators, there is a dispersion tothe correlation coefficients that range from a lowof 0.54 (for q13 with ADB) to a high of 0.89 (forq13 with AfDB and q6 with the World GovernanceIndicators). Several possible reasons may accountfor this dispersion. First, as mentioned, exceptfor the AfDB and the ADB, the other indicatorsare not assessing the exact same criteria as theCPIA. Second, judgment is involved in the ratingexercise (by the Bank and by virtually all the otherinstitutions; see table 3.2), which is exacerbatedwhen there is not enough information availableon the criteria.4 4

R e l i a b i l i t y o f t h e C P I A R at i n g sTable 3.2: Other Indicators—Expert Judgment or Hard Data?External data sourceCPIA CriterionADB q1 to q16Expert Judgment/SurveyHard dataExpert Judgment/Surveyand Hard DataAfDB q1 to q16 ✓Global Competitiveness Index q4 ✓International Country Risk Guide q12 ✓q5q6q9q12q15q16Enabling Trade Index q4 ✓q6Doing Business q5 ✓Bertelsmann Transformation Index q6 ✓q6q10q12q16Index of Economic Freedom q6 ✓Worldwide Governance Indicators q6 ✓Corruption Perception Index q16 ✓q12q16q12q15q16Ibrahim Index of African Governance q16 ✓Environmental Sustainability Index q11 ✓Gender Gap Index q7 ✓Gender Empowerment Measure q7 ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓4 5

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tOther comparatorindicators correlatebetter with the Bank’sCPIA ratings thanwith the ratings by theAfDB and the ADB.For example, even though the ADB uses the sameCPIA questionnaire as the Bank, the correlationcoefficients for two criteria, q8 and q13, arerelatively low (the respective correlation coefficientsare 0.58 and 0.54). Very different rankingsfor two to three countries (of 26 countries) areresponsible for the relatively low correlationsoverall. In the case of q8, these are Micronesia,Cambodia, and Timor-Leste, and for q13, theseare Azerbaijan and Tonga. Two of these countriesare Pacific Islands, for which the Bank has littleup-to-date or firsthand information (accordingto region and network reviewers who wereinterviewed for this evaluation).For all of the criteria for which there are comparableindicators other than those from ADB andAfDB (that is, all criteria except for q1, q2, q3,q8, q13, and q14 as discussed above),comparisons of rank correlations wereundertaken to assess whether theseother indicators correlate better 5 withthe Bank’s CPIA or with those of theother two development banks. (Theratings by the AfDB and the ADB arethe points of reference here because they are themost comparable with the Bank’s ratings bothbecause of their content, and because all threeinstitutions take into account country context inthe ratings whereas the other indicators do not.)The comparisons indicate that, overwhelmingly,these other indicators correlate betterwith the Bank’s ratings than with ratings byAfDB and ADB (table 3.3).Comparing changes in ratingsThe two previous sections indicate that thelevel of the CPIA ratings (and implied rankings)compare relatively well with other indicators.This section examines how well the changes inratings compare with changes in the comparatorindicators between 2006 and 2007.Strictly speaking, a similar assessment of a particularcriterion by different institutions wouldimply that a change or no change in ratings byone institution would be associated with similarmovements in ratings by the other institutions.Because the timing of the CPIA assessmentexercise varies across these institutions, aTable 3.3: Are Other Comparator Indicators Closer to the Bank or to AfDB and ADB?Majority of other indicators correlates better withMajority of other indicators correlates better withCPIA criterionBank AfDB Bank ADBq4 n.a. n.a. ✓q5 ✓ ✓q6 ✓ No differenceq7 ✓ ✓q9 No difference No differenceq10 No difference ✓q11✓q12 ✓ ✓q15 ✓ ✓q16 ✓ ✓4 6

R e l i a b i l i t y o f t h e C P I A R at i n g schange in the assessment by one institution maynot always be associated with a similar change inassessment by the other institution. However,at the very least, these assessments—if they aresimilar—would not contradict each other or,in other words, the ratings would not move inopposite directions.The comparisons of changes in ratings indicatethat the Bank’s assessments are very similar tothose of the AfDB and the ADB. Only for a fewcriteria and a very small share of countries dothe ratings of the Bank and AfDB/ADB move inopposite directions (table 3.4).In the comparison with the AfDB, for The Bank’s CPIA ratings10 of the 15 criteria (q4 is excluded and those of AfDBfrom the analysis for reasons stated and ADB correlateearlier), none of the countries has well in terms ofratings that move in the opposite direction of change.direction. For four criteria—q2(fiscal policy), q7 (gender), q8 (equity of publicresource use) and q13 (quality of budgetaryand financial management)—the ratingsmove in the opposite direction for only 2 percentof the countries (one out of 50 countries). Forq1 (macroeconomic management), a slightlyhigher 4 percent of the countries (2 out of50 countries) have ratings that move in theTable 3.4. Comparison of Changes in CPIA Ratings 2006–07 between the Bank,AfDB, and ADBComparison with AfDBComparison with ADBCriterionChange inthe samedirection(%)Change inthe oppositedirection (%)Change for oneinstitution butno change inthe other (%)Change in thesame direction(%)Change inthe oppositedirection (%)Change for oneinstitution butno change inthe other (%)q1 60.0 4.0 36.0 50.0 4.2 45.8q2 42.0 2.0 56.0 41.7 0.0 58.3q3 64.0 0.0 36.0 33.3 0.0 66.7q4 n.a. n.a. n.a. 58.3 0.0 41.7q5 72.0 0.0 28.0 54.2 4.2 41.7q6 66.0 0.0 34.0 75.0 0.0 25.0q7 58.0 2.0 40.0 62.5 0.0 37.5q8 52.0 2.0 46.0 54.2 0.0 45.8q9 66.0 0.0 34.0 66.7 0.0 33.3q10 74.0 0.0 26.0 50.0 0.0 50.0q11 64.0 0.0 36.0 66.7 4.2 29.2q12 80.0 0.0 20.0 50.0 0.0 50.0q13 56.0 2.0 42.0 33.3 0.0 66.7q14 72.0 0.0 28.0 54.2 0.0 45.8q15 70.0 0.0 30.0 70.8 0.0 29.2q16 56.0 0.0 44.0 54.2 0.0 45.8Source: IEG.Note: q4 is excluded from this analysis because of the additional dimension of regional integration that is included in q4 by AfDB but not the Bank. ADB= Asian Development Bank; AfDB=African Development Bank.4 7

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tCPIA ratings correlatebetter with othercomparator indicatorsfor IBRD than forIDA countries.opposite direction. In the comparison withADB, for 13 of 16 criteria, none of the countrieshave ratings move in the opposite direction. Forthe remaining 3 criteria—q1 (macroeconomicmanagement), q5 (financial sector), and q11(environmental sustainability), 4 percent ofthe countries (1 of 26 countries) have ratingsmove in the opposite direction.The same comparison with the other indicatorsis not reported here because the CPIA ratings (bythe Bank and the other two development banks)are more discrete (with intervals of 0.5) than theother indicators. This implies that changes in theratings of the other indicators may not correspondto changes in CPIA ratings, even if the assessmentsare similar to those of the Bank. In other words,small changes in policies and institutions may leadto a change in a rating that is on a more continuousscale (such as the Global Competitiveness Index),whereas the same small change would not bereflected in changes in the CPIA rating because ittakes a relatively significant change in policies andinstitutions for the CPIA rating to change by 0.5.Therefore, comparisons of changes in CPIA ratingswith changes in the ratings of other indicators couldonly be made for instances where the CPIA ratingschange. They cannot be made in cases when CPIAratings do not change. This restricts the numberof observations significantly, because CPIA ratingsdo not change very much over time. The restrictednumber of observations per criterion (these are atmost slightly above 20, and in many instances wellbelow 20) weakens the confidence in the analysis;hence the results are not reported here.IBRD versus IDA ratingsThe Bank’s CPIA ratings correlate better withratings from AfDB and other institutions for IBRDcountries than for IDA countries 6 (table 3.5). AfDBratings are closer to those of the Bank for twiceas many criteria for IBRD than for IDA countries(6 versus 3 criteria). Ratings of otherindicators are overwhelmingly closerto Bank ratings for IBRD than for IDAcountries. They are closer to Bankratings for 8 criteria for IBRD countriesand for no criteria for IDA countries.Two possible reasons may account for thesefindings. First, in general, more informationis available on IBRD countries than on IDAcountries (see appendix J) which increases thelikelihood of different institutions (AfDB as wellas other institutions) having similar assessmentson IBRD countries as the Bank. Second, the stageof development (taken into account by the Bankand the AfDB) is more pertinent for IDA thanfor IBRD countries, and the additional judgmentinvolved in accounting for the stage of developmentwould likely make ratings for IDA countriesless comparable. Regardless of whether oneor both reasons are valid here, CPIA ratingscorrelate better with those of other indicatorsfor IBRD than for IDA countries.Conclusions on comparability of CPIA withother indicatorsThe findings indicate that CPIA ratings for all16 criteria correlate relatively well with those ofsimilar indicators in terms of both the relativerankings of countries as well as the direction ofchange. The rank correlations of CPIA ratingswith ratings of other indicators average between0.7 and 0.8 for each of the 16 CPIA criteria. Ratingsof other indicators correlate better with CPIAratings by the Bank than those by the AfDB andthe ADB. Finally, CPIA ratings correlate better forIBRD than for IDA countries with ratings of otherindicators.CPIA Ratings Generation ProcessThe central determinant of CPIA ratings is theprofessional judgment of Bank staff, who can alsodraw on other indicators (including outcomeindicators/hard data) provided as guidepostsin the CPIA Questionnaire. The ratings areproduced in a multistep process, which entailstwo levels of review—first at the regional level,and then at the network level (see box 3.2).In cases where the regions and the networksdisagree over the final ratings, the networks havethe final say unless the regions have supportingevidence.Regional reviewInterviews with regional reviewers indicate thatthere is no one standard review practice across4 8

R e l i a b i l i t y o f t h e C P I A R at i n g sTable 3.5: Rank Correlations between the CPIA and Other Indicators:IBRD versus IDA CountriesBank ratings correlate better withAfDB ratings forBank ratings correlate better withother Indicators forCPIA CriterionIBRD countries IDA countries IBRD countries IDA countriesq1 ✓ n.a. n.a.q2 No difference n.a. n.a.q3 ✓ n.a. n.a.q4 n.a. n.a. ✓q5 ✓ ✓q6 No difference ✓q7 ✓ No differenceq8 No difference n.a. n.a.q9 No difference ✓q10 ✓ No differenceq11 No difference ✓q12 ✓ ✓q13 ✓ n.a. n.a.q14 ✓ n.a. n.a.q15 No difference ✓q16 ✓ ✓regions. One factor that influences the regionalreview practices is the size of the region.For regions with numerous countries, such as theAfrica and the Europe and Central Asia Regions,reviewers undertake statistical exercises usingexternal indicators to review country team ratingproposals. In the Africa Region, sector specialistsundertake such exercises, which are used asinputs in the regionwide review process beforethe first round of rating proposals are submittedto Operations Policy and Country Studies (OPCS).In smaller regions, such as the Middle East andNorth Africa and South Asia, sectoral staff aregenerally knowledgeable about theirsectors for many countries in the There is no one standardregion, whereas staff in the Chief CPIA review practiceEconomist’s office are knowledgeable across regions.about all the countries in the region.Hence these regional reviews do not entail crosscountrystatistical exercises.Further, though a regional review is meant toensure intraregional rating comparability priorto submitting the first round of rating proposalsto OPCS, interviews with regional reviewersindicate that not all regions do that. The varyingextents to which regions undertake this reviewmay account for the varying extents to which4 9

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tBox 3.2: The Process of Preparing CPIA RatingsThe process begins with a benchmarking phase, which entailsrating a small representative sample of countries drawn fromall the regions (see chapter 1, “Other Changes in the CPIA”).This is followed by a roll-out phase, during which the restof the countries are rated. Both phases entail a multistepprocedure.In the first step, the country teams generate a set of proposedratings for their respective countries. This step is usually led bycountry economists with participation from sector specialists andcountry management.In the second step, the Regional Chief Economist offices reviewand revise (as necessary) the ratings for the countries within therespective regions to ensure cross-country comparability withineach region.In the third step, the network anchors and other central unitsreview the ratings at the Bank-wide (global) level to ensure crossregionalcomparability of ratings.The fourth step is somewhat different for the benchmarkingversus the roll-out phase. For the benchmarking phase, the fourthstep entails a meeting of representatives from Operations Policyand Country Services (OPCS), the regions, networks, and centraldepartments to review the proposed ratings for all of the criteriaand for all of the benchmark countries, after which the ratings are“frozen” and the roll-out phase proceeds. For the fourth step ofthe roll-out phase, most of the ratings are finalized through virtualcommunication because of the large number of countries involved.Meetings are only held to discuss the few cases that have not beenresolved by virtual communication.Source: IEG, based on interview with OPCS.networks disagree with the regions over theinitial rating proposals (table 3.7).All regional reviewers were asked the open-endedquestion of which criteria are difficult to assess.Reviewers found that the criteria in the publicsector management and institutions cluster werethe most difficult, pointing to the lack of dataand the judgment involved. One of the networkreviewers for this cluster indicated that a lot ofjudgment is involved in most of the criteria inthat cluster and pointed out the criterion ontransparency, accountability, and corruptionas an example.Network review Network reviewpractices also vary. The review practices of the networksvary depending on various factors,including: (i) the extent to which other quantitativeindicators are available for cross-checkingthe CPIA ratings; (ii) the extent to which anyother information is available on the criteria atall; (iii) the importance the particular networkaccords to the exercise (and hence the amountof resources devoted to it); and (iv) the clarityof the criteria content and associated ease ofassessment. These factors affect the extent ofthe review, which varies quite significantly acrosscriteria (box 3.3).Expert judgment and potential conflict ofinterestAs indicated by the external panel in the 2004review of the CPIA (World Bank 2004a), thedepth of country knowledge by Bank staff is amajor strength of the exercise. The practice ofrelying on expert judgment for ratings is alsoused by virtually all of the other indicators againstwhich the CPIA was compared in the previoussection (table 3.2).Although the expert judgment of Bank staff isclearly an asset in the CPIA exercise, at the sametime there is a potential conflict of interest inhaving staff provide ratings, particularly for IDAcountries. This potential for conflict of interestarises from the fact that ratings produced bystaff are in turn used for allocating IDA resourcesfor the same countries on which the workprograms of those staff depend. Therefore, staffmay potentially be upwardly biased in assigningratings for their countries. The regional review ismeant to adjust for such potential biases at theregional level, although there could still be issues5 0

R e l i a b i l i t y o f t h e C P I A R at i n g sBox 3.3: The Network Reviews of CPIA RatingsFor the economic management cluster (criteria q1 to q3), thereviewers read every write-up submitted by the region, as wellas reports from the International Monetary Fund, Debt SustainabilityAssessments, and private sector reports for the countrybeing reviewed. Many reviewers in the Economic Policy andDebt Department (PRMED) are involved in this exercise, witheach reviewer assigned about six countries. A coordinatorthen reviews about 60 percent of all the reviewers’ commentsto ensure consistency of ratings across countries.Perhaps because of the resource-intensiveness of the review,these were also the criteria on which there were the most comments.In 2007, for each of the criteria q1, q2, and q3, the networkcommented on 86 percent of the countries, compared with commentson an average of 38 percent of the countries for all 16 CPIAcriteria (appendix K). Further, these were also the criteria for whichthe network disagreed with the regional proposals for a highershare of countries than for most other criteria (see table 3.7).For the criteria on trade (q4), business regulatory environment(q5), financial sector (q6) up to last year, gender (q7), propertyrights and rule-based governance (q12), quality of budgetary andfinancial management (q13), parts (c) and (d) of quality of publicadministration (q15), and transparency, accountability, and corruptionin the public sector (q16), the networks use other quantitativeindicators to cross-check the ratings.For the gender criterion (q7), the network actually first generatethe ratings based on quantitative indicators which are passed ontothe region for review. These initial gender ratings are then adjustedif country teams provide additional country-specific informationthat is not captured by the ratings. Some networks supplementassessments based on quantitative indicators with the write-upssubmitted by the regions.For the financial sector (q5) for fiscal 2009, building humanresources (q9), and efficiency of revenue mobilization (q14), thenetworks review all of the write-ups that were submitted. Thenetwork reviewer for q14 supplements these with quantitativeindicators from various other sources.For the equity of public resource use (q8) and social protectionand labor (q10), the networks' review focuses only oncountries for which the proposed ratings are different from theprevious years. For the environment (q11), the network reliesmore on the region’s judgment because it has little other informationon the criterion.Source: IEG, based on interviews with network reviewers.with the levels of the ratings even if the relativerankings of countries are adjusted at the regionallevel.The network review—among other functions—ismeant to adjust potential biases in the levels ofratings across regions (box 3.3). The evidence fromthe 2007 review process indicates that there was notmuch difference between IBRD and IDA countriesin terms of the extent of network disagreementwith the regions’ initial rating proposals. For allcountries, the networks disagreed with about 12.5percent of the initial regional ratings proposedfor IDA countries, compared with only a slightlylower share of 11.8 percent of the ratings for IBRDcountries (table 3.6).The differences were much more significantwhen the comparison was made at the regionallevel. For every region except Latin Americaand the Caribbean, the networks challengedinitial regional proposals for a larger share ofIDA than IBRD countries. For both IBRD andIDA countries, there was greater disagreementbetween network and regions for Europe andCentral Asia than for all the other regions (for 16and 20 percent of Europe and CentralAsia countries, respectively). For 7 ofthe 16 criteria, the networks disagreedmore often with regional proposals ofratings for IBRD countries, whereasfor 9 of the 16 criteria, they disagreedmore often with ratings for IDAcountries (table 3.7).Ratings were more likely to be challenged bythe networks when the regions proposed anincrease from 2006, and much more so for IDAthan for IBRD countries. Specifically, when thenetworks challenged regional proposals, it wasThe networks challengedthe regional proposalsmore often for IDAthan for IBRD countriesfor all regions exceptLatin America andthe Caribbean.5 1

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tTable 3.6: Numbers and Shares of Initial Regional Rating Proposals on Which the NetworksDisagreed with the Regions, by Criteria, for 2007CPIA criterionNo. of timesnetworksdifferedTotal For IBRD countries For IDA countriesShare of allratings (%)No. of timesnetworksdifferedShare of allratings (%)No. of timesnetworksdifferedShare of allratings (%)q1 24 17.1 12 18.5 12 16.0q2 28 20.0 12 18.5 16 21.3q3 27 19.3 12 18.5 15 20.0q4 12 8.6 6 9.2 6 8.0q5 15 10.7 9 13.8 6 8.0q6 11 7.9 6 9.2 5 6.7q7 28 20.0 11 16.9 17 22.7q8 8 5.7 3 4.6 5 6.7q9 22 15.7 10 15.4 12 16.0q10 6 4.3 2 3.1 4 5.3q11 10 7.1 4 6.2 6 8.0q12 10 7.1 5 7.7 5 6.7q13 17 12.1 4 6.2 13 17.3q14 22 15.7 12 18.5 10 13.3q15 13 9.3 7 10.8 6 8.0q16 20 14.3 8 12.3 12 16.0Total 273 12.2 123 11.8 150 12.5Source: IEG, based on World Bank data.Note: IBRD= International Bank for Reconstruction and Development; IDA= International Development Association.Networks challenged found that in 59 percent of the timeregional proposals more the regions had proposed an increase.often for Europe and This ratio was 66 percent for IDACentral Asia—for 18 countries compared to a much lowerpercent of the ratings 50 percent for IBRD countries (tablecompared to 12 percent 3.8). This indicates that the networksfor all regions. perceived more of an upward bias inthe ratings for IDA countries than forIBRD countries.The role of the networksin validating the ratingshelps to minimizethe potential for biasin the ratings.Regarding the instances in which thenetworks challenged a rating increasefrom the regions, the networksprevailed in an overwhelming majorityof 77 percent of the time. The share washigher for IBRD countries (86 percent) than forIDA countries (73 percent) (table 3.9). Thus, theconclusion can be drawn that for those 73 and 86percent of instances, there was indeed an upwardbias in ratings. However, these instances made uponly 6 percent of the ratings for IDA countries andabout 5 percent of the ratings for IBRD countries,which implies that there was not a severe upwardbias in ratings for either group of countries.Taking into account all disagreements—that is,including also instances where regions proposedno change or a decrease in ratings in addition toan increase in ratings—the 73 percent in whichnetwork views prevailed dropped somewhat to5 2

R e l i a b i l i t y o f t h e C P I A R at i n g sTable 3.7: Number and Share of Initial Regional Rating Proposals on Which the NetworksDisagreed with the Regions, by Region and IBRD and IDA Countries for 2007Number of rating proposals on which networksdisagreed with regionsShare of rating proposals on which networksdisagreed with regions (%)RegionTotal IBRD IDA Total IBRD IDAAfrica 82 10 72 11.4 7.8 12.2East Asia and Pacific 28 6 22 8.8 4.2 12.5Europe and CentralAsiaLatin America andCaribbeanMiddle East andNorth Africa81 55 26 17.5 16.4 20.355 40 15 12.3 13.2 10.416 12 4 10.0 9.4 12.5South Asia 11 n.a. 11 8.6 n.a. 8.6Total 273 123 150 12.2 11.8 12.5Source: IEG, based on data from OPCS.Note: IBRD= International Bank for Reconstruction and Development; IDA= International Development Association.68 percent, which is still high (table 3.10). Insum, the evidence reflects the central role of thenetworks in the review process, which helps tominimize potential biases in ratings.The networks prevailed in the majority of the casesfor all regions except South Asia and prevailedmost often for Latin America and the Caribbean.The networks also prevailed more often for IBRDthan for IDA countries overall (77 versus 61percent of the time). This could either mean thatthe regions have more supporting evidence forthe ratings they proposed for IDA than for IBRDcountries, or that the regions make a bigger effortfor IDA countries because IDA funds are involved(this evaluation has obtained anecdotal evidencefrom interviews with World Bank staff on this).The nature of the network review (that is, theresource-intensiveness of the review) appears tohave an effect on the extent of the review, andhence possibly on the quality of the ratings. Theevidence from the 2007 review process indicatedthat four criteria stood out as having been morerigorously reviewed than other criteria, includingthe three economic management criteria andthe criterion on building human resources. Forthese four criteria, the networks commented onthe initial regional rating proposals even whenthe regions did not propose a change in ratingsfrom the previous year (table 3.11). As discussedin box 3.3, these were also the criteria on whichthe networks had invested the most time. For thefour criteria the reviewer read all of the write-ups.Additionally, for the three criteria on economicmanagement, each reviewer was responsible forTable 3.8: Network Disagreements with InitialRegional Rating ProposalsRegional proposalsAll countries(%)IBRDcountries(%)IDAcountries(%)Lower rating than 2006 1.9 2.7 1.3Same rating as 2006 39.2 47.8 32.7Higher rating than 2006 58.9 49.6 66.0Source: IEG based on World Bank data.Note: IBRD= International Bank for Reconstruction and Development; IDA= International DevelopmentAssociation.5 3

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tTable 3.9: Share of Instances Where NetworksPrevailed When Networks Disagreed with Regionsover Proposed Increases in Ratings from 2006RegionAll countries(%)IBRD countries(%)IDA countries(%)Africa 74.5 83.3 73.5East Asiaand PacificEurope andCentral AsiaLatinAmerica andCaribbeanMiddle Eastand NorthAfrica80.0 100.0 78.669.8 82.6 55.092.6 89.5 100.081.8 85.7 75.0South Asia 75.0 n.a. 75.0Total 77.4 85.7 72.7Source: IEG, based on OPCS data.Note: IBRD= International Bank for Reconstruction and Development; IDA= International DevelopmentAssociation.about six countries for which he or she also readreports from the International Monetary Fund(IMF), Debt Sustainability Assessments, and theprivate sector reports.ConclusionsAnalysis of the 2007 review process indicates thatthe networks perceived more of an upward biasin ratings for IDA than for IBRD countries. Thisupward bias did not seem very severe, as it couldonly be detected in about 6 percent of the ratingsfor IDA countries and about 5 percent of theratings for IBRD countries. Analysis of the 2007review process also indicates that the networks didhave a central role in the ratings review process,as they prevailed in the majority—although notall—of the cases when there were initial disagreementsbetween them and the regions. This helpsto minimize potential biases in the ratings. Finally,the quality of the ratings is likely to be enhancedby greater intensiveness in the network review,which would entail more resources than havebeen provided for the exercise.Country contextThe greater comparability of the ratings on IBRDthan IDA countries with other indicators highlightsone issue with respect to the CPIA ratings generationprocess—that aspect of country contextthat refers to the stage of development. Specifically,the questionnaire stated, “The criteria weredeveloped to ensure that, to the extent possible,their contents are developmental neutral; thatthe higher scores do not set unduly demandingstandards, and can be attained by a country that,Table 3.10: Share of Instances Where Networks Prevailed When Networks Disagreed withRegions, 2007Region All countries (%) IBRD countries (%) IDA countries (%)Africa 68.3 90.0 65.3East Asia and Pacific 60.7 66.7 59.1Europe and Central Asia 63.0 70.9 46.2Latin America and Caribbean 87.3 87.5 86.7Middle East and North Africa 68.8 66.7 75.0South Asia 27.3 n.a. 27.3Total 68.1 77.2 60.7Source: IEG, based on OPCS data.Note: IBRD= International Bank for Reconstruction and Development; IDA= International Development Association.5 4

R e l i a b i l i t y o f t h e C P I A R at i n g sTable 3.11: Shares of Countries on Which Networks Commented in 2007 When Regions Proposedthe Same Ratings as in 2006CPIA criterion All countries (%) IBRD countries (%) IDA countries (%)q1 89.2 88.9 89.6q2 84.5 89.6 80.0q3 85.3 88.6 82.4q4 50.8 58.5 44.8q5 3.4 5.9 1.5q6 3.6 5.7 1.7q7 28.8 28.8 28.8q8 9.2 9.3 9.2q9 83.8 88.5 80.0q10 44.6 44.6 44.6q11 0.0 0.0 0.0q12 13.7 16.7 11.4q13 1.0 0.0 1.9q14 14.0 18.0 10.9q15 5.0 7.4 3.1q16 8.4 8.0 8.8Total 31.5 33.7 29.6Source: IEG, based on data from OPCS.Note: IBRD= International Bank for Reconstruction and Development; IDA= International Development Association.given its stage of development [italics by IEG],has a policy and institutional framework thatstrongly fosters growth and poverty reduction.”(CPIA Questionnaires 2004–08). (The mostrecent CPIA Questionnaire available at the timeof this writing was for 2008).Different practices are adopted by the regionsand networks with respect to country context.Discussions with regional reviewers indicatethat none of the regions take country contextinto account in the rating exercise. Discussionswith network reviewers indicate that “countrycontext” is interpreted quite differently by differentnetwork reviewers. Some interpret it to meanthat the country-specific information providedby the country teams needs to be taken intoaccount in the assessment. Some interpret it tomean that different policies and institutions canachieve similar results. Still others interpret it tomean the stage of development, but account forit in different ways, with some ways being moresubjective than others. Those who account forthe stage of development objectivelyadjust the indicators they are assessingby per capita incomes (for the gendercriterion, and for the finance criterionuntil two years ago). Regardless ofhow country context is interpreted,many network reviewers find theconcept difficult to implement, andhad comments such as “...this is thesingle toughest thing.”Aside from the mixed interpretation of “countrycontext”—which can distort the quality of theratings—the fact that judgment is involved inaccounting for “country context” introducesThe greater comparabilityof the ratings for IBRDthan for IDA countrieswith other indicatorshighlights the problemwith accounting for thestage of developmentin the CPIA ratings.5 5

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tRegardless of how it isinterpreted, networkreviewers find the conceptof country contextdifficult to implement.further subjectivity to an exercise thatalready relies centrally on judgment.Minimizing the amount of subjectivityinvolved in the rating exercise wouldhelp to enhance the quality of theratings.The first interpretation of country context bynetwork reviewers—taking into account countryspecificinformation—is reasonable. After all,the deep country knowledge of Bank staff is themajor value added that the Bank brings to theCPIA rating exercise.Accounting for the An example can be found in the genderstage of development is criterion, which assesses, amongproblematic because of other things, the share and growththe judgment involved rate of parliamentary seats occupiedand the uneven practice by women. However, a straightforwarduse of these indicators couldacross the Bank.be misleading. For example, a recentstudy on Bangladesh revealed that althoughgender quotas increased the total number ofwomen in political arenas, their representation inthe decision-making process is still not ensuredas elected female representatives in Bangladeshface social, cultural, and religious challengeswhich hinder their participation (Panday 2008).This is the very kind of useful qualitative informationthat the country team could potentiallyprovide in terms of “country context.”Regarding the second interpretation that differentpolicies and institutions can lead to similarresults, the literature review conducted for thisevaluation indicates that, by and large, the CPIAcriteria overlap with what are considered as theconsensus determinants of growth and povertyreduction by the wider research community, andnot just by the Bank (chapter 2). However, thetrade criterion is problematic and needs to berevised, as discussed in chapter 2.Implementation of the third interpretation ofcountry context—the stage of development—is also problematic. The simplest way would beto adjust the ratings quantitatively, using percapita incomes. The issue then arises as to whichcriteria or subcriteria to adjust, because the stageof development is only pertinent for some (sub)criteria but not others. For example, the developmentof the financial sector clearly dependson the stage of development of the country.Therefore, it is reasonable to adjust the indicatorbeing assessed (for example, private sector creditas a GDP) by per capita incomes, which hadpreviously been, but is no longer, done by thenetwork reviewer. It is not clear how much thestage of development matters for other criteria.Thus, deciding which criteria or subcriteria toadjust for the stage of development would itselfbe controversial.In addition to which (sub)criteria to adjustfor stage of development, how to adjust such(sub)criteria is also an issue. Although it ismore straightforward to adjust quantitativeindicators (although the methodology can besubject to debate), when it comes to qualitativeindicators (and many CPIA criteria areassessed on such indicators), it is very difficultto adjust and the process could be open to a lotof arbitrariness.The adjustment of CPIA ratings by the stage ofdevelopment has affected the quality of theseratings, to the extent that some network reviewersindicated to the IEG team that they do notuse the ratings for their own analytical work. Incontrast, network reviewers who do not adjustthe CPIA ratings for the stage of developmentindicated to the IEG team that they do use theratings for their own analytical work, reflectingthe confidence they have in the ratings.IEG recommends excluding accounting for thestage of development from the CPIA exercise. Ifthis cannot be done, at the very least it is importantto clarify and justify in the CPIA guidelines whichcriteria need to take into account the stage ofdevelopment and how such adjustments need tobe made.5 6

Chapter 4

Quay crane on docks, Sri Lanka. Photo by Dominic Sansoni, World Bank

Findings andRecommendationsThe CPIA criteria are largely relevant for sustaining growth andimproving welfare. By and large, the CPIA covers the determinants—policies and institutions—of growth and poverty reduction identifiedin the literature. However, some criteria can be streamlined and one needs tobe added. The assessment of certain criteria needs to be coordinated, and thecontent of all criteria reviewed (see recommendations later in this chapter).OverviewThis evaluation takes the premise that beyondinforming IDA allocation, the CPIA is useful asa broad indicator of development effectiveness.It reviews the appropriateness of the CPIA asan indicator that assesses the conduciveness ofa country's policies and institutions to fosteringpoverty reduction, sustainable growth, andthe effective use of development assistance. Itassesses the relevance of the content of the CPIAthrough a review of the economics literature.It also assesses the reliability of CPIA ratings intwo ways—through comparing CPIA ratings withsimilar indicators, and through reviewing theCPIA ratings generation process. Based on theseassessments, the main findings and recommendationsare as follows.Main FindingsCPIA contentThe evidence is mixed as to whether theCPIA criteria are relevant for aid effectivenessas defined in the literature—that is,whether the criteria represent the policiesand institutions that are important for aidto lead to growth. Much of the literature onaid effectiveness uses cross-country empirics toestimate the impact of aid on growth, with growthrepresenting aid effectiveness. The review of theliterature indicates that there is limited consensuson the impact of aid on growth itself, and on theconditions under which aid can have a positiveimpact on growth. Also, this evaluation could notestimate the impact of IDA assistance and CPIAratings on growth because the restructuring ofthe CPIA in 2004 has resulted in a discontinuityin the CPIA series.However, CPIA ratings are found to bepositively associated with Bank loan performance.Specifically, empirical analysis conductedfor this evaluation finds that the ratings of theoverall CPIA as well as those for each of the CPIAclusters are negatively associated with the shareof problem loans (that are in turn correlated withloan performance).5 9

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tEmpirical analysis also indicates that thereis insufficient evidence from the data toconclude that cluster D associates betterwith loan performance than the otherthree clusters. The new country performancerating that is used for IDA allocations has madeexplicit the relative weights applied to the differentclusters of the CPIA, that is, 8 percent on eachof CPIA clusters A, B, and C, and 68 percent onCPIA cluster D. Neither the literature review onthe determinants of growth, poverty reductionand development effectiveness, nor the empiricalanalysis conducted by IEG, has providedevidence to justify these (or any other) specificweights. It can therefore be surmised that theway the CPIA is currently being used for IDAallocation—that is, with a large emphasis oncluster D—seems to be driven much more byfiduciary and possibly other concerns of donorsthan by the objectives of achieving sustainedgrowth and poverty reduction.The CPIA strives to allow for country specificity,although there are some potentialpitfalls. An important aspect of country specificityis that different policies and institutions canproduce similar outcomes. The CPIA strives toprovide for this aspect of country specificity inits instruction to staff: when assessing policiesand institutions, outcomes need to be taken intoaccount. Indeed, outcome indicators are includedin the assessment of certain CPIA criteria (forexample, finance and gender), but they could beadded to other criteria, in particular trade.The CPIA does not adequately allow forcountry specificity in its trade criterion. Theway in which the trade criterion is specified doesnot allow for different approaches to trade liberalizationthat have proven successful in countryexperiences. The specification of particular tariffrates for different ratings reflects a one-size-fitallapproach to trade liberalization that is notsupported by country experience. Revising thiscriterion by changing the way trade restrictivenessis assessed, and including an assessmentof export performance as the outcome variable,would allow for more country specificity to beincorporated into the criterion.The trade criterion does not reflect theimportance of complementary institutionsfor improving trade performance.Incorporating export performance in theassessment of the trade criterion would alsoreflect the evidence that integrating into theglobal economy—an important determinantof growth—requires integration on both theexport and import fronts. Country experiencefurther indicates that complementary factors—including trade facilitation—are also importantfor export growth and, in fact, more importantthan further tariff reduction once countriesreach moderate tariff levels (which practicallyall countries currently have). Yet not only doesthe trade criterion of the CPIA focus mostly onthe import side, but the much larger weightaccorded to trade restrictiveness (two-thirds)than to trade facilitation (one-third) also doesnot give enough importance to complementaryfactors.Accounting for country specificity requiressubstantial judgment. Incorporating outcomevariables in the assessment of CPIA criteria allowsfor country specificity, although it needs to berecognized that this entails substantial judgment.On the one hand, the reason the CPIA focuseson assessing policies and institutions is to avoidpenalizing countries for not achieving certainoutcomes because of exogenous factors. On theother hand, assessing outcomes could penalizecountries at a lower stage of development for notachieving those outcomes (such as the share ofprivate sector credit as a share of GDP). Thus,substantial judgment is needed to take these twoaspects into account in a balanced fashion.The debate over the weighting scheme isnot very relevant for the use of the CPIA asa broad index, although it is very relevantin its use in the PBA formula. With respect tothe use of the CPIA as a broad index of policiesand institutions, the debate over the weightingscheme is not very relevant given the highcorrelation between the ratings of the CPIAclusters. However, the weights applied to thedifferent CPIA clusters do matter for the allocationof IDA funds.6 0

F i n d i n g s a n d R e c o m m e n d at i o n sThese findings raise the question of the usefulnessof aggregating the different CPIA clustersinto an overall index according to any predeterminedweighting scheme. In the case of the CPIAas a broad index of development effectiveness, itdoes not allow for country specificity which wouldimply different weights on the clusters dependingon the initial conditions and the countries’stage of development. In the case of the CPIA asan indicator for the allocation of IDA funds, theoverall CPIA index is no longer used as such.The CPIA is missing an assessment on otherdisadvantaged socioeconomic groups asidefrom gender. Currently, only gender is beingassessed with respect to equality. Yet countryevidence indicates that social exclusion of othergroups could have severe poverty and growthimplications.Important interlinkages between certaincriteria are not reflected in the CPIA. Countryevidence indicates that intersectoral labor mobilityneeds to be ensured before trade liberalizationproceeds. Otherwise, trade liberalization couldexacerbate poverty. Similarly, the assessmentof fiscal policy (q2) and the quality of budgetaryand financial management (q13) needs togo hand-in-hand so that the fiscal aspect of thecountry in its entirety is realistically captured.Assessment or reporting of certain CPIAcriteria can be streamlined or restructured.The current content of the criterion onequity of public resource use is redundant. Thisdoes not mean that equity is not important. Infact, equity has been identified in the literatureas one of the determinants of growth on whichthere is consensus. However, the measuresidentified in the literature as being importantfor equity—property rights, access to credit,access to education, gender equality, and incometransfers—are already covered by other CPIAcriteria. The criterion is currently assessed ontwo fronts—the public expenditure and publicrevenues sides. The former is covered by otherCPIA criteria, whereas the assessment of thelatter requires incidence analysis of taxes, whichis rarely done. As a result, Bank staff does nothave enough information to meaningfully ratethis subcomponent.There are overlaps in the assessment of somecriteria. Tax policy is an intrinsic part of fiscalpolicy and can be assessed as part of it rather thanseparately, as is the case now. Judicial independenceis assessed in two different criteria andcorruption in three different criteria of cluster D,all of which can be streamlined.The assessment of the environment criterion isonerous. It requires country teams to answer 85questions to arrive at one rating.The three economic management criteria—macroeconomic management, fiscal policy,and debt policy—are conceptually not distinctfrom each other, and hence need to be, andindeed are assessed together. Yet separate scoresare prepared and published for each of the threecriteria (for IDA countries), which could lead toconfusion.The in-depth literature review of thefinancial sector criterion reveals room forimprovement. Although the criterion coversthe dimensions along which finance is currentlythought to be important—stability, depth andefficiency, and access—the relative weights of thethree dimensions (which are currently equallyweighted) need to be revisited. This followsfrom the considerable evidence of a large impactof banking crises on output losses and on thenational budget. At the same time, the evidenceon microfinance is mixed. Further, there isa greater focus in this criterion on assessingintermediate outcomes rather than policies andinstitutions, in particular regarding the financialdepth dimension. It would be useful to includepolicies and institutions for fostering an enablingenvironment for the financial sector here,namely the legal, contractual, informational, andgovernance framework. Also, the indicators forassessing financial stability can be strengthened.CPIA ratingsThe CPIA ratings correlate relatively wellwith similar indicators in terms of relative6 1

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n trankings of countries and direction ofchange. For each of the 16 CPIA criteria, therank correlations of CPIA ratings with similarindicators average between 0.7 and 0.8. Otherindicators are found to correlate better with theCPIA ratings by the Bank than by the AfDB andthe ADB, which are the closest comparators tothe Bank, as they use almost exactly the sameCPIA guidelines as the Bank.CPIA ratings correlate better with otherindicators for IBRD than for IDA countries.This could be because more informationis available on IBRD countries than on IDAcountries, which increases the likelihood ofdifferent institutions having similar assessmentson IBRD countries. This could also bebecause the Bank takes into account the stage ofdevelopment when rating countries. This is morepertinent for IDA countries and entails judgment,which would likely make ratings for IDA countriesless comparable with those by other institutions.Accounting for the stage of developmentin the CPIA ratings is problematic. As noted,accounting for the stage of development couldhave affected the quality of the CPIA ratings,not only because of the judgment involved, butalso because of the different practices employedacross the Bank. None of the regional reviewersand only some of the network reviewers takethe stage of development into account in theirratings. The network reviewers who take stageof development into account do it in a variety ofways—some by adjusting quantitative indicatorswith per capita incomes, and some by their ownjudgment. Further, accounting for the stage ofdevelopment means that the CPIA is no longeran index in the true sense of the word.The strength of the CPIA ratings is Bankstaff ’s professional judgment. The centraldeterminant of the ratings is the expert judgmentof Bank staff (with deep country knowledge),which is clearly the major asset of this exercise.At the same time, however, there is a potentialconflict of interest in having Bank staff provideratings that are used for allocating IDA resourcesto those countries on which the work programsof Bank staff depend. However, analysis of the2007 review process indicates that there did notseem to be much of an upward bias in ratings foreither IDA or IBRD countries. Such an upwardbias could be detected in only 6 percent of theratings for IDA countries, and a slightly lower 5percent of the ratings for IBRD countries.The multistep review process of the CPIA,with the networks having the central rolein the validation of the ratings, helps guardagainst the potential biases in ratings. Specifically,the networks were found to prevail for a largemajority of the time when they disagreed with theregions over the latter’s initial rating proposals in2007. Interestingly, the networks prevailed moreoften for IBRD countries than for IDA countries.This could either mean that the regions have moresupporting evidence for the ratings they proposedfor IDA than for IBRD countries, or that theregions put up a greater effort for IDA countriesbecause IDA funds are involved (this evaluationhas obtained anecdotal evidence on that front).Both the regional and network reviewerspointed out the high degree of judgmentinvolved in rating the criteria in the publicsector management and institutionscluster (D). This calls into further questionthe large weight this cluster has in the IDAallocation formula. All of the regional reviewerswho were asked the open-ended questionof which criteria they found particularly difficultto rate gave cluster D as their response. One ofthe network reviewers for cluster D indicatedthat a lot of judgment was involved in most ofthe criteria in that cluster and pointed to thecriterion on transparency, accountability, andcorruption as an example.RecommendationsBased on the above findings, IEG has severalrecommendations to Bank managementfor improving the CPIA as an indicator thatrepresents the policies and institutions thatare important for sustaining growth, fosteringpoverty reduction (or enhancing welfare morebroadly), and the effective use of developmentassistance.6 2

F i n d i n g s a n d R e c o m m e n d at i o n sThese recommendations are aimed at enhancingthe CPIA as a broad indicator of policies andinstitutions, more than just as an indicator for theallocation of IDA funds. If the CPIA were viewedonly in the latter context, then the questioncould be raised as to the necessity of rating IBRDcountries.Adoption of these recommendations could resultin a discontinuity in the CPIA ratings, which Bankmanagement has been trying to avoid. However,it is important that the CPIA reflect the latestthinking in development paradigm and lessonslearned (both of which are stated intentionsof the Bank regarding the CPIA). It would alsoprovide the opportunity to address an issue thatsome network reviewers have raised regardingthe quality of the ratings for some criteria becauseof what they perceive as inflated baseline ratingsfrom a few years ago. The proposed recommendationsare as follows:Disclose ratings for IBRD countries. Disclosureis important for accountability and transparencyand would further enhance the quality ofthe ratings.Remove accounting for the stage ofdevelopment in the CPIA rating exercise.If this cannot be done, at the very least the CPIAguidelines need to specify and justify whichcriteria should take into account the stage ofdevelopment and how the adjustments shouldbe made.Undertake a thorough review of eachCPIA criterion and revise as necessary. It isrecommended that the review entail an in-depthliterature review for each criterion and reflectthe latest thinking on development and lessonslearned. The review needs to take into accountthe balance between liberalization and regulation.It also needs to examine the clustering ofthe criteria, in particular having social sectorsand the environment in one cluster. Guidepostsfor assessing the criteria need to be reviewed atthe same time. It is also recommended that thefollowing be taken into account in the review andrevisions:• Revision of the trade criterion (q4). A subcomponenton exports needs to be added thatassesses export performance and policies andinstitutions to reduce anti-export bias (such asexport rebate and duty drawback). This newsubcomponent, and the existing subcomponentson trade restrictiveness and trade facilitation,need to all receive equal weights (thatis, one-third weight each). The tariff rates inthe trade restrictiveness subcomponent needto be revised to reflect country experiencethat at moderate tariff levels (which almostall countries have), further tariff reduction isless important than complementary factors(such as macroeconomic stability and tradefacilitation) for global integration.• Dropping or reformulating the criterion on equityof public resource use (q8), as the currentcontent is already covered by other criteria.• Addition of an assessment of other marginalizedsocioeconomic groups to the CPIA. Theassessment of other marginalized socioeconomicgroups could either be added as a newcriterion (in place of the criterion on equity ofpublic resource use, which IEG recommendsdropping) or added to a reformulated criterionon equity of public resource use.• Revision of the financial sector criterion (q5).The weights on the three subcomponents—stability, depth and efficiency, access—needto be revisited in light of the importance ofstability and the mixed evidence on microfinance.Policies, regulations, and institutionsfor fostering an enabling environment for thefinancial sector need to be added. The indicatorsused in the assessment of financial stabilityneed to be strengthened.• Combining the assessment of tax policy(q14a) with the assessment of fiscal policy.That part of tax policy that assesses importtariffs is already being assessed in the tradecriterion, whereas the part on export rebateand duty drawback need to be incorporatedinto the revised trade criterion as suggestedabove.• Streamlining the assessment of judicial independencein the public sector managementand institutions cluster. Currently, judicial independenceis assessed in both the criterion6 3

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n ton property rights and governance (q12) andthe criterion on transparency, accountability,and corruption in the public sector (q16).• Streamlining the assessment of corruption inthe public sector management and institutionscluster. Currently corruption is assessed in thecriteria on efficiency of revenue mobilization(q14), the quality of public administration(q15), and transparency, accountability, andcorruption in the public sector (q16).• Strengthening the assessment of the environmentcriterion (q11) and make the processmore efficient. Currently, staff need to answer85 questions to arrive at one rating.• Reporting only the consolidated score for theeconomic management cluster.Consider not producing an overall CPIAindex although continue to produce andpublish the separate CPIA components.IDA is already using the components separatelyin the PBA formula. With respect to the use ofthe CPIA as a broad index of policies and institutions,this would allow for country specificity,as different weights could be assigned to thedifferent clusters depending on the country’sinitial conditions and stage of development.Producing the different components of theCPIA without assigning weights to them toarrive at an aggregate index would allow fordifferent weights to be applied according tocountry contexts and use.6 4

Appendixes6 5

6 6Landscape view of water, glaciers and mountains, Chile. Photo by Curt Carnemark/World Bank.

Appendix A. 2008 Country Policy and Institutional Assessment CriteriaCluster(weight in index)Economicmanagement(0.25)Structuralpolicies(0.25)q1q2q3q4q5Criteria(weight in index)Macroeconomicmanagement(0.083)Fiscal policy(0.083)Debt policy(0.083)Trade(0.083)Financial Sector(0.083)Indicators(weight in index)• Monetary/exchange rate policy with clearly defined price stability objectives• Aggregate demand policies focus on maintaining short- and medium-term external balance.• Avoid crowding out private investment• Primary balance managed to ensure sustainability of public finances• Public expenditure/revenue can be adjusted to absorb shocks• Provision of public goods including infrastructure consistent with medium-term growth• Debt burden indicators do not signal debt servicing difficulties• External and internal debt contracted with view to achieving/maintaining debt sustainability• Coordination between debt management and other macroeconomic policies• Debt management unit well established, has adequate system for recording and monitoringdebt, and good analytical capacity as indicated by regular analytical work on debt• Accurate, timely, and publicly available debt data• Government has clear financing strategy and the legal framework for borrowing is clearlydefined.• 75 percent weight for trade restrictiveness: (0.063)• Average tariff rates, number of tariff bands, maximum tariff band• Internal taxes do not discriminate between imported and local products• Transparency and predictability of trade regime including in the use of non-tariff barriers• 25 percent weight for customs/trade facilitation: (0.02)• Reputation of customs with respect to professionalism and corruption• Use of risk management, information technology (IT), physical examination• Processing of collections and refunds• Documentation of customs procedures• Resolutions of appeals of customs decisions.Financial stability• Banking sector’s vulnerability to shocks• Banking system soundness (share of non-performing loans [NPLs] and level of capital at risk)• Adherence to Basel Core Principles• Quality of risk management in financial institutions• Quality of supervision.Financial sector efficiency, depth, and resource mobilization• Size and reach of financial markets• Development of capital markets• Interest rate spreads• Private sector credit/GDP• Efficiency of microfinance.(continued on next page)6 7

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tCluster(weight in index)Structuralpolicies(0.25)(cont.)Policiesfor SocialInclusion/Equity(0.25)q6q7q8q9Criteria(weight in index)BusinessRegulatoryEnvironment(0.083)Gender Equality(0.05)Equity of PublicResource Use(0.05)Building HumanResources(0.05)Indicators(weight in index)Access to financial services• Development of payment, clearance, and credit reporting systems• Share of population with access to formal sector financial services• Access of small and medium enterprises (SMEs) to finance• Legal and regulatory framework supporting access to finance.Regulations affecting entry, exit, and competition (0.028)• Bans on, or, investment licensing requirements• Entry and exit procedures• Legal framework (and implementation thereof) to address anti-competitive conduct by firms• Procurement by public sector firms.Regulations of ongoing business operations (0.028)• Operational licensing, permits, compliance and inspection requirements including taxes andcustoms• State intervention in goods markets (state ownership in competitive sectors, price controls,state making administrative allocation/decisions about production)• Corporate governance laws (and enforcement thereof) to encourage disclosure and protectshareholders rights.Regulations of goods and factor markets (0.028)• Employment law provides for flexibility in hiring and firing• State intervention in labor and land markets limited to regulation and/or legislation tosmooth out market imperfections• Procedures to register property are simple and low-cost.Human capital development (0.017)• Differences (between male and female) in primary completion rates, and access tosecondary education (female to male enrollment)• Access to delivery care and family planning services• Adolescent fertility rate.Access to economic and productive resources (0.017)• Gender disparities in labor force participation, land tenure, property ownership, andinheritance practices.Status and protection under the law (0.017)• The law gives men and women equal individual and family rights• Violence against women considered a crime• Gender disparities in political participation at the national level.Government spending (0.033)• Identification of individuals, groups, or localities that are poor, vulnerable, or have unequalaccess to services and opportunities• Adoption of national development strategy with explicit interventions to assist groupsidentified above• Systematic tracking of composition and incidence of public expenditures and their resultsfeedback into subsequent allocations.Revenue collection (0.017)• Incidence of major taxes (progressive or regressive) and their alignment with povertyreduction priorities.Health and nutrition including reproductive health (0.017)• Equitable access to basic health services• Prevention of malnutrition.6 8

A p p e n d i x A . 2 0 0 8 C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n t C r i t e r i aCluster(weight in index)Policiesfor SocialInclusion/Equity(0.25)(cont.)PublicSectorManagementandInstitutions(0.25)q9(cont.)q10q11q12Criteria(weight in index)Building HumanResources(0.05)SocialProtection andLabor (0.05)Policies andInstitutions forEnvironmentalSustainability(0.05)Property Rightsand Rule-BasedGovernance(0.05)Indicators(weight in index)Education (0.017)• Sustained progress toward universal basic education, literacy, and more equitable access toearly child development program services• Standards for teacher preparation, student learning, and oversight of private/nongovernmentalorganization (NGO) providers• Systematic tracking of school performance and student learning outcomes and feedback toschools and parents• Policies for post-basic education and training services• Quality, equity of access, and efficiency of resource use.Human immunodeficiency virus/Acquired immunodeficiency syndrome (HIV/AIDS), tuberculosis,malaria• Prevention, treatment, care and support of HIV/AIDS, tuberculosis, and malaria• Track disease prevalence, resources, and program implementation.• Quality and timeliness of services• Focus on the poor• Cost-effective use of public resourcesSocial safety net programs (0.01)• Social protection programs provide income support to poor and vulnerable groups.Protection of basic labor standards (0.01)• Ratification and implementation of international core labor standards.Labor market regulations (0.01)• Labor market regulations on health and safety, working conditions, and hiring and firing.Community-driven initiatives (0.01)• Encourage and support communities’ own development initiatives or local accountabilitymechanisms.Pension and old-age savings programs (0.01)• Pension and savings programs provide income security to most potentially vulnerablegroups.• Regulations and policies (and implementation thereof) for pollution and natural resource• Information widely available• Priority setting• Sector ministries incorporate environmental concerns.Legal basis for secure property and contract rights (0.017)• Transparent and well-protected property rights• Current and non-corrupt property registries• Enforced contracts.Predictability, transparency, and impartiality of laws and regulations affecting economic activity(0.017)• Transparent and predictable laws and regulations affecting businesses and individuals• Low-cost means for pursuing small claims• Impartial and predictable applications of laws and regulations.Crime and violence as impediment to economic activity (0.017)• Well-functioning and accountable police force protects citizens and their property fromcrime and violence.(continued on next page)6 9

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tCluster(weight in index)PublicSectorManagementandInstitutions(0.25)(cont.)q13q14q15q16Criteria(weight in index)Quality ofBudgetaryand FinancialManagement(0.05)Efficiencyof RevenueMobilization(0.05)Qualityof PublicAdministration(0.05)Transparency,Accountability,and Corruptionin the PublicSector (0.05)Indicators(weight in index)Comprehensive and credible budget linked to policy priorities (0.017)• Multiyear expenditure projections integrated into budget formulation process• Spending ministries and the legislature consulted in budget formulation, adhering to fixedbudget calendar• Budget classification system comprehensive and consistent with international standards• Minimal and transparent off-budget items.Financial management (0.017)• Budget implemented as planned• Budget monitoring based on management information systems• Negligible payment arrears.Fiscal reporting (0.017)• Reconciliation of banking and fiscal records• Regular in-year fiscal reporting• Timely preparation of public accounts• Timely auditing of accounts and appropriate action taken on budget reports and auditfindings.Tax policy (0.025)• Bulk of revenues from low-distortion taxes such as sales/value-added tax (VAT), property,and so forth• Low and relatively uniform import taxes• Functional export rebate or duty drawback• Broad tax base• Few arbitrary exemptions.Tax administration (0.025)• Rule-based tax administration• Low administrative and compliance costs• Taxpayer service and information program• Efficient and effective appeals mechanism.Policy coordination and responsiveness (0.0125)• Effective coordination mechanism to ensure high degree of policy consistency.Service delivery and operational efficiency (0.0125)• Organizational structures along functional lines with little duplication; regular review ofbusiness processes to ensure efficient decision making.Merit and ethics (0.0125)• Hiring and promotion based on merit and performance; ethics prevail.Pay adequacy and management of the wage bill (0.0125)• Sustainable wage bill that does not crowd out public services spending• Pay and benefit levels adequate• Flexibility in paying higher wages for hard-to-fill positions.Accountability of the executive to oversight institutions and of public employees for theirperformance (0.017)• Strong public service ethic reinforced by audits, inspections, and adverse publicity forperformance failures• Independent and impartial judiciary• Corruption monitored and sanctions implemented.Access of civil society to information on public affairs (0.017)• Results and costs of government decisions clear and communicated to public• Citizens can access government documents at nominal cost• Media independent of government and can fulfill critical oversight roles.State capture by narrow vested interests (0.017)• Conflict of interest and ethics rules for public servants observed and enforced• Top government officials required to disclose income and assets and can be prosecuted formalfeasance.7 0

Appendix B: Public Sector Literature ReviewCountry Policyand InstitutionalAssessment (CPIA)Criteria/IndicatorsTransparent andwell-protectedproperty rightsCurrent and noncorruptregistriesSustained growthLiterature on:Poverty reductionProperty Rights and Rule-Based GovernanceLegal basis for secure property and contract rights• Property rights associated with per capitaincomes (Besley 1995).• Property rights associated with investments(Acemoglu, Johnson, and Robinson 2005;Bardhan 2006b).• Existence of market exchange presupposesproperty rights (Rodrik 2003).• Positive correlation between property rightsand growth—cross-country studies (Knack andKeefer 1995; Mauro 1995; Hall and Jones 1999;Rodrik 1999; De Soto 2000; Rodrik, Subramanian,and Trebbi 2002; Summers 2003; Kerekes andWilliamson 2008).• Positive correlation between property rights andgrowth—micro studies (Mazingo 1999; Johnson,McMillan, and Woodruff 2002).• Use of registration system to establish identityof property owners (Shavell 2003).• De Soto (2000) used counter-examples fromMalawi and Peru to underline the importance ofa current and non-corrupt registry system.Enforced contracts • Contract enforcement and growth (North 1990;Knack and Keefer 1995; Levine 1998; Hall andJones 1999; Kaufmann, Kraay, and Mastruzzi1999; Hellman, Jones, and Kaufmann 2003;Summers 2003).• Governments can help thepoor with access to creditby establishing functioningproperty rights (Fleisig1995).• Obtaining property rightsover land in urban areashelps poor households gainaccess to credit (De Soto2000; Fields and others2002).• Giving the poor land rightsenhances their ability toutilize and invest in landthey cultivate (Deolalikarand others 2002).• Increase in protection ofproperty rights across theglobe of half of one standarddeviation would halve globalpoverty (Besley and Burgess2003).• Increased protection ofproperty rights has strongeffects in reducing poverty(Acemoglu, Johnson, andRobinson 2001).Developmenteffectiveness(continued on next page)7 1

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tCountry Policyand InstitutionalAssessment (CPIA)Criteria/IndicatorsTransparent andpredictable laws andregulations affectingbusinesses andindividualsLow-cost meansfor pursuing smallclaimsImpartial andpredictableapplications of lawsand regulationsWell-functioning andaccountable policeforce to protectcitizens and theirproperty from crimeand violenceSustained growthLiterature on:Poverty reductionPredictability, transparency, and impartiality of laws and regulations affecting economic activity• Laws and regulations should be transparent toenable very imperfect courts to verify violationsand correct wrongs (Hay, Shleifer, and Vishny1996).• Unpredictable regulations are a disincentive toinvestment and hence lower growth: example(Gyimah-Brempong and Munoz de Camacho2006).• Growth depends positively on the rule of law(Barro 2003).• La Porta, Lopez-de-Silanes, and Shleifer (2008)emphasize that costly contract claims donot protect investors [and hence would be adisincentive to investment].• Predictability and impartiality characterizesthe quality of the law enforcement, and richercountries have higher quality law enforcement(La Porta and others 1998).Crime and violence as impediment to economic activity• The police (and the courts) are most directlyinvolved in determining and defending propertyrights (Andvig and Fjeldstad 2008).• High crime rates may have devastating impactson investments and economic growth (Andvigand Fjeldstad 2008).• Crime has direct costs on firms through theftlosses and security-related expenses, whichreduce competitiveness and lower investment(Bourguignon 1998).• Security issues must be addressed to fullycomprehend the nature and possibilities forsocioeconomic development (Londono 1996;Moser 1996; Moser and Holland 1997; Inter-American Development Bank 1997; Ayers 1997;Buvinic, Morrison, and Shifter 1998; Call 2000).• Crime and violence are major obstacles todevelopment objectives including lost growth inLatin America and the Caribbean (Schneidman1996; Ayers 1997; Buvinic and Morrison 2000)and through losses in human capital (Heinemannand Verner 2006).Quality of budgetary and financial management• Police corruption especiallyin slum areas of poorercountries may increaseuncertainty of propertyrights of the very poor(Andvig and Fjeldstad 2008).• Violence disproportionatelyaffects the poor in LatinAmerica, eroding theirassets and livelihoods(Heinemann and Verner2006).• Victims of crimes are morelikely to be the poorer part ofthe population (Bourguignon1999; Deolalikar and others2002).DevelopmenteffectivenessGood budgetary and financial management is of particular importance in developing countries because the absence of aggregate fiscaldiscipline could result in large unsustainable deficits that translate into an unstable macroeconomic environment (high inflation, high interestrates, burgeoning current account deficits) that ultimately retard growth (Fischer 1991; Easterly, Rodriguez and Schmidt-Hebbel, 1995;Gupta and others 2005). More effective public expenditure management and macroeconomic and budget stability are important for publicexpenditures to better serve the poor (Foster and others 2002).7 2

A p p e n d i x B : P u b l i c S e c t o r L i t e r at u r e R e v i e wCountry Policyand InstitutionalAssessment (CPIA)Criteria/IndicatorsMultiyearexpenditureprojectionsintegrated intobudget formulationprocessSpending ministriesand the legislatureconsulted in budgetformulation, adheringto fixed budgetcalendarBudget classificationsystemcomprehensiveand consistentwith internationalstandardsMinimal andtransparent offbudgetitemsBudget implementedas plannedBudget monitoringbased onmanagementinformation systemsNegligible paymentsarrearsSustained growthLiterature on:Poverty reductionComprehensive and credible budget linked to policy priorities• Aggregate fiscal discipline depends on theexistence of a medium-term expenditureframework (Campos and Pradhan 1996).• Rationale for multiyear budget approach basedon several potential benefits (Boex, Martinez-Vasquez, and McNab 2000).• Consulting spending ministries is important forthem to have the correct perception of their budgetconstraints, which in turn reduces the possibilityof excess spending (resulting from “aid illusion”)(McGillivray and Morrissey 2001a, 2001b).• Explicit rules that put specific limits on spendingand borrowing and that impose penalties onoverspending by line ministries give centralministries more leverage over claimants (Camposand Pradhan1996).Evidence is mixed:• Serving the poor moreeffectively through publicexpenditure requires amedium–term process forbudget allocation (Fosterand others 2002).• Changes in budgetingand expenditure planningunlikely to generatesignificantly improvedperformance where corefunctions continue tooperate inadequately(Fozzard and Foster 2001).Developmenteffectiveness• No evidence available • No evidence available • No evidence available• Integration of all expenditures in budget canhelp improve accountability and transparencyby imposing political costs on politicians andbureaucrats for violating rules, and raise qualityof budgetary and financial management (Camposand Pradhan 1996).Financial management• No evidence available • No evidence available • No evidence available• No evidence available • No evidence available • No evidence available• No evidence available • No evidence available • No evidence available(continued on next page)7 3

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tCountry Policyand InstitutionalAssessment (CPIA)Criteria/IndicatorsSustained growthLiterature on:Poverty reductionDevelopmenteffectivenessFiscal Reporting. This seems to be important for economic growth. In Latin America, countries with better fiscal transparency and additionalspending controls have average fiscal surpluses of 1.7 percent of gross domestic product (GDP), whereas those with fewer spending controlsand lowest levels of transparency have average deficits of 1.8 percent of GDP (Alesina 1997). Fiscal transparency—and transparency ofthe banking sector—is crucial in reducing vulnerability to economic shocks, particularly following the Asian Economic Crisis (InternationalMonetary Fund 2000). The latter stemmed from inadequate disclosure of risks by government and banks (Fozzard and Foster 2001). Fiscalreporting is a core element of transparency, empowerment, and accountability (World Bank 2004g).Reconciliation ofbanking and fiscalrecords• No evidence available • No evidence available • No evidence availableRegular in-year fiscalreportingTimely preparation ofpublic accountsTimely auditingof accounts andappropriate actiontaken on budgetreports and auditfindingsTax policyBulk of revenuesfrom low-distortiontaxes (sales/VAT,property, and so on)• No evidence available • No evidence available • No evidence available• No evidence available • No evidence available • No evidence available• No evidence available • No evidence available • No evidence availableEfficiency of revenue mobilizationValue-Added Tax (VAT)• VAT is central to a good tax system in mostcountries (Bird 2005; Bahl and Bird 2008).• It is a low distortionary manner to raise taxes,non-cascading, and does not interfere withproduction efficiency (Burgess and Stern 1993).• It is also a good instrument to reduce potentialfor corruption as it minimizes contact betweentaxpayers and tax administrators and movestoward self-assessment (Bahl and Bird 2008).• It is a broad-based tax which is an important partof tax policy (Ames and others 2001).Land (Property) Tax• If well designed and political opposition is wellhandled, property taxes and especially land taxescan raise substantial revenues (Heady 2001).• Property taxation is a potential source ofsignificant income for many municipal andmetropolitan authorities to whom centralgovernments have devolved increasingresponsibilities without commensurate increasesin fiscal transfers (Bird and Slack 2002; Dillinger1992; Mikesell 2003).• Taxing property is much easier now because ofdigital databases of modern property registries(Fjeldstad and Moore 2007).VAT• Although VAT has longbeen known to be likelyregressive (Ahmad andStern 1987; Cnossen 2004;Bird 2005), a recent surveyof studies of consumptiontax incidence have shownsignificantly less regressiveresults than those reportedfor similar taxes in earliersurveys (Bird and Gendron2006).• VATs in developing countriesare not always, or notnecessarily, regressive (Birdand Gendron 2007).• Even when VAT isregressive, it is found to bemore progressive than theimport and excise taxes thatit replaced (Gemmell andMorrissey 2003).7 4

A p p e n d i x B : P u b l i c S e c t o r L i t e r at u r e R e v i e wCountry Policyand InstitutionalAssessment (CPIA)Criteria/IndicatorsTax policy(cont.)Broad tax baseand few arbitraryexemptionsSustained growth• Exemptions in many developing countriesgenerally protect the interest of powerful groups,so reducing exemptions not only raises revenues,but also improves economic efficiency andincome distribution (Heady 2001).• Most developing countries do not have theright conditions (macroeconomic stability anda stable political and administrative system)under which exemptions work, with the resultthat exemptions reduce revenues and complicatethe fiscal system without achieving their statedobjectives (Bird and Zolt 2007).• Tax incentives (through exemptions) are oftendistortive and inefficient and divert scarceresources into less than optimal use (McLure1999).• There may be a limited role for simple incentives,for example as part of a growth-oriented fiscalstrategy as the East Asian experience suggests(Bird and Chen 1998).Literature on:Poverty reduction• To the extent that thepoorest sectors of thesociety remain outside ofthe market economy, a VATmay be broadly progressive(Bird and Gendron 2007; Birdand Zolt 2007).Property Tax• Property incomes andproperty wealth aresignificantly undertaxedand an important sourceof inequity (Fjeldstad andMoore 2007).• Exemptions in manydeveloping countriesgenerally protect theinterest of powerful groups,so reducing exemptions notonly raises revenues, butalso improves economicefficiency and incomedistribution (Heady 2001).• Exempting only five narrowlydefined items in Jamaicacut the VAT burden on thelowest 40 percent of theincome distribution in half(Bird and Miller1989).DevelopmenteffectivenessTax administration. Reforms to tax administration are just as important as tax policy reforms for overall fiscal reform (Mookherjee and Das-Gupta 1995; Devas, Delay, and Hubbard 2001). It is estimated that two-thirds of the rapid increase in Argentina’s tax revenues (from 13 to 23percent of GDP) over the 1989–92 period was attributable to improved administration effort.Rule-based taxadministration• An efficient tax administration has to be rulebased—the legal environment is important (Bird2003).• A stable transparent tax system inspires moreconfidence in its fairness and will result ingreater compliance (Boskin 2006).• A less discretionary tax administration is likelyto encourage political mobilization of taxpayersaround taxation issues, and reduce temptationsto pursue corrupt deals (Moore 2004).(continued on next page)7 5

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tCountry Policyand InstitutionalAssessment (CPIA)Criteria/IndicatorsLow administrativeand compliance costsTax payer serviceand informationprogramEfficient andeffective appealsmechanismSustained growth• One of the most efficient ways to facilitate taxcompliance is to decrease compliance costs (Bird2003).• Low compliance costs also lower the potential ofcorruption as they lower the amount of a bribe ataxpayer might be willing to pay to avoid taxes(Bahl and Bird 2008).• It lowers the costs of operating in the formalsector and hence facilitates growth (Bird 2008).• It has a direct effect on economic growth sincehigh compliance costs divert resources towardadministering and complying with taxes (Bird2008).• Provision of extensive information for taxpayersreduces compliance costs (Bird 2003; Braithwaite2003; Fjeldstad and Moore 2007).• Taxpayer information programs reduce thepotential of corruption, and of non-compliance(Bahl and Bird 2008).• A time-bound appeal procedure reduces delaysin payments (Mexico versus India) (Mookherjeeand Das-Gupta 1995).Literature on:Quality of public administrationPoverty reductionDevelopmenteffectivenessThis is a very important element in the literature on sustained growth. The literature finds that administrative reforms can foster fastereconomic growth and sustain poverty reduction by removing the obstacles to private sector development that a poorly performing public sectorcreates. Reforms can also: increase public resources for priority spending; reduce corruption; and increase accountability of the public sector.A seminal paper (Mauro 1995) finds that the efficiency of the bureaucracy is associated with better rates of investment and growth. Deolalikarand others (2002) underline that administrative reforms, including reform of the bureaucracy and civil service, are one of the major areas ofreform involving public institutions for poverty reduction.Effectivecoordination andorganizationalstructures alongfunctional linesMerit and ethicsHiring and promotionbased on merit andperformance; ethicsprevailPolicy coordination and responsiveness; service delivery and operational efficiency• Improving the quality of the bureaucracy requiresimproving coordination among agencies withoverlapping functions; organizational structuresshould be formed along functional lines(Deolalikar and others 2002).• Inefficiency of public administration stems fromconstraints imposed by the civil service systemon human resource management, especiallyhiring, firing, promotion, and rewards (Devas,Delay, and Hubbard 2001).7 6

A p p e n d i x B : P u b l i c S e c t o r L i t e r at u r e R e v i e wCountry Policyand InstitutionalAssessment (CPIA)Criteria/IndicatorsQuality of publicadministration(cont.)Sustainable wagebill does not crowdout public servicesspendingPay adequacy andmanagement of thewage billSustained growth• Using a “Weberianness Scale” that measuresmeritocratic recruitment and long-term,predictable, rewarding careers, Evans andRauch (1999) found that these characteristicssignificantly enhance prospects for economicgrowth even after taking into account initiallevels of per capita GDP and human capital. Thisis because longer–term horizons associated withpredictable, rewarding careers will increasethe bureaucracy’s propensity to advocate publicsector infrastructure rather than consumptionexpenditures. Meritocratic recruitment alsoincreases competence.• Effective bureaucracy is important, or evenessential, for implementing or maintaininga policy environment conducive to economicgrowth (Rauch and Evans 2000).Literature on:Poverty reductionPay adequacy and management of wage bill• Containing salary expenditures (throughdownsizing) will help increase public resourcesfor priority spending (Deolalikar and others 2002;Bardhan 2006b).Mixed evidence:• Seminal paper (Becker and Stigler 1974) showsthat high wages paired with non-zero auditprobability could be used to deter misbehaviorand corruption.• Empirically, no evidence that wages detercorruption (Van Rijckeghem and Weder 1997;Rauch and Evans 2000; Treisman 2000).• Higher wages are correlated with highercorruption (La Porta and others 1999).• Di Tella and Schargrodsky (2003) explain theapparent empirical failure of the Becker-Stiglerhypothesis because the empirical studies includeobservations from environments with no activeaudits where the probability of being punishedfor corruption is near zero, or very high auditinglevels where the probability of being punished isnear one.• Incentive pay structure is one of the mosteffective ways of fighting corruption (Bardhan1997, 2006a; Chand and Moene 1999; VanRijckeghem and Weder 2001; Di Tella andSchargrodsky 2003; Andvig and Fjeldstad 2008).Developmenteffectiveness(continued on next page)7 7

A p p e n d i x B : P u b l i c S e c t o r L i t e r at u r e R e v i e wCountry Policyand InstitutionalAssessment (CPIA)Criteria/IndicatorsTransparency,accountability, andcorruption in thepublic sector(cont.)Corruption monitoredand sanctionsimplementedSustained growthThere are two strands of literature, with the strandthat finds corruption having a negative impact ongrowth dominating.Corruption has a positive impact on growth:• Where there are pre-existing policy distortionsincluding pervasive and cumbersome regulations,corruption may help efficiency and growth (Leff1964; Huntington 1968; Lui 1985; Beck andMaher 1986; Lien 1986; Bardhan 1997).Corruption has a negative impact on growth when:• Corruption is directly related to variations in thegrowth of per capita income (Knack and Keefer1995).• Corruption and red tape are significantlyassociated with increased levels of investment,which is shown empirically to be one of the mostpowerful predictors of growth (Mauro 1995).• Corruption has adverse effects on investmentand growth (Bardhan 1997, 2006a).• Corruption is associated with higher militaryspending as a share of GDP, hence reductionin corruption will improve composition ofgovernment spending toward more productive,non-military outlays (Gupta, de Mello, andSharan 2000).• Higher bribes imply lower profitability ofproductive relative to rent-seeking investments(Kaufmann, Kraay, and Zoido-Lobaton 1999; Tanzi1998; Acemoglu and Verdier 2000).• Corruption increases uncertainty, hence reducinginvestment in physical and human capital (Wei2000; Alesina and Weder 2002).• Societies’ willingness to tax themselves dependson perception that government institutions arehonest; hence corruption, VAT evasion, and sizeof underground economy are found to be closelylinked (Bird and Gendron 2006).• Corrupt police may have a negative impact ongrowth by taxing businesses through predation,or by supplying protection in competition or incooperation with organized crime units (Andvigand Fjeldstad 2008); police corruption is shownto have significant negative impacts on businessactivities (Wei 2000).Literature on:Poverty reductionCorruption increases incomeinequality: a worsening inthe corruption index by onestandard deviation increasesthe Gini coefficient by 5.4points (Gupta and others 1998).Corruption increases incomeinequality in a sample ofdeveloping countries anddecreases the share ofgovernment expenditures oneducation and health (Gupta,Davoodi, and Terme 2002).DevelopmenteffectivenessAid has a larger effectin displacing domesticrevenues in countrieswith higher levels ofcorruption—doubling ofgrants as a share of GDPis associated with a 1.3percentage point declinein revenues as a share ofGDP in relatively corruptcountries, and as muchas 3.8 percentage pointdecrease in the mostcorrupt countries (Guptaand others 2003).(continued on next page)7 9

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tCountry Policyand InstitutionalAssessment (CPIA)Criteria/IndicatorsLiterature on:Sustained growthPoverty reductionAccess of civil society to information on public affairsDevelopmenteffectivenessMedia independentof governmentand fulfill criticaloversight roles• Free flow of information pressures (evennondemocratic) governments to public action(Dreze and Sen 1995).• Local newspapers increase responsiveness ofIndian state governments to natural disasters(Besley and Burgess 2002).• Media campaign via radio and newspapersinforming local communities of their entitlementto school funds from the central governmentof Uganda (along with increased governmentmonitoring) reduced diversion of funds byintermediating provincial governments from 80to 20 percent (Reinikka and Svensson 2004).• Negative correlation between press freedom andcorruption (Ahrend 2001; Brunetti and Weder1999).• Mass media makes governments moreaccountable (Besley, Burgess, and Prat 2002).• Media can provide a counterbalance to thepower of vested interests especially by informingvoters (Dyck, Moss, and Zingales 2008).• No country with a free presshas ever had a major famine(Dreze and Sen 1989).• Provision of moreinformation (forexample throughthe use of localnewspapers)increases aidefficiency and reducesthe negative impactof aid volatility onaid efficiency (Cagé2009b).State capture by narrow vested interests. Vested interests are a restraint for economic growth (Mokyr 1990; Maine 1980; and Landes1983). Vested interests (such as those arising from import substitution policies or capital taxes) can be detrimental to economic growth (Kruselland Rios-Rull 1996; Acemoglu and Robinson 2000).Top governmentofficials requiredto disclose incomeand assets and canbe prosecuted formalfeasance• Disclosure is correlated with lower corruptionwhen it is public, identifies sources of incomeand conflicts of interest, and when a country is ademocracy; but there is no significant evidenceof benefits from disclosure of values of income,consumption, and wealth (Djankov and others2009).8 0

Appendix C: 2007 CPIA Criteria ON Economic Management, Structural Policies,and Policies for Social Inclusion/Equity and Evidence in the LiteratureCPIA criteria/indicators Sustained growth Poverty reductionMacroeconomicManagement(0.083)Fiscal Policy(0.083)Debt Policy(0.083)• Monetary/exchange ratepolicy with clearly definedprice stability objectives.• Aggregate demand policiesfocus on maintaining shortandmedium-term externalbalance.• Avoid crowding out privateinvestment.• Primary balance managedto ensure sustainability ofpublic finances.• Public expenditure/revenuecan be adjusted to absorbshocks.• Provision of public goodsincluding infrastructureconsistent with mediumtermgrowth.• Debt burden indicators donot signal debt servicingdifficulties.• External and internal debtcontracted with view toachieving/maintaining debtsustainability.• Coordination between debtmanagement and othermacroeconomic policies.• Sound money andfiscal solvency (Rodrik2003).• Keep inflation lowand stable; monetarypolicy stanceconsistent with lowand stable inflation(World Bank 2005).• Transparency,sustainable solvency,flexibility, pro-growthstructure of budgets(World Bank 2005).• Infrastructure(Easterly 2001; WorldBank 1994; Sachs2005, 2008; Collier2007).• Reduce recourse toexternal debt (WorldBank 2005).• Adverse impact ofinflation on the poor(Easterly and Fisher2001).• Inflation hurts thepoor (Dollar and Kraay2002; Ravallion andDatt 2002).• Potential effect ofinflation on poverty,once controlled fordirect effect of growthon poverty, is mixed(Epaulard 2003).• Public investment inbasic infrastructurefacilitates access ofthe poor to marketsor to basic socialservices (Loayza 1996;Calderon and Serven2003, 2004).Developmenteffectiveness• Randomized literatureon infrastructure:water and sanitationinfrastructure (Ashraf,Berry, and Shapiro2007; Kremer andothers 2008).(continued on next page)8 1

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tCPIA criteria/indicators Sustained growth Poverty reductionDebt Policy(0.083)(cont.)Trade(0.083)• Debt management unit wellestablished, has adequatesystem for recording andmonitoring debt, andgood analytical capacityas indicated by regularanalytical work on debt.• Accurate, timely, andpublicly available debtdata.• Government has clearfinancing strategy andthe legal frameworkfor borrowing is clearlydefined.75 percent weight for traderestrictiveness: (0.063)• Average tariff rates,number of tariff bands,maximum tariff band.• Internal taxes do notdiscriminate betweenimported and localproducts.• Transparency andpredictability of traderegime including in the useof non-tariff barriers.25 percent weight forcustoms/trade facilitation:(0.02)• Reputation of customs withrespect to professionalismand corruption.• Use of risk management,information technology,physical examination.• Processing of collectionsand refunds.• Need to take intoaccount promotion ofexport growth (WorldBank 2005, 2008a).• Need to havecomplementarypolicies (World Bank2005).• No conclusiveevidence in therelationship betweentrade liberalizationand poverty (Winters,McCulloch, andMcKay 2004; Harrison2006; Ravallion 2004).• For trade liberalizationto have positiveimpact on povertyreduction, needsto be accompaniedby improvedinfrastructure,adequate competitionpolicies, enhancedaccess to credit,better educationand health, andlow marketing orintermediation costs(Balat, Brambilla, andPorto 2007; Harrison2006).• When trade reformsare accompaniedby labor marketreforms (hire andfire, relocation), theadverse impact oftrade liberalization onpoverty disappears(Goldberg and Pavcnik2005).Developmenteffectiveness8 2

A p p e n d i x C : 2 0 0 7 C P I A C r i t e r i a F O R C L U S T E R S A , B a n d C a n d E v i d e n c e i n t h e l i t e r at u r eCPIA criteria/indicators Sustained growth Poverty reductionTrade(0.083)(cont.)Financial Sector(0.083)• Documentation of customsprocedures.• Resolutions of appeals ofcustoms decisions.Financial stability• Banking sector vulnerabilityto shocks• Banking system soundness(share of non-performingloans and level of capitalat risk).• Adherence to Basel CorePrinciples.• Quality of risk managementin financial institutions.• Quality of supervision.Financial sector efficiency,depth, and resourcemobilization• Size and reach of financialmarkets.• Development of capitalmarkets.• Interest rate spreads• Private sector credit/grossdomestic product (GDP).• Efficiency of microfinance.Access to financial services• Development of payment,clearance, and creditreporting systems.• Share of population withaccess to formal sectorfinancial services.• Access of small andmedium enterprises (SMEs)to finance.• Legal and regulatoryframework supportingaccess to finance.• Considerable evidenceof large impactof banking criseson output losses(Hoggarth, Reis, andSaporta 2002).• Finance is adeterminant ofsustained growth(Aghion, Howitt, andMayer-Foulkes 2005;Krebs 2003; Levine1997, 2003, 2005;Levine, Loayza, andBeck 2000).• Efficient domesticfinancial system isimportant for growth(World Bank 2005).• Cross-countryregressions donot suggest anystrong influence ofhousehold financialaccess on growthor poverty reduction(Honohan 2008a).• Access of SMEs tofinance promotes firmentry, firm growth,innovation, and sizedistribution of firms(Beck, Demirguc-Kunt,and Levine 2005;Ayyagari, Demirguc-Kunt, and Maksimovic2007; Klapper, Laeven,and Rajan 2006).• Negative impact oftrade reforms onpoverty is due tolimited labor mobility(Topalova 2004, 2005).• Crises do notsystematically worsenthe Gini coefficientalthough the poorare likely less able toabsorb adverse shocks(Honohan 2004a).• Financial (banking)depth is negativelyassociated withheadcount poverty(Honohan 2004b).• Impact of nationalincome volatilityon child labor isinsignificant forcountries with deepfinancial systems(Dehejia and Gatti2002).• Financial developmentreduces incomeinequality bydisproportionatelyboosting the incomesof the poor (Beck,Demirguc-Kunt, andLevine 2004).• Emergence ofmicrofinance as asource of credit isboth efficient andequitable as it hasenabled the poorto invest, therebypromoting growthand reducing poverty(Khandeker 2005).Developmenteffectiveness(continued on next page)8 3

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tCPIA criteria/indicators Sustained growth Poverty reductionFinancial Sector(0.083)(cont.)BusinessRegulatoryEnvironment(0.083)Regulations affecting entry,exit, and competition (0.028)• Bans on or investmentlicensing requirements.• Procedures to enter or exit.• Legal framework (andimplementation thereof) toaddress anti-competitiveconduct by firms.• Procurement by publicsector firms.Regulations of ongoingbusiness operations (0.028)• Operational licensing,permits, compliance andinspection requirementsincluding taxes andcustoms.• State intervention in goodsmarkets (state ownershipin competitive sectors,price controls, statemaking administrativeallocation/decisions aboutproduction).• Corporate governance laws(and enforcement thereof)to encourage disclosureand protect shareholdersrights.Regulations of goods andfactor markets (0.028)• Employment law providesfor flexibility in hiring andfiring.• Informational,regulatory, legal, andjudicial framework(World Bank 2005).• Red tape (Mauro1995).• Reducing risks forprivate investors(World Bank 2000c).• Clear and transparentbusiness environment(World Bank 2000c).• Importance of investorprotection (La Portaand others 2000).• Importance of labormarket reforms forminimizing adverseeffects of trade reformon the poor (Goldbergand Pavcnik 2005;Topalova 2004, 2005;Welch, MacMillan,and Rodrik 2004).• Getting the labormarket right (WorldBank 2008a; Ocampo2003; Cardenas,Ocampo, and Thorp2000).• Massive socialbanking experimentin India has led tosignificant falls inrural poverty (Burgessand Pande 2005).• Impact assessmentsshow microfinancein general helps thepoor (Hossain 1988;Hashemi, Schuler, andRiley 1996; Khandker1998, 2005; Khandkerand Pitt 2003).• Improve climatefor doing business(Besley and Burgess2003).Developmenteffectiveness8 4

A p p e n d i x C : 2 0 0 7 C P I A C r i t e r i a F O R C L U S T E R S A , B a n d C a n d E v i d e n c e i n t h e l i t e r at u r eCPIA criteria/indicators Sustained growth Poverty reductionBusinessRegulatoryEnvironment(0.083)(cont.)Gender Equality(0.05)Equity of PublicResource Use(0.05)• State intervention in laborand land markets limitedto regulation and/orlegislation to smooth outmarket imperfections.• Procedures to registerproperty are simple andlow cost.Human capital development(0.017)• Differences (betweenmale and female) inprimary completion rates,and access to secondaryeducation (female to maleenrollment).• Access to delivery care andfamily planning services.• Adolescent fertility rate.Access to economic andproductive resources (0.017)• Gender disparities in laborforce participation, landtenure, property ownershipand inheritance practices.Status and protection underthe law (0.017)• The law gives men andwomen equal individualand family rights.• Violence against womenconsidered a crime.• Gender disparities inpolitical participation atnational level.Government spending (0.033)• Identification of individuals,groups or localities that arepoor, vulnerable, or haveunequal access to servicesand opportunities.• Adoption of nationaldevelopment strategy withexplicit interventions toassist groups identifiedabove.• Systematic tracking ofcomposition and incidenceof public expenditures andtheir results feed back intosubsequent allocations.• Education for girls(World Bank 2006b,2008a).• Fertility (Easterly2001).• Inequality isnegatively related togrowth: conceptualpapers (Todaro 1997;Aghion, Caroli, andGarcia-Penalosa 1999;Bardhan 2000; Hoffand Stiglitz 2001).• Inequality isnegatively relatedto growth: empiricalevidence (Alesina andRodrick 1994; Perotti1992, 1993, 1996;Persson and Tabellini1994).• Higher genderinequality increasespoverty, and femaleliteracy is one ofthe most importantdeterminants of theeffects of growth onincome poverty (WorldBank 2001a; Ravallionand Datt 2002).• Educating girls andintegrating theminto the workforceis one way to breakthe intergenerationalcycle of poverty(World Bank 2008a).• Inequality reducesthe poverty reductionimpact on growth(Ravallion 2001).Developmenteffectiveness(continued on next page)8 5

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tCPIA criteria/indicators Sustained growth Poverty reductionEquity of PublicResource Use(0.05)(cont.)Building HumanResources (0.05)Revenue collection (0.017)• Incidence of major taxes(progressive or regressive)and their alignment withpoverty reduction priorities.Health and nutrition includingreproductive health (0.017)• Equitable access to basichealth services.• Prevention of malnutrition.• Channels throughwhich inequalityaffects growth: creditconstraints (Perotti1992; Bardhan 2000;the World Bank 2005);political instability,crime, insecurityof property rights(Alesina and Perotti1996; Bardhan 2000);volatility (Aghion,Caroli, and Garcia-Penalosa 1999);social tension andviolent redistribution(Bourguignon 2004).• Micropanel studiesshow householdswith few physicaland human assetscaught in povertytrap reducingchance of economicadvancement andlowering overallgrowth (Christiaensen,Demery, andPaternostro 2002;Woolard and Klasen2005).• Redistributionmeasures—incometransfers (Skoufias,Davis, and de la Vega2001; BourguignonFerreira, and Leite2003).Mixed evidence:• Higher life expectancyraises growth(Jamison, Sachs,and Wang 2001), percapita income (Weil2005), and incentiveto acquire schooling(Kalemli-Ozcan, Ryder,and Weil 2000).Developmenteffectiveness• Positive effects ofhealth interventionsby aid agencies andnon-governmentalorganizations (NGOs)(see appendix E).8 6

A p p e n d i x C : 2 0 0 7 C P I A C r i t e r i a F O R C L U S T E R S A , B a n d C a n d E v i d e n c e i n t h e l i t e r at u r eCPIA criteria/indicators Sustained growth Poverty reductionBuilding HumanResources (0.05)(cont.)Education (0.017)• Sustained progress towarduniversal basic education,literacy, and more equitableaccess to early childdevelopment programservices.• Standards for teacherpreparation, studentlearning, and oversight ofprivate/NGO providers.• Systematic tracking ofschool performance andstudent learning outcomesand feedback to schoolsand parents.• Policies for post-basiceducation and trainingservices.• Quality, equity of access,and efficiency of resourceuse.Human immunodeficiencyvirus/acquiredimmunodeficiencysyndrome (HIV/AIDS),tuberculosis, malaria• Prevention, treatment, careand support of HIV/AIDS,tuberculosis, and malaria.• Track disease prevalence,resources, and programimplementation.• Major internationalhealth improvementssince 1940s have ledto a larger increasein population thanincomes, hence thereis no evidence thathealth improvementsraised income percapita (Acemoglu andJohnson 2007).• Educationalattainment (Easterly2001).• Primary enrollmentrate (Hanushek andKimko 2000; Barro andSala-i-Martin 2003;Doppelhofer, Miller,and Sala-I Martin2004; Hanushek andWoessmann 2008).• Secondary andtertiary education(World Bank 1999;Vandenbussche,Aghion, and Meghir2006; Aghion andHowitt 2009).Mixed evidence:• Greater theprevalence of malaria,lower the per capitaincome (Sachs 2003).• Adequate andeffective delivery ofeducation, health, andsocial infrastructure(World Bank 2004e).• Education enablespeople to makeuse of economicopportunities createdby the growth process(Dreze and Sen 2002).• Improving literacyfacilitates more propoorgrowth (Chhibberand Nayyar 2007).• Poor educationaloutcomes reducethe poverty-reducingimpact of growth(Menezes-Filho andVasconcellos 2004).• Investment ineducation can be usedto attack poverty as amethod to redistributeto the poor (Besleyand Burgess 2003).Developmenteffectiveness• Randomized literaturehas found a number ofaid interventions to beeffective in education(see appendix E).(continued on next page)8 7

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tCPIA criteria/indicators Sustained growth Poverty reductionBuilding HumanResources (0.05)(cont.)Social Protectionand Labor (0.05)• Quality and timeliness ofservices.• Focus on the poor.• Cost-effective use of publicresources.Social safety net programs(0.01)• Social protection programsprovide income support topoor and vulnerable groups.Protection of basic laborstandards (0.01)• Ratification andimplementation ofinternational core laborstandards.• Short-run eradicationof malaria lowersper capita income(because malariaaffects mainly youngchildren) whereasit raises per capitaincome only slightlyover long run (Ashraf,Lester, and Weil2008).• Eradication oftuberculosis raises percapita income slightlyin both short and longrun (Ashraf, Lester,and Weil 2008).• HIV/AIDS associatedwith small reductionin per capita incomeover long-run(Kambou, Devarajan,and Over 1992; Over1992; Cuddington1993; Cuddingtonand Hancock 1994;Haacker 2002).• HIV/AIDs has noimpact on per capitaincome (Bloom andMahal 1997).• HIV/AIDS has largenegative impact onper capita income(Bell, Devarajan, andGersbach 2006).• HIV/AIDS increasesper capita incomeof survivors (Young2004).• Getting the labormarket right to fostergrowth and, at thesame time, protectingpeople (not jobs)through establishingsocial safety nets(World Bank 2008a;Ocampo 2003).Child labor may leadto intergenerationaltransmission of poverty:“child labor traps”(Barham and others1995; Ilahi, Orazem, andSedlacek 2001; Emersonand Souza 2003;Edmonds 2007).Developmenteffectiveness8 8

A p p e n d i x C : 2 0 0 7 C P I A C r i t e r i a F O R C L U S T E R S A , B a n d C a n d E v i d e n c e i n t h e l i t e r at u r eCPIA criteria/indicators Sustained growth Poverty reductionSocial Protectionand Labor (0.05)(cont.)Labor market regulations(0.01)• Labor market regulationson health and safety,working conditions, andhiring and firing.• Empowerment ofcommunities asa determinant ofsustained growth(Stern 2001; WorldBank 2000c, 2004e).DevelopmenteffectivenessCommunity-driven initiatives(0.01)• Encourage and supportcommunities’ owndevelopment initiativesor local accountabilitymechanisms.Pension and old-age savingsprograms (0.01)• Pension and savingsprograms provide incomesecurity to most potentiallyvulnerable groups.Policies andInstitutions forEnvironmentalSustainability(0.05)• Regulations and policies(and implementationthereof) for pollution andnatural resources.• Information widelyavailable.• Priority setting.• Sector ministriesincorporate environmentalconcerns.• Debate on whetherenvironmentalregulation canraise growth rate(Bovenberg andSmulders 1995, 1996;Hettich 1998; andEsty and Porter 2005;versus Fullerton andKim 2006).• Policies reducingenvironmental risknot necessarilyprogressive (Parry andothers 2005; Fullerton2008).8 9

9 0Water supply in Marracuene, Mozambique. Photo by Eric Miller/World Bank

Appendix D: Literature Review on Aid EffectivenessA. Aid ➔ Growth with No ConditionsArticle (year)Type of aidShort /Long-termimpactPeriod ofanalysisHansen and Tarp (2000) All aid Short-term (4 years) 1974–1993Dalgaard and Hansen(2001)All aid Short-term (4 years) 1974–1993Hansen and Tarp (2001) All aid Short-term (4 years) 1974–1993Ram (2003) Bilateral Short-term (4 years) 1970–1993Clemens, Radelet, andBhavnani (2004)Short impact aid (budget support or “program” aid for anypurpose, and project aid given for real sector investments forinfrastructure or to directly support production in transportation,communications, energy, banking, agriculture, and industry).Short-term (4 years) 1970–2001Headey (2005) Multilateral and bilateral, separately Short-term (4 years) 1970–2001Reddy and Minoiu (2006) Developmental (multilateral and bilateral) Long-term (2 decades) 1960–20009 1

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tB. Aid ➔ Growth with ConditionsArticle (year)Burnside and Dollar (2000)Guillaumont and Chauvet(2001)Collier and Dehn (2001)Type ofaidAll aid(net flow)Short /Long-termimpactPeriod ofanalysisConditionsShort-term (4 years) 1970–1993 Index of policies: trade openness (Sachs-Warner),fiscal policy (budget surplus), monetary policy(inflation rate).All aid Long-term (12 years) 1970–1993 Index of environment/vulnerability: climaticshocks (stability of agricultural value added),trade shocks (stability of real value of exports,trend of terms of trade), and structural exposureto these shocks (population size).All aid(net flow)Short-term (4 years) 1970–1993 Countries suffering sharp price drops in keycommodity exports.Collier and Dollar (2002) All aid Short-term (4 years) 1974–1997 Country Policy and Institutional Assessment(CPIA) (overall for 20 components).Collier and Hoeffler (2004) All aid Short-term (4 years) 1974–1997 A few years after civil war with good policies(first social, then sectoral, then macro).Dalgaard, Hansen, and Tarp(2003)All aid Short-term (4 years) 1970–1993 Countries outside of the tropics.C. Aid Has No Impact on GrowthArticle (year) Type of aid Short/Long-term impact Period of analysisBoone (1994)Easterly, Levine, and Roodman(2004)All aid Short-term (4 years) 1970–1997Easterly (2003) All aid Long-term (12 or 24 years) 1970–1993Roodman (2007a) All aid Tested Hansen and Tarp (2001) in section A andall six papers in section B aboveRajan and Subramanian (2008) All aid Medium (10–20 years) and long-term (30–40years)1960–2000D. Aid Has Negative Impact on GrowthArticle (year) Type of aid Short/Long-term impact Period of analysisRam (2003) Multilateral Short-term 1970–1993Reddy and Minoiu (2006) Geopolitical (all aid-developmental aid) Long-term (2 decades) 1960–20009 2

Appendix E. Examples of Positive Impacts of Aid Projects from RandomizedEvaluations in Education, Health, Infrastructure, and AgricultureEducationAngrist and others (2002) find that lottery winnersof vouchers for private schools in Colombiahad 0.12–0.16 additional years of schooling,test scores that were higher by 0.2 standarddeviations, and higher secondary school completion(the last finding was confirmed in a follow-upstudy by Angrist, Bettinger, and Kremer (2006).Vermeersch and Kremer (2003) find that a schoolmeals program in preschools in Kenya raisedattendance rates from 21 to 29 percent. Kremerand Holla (2008) find that a merit scholarshipfor high school girls in Kenya seemed to inducegreater study effort and raised the girls’ testscores, and even had some externalities to boys’performance in the same classroom.More generally, as underlined by Kremer andHolla (2008), evidence is accumulating on theeffectiveness of certain school inputs such asextra teachers and textbooks (for example, seeBanerjee and others 2005; Duflo, Kremer, andRobinson 2007; and Glewwe, Kremer, and Moulin2007); provider incentives (Glewwe, Holla, andKremer 2008 and Muralidharan and Sundararaman2007); remedial education (Banerjeeand others 2007; Duflo, Kremer, and Robinson2007; He, Linden, and MacLeod 2007); citizens’report cards; the hiring of contract teachers;or increased oversight of local school committees(Bjorkman and Svensson 2009; and Duflo,Kremer, and Robinson 2007); and school choiceprograms (Angrist, Bettinger, and Kremer 2002,2006; Bettinger, Kremer, and Saavedra 2008).cash-for-schooling program in Mexico, whichhad health components, also had a major healthimpact. The Bobonis, Miguel, and Puri-Sharma(2006) study on anemia and school participationfinds that iron supplements and de-wormingdrugs were effective in increasing children’sweight-for-height and weight-for-age scores.Another area in which randomized evaluationsfound success is in preventing or treating infantdiarrhea (Zwane and Kremer 2007).InfrastructureAshraf, Berry, and Shapiro (2007) find that waterpurification tablets in Zambia are an inexpensiveway of avoiding waterborne illness. Kremer andothers (2008) show that investment in protectingnaturally occurring springs from contaminationled to dramatic improvements in water quality inrural Kenya.AgricultureDuflo, Kremer, and Robinson (2007) findthat selling vouchers earmarked for fertilizerpurchases to the farmers right after harvestsolved the problem of the farmers not settingaside funds for fertilizer for the next season.Duflo, Kremer, and Robinson (2008) find a largepositive return to fertilizer use in maize farmsin Kenya, whereas too little or too much fertilizerrenders the return unfavorable. Conleyand Udry (2007) find that farmers learn howmuch fertilizer to apply from their successfulneighbors in a new technology for pineapplegrowing in Ghana.HealthGertler (2004) finds that the Programa deEducación, Salud y Alimentación (PROGRESA)9 3

9 4Fisherman, Colombia. Photo by Edwin Huffman/World Bank

Appendix F. Review of Financial Sector CriterionA review of the literature indicates that thefinancial sector criterion does cover thedimensions along which finance is currentlythought to be important: stability, that is, depthand efficiency; and access. However, not allthree finance dimensions are equally importantfor growth and poverty reduction. Further, theway some of the dimensions are currently beingassessed can be strengthened.Financial StabilityThere is considerable evidence of a large impactof banking crises on output losses (Hoggarth,Reis, and Saporta 2002) 1 as well as on the nationalbudget (Laeven and Valenciana 2008; Honohan2008b). 2 On the other hand, the evidence as towhich policies work to limit banking crises isambiguous and controversial (discussed below).In light of the controversies over what constituteappropriate policies for financial stability, the factthat the Country Policy and Institutional Assessment(CPIA) assesses intermediate outcomesin addition to policies and institutions (it issupposed to only assess the latter) is perhapsreasonable.There is quite a lot of controversy in the literatureon the current CPIA indicators used for assessingthe stability of the financial system. The indicatoron the banking sector’s vulnerability to shocks,although relevant, is extremely vague. It evokesthe concept of stress testing, 3 which has notbeen very effective, as clearly demonstrated bythe recent wave of systemic failures in matureeconomies, all of whose prudential regulatorswere using stress tests.The indicator for banking system soundnessspecifies two alternative intermediate outcomemeasures related to non-performing loans(NPLs). 4 Although NPLs may predict crises tosome extent, they are typically a lagging indicator,with high values suggestive of a problem that hasalready crystallized. Hence these NPL measuresare crude and inadequate even as indicators ofcurrent or imminent problems. This is clearlyreflected to the fact that NPLs for residentialmortgages did not provide a sensitive earlywarning system in the recent crisis in advancedeconomies.The indicator on adherence to Basel Core Principles(BCP) does embody a set of regulatorypolicies and practices on most points on whichthere is broad consensus.However, the econometric evidence betweenBCP compliance and financial sector performanceis not very strong.The indicator on tools and resources for bankingsupervision is subject to a lot of controversy.An extensive data collection and econometricexercise (Barth, Caprio, and Levine 2006) fails touncover any statistically significant relationshipbetween their measures of banking supervisionand regulation on the one hand, and financialsector performance including stability on theother. The alternative approach that emphasizesmarket discipline (enforced through regulationsthat provide market participants with informationand incentives to monitor firms) has not workedeither as evidenced by the current financial crisis.What is clearly missing in the CPIA is explicitattention to the information (for example,accounting and disclosure requirements) andincentive structures (for example, deposit9 5

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tinsurance) for market discipline. This is supportedby an extensive literature on moral hazard ofdeposit insurance schemes and other crisismanagement policies which show that fiscal costsof crises are higher in cases where policy designneglects moral hazard (Honohan and Klingebiel2003; Demirguc-Kunt, Kane, and Laeven 2006).Financial Sector Efficiency, Depth, andResource MobilizationThere is a convincing literature that financialdepth (usually measured as outstanding bankcredit to the private sector as a share of grossdomestic product [GDP] drives economicgrowth (Honohan 2009). At the same time, thereis a wide range of policies both in finance andin other sectors that influence the depth andefficiency of finance. In this light, it is perhapsnot unreasonable that most of the indicatorsunder this dimension are outcome indicators.Yet many of the indicators specified in the CPIAare problematic.The indicator “size and reach of financial markets”is vague, and is already captured by anotherindicator “private sector credit as a share of GDP.”The latter corresponds directly to the measure offinancial depth most commonly used in empiricalanalyses of the finance-growth link. However,private sector credit as a share of GDP is not anideal measure. Rapid increases in financial depthare often unsustainable, and yet can be mistakenfor finance-supported sustainable growth(Honohan 2004b). Further, the term “reach”normally refers to access, which is covered in thethird dimension of the CPIA finance criterion.The indicator “development of capital markets”is typically measured in terms of overall stockmarket capitalization. But the free-float of sharesmay be a tiny fraction of total capitalization, sosome correction needs to be made to addressthis. Also there is little or no empirical evidencethat confirms a causal link between marketcapitalization and subsequent economic growth.One paper (Levine and Zervos 1998) finds arelationship between stock market turnover andgrowth, although this is also not very stronglysupported by the evidence.The indicator “interest rate spreads” measuresefficiency of intermediation, which is affectedby both administrative efficiency as well as thedegree of competition in the sector. Policy inthe form of taxation of financial intermediation(implicit or explicit) and in the form of administrativecontrols (such as holding interest spreadsbelow breakeven rates) can strongly influencethe spread, so there is no unambiguous linkbetween spreads and sustainable growth.The indicator efficiency of microfinance is vague,as efficiency presumably relates to the cost ofachieving some goal. Yet the CPIA instructionsneither specify the goal nor how to judge thecost. Further, this indicator might fit better underthe access dimension.In addition to these problems with the outcomeindicators, this dimension has also neglected thepolicies that are important for achieving financialdepth. A recent paper (Tressel and Detragiache2008) using a large cross-country panel findsthat policies that facilitate entry, liberalize pricingand the provision of services, and reduce stateownership in finance do help deepen finance—although only where complementary deep legalinstitutions protecting private property fromexpropriation are present.Access to Financial ServicesThere are individual success stories of microfinance(Littlefield, Morduch, and Hashemi 2003),but because of the formidable selection biases inthe data, there is little firm evidence of a sizeablegrowth or poverty reduction impact of microfinance.Indeed, the indications are that overallfinancial development, as measured by financialdepth, has had a larger and more certain impactnot just on growth but also on poverty reductionthan the expansion of microfinance (World Bank2008b).Aside from the lack of strong empirical evidenceof this dimension on growth and povertyreduction, the indicators specified under thisdimension also suffer from a lack of clarity andspecificity. The indicator on the payments systemis unclear. To the extent that it refers to the9 6

A p p e n d i x F. R e v i e w o f F i n a n c i a l S e c t o r C r i t e r i o nwholesale payments system, it is not an accessissue but rather relates to the cost and securityof a minor part of the banking system (Honohan2009). At the same time, however, there is noempirical evidence that a wholesale paymentssystem failure could trigger a banking collapse.To the extent that the payments system refers tothe retail system, it relates to access but it is notclear what is being assessed in the CPIA.The indicator on the credit reporting system isrelevant, as credit registries are known to have thepotential to expand access to finance, particularlyto small and medium enterprises (SMEs) (Miller2003; Berger, Frame, and Miller 2005; Djankov,McLiesh, and Shleifer 2007). However, the CPIAfalls short in specifying how this is to be assessed,although the literature has established a standardset of good practices. Such good practices includefacilitating entry of private providers of registrieswhere possible, including as wide a set ofinformation providers as possible, and ensuringprivacy laws do not unduly constrain reporting of“positive information” (Miller 2003). There is alsoevidence that quality matters in ensuring that acredit registry does expand availability of finance(Brown, Jappelli, and Pagano 2009, and Lutoto,McIntosh, and Wydick 2007).Regarding the indicators on access, cross-countryregressions do not suggest any strong influenceof household financial access on growth or onpoverty (taking into account the average level ofincome) (Honohan 2008a). However, access ofsmall and medium enterprises (SMEs) to formalsector financial services has been found to beimportant for firm entry, firm growth, innovation,and size distribution of firms (Beck, Demirguc-Kunt, and Levine 2005; Ayyagari, Demirguc-Kunt,and Maksimovic 2007).The indicator regarding the legal and regulatoryframework supporting access to finance isvague. Also, the cross-country empirical literaturesuggests that although legal and politicalinstitutions that protect private property againstthe state are important for financial depth, theprotection of contracts between private agentsis important for determining access (Beck,Demirguc-Kunt, and Levine 2005; World Bank2008b).RecommendationsThe above review suggests that the CPIA indicatorsfor all three dimensions could be strengthened.Suggestions for such strengthening arepresented in appendix box F. 1.9 7

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tAppendix Box F.1: Suggestions for Strengthening of the Country Policy and InstitutionalAssessment (CPIA) Indicators for the Financial SectorEach of the three dimensions currently covered by the CPIAfinancial sector criterion can be revised and strengthened, asfollows:Financial StabilityWhether policy creates good incentives for prudential managementof financial firms. This would include enforcement ofprudential regulations and the safety net policy (including thenature and extent of deposit insurance).How good are supervisory powers and effectiveness (includingthe tools and resources for risk assessment)? This would entailassessment of the compliance with Basel Core Principles (BCP)(already included in the CPIA), which is a useful summary forbanking. However, non-bank finance also needs to be consideredwhere it is sizeable.The vulnerability of financial institutions to shocks. This can beinformed by measures of bank capitalization (for example, whetherregulatory minima sufficient for the economy’s risk, well measuredand enforced; whether all banks satisfy regulatory minima; how iscapital measured). Non-performing loans (NPLs) (especially netof provision) should be taken into account but supplemented byexposure to foreign exchange risk. Rapid growth of credit shouldbe monitored as a possible warning sign.Financial Depth and EfficiencyDepth. The key measure is the size of private sector creditas a share of gross domestic product (GDP), adjusted for thecountry’s overall level of development. (However, rapid growthin this indicator should be assessed as negative). Other measuresare market capitalization (total and free-float) and stockmarket turnover.Efficiency of intermediation. The measures are intermediationspreads (ideally distinguishing what is attributable to taxesand quasi-taxes, inadequate legal protection, exogenous riskand market power), and bank operating costs (for example asa share of total assets).Policy barriers to efficiency and depth. Among the relevant indicatorsare: distorting financial sector taxes and quasi-taxes onintermediation including binding interest rate ceilings; and directedcredit programs. The preconditions for the BCP (which are notgraded) also provide a useful summary of key policy dimensionsas relevant to depth and efficiency as to stability.AccessPolicy. Has policy created an enabling environment for expandingoutreach of the financial system? For example, is the legalframework for debt recovery and insolvency economically andadministratively efficient (including time taken and cost of insolvencyand debt recovery)? Is there a good credit reportingsystem? Are there legal or regulatory barriers to creating one?Is there an adequate legal basis for the leasing industry? Isthe regulatory framework for microfinance supportive? Is antimoneylaundering and combating the financing of terrorism(AML-CFT) policy well adapted to protecting financial serviceproviders focusing on the poor from undue costs (for example,are microfinance firms able to operate depository and moneytransmission services without crippling regulatory burden?)(Isern and others 2005).Outcomes. Measures could include: the percentage of the populationwith access to formal financial services; how good theaccess of small and medium enterprises (SMEs) to financialservices is (such as from responses to the Investment ClimateAssessment surveys); how financially secure the microfinanceindustry is; and whether retail money transmission costs (internaland international) are low.Source: Honohan 2009.9 8

Appendix G: Country Policy and Institutional Assessment (CPIA)and Loan PerformanceThis appendix presents the econometricanalysis undertaken to estimate the relationshipbetween the CPIA and Bank loan performance.The methodology draws on a similar exerciseby Dollar and Levin (2005) who find a strongassociation between institutional quality and loansuccess rate using the property rights/rule of lawmeasure from the International Country RiskGuide and democracy measure from FreedomHouse as proxies for institutional quality, and(Independent Evaluation Group) IEG’s loanoutcome ratings as a measure of loan performance.This analysis uses a very similar specificationto that of Dollar and Levin, but a differentmeasure of loan performance (see below) andthe CPIA for the institutional quality variables.DataPolicy and institutionsThe average overall CPIA rating for the countryfrom 2004–07 was used as the explanatoryvariable. The average ratings for each of the fourCPIA clusters were also used alternatively asexplanatory variables. The time period 2004–07was selected because of the major restructuringof the CPIA in 2004, and in particular because ofthe change in the definition of the rating scale,which resulted in a discontinuity in the data thatyear.Loan performanceThis analysis uses the Implementation Status andResults Report (ISR) flags for problem loans asthe proxy for loan performance because of theneed to match the loan performance data withthe CPIA data which confines the analysis to thisperiod of 2004–07. Given this restriction, it is notpossible to use IEG loan outcome ratings becausethe analysis would have been restricted to loansapproved and closed between 2004 and 2007.This, in turn, would have reduced the samplesize substantially as well as biased the samplebecause those loans would have consisted mostlyof development policy loans. Estimates basedon data for loans approved since 1998 found arelatively good correlation between the shareof problem loans and IEG loan outcome ratings(the correlation coefficient is 0.63).The flags for problem loans are based onself-evaluation by Bank staff of loans whereimplementation progress or developmentobjective is rated unsatisfactory. For this analysis,a loan is identified as a problem if it has beenflagged as a problem loan anytime between 2004and 2007. The sample is restricted to all countriesthat had at least three loans approved duringthe period. For each country the share of loans(approved between 2004 and 2007) that hadbeen flagged as problem loans was used as thedependent variable. 1Control variablesThe following control variables are used in theeconometric analysis: 2• Average aid as a share of gross domestic product(following Dollar and Levin 2005) whichcould influence loan performance in oppositeways. On the one hand, there could be increasingreturns in the sense that a lot of aidmay create a better environment of supportingservices and resources. On the other hand,there could be diminishing returns or absorptivecapacity constraints. The data are fromthe Organisation for Economic Co-operationand Development (OECD) Development AssistanceCommittee (DAC) database.9 9

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n t• Geographic constraints to development asrepresented by the share of a country’s territoryin the tropics that has been found to affecteconomic performance (Dollar and Levin2005, following Gallup, Sachs, and Mellinger1999).• Extent of ethnic fractionalization that has beenfound to affect political stability and policiesand institutions (see, for example, Mauro1995; Easterly and Levine 1997; Alesina andothers 2003).• Regional dummies to control for region-specificeffects on loan performance.Econometric Specification and ResultsOrdinary least squares regressions 3 are used forthe estimations. The results are presented inappendix table G.1. There is a significant negativeassociation between the overall CPIA rating of acountry and the percentage of problem loans inthat country. The results are similar when each ofthe four CPIA cluster ratings are used separatelyinstead of using the overall CPIA rating. There issome evidence that a higher level of aid as a shareof GDP is associated with better portfolio performance,and countries with a higher proportionof land in tropics are more likely to have a higherproportion of problem loans. However, therespective coefficients are not always significantacross the different specifications.Attempts were made to estimate the relativeimportance of the four CPIA clusters on the Bank’sloan portfolio performance. This was difficultbecause of the high degree of correlation amongthe ratings of the four clusters (table G.2). Indeedwhen all four clusters were included as explanatoryvariables along with other control variables,none of them are found to be significant and theirsigns are not consistent across specifications—amanifestation of multicollinearity.To reduce the multicollinearity problem, specificationswere estimated using ratings from all thepossible combinations of two CPIA clusters asexplanatory variables, in turn. The coefficientson all the combinations of the two CPIA variablesare not significant but their signs are consistentlynegative across specifications. An F-test of thejoint null hypothesis that the coefficients of bothCPIA variables are equal to zero was rejected,which implies that one or both CPIA ratings havesignificant negative association with the shareof problem projects. However, their individualcontributions cannot be discerned because oftheir high degree of correlation with each other.Further, an F-test of the null hypothesis that thecoefficient of the CPIA rating for one cluster isnot different from the coefficient of the rating ofthe other CPIA cluster cannot be rejected. Similarresults were obtained when the share of problemprojects in a country was regressed on the averageratings of the first three clusters combined and theratings for cluster D. In sum, there is not sufficientevidence to conclude that the policies and institutionsmeasured by one cluster are relatively moreimportant than those measured by the otherclusters for better project performance.ConclusionsThe analysis finds that higher CPIA ratingsare associated with better Bank loan portfolioperformance. This association is found foroverall CPIA ratings as well as ratings for eachof the four CPIA clusters. There is insufficientevidence to conclude that any one of the fourCPIA clusters is more important than the othersfor loan performance.1 0 0

A p p e n d i x G : C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n t ( C P I A ) a n d L o a n P e r f o r m a n c eTable G.1: CPIA and Loan Performance: Ordinary Least Squares RegressionsDependent Variable: Percent of Problem LoansIndependentVariablesOverall CPIA CPIA Cluster A CPIA Cluster B CPIA Cluster C CPIA Cluster D(1) (2) (3) (1) (2) (3) (1) (2) (3) (1) (2) (3) (1) (2) (3)CPIA ratings –13.4*** –16.1* ** –17.9** –9.1*** –9.0** –10.1*** –11.4*** –14.4*** –16.0*** –9.7** –13.7** –14.4*** –10.5** –11.4** –13.4***Aid/GDP –0.5* –0.4* –0.4 –0.3 –0.5 –0.4 –0.4 –0.3 –0.4 –0.3% Tropical Land 10.5 10.6 12.5 13.6* 10.3EthnicFractionalization1.7 6.0 2.5 –0.7 2.8Africa Dummy –18.8*** –18.7** –16.7*** –16.5*** –17.7** –14.0** –18.8*** –19.5** –17.3*** –17.7*** –18.9** –16.9*** –16.6*** –16.3** –14.4**Europe and CentralAsia Dummy–20.6*** –15.5* –21.9*** –19.8*** –13.5 –19.8*** –21.1*** –14.3 –22.4*** –20.5*** –13.3 –21.9*** –22.2*** –16.9* –23.6***East Asia and PacificDummy–25.2*** –25.7*** –23.6*** –21.6*** –21.6** –19.5** –26.5*** –27.4*** –26.1*** –23.7*** –25.1*** –23.0*** –24.2*** –24.7*** –22.8***Middle East and NorthAfrica Dummy–26.2*** –16.3 –28.0*** –25.0*** –14.4 –25.5*** –25.7*** –15.2 –27.6*** –25.2*** –13.6 –27.2*** –24.2*** –13.4 –25.1***South Asia Dummy –22.8*** –17.2* –22.5*** –20.0** –13.9 –18.8** –23.9*** –16.4 –24.5*** –21.5** –15.8 –21.7** –21.1** –15.2 –20.4**Constant 85.5*** 90.9*** 104.1*** 71.1 *** 63.1*** 75.9*** 78.9*** 83.6** 98.5*** 71.0*** 79.3*** 89.9*** 71.0*** 68.6*** 82.5***Observations 92 79 87 92 79 87 92 79 87 92 79 87 92 79 87R-squared 0.23 0.29 0.26 0.22 0.26 0.22 0.21 0.28 0.24 0.19 0.26 0.22 0.21 0.25 0.23Source: IEG estimates.Note: *** significant at 1% level; ** significant at 5% level; * significant at 10% level.1 0 1

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tTable G.2: Correlations among Average 2004–07 CPIA Ratings for the Four Country Policy and InstitutionalAssessment (CPIA) ClustersCPIA Cluster A Cluster B Cluster C Cluster D Overall CPIACluster A 1.00Cluster B 0.61 1.00Cluster C 0.66 0.74 1.00Cluster D 0.74 0.81 0.84 1.00Overall CPIA 0.86 0.87 0.90 0.94 1.00Source: IEG estimates.Notes: a. Based on 92 observations. b. All correlations are significant at 1% level1 0 2

Appendix H: Comparing Country Policy and Institutional Assessment Ratingsby the World Bank, African Development Bank, and Asian Development BankComparison of the CPIA ratings (rather thanrelative rankings implied by the ratings) canonly be done for the CPIA ratings by the WorldBank, the AfDB, and the ADB. Ratings by thesethree development banks are on the exact samescale, with each rating having the exact samemeaning, which is not the case with the otherindicators.Generally speaking, it is found that the 2007 CPIAratings by the AfDB and ADB are higher thanthe CPIA ratings by the Bank. Specifically, theratings by the AfDB were higher than those bythe Bank for all 16 CPIA criteria; for 12 of the 16,the ratings were statistically significantly higherat the 5 percent level. The ratings by the ADBwere higher than those of the Bank for 14 ofthe 16 CPIA criteria; for 5 of the 14, the ratingswere statistically significantly higher. For thetwo criteria for which the Bank’s ratings werehigher than those of the ADB, the differences inthe ratings were not significantly different (seeappendix table H.1).1 0 3

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tAppendix H.1: 2007 CPIA Ratings by the World Bank, AfDB and ADBAverage for common World Bankand AFDB member countriesAverage for common World Bankand ADB member countriesCriterion World Bank AfDB World Bank ADBq1 3.64 3.99 3.83 4.00q2 3.38 3.74 3.44 3.67q3 3.31 3.67 3.90 4.00q4 3.58 3.62 3.87 3.65q5 3.20 3.48 3.17 3.42q6 3.25 3.38 3.35 3.27q7 3.24 3.55 3.56 3.62q8 3.24 3.53 3.58 3.63q9 3.31 3.45 3.48 3.58q10 3.01 3.22 3.23 3.29q11 3.24 3.45 3.13 3.25q12 2.90 3.23 3.00 3.21q13 3.16 3.52 3.29 3.60q14 3.46 3.55 3.35 3.58q15 3.02 3.27 3.10 3.13q16 2.85 3.24 2.87 3.06Source: World Bank, AfDB, and ADB.Note: Bold figures denote ratings that are significantly different at the 5 percent level between the World Bank and the AfDB, and the World Bank and the ADB. ADB= Asian DevelopmentBank; AfDB= African Development Bank.1 0 4

Appendix I: Comparator INDICES of the CPIA1 0 5

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tComparatorIndicesTrade (q4)FinancialSector (q5)WorldwideGovernanceIndicatorDoing Business (5) Gettingcredit (Strengthof legal rightsindex, depthof creditinformationindex).Index ofEconomicFreedom:HeritageFoundationInternationalCountry RiskGuide (ICRG):PRS GroupGlobal GenderGap Report(WorldEconomicForum)TransparencyInternationalBusinessRegulatoryEnvironment(q6)RegulatoryqualityDoing BusinessIndicators:(1) Starting aBusiness(2) Dealing withconstructionpermits(3) Employingworkers(6) Protectinginvestors(7) Paying taxes(10) Closing abusinessBusinessfreedom (datafrom DoingBusiness)GenderEquwality(q7)Gender GapIndexEquity ofPublicResource Use(q8)BuildingHumanResources (q9)CPIA CRITERIASocialProtectionand Labor(q10)Policies andInstitutionsfor EnvironmentalSustainability(q11)PropertyRights andRule BasedGovernance(q12)Quality ofBudgetaryand FinancialManagement(q13)Efficiencyof RevenueMobilization(q14)Qualityof PublicAdministration(q15)Transparency;Accountability,and Corruptionin the PublicSector (q16)Rule of Law GovernmenteffectivenessControl ofcorruptionProperty Rights Freedom fromCorruption (datafrom TransparencyInternational);property rightsInvestmentprofile (contractviability; profitrepatriation;payment delays);law and orderBureaucracyQualityCorruptionCorruptionPerception Index1 0 6

A p p e n d i x I : C o m pa r at o r I N D I C E S o f t h e C P I AComparatorIndicesGenderEmpowermentMeasureUnited NationsDevelopmentProgramme(UNDP) HumanDevelopmentReportIbrahim Indexof AfricanGovernanceBertelsmannTransformationIndexGlobal EnablingTrade Report(WorldEconomicForum)2005EnvironmentalSustainabilityIndexGlobalCompetitivenessIndexTrade (q4)Enabling TradeIndex:(1) Marketaccess—tariffsand non-tariffbarriers,(2) BorderadministrationPrevalence oftrade barriersTrade-weightedtariff rate (harddata)Burden ofcustomsproceduresFinancialSector (q5)Interest ratespread (harddata)Financial marketsophisticationFinancingthrough localequity marketCPIA CRITERIABusinessRegulatoryEnvironment(q6)GenderEquwality(q7)Equity ofPublicResource Use(q8)BuildingHumanResources (q9)SocialProtectionand Labor(q10)GenderEmpowermentMeasurePrivate property Social safetynetsEnabling TradeIndex: (4)the businessenvironment—regulatoryenvironmentBurden ofgovernmentregulationBusiness impactof malariaStrength ofauditing andreportingstandardsMalariaincidence (harddata)Efficacy ofcorporate boardsBusiness impactof tuberculosisPolicies andInstitutionsfor EnvironmentalSustainability(q11)Social andinstitutionalcapacityPropertyRights andRule BasedGovernance(q12)Quality ofBudgetaryand FinancialManagement(q13)Efficiencyof RevenueMobilization(q14)Qualityof PublicAdministration(q15)Transparency;Accountability,and Corruptionin the PublicSector (q16)Rule of law,transparency, andcorruptionPrivate property Q3.2 Does anindependentjudiciary exist?Q3.3 Are therelegal or politicalpenalties forofficeholderswho abuse theirposition?Property rightsEfficiency oflegal frameworkBusiness costsof crime andviolence(continued on next page)1 0 7

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tComparatorIndicesGlobalCompetitivenessIndex (cont.)Source: IEG.Trade (q4)FinancialSector (q5)Ease of accessto loansSoundness ofbanksRegulationof securitiesexchangesLegal rightsindex (harddata)BusinessRegulatoryEnvironment(q6)Protectionof minorityshareholders’interestsIntensity of localcompetitionExtent of marketdominanceEffectiveness ofanti-monopolypolicyExtent and effectof taxationTotal tax rate(hard data)Number ofproceduresrequired to starta business (harddata)Time required tostart a business(hard data)Strengthof investorprotection (harddata)Cooperation inlabor-employerrelationsFlexibilityof wagedeterminationHiring and firingpracticesGenderEquwality(q7)Equity ofPublicResource Use(q8)BuildingHumanResources (q9)Tuberculosisincidence (harddata)Business impactof HIV/AIDSHumanimmunodeficiencyvirus (HIV)prevalence (harddata)Infant mortality(hard data)Life expectancy(hard data)Quality of primaryeducationPrimaryenrollment (harddata)Educationexpenditure (harddata)Secondaryenrollment (harddata)Tertiaryenrollment (harddata)Quality of theeducationalsystemCPIA CRITERIASocialProtectionand Labor(q10)Policies andInstitutionsfor EnvironmentalSustainability(q11)PropertyRights andRule BasedGovernance(q12)Reliability ofpolice servicesQuality ofBudgetaryand FinancialManagement(q13)Efficiencyof RevenueMobilization(q14)Qualityof PublicAdministration(q15)Transparency;Accountability,and Corruptionin the PublicSector (q16)1 0 8

Appendix J. Number of IDA and IBRD Countries forwhich External Data ARE Available1 0 9

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tCPIA ICRG GCI ETI DBBertelsmannTransformationIndexIndex ofEconomicFreedom WGICorruptionPerceptionIndexIbrahim Indexof AfricanGovernanceEnvironmentalSustainabilityIndexGender GapIndexGenderEmpowermentMeasureq1 IDAIBRDq2 IDAIBRDq3 IDAIBRDq4 IDA 39 37IBRD 52 48q5 IDA 39 75IBRD 52 64q6 IDA 38 37 75 56 55 75IBRD 52 48 64 55 59 65q7 IDA 40 17IBRD 53 43q8 IDAIBRDq9 IDA 39IBRD 52q10 IDA 56IBRD 55q11 IDA 57IBRD 53q12 IDA 42 40 56 55 75IBRD 53 52 55 59 65q13 IDAIBRDq14 IDAIBRDq15 IDA 42 75IBRD 53 65q16 IDA 42 56 55 75 74 38IBRD 53 55 59 65 61 8Source: IEG.Note: DB= Doing business; ETI=Enabling Trade Index; GCI= Global Competitiveness Index; IBRD= International Bank for Reconstruction and Development; ICRG= International Country Risk Guide; IDA= International Development Association; WGI=World Governance Indicators.1 1 0

Appendix K. Comments by Network on Regional CPIA Rating Proposals, 2007CPIA criterionNumber ofnetworkcommentsIBRD IDA TotalShare ofcountrieswith networkcomments(%)Number ofnetworkcommentsShare ofcountrieswith networkcomments(%)Number ofnetworkcommentsShare ofcountrieswith networkcomments(%)q1 59 90.8 62 82.7 121 86.4q2 59 90.8 62 82.7 121 86.4q3 59 90.8 62 82.7 121 86.4q4 43 66.2 37 49.3 80 57.1q5 10 15.4 8 10.7 18 12.9q6 7 10.8 7 9.3 14 10.0q7 16 24.6 25 33.3 41 29.3q8 12 18.5 10 13.3 22 15.7q9 50 76.9 48 64.0 98 70.0q10 34 52.3 38 50.7 72 51.4q11 4 6.2 7 9.3 11 7.9q12 13 20.0 11 14.7 24 17.1q13 10 15.4 19 25.3 29 20.7q14 14 21.5 13 17.3 27 19.3q15 7 10.8 6 8.0 13 9.3q16 10 15.4 14 18.7 24 17.1Total 407 39.1 429 35.8 836 37.3Source: IEG calculations based on World Bank information.Note: IBRD= International Bank for Reconstruction and Development;; IDA= International Development Association.1 1 1

1 1 2Elephants, Kenya. Photo by Curt Carnemark/World Bank

EndnotesChapter 11. This is an evaluation of the implementation ofthe IDA10–12 Replenishment Agreements.2. The previous name of the Independent EvaluationGroup (IEG) name was the Operations EvaluationDepartment (OED). The name changed from OED toIEG in December 2005. Relevant documents pertainingto the CPIA and other evaluations from the earlierperiod may still be catalogued under OED in somedatabases.3. The external panel consisted of nine academicsand public officials from developed and developingcountries. The panel met at the Bank on February17–18, 2004, and reviewed the coverage of the CPIAsystem, methodology, database, and cross-countrycomparability. The panel submitted its final report toManagement on April 2, 2004. See World Bank 2004a.4. The cluster was renamed from “public finance/civil administration” in 1997.5. The criteria on financial stability and financialsector depth, efficiency and resource mobilizationwere collapsed into a new financial sector criterion,and the competitive environment for the privatesector and goods and factor markets criteria werecollapsed into the new business regulatory environmentcriterion.6. One of these is the criterion on managementand sustainability of the development program,which is covered in almost all of the other criteria. Theother is the criterion on monitoring and analysis ofpoverty outcomes and impacts. This criterion coversthe availability of up-to-date household surveys andanalysis, which is necessary for the criterion on equityof public resource use, and the criterion on buildinghuman resources. (World Bank 2004d).7. There were some changes in the instructionsfor some criteria mainly aimed at reducing overlapbetween criteria, and improving consistency.8. World Bank (2008e, annex 1, p. 2).Chapter 21. These are some of the key elements of theso-called Washington Consensus view.2. The point of view of Rodrik and others seems tobe accepted by the majority of the economists today(even if this consensus is obviously quite recent).Aghion and Howitt (2009) reconcile new growth theorywith what they call “Gerschenkron’s views,” therebyaddressing development economists’ concern thatgrowth theory can only deliver universal, one-sizefits-allpolicy prescriptions (legal reform to enforceproperty rights, investment climate favorable toentrepreneurship, education, macrostability, and soon) to maximize the growth prospects of a countryor sector, and does not apprehend structural transformationsin the process of convergence. (New growththeory calls for better property rights protection andhigher education investment in all countries under alllatitudes. Gerschenkron’s view is the idea that relativelybackward economies could more rapidly catch up withmore advanced economies by introducing appropriateinstitutions that are growth enhancing at an early stageof development but may cease to be so at a later stage).More specifically, they analyze some general implicationsof the notion of “distance-dependent” appropriateinstitutions, by which they mean institutions that aregrowth enhancing only for countries at a certain stageof technological development. In particular, they showhow the failure to adapt institutions to technologicaldevelopment may generate non-convergence trapswhereby a country’s average productivity (or per capitaGDP) remains bounded away from frontier levels.3. The theoretical and empirical analyses ofVandenbussche, Aghion, and Meghir (2006) suggestthat countries with productivities far from the technologicalfrontier should put more emphasis on primary/secondary education, whereas countries closer tothe frontier should put more emphasis on tertiaryeducation.1 1 3

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n t4. See Aghion and Howitt (2009) for a discussion.For the neoclassical and endogenous growth theories,it is investment in physical and human capital thatdrives growth, whereas for the product variety andSchumpeterian theories, it is investment in technology(in the form of research). In the hybrid model, investmentin capital and technology are both important.5. The discussion in this paragraph is drawn fromWorld Bank (2008a).6. Bourguignon (2004) indicated that a seriousevaluation of Mexico’s Progresa/Oportunidades andBrazil’s Bolsa Escola/Bolsa Families, essentially meanstestedincome transfer programs with conditionalities,finds that these programs were effective in raisingschool enrollment rates and health outcomes intargeted populations. The sources cited for this wereSkoufias (2001) on Progresa, and Bourguignon,Ferreira, and Leite (2003) on Bolsa Escola; and WorldBank (2004g).7. This has especially been the case with EastAsian countries. See World Bank (1993).8. An early study of Grameen Bank finds that itgenerated employment and income for the poor,especially women (Hossain 1988). The most comprehensiveimpact studies of microfinance, a joint researchproject of the Bangladesh Institute of DevelopmentStudies and the World Bank find strong evidence thatthe programs help the poor through asset building(and consumption smoothing) (Khandker 1998; Pittand Khandker 2003).9. A very recent paper (Freund and Rocha 2010)estimates that for African exports, a one-day reduction ininland travel times translates into nearly a 3 percentagepoint reduction in all importing country tariffs.10. This is in accordance with the so-called Lerner’ssymmetry, whereby taxing imports has the same effecton international trade as taxing exports.11. The phenomenon is called the “environmentalKuznets curve” whereby pollution levels, initially fairlylow, rise as national income increases up to a certainpoint, then begin to decline with further economicgrowth as cleaner technologies are adopted andenvironmental sanitation infrastructure investmentsbegin to catch up with expanding local needs (IEG2008, chapter 1, footnote 3).12. The allocations were calculated using the PBAformula. The GNI per capita and population data arefrom World Development Indicators, and the portfolioperformance ratings are constructed using datafrom The World Bank’s internal database. Because thesimulations focused on changes in allocations, theydid not require data on the overall IDA envelope.13. In fiscal 2009, the post-conflict countries areAfghanistan, Angola, Burundi, Democratic Republicof Congo, Republic of Congo, Côte d’Ivoire, Eritrea,Liberia, and Timor-Leste. The re-engaging countriesare Central African Republic, Haiti, and Togo. Thecapped countries are India and Pakistan.14. References are cited in Kanbur (2004).15. A 1 percent rate of growth in average householdincome or consumption brought anything from amodest drop in the poverty rate of 0.6 percent to amore dramatic 3.5 percent annual decline. (Ravallion2001).16. This is a term coined by Ghura, Leite, andTsangarides (2002).17. Besley and Burgess (2003) underline thatexpanding access to credit for the poor may increasethe elasticity between economic growth and povertyreduction, in addition to being a form of redistribution.18. Honahan (2004b), based on analysis of a crosssectionof some 70-odd developing countries.19. According to Winters, McCulloch, and McKay(2004), these channels are: (i) job opportunities andwages of the poor; (ii) prices that poor consumerspay for the goods they buy; (iii) government revenuesand in turn social expenditures; (iv) income instabilityand workers’ chances of becoming poor. Goldbergand Pavcnik (2004) identify three other channels:(i) participation of household members in thelabor market; (ii) household consumption; and (iii)household production.20. Porto (2005) shows that reducing such barrierswould reduce poverty from an initial headcount ratioof 48.3 percent to a poverty rate from 43.3 to 45.5percent. In other words, it would lift between 100,000to 180,000 Moldovan citizens (out of a population of3.5 million) out of poverty.21. Notably, this has followed Amartya Sen’s workon poverty and freedom (Sen 1997, 1999).22. It is recognized that economic developmentencompasses more dimensions than just growth,although growth is certainly the key dimension, andmoreover the only dimension for which a theoreticalfoundation exists. Poverty reduction is anotherdimension of economic development, and it is alsorecognized that although growth is not sufficientfor poverty reduction, it is necessary. Therefore, the1 1 4

e n d n o t e sdiscussion on the theoretical and empirical literatureof the impact of aid on growth also pertains to theimpact of aid on reduction of income poverty.23. These are the Harrod-Domar model and itsextension the Solow growth model.24. Boone (1994). According to Clemens, Radelet,and Bhavnani (2004), parts of the analysis of Boone(1994) were published in Boone (1996) without thegrowth regressions.25. The exception was small countries, where aidflows which made up a large share of GDP were foundto lead to higher investments, which Boone attributedto the lack of fungibility of aid flows. Boone providedan example that in a small country one dam or a largepublic infrastructure project can represent a sizeableportion of GNP, and the project is unlikely to be fungible.26. Other innovations in this later generation ofempirical literature include: larger datasets that covermore countries and more years; accounting for theendogeneity of aid (that is, in addition to growthbeing an outcome of aid, aid can also be influenced bythe growth performances of countries); and allowingfor a non-linear aid-growth relationship (in particulardiminishing returns to aid, or in other words, decreasingimpact of aid on growth after reaching a point ofmaximum impact).27. In Burnside and Dollar (1997, 2000), fiscalpolicy is measured by the budget surplus (followingEasterly and Rebelo 1993), monetary policy by theinflation rate (following Fischer 1993) and trade bytrade openness (as measured by Sachs and Warner1995).28. Including the relative size of the donor versusthe recipient (the larger the ratio, the greater theinfluence, and presumably the larger amounts of aid),commonality of language, and colonial ties (eithercurrent or past).29. Roodman (2007a). He performed the tests onBurnside and Dollar (2000), Collier and Dehn (2001),Collier and Dollar (2002), Collier and Hoeffler (2004),Hansen and Tarp (2001), Dalgaard, Hansen, and Tarp(2004), and Guillaumont and Chauvet (2001). Thetests were designed to minimize arbitrariness andwere derived mainly from the differences among thepapers themselves. The tests included: (i) changingthe control set (using the control sets of other papersbeing tested); (ii) redefining aid; (iii) redefining goodpolicy; (iv) changing periodization; (v) changingoutliers; and (vi) expanding the sample.30. Such problems include difficulties with: (i)accounting for endogeneity (in particular, whichinstruments to use); (ii) the specification of the regressionequation (including the timing of the impact ofaid on growth); and (iii) the treatment of outliers; datalimitations, among others.31. Headey (2005) and Reddy and Minoiu (2006)find that multilateral or developmental aid has apositive impact on growth, whereas geopolitical andbilateral aid do not (except for bilateral aid that is notpolitically motivated). Ram (2003) finds that bilateralbut not multilateral aid has a large and positive impacton growth (although this paper has been criticized fornot taking the endogeneity of aid into account).32. Clemens, Radelet, and Bhavnani (2004) alsofind that the impact on growth is somewhat largerin countries with stronger institutions or longer lifeexpectancies.33. The concept of such evaluations originatesfrom the scientific/medical realm of randomized trials,whereby the effects of medicines are tested by comparingthe group of patients who received the medicinewith a group who received the placebo. The selectionof the “comparable” group is therefore key and, in thecontext of aid project evaluations, needs to ensure thatthe differential effects between the two groups couldnot be due to some characteristic of the recipientgroup that is not present in the control group.34. A request made by the IEG team to the networkon April 23, 2009, for the number of responses to theenvironment questionnaire for 2008 was not answered.35. This is constructed by applying the relevantweights to the ratings for market access and borderadministration, respectively, of the Enabling TradeIndex.36. Although the rank correlation coefficient withone based on the Global Competitiveness Index (GCI)will remain unchanged at 0.70. The one based on theGCI is constructed by applying the relevant weights tothe combined rating for tariffs and trade barriers (50percent each) and the rating for burden of customsadministration.37. As in the simulation on dropping q8, thissimulation is conducted based on a CPIA implied bythe IDA allocation formula—that is, 25 percent weightgiven to CPIA clusters A through C, and 75 percentweight on cluster D.38. There is room for debate as to the causalitybetween every banking crisis and output losses, as1 1 5

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tthere have certainly been instances in the past wherefiscal or political crises have triggered banking crises,such as Argentina in 1981, in the transition countries,and in several African countries.39. At the same time, however, what evidencethere is does not suggest that crises systematicallyworsen the Gini coefficient (Honohan 2004a), that is,that banking crises have a disproportionate effect onthe poor, although the poor are likely less able thanothers to absorb adverse shocks.40. One is the stock of NPLs as a share of totalloans; the other, the “level of capital at risk,” is a netfigure which subtracts provisions already taken againstthese NPLs.Chapter 31. For the criteria on gender (q7), property rightsand rule-based governance (q12), and quality ofbudgetary and financial management (q13), the2007 AfDB questionnaire is identical to the pre-2005Bank CPIA questionnaire that has since been updated,although the differences are not substantial. Further,the Bank’s ratings for the criterion on policies andinstitutions for environmental sustainability (q11)are relatively stringent compared with those of theAfDB. For example, the Bank gives a “1” rating on q11if “For both pollution and natural resource issues:regulations and policies are lacking. Environmentalinformation is not published. Environmental Assessmentlegislation is lacking. No data are available forpriority setting. Sector ministries do not incorporateenvironmental concerns.” A “1” rating is given by AfDBif “For two years or more, government policies have anegative effect on environment (for example agriculturepolicies that stimulate expansion into marginalland or tropical forest; subsidized prices on the exploitationof scarce and/or non-renewable resources).Government has no environmental action plans orsimilar national framework, and no institutions tosustainably manage the environment and support thevarious dimensions of sustainable development.”2. It is difficult to find good comparators forthe economic management criteria because bothoutcomes and policies are taken into account in theCPIA ratings, whereas other indicators focus only onoutcomes (such as the economic risk index producedby the International Country Risk Guide).3. The IEG team examined two indicators that arepossible comparators for q13. These are one of theglobal integrity indicators produced by Global Integritythat rate the budget process, and the Open BudgetIndex produced by the Open Budget Initiative that ratesthe transparency of the budget. However, both theseindicators comprise only part of what is being assessedunder q13, so are not strictly comparable with q13, andhence the team did not include them as comparators.4. Rank correlations are used instead of pair-wisecorrelations for comparing the different indicatorsbecause of the less restrictive assumption underlyingrank correlations (they do not require the assumptionof a linear relationship between the indicators).Nonetheless, the IEG team for this evaluationperformed both types of correlations and obtainedvery similar results from them.5. A better correlation denotes a rank correlationcoefficient that is at least 5 percent higher.6. Ratings from ADB are excluded from thiscomparison because only 3 IBRD countries are ratedby ADB, whereas 12 are rated by AfDB.Appendix F1. There is room for debate as to the causalitybetween every banking crisis and output losses, asthere have certainly been instances in the past wherefiscal or political crises have triggered banking crises,such as Argentina in 1981, in the transition countries,and in several African countries.2. At the same time, however, what evidence thereis does not suggest that crises systematically worsenthe Gini coefficient (Honohan 2004a), that is, thatbanking crises have a disproportionate effect on thepoor, although the poor are likely less able than othersto absorb adverse shocks.3. Stress tests model the impact of extreme butplausible shocks, and measure the capacity of banks toabsorb the shocks with available liquidity and capital.4. One is the stock of NPLs as a share of total loans;the other, the “level of capital at risk,” is a net figurewhich subtracts provisions already taken against theseNPLs.Appendix G1. For loans approved from 1998 onward, thecorrelation between the share of problem projects in acountry and the share of loans receiving an unsatisfactoryIEG outcome rating for that country is 0.63.2. The level of per capita GDP was not includedas a control variable due to its high correlation with1 1 6

e n d n o t e sthe overall CPIA rating. The percent of problem loansmight also depend on the quality of the Bank staffworking on those loans; ratings on task team qualityare provided by the Quality Assurance Group (QAG)in its quality at entry ratings for loans. However, dueto limited data availability, the quality of Bank staff wasnot included as a control variable.3. To account for the possible endogeneity of boththe CPIA ratings and the Aid to GDP variable, two-stageleast square specifications were estimated. The CPIAand Aid to GDP variable were instrumented using theshare of population speaking English, the share ofpopulation speaking a continental European language,distance from equator, level of population, and each ofthe above four instruments multiplied by population(see Dollar and Levin 2005). The first-stage regressionssuggest that the instruments are weak (the F-statisticsare relatively low). The Durbin-Wu-Hausmantest for endogeneity suggests that both variables areexogenous and the OLS estimates are consistent.1 1 7

1 1 8Women preparing food, Nigeria. Photo by Curt Carnemark/World Bank

ReferencesAcemoglu, Daron, and Simon Johnson. 2007.“Disease and Development: The Effect of LifeExpectancy on Economic Growth.” Journalof Political Economy 115 (61): 925–985.Acemoglu, Daron, and James A. Robinson. 2000.“Political Losers as a Barrier to EconomicDevelopment.” American Economic ReviewPapers and Proceedings 90: 126–44.Acemoglu, Daron, and Thierry A. Verdier. 2000.“The Choice Between Market Failures andCorruption.” American Economic Review 90(1): 194–211.———. 2001. “The Colonial Origins of ComparativeDevelopment: An Empirical Investigation.”American Economic Review 91(5): 1369–401.Acemoglu, Daron, Simon Johnson, and James A.Robinson. 2005. “The Rise of Europe: AtlanticTrade, Institutional Change, and EconomicGrowth.” American Economic Review 95 (3):546–79.Adsera, Alicia, Carles Boix, and Mark Payne. 2003.“Are You Being Served? Political Accountabilityand Quality of Government.” Journal ofLaw, Economics, & Organization 19 (2):445–90.Agenor, Pierre-Richard. 2004. “Does GlobalizationHurt the Poor?” International Economicsand Economic Policy 1 (1): 21–51.Aghion, Philippe, and Peter Howitt. 2009. TheEconomics of Growth. Cambridge, Mass.: TheMIT Press.Aghion, Philippe, Eve Caroli, and Cecilia García-Peñalosa. 1999. “Inequality and EconomicGrowth: The Perspective of the New GrowthTheories.” Journal of Economic Literature37 (4): 1615–60.Aghion, Philippe, Peter Howitt, and DavidMayer-Foulkes. 2005. “The Effect of FinancialDevelopment on Convergence: Theory andEvidence.” Quarterly Journal of Economics120 (1): 173–222.Ahmad, Ehtisham, and Nicholas Stern. 1987.“Alternative Sources of Government Revenue:Illustrations from India, 1979–80.” In TheTheory of Taxation for Developing Countries.David M. G. Newbery, and Nicholas H. Stern(editors), New York: Published for The WorldBank by Oxford University Press.Ahrend, Rudiger. 2001. “Press Freedom, HumanCapital and Corruption.” Mimeo.Alesina, Alberto. 1997. “Politics, Procedures andBudget Deficits.” Background Paper preparedfor The World Development Report 1997.World Bank, Washington, DC.Alesina, Alberto, and George-Marios Angeletos.2005. “Corruption, Inequality and Fairness.”National Bureau of Economic Research(NBER) Working Paper No. 11399.Alesina, Alberto, and Roberto Perotti. 1996.“Income Distribution, Political Instability, andInvestment.” European Economic Review 40(6): 1203–28.Alesina, Alberto, and Dani Rodrik. 1994.“Distributive Politics and Economic Growth.”Quarterly Journal of Economics 109 (2):465–90.Alesina, Alberto, and Beatrice Weder. 2002. “DoCorrupt Governments Receive Less ForeignAid?” The American Economic Review 92 (4):1126–37.Alesina, Alberto, Arnaud Devleeschauwer,William Easterly, Sergio Kurlat, and RomainWacziarg. 2003. “Fractionalization.” Journalof Economic Growth 8 (2): 155–94.Ames, Brian, Ward Brown, Shanta Devarajan, andAlejandro Izquierdo. 2001. “MacroeconomicPolicy and Poverty Reduction.” Prepared by1 1 9

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tThe International Monetary Fund and TheWorld Bank.Anderson, Kym. 2003. “Trade Liberalization,Agriculture, and Poverty in Low-incomeCountries.” WIDER Discussion Paper No.2003/25. United Nations University.Andvig, Jens, and Odd-Helge Fjeldstad. 2008.“Crime, Poverty and Police Corruption inDeveloping Countries.” Christian MichelsenInstitute (CMI) Working Paper No. 2008:7.Angrist, Joshua, Eric Bettinger, and MichaelKremer. 2006. “Long-Term EducationalConsequences of Secondary School Vouchers:Evidence from Administrative Records inColombia.” American Economic Review 96(3): 847–62.Angrist, Joshua, Eric Bettinger, Erik Bloom,Elizabeth King, and Michael Kremer. 2002.“Vouchers for Private Schooling in Colombia:Evidence from a Randomized Natural Experiment.”American Economic Review 92 (5):1535–58.Ashraf, Nava, James N. Berry, and Jesse Shapiro.2007. “Can Higher Prices Stimulate ProductUse? Evidence from a Field Experiment inZambia.” NBER Working Paper No. 13247.Ashraf, Quamrul, Ashley Lester, and David Weil.2008. “When Does Improving Health RaiseGDP?” NBER Working Paper No. 14449.Ayers, Robert. 1997. “Crime and Violence asDevelopment Issues in Latin America and theCaribbean.” Paper prepared for a World BankSeminar on Urban Criminal Violence, Rio deJaneiro, March 2–4.Ayyagari, Meghana, Asli Demirguc-Kunt, andVojislav Maksimovic. 2007. “Firm Innovationin Emerging Markets: Role of Governance andFinance.” Policy Research Working Paper No.4157. World Bank, Washington, DC.Azfar, Omar, Young Lee, and Anand Swamy. 2001.“The Causes and Consequences of Corruption.”Annals of the American Academy ofPolitical and Social Science 573: 42–56.Bahl, Roy, and Richard Bird. 2008. “Tax Policyin Developing Countries: Looking Backand Forward.” Institute for InternationalBusiness Working Paper No. 13. Joseph L.Rotman School of Management, Universityof Toronto.Balat, Jorge, Irene Brambilla, and Guido Porto.2007. “Realizing the Gains from Trade: ExportCrops, Marketing Costs, and Poverty.” NBERWorking Paper No. 13395.Banerjee, Abhijit, Paul Gertler, and MaitreshGhatak. 2002. “Empowerment and Efficiency:Tenancy Reform in West Bengal.” The Journalof Political Economy 110 (2): 239–80.Banerjee, Abhijit, Shawn Cole, Esther Duflo, andLeigh Linden. 2005. “Remedying EducationEvidence from Two Randomized Experimentsin India.” NBER Working Paper No. 11904.———. 2007. “Remedying Education: Evidencefrom Two Randomized Experiments in India.”Quarterly Journal of Economics, 122 (3):1235–64.Bardhan, Pranab. 1997. “Corruption andDevelopment: A Review of Issues.” Journalof Economic Literature 35 (3): 1320–46.———. 2000. “Social Justice in the GlobalEconomy.” International Labor OrganizationSocial Policy Lecture, University of theWestern Cape, South Africa. September 1–6.———. 2006a. “The Economist’s Approach tothe Problem of Corruption.” World Development34 (2): 341–48.———. 2006b. “Institutional Economics ofDevelopment: Some General Reflections.”Opening Lecture at CESifo/BREAD Conferenceon Institutional Economics at SanServolo, Italy.———. 2006c. “Institutions, Trade and Development.”Paper prepared for UNCTAD, Geneva,Switzerland.Barham, Vicky, Robin Boadway, Maurice Marchand,and Pierre Pestieau. 1995. “Educationand the Poverty Trap.” European EconomicReview 39 (7): 1257–75.Barro, Robert. 1999. “Determinants of Democracy.”Journal of Political Economy 107 (6) 2.———. 2003. “Determinants of EconomicGrowth in a Panel of Countries.” Annals ofEconomics and Finance 4: 231–74.Barro, Robert, and Xavier Sala-i Martin. 2003.Economic Growth. Cambridge, Mass.: TheMIT Press.Barth, James, Gerard Caprio, and Ross Levine.2006. Rethinking Bank Regulation: Till AngelsGovern. New York: Cambridge University Press.1 2 0

R E F E R E N C E SBeck, Paul, and Michael Maher. 1986. “A Comparisonof Bribery and Bidding in Thin Market,”Economics Letters 20: 1–5.Beck, Thorsten, Asli Demirguc-Kunt, and RossLevine. 2004. “Finance, Inequality, andPoverty: Cross-Country Evidence.” NBERWorking Paper No. 10979.———. 2005. “Law and Firms’ Access to Finance.”American Law and Economics Review 7 (1):211–52.Becker, Gary, and George Stigler. 1974. “LawEnforcement, Malfeasance and the Compensationof Enforcers.” Journal of Legal Studies3 (1): 1–19.Bell, Clive, Shantayanan Devarajan, and HansGersbach. 2006. “The Long-Run EconomicCosts of AIDS: A Model with an Applicationto South Africa.” The World Bank EconomicReview 20 (1): 55–89.Berger, Allen N., W. Scott Frame, and Nathan H.Miller. 2005. “Credit Scoring and the Availability,Price, and Risk of Small Business Credit.”Journal of Money, Credit, and Banking 37(2): 191–222.Besley, Timothy. 1995. “Property Rights andInvestment Incentives: Theory and Evidencefrom Ghana.” The Journal of PoliticalEconomy 103 (5): 903–37.Besley, Timothy, and Robin Burgess. 2000. “LandReform, Poverty Reduction, and Growth:Evidence from India.” Quarterly Journal ofEconomics 115 (2): 389–430.———. 2002. “The Political Economy of GovernmentResponsiveness: Theory and Evidence.”Quarterly Journal of Economics 117 (4):1415–51.———. 2003. “Halving Global Poverty.” TheJournal of Economic Perspectives 17 (3):3–22.Besley, Timothy, and Anne Case. 1995. “DoesElectoral Accountability Affect EconomicPolicy Choices? Evidence from GubernatorialTerm Limits.” Quarterly Journal of Economics110 (3): 769–98.Besley, Timothy, Robin Burgess, and Andrea Prat.2002. “Mass Media and Political Accountability.”In The Right to Know: Institutions andthe Media, Roumeen Islam (ed.). World Bank,Washington, DC.Bettinger, Eric, Michael Kremer, and JuanSaavedra. 2008. “Are Educational VouchersOnly Redistributive?” Draft, prepared for theCESifo/PEPG joint conference “EconomicIncentives: Do They Work in Education?”Munich, Germany, May 16–17.Bird, Richard. 2003. “Administrative Dimensionsof Tax Reform.” International Tax ProgramPaper No. 0302. Joseph L. Rotman School ofManagement, University of Toronto.———. 2005. “Value-Added Taxes in Developingand Transitional Countries: Lessons andQuestions.” International Tax Program PaperNo. 0505. Joseph L. Rotman School of Management,University of Toronto.———. 2008. “Tax Challenges Facing DevelopingCountries.” Institute for InternationalBusiness Working Paper No. 12.Bird, Richard, and Duanjie Chen. 1998. “TheFiscal Framework for Business in Asia.” InFiscal Framework and Financial Systemin East Asia: How Much Do They Matter?,Wendy Dobson (ed.). Toronto: University ofToronto Press.Bird, Richard, and Pierre-Pascal Gendron. 2006.“Is VAT the Best Way to Impose a GeneralConsumption Tax in Developing Countries?”International Tax Program Paper No. 0602.Joseph L. Rotman School of Management,University of Toronto.———. 2007. The VAT in Developing and TransitionalCountries. New York: CambridgeUniversity Press.Bird, Richard, and Barbara Miller. 1989. “TheIncidence of Indirect Taxation on Low-IncomeHouseholds in Jamaica.” Economic Developmentand Cultural Change 37 (2): 393–409.Bird, Richard, and Enid Slack. 2002. “Land andProperty Taxation around the World: AReview.” Journal of Property Tax Assessmentand Administration 7 (3): 31–80.Bird, Richard, and Eric Zolt. 2007. “Tax Policyin Emerging Countries.” International TaxProgram Paper No. 0707. Joseph L. RotmanSchool of Management, University of Toronto.Bjorkman, Martina, and Jakob Svensson. 2009.“Power to the People: Evidence from aRandomized Field Experiment of Community-Based Monitoring in Uganda.” Policy Research1 2 1

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tWorking Paper No. 4268. World Bank,Washington, D.C.Bloom, David E. and Ajay S. Mahal. 1995. “Doesthe AIDS Epidemic Really Threaten EconomicGrowth?” NBER Working Paper No. 5148.———. 1997. “Does the AIDS Epidemic ThreatenEconomic Growth?” Journal of Econometrics77 (1): 105–24.Bobonis, Gustavo J., Edward Miguel, and CharuPuri-Sharma. 2006. “Anemia and School Participation.”Journal of Human Resources 41(4): 692–721.Bodewig, Christian, and Akshay Sethi. 2005.Poverty, Social Exclusion and Ethnicity inSerbia and Montenegro: The Case of Roma.World Bank, Washington, DC.Boex, L., F. Jameson, Jorge Martinez-Vazquez,and Robert McNab. 2000. “Multi-Year Budgeting:A Review of International Practices andApplications for Developing and TransitionalEconomies.” Public Budgeting and Finance20 (2).Boone, Peter. 1994. “The Impact of Foreign Aidon Savings and Growth.” Center for EconomicPerformance Working Paper No. 677, LondonSchool of Economics.———. 1996. “Politics and the Effectiveness ofForeign Aid.” European Economic Review 40(2): 289–329.Boskin, Michael. 2006. “Eight Lessons for PublicFinance in Developing Economies.” ChinaDevelopment Forum, State Council of thePRC, Beijing, March.Bourguignon, Francois. 1998. “Crime as a SocialCost of Poverty and Inequality: A ReviewFocusing on Developing Countries.” DELTA,Paris. Mimeo.———. 1999. “Crime, Violence and InequitableDevelopment.” Paper prepared for theAnnual World Bank Conference on DevelopmentEconomics, Washington, D.C., April28–30.———. 2004. “The Poverty-Growth-InequalityTriangle.” Paper presented at the IndianCouncil for Research on InternationalEconomic Relations, New Delhi, February 4.Bourguignon, Francois, Francisco Ferreira, andPhillippe Leite. 2003. “Conditional CashTransfers, Schooling and Child Labor: Micro-simulating Brazil’s Bolsa Escola Program.”World Bank Economic Review 17 (2).Bovenberg, A. Lans, and Sjak Smulders. 1995.“Environmental Quality and Pollution-Augmenting Technical Change in a Two-SectorEndogenous Growth Model.” Journal ofPublic Economics 57 (3): 369–91.———. 1996. “Transitional Impacts of EnvironmentalPolicy in an Endogenous Growth Model.”International Economic Review 37 (4): 861–93.Braithwaite, Valerie. 2003. Taxing Democracy.Understanding Tax Avoidance and Evasion.Aldershot and Burlington, VT: Ashgate.Brautigam, Deborah. 2000. Aid Dependence andGovernance. Stockholm: Almqvist & Wiksell.International for the Swedish Ministry ofForeign Affairs.Breen, Richard, and Cecilia García-Peñalosa.2005. “Income Inequality and MacroeconomicVolatility: An Empirical Investigation.” Reviewof Development Economics 9 (3): 380–98.Brown, Michael, Tulio Jappelli, and MarcoPagano. 2009. “Information Sharing andCredit: Firm-Level Evidence from TransitionCountries.” Journal of Financial Intermediation18 (2): 151–72.Brunetti, Aymo, and Beatrice Weder. 1999. “A FreePress is Bad News for Corruption.” Mimeo.Basel, Switzerland: University of Basel.Burgess, Robin, and Rohini Pande. 2005. “DoRural Banks Matter? Evidence from the IndianSocial Banking Experiment.” AmericanEconomic Review 95 (3): 780–95.Burgess, Robin, and Nicholas Stern. 1993.“Taxation and Development.” Journal ofEconomic Literature 31 (2): 762–830.Burnside, Craig, and David Dollar. 1997. “Aid,Policies, and Growth.” Policy ResearchWorking Paper No. 1777. World Bank,Washington, DC.———. 2000. “Aid, Policies, and Growth.”American Economic Review 90 (4): 847–68.———. 2004. “Aid, Policies, and Growth: Reply.”American Economic Review 94 (3): 781–84.Buvinic, Mayra, and Andrew R. Morrison. 2000.“Living in a More Violent World.” ForeignPolicy 118: 58–72.Buvinic, Mayra, Andrew R. Morrison, and MichaelShifter. 1998. “Violence in Latin America and1 2 2

R E F E R E N C E Sthe Caribbean: A Framework for Action.” Paperpresented at an Inter-American DevelopmentBank (IDB) Seminar on “BuildingPeaceful Societies: A Framework for Action.”Cartagena, Columbia.Cagé, Julia. 2009a. “Literature Review for aSpecial Study by the Independent EvaluationGroup on the Country Policy and InstitutionalAssessment of The World Bank.” BackgroundPaper for this CPIA evaluation.———. 2009b. “Asymmetric Information,Rent Extraction and Aid Efficiency.” AgenceFrançaise de Développement Working Paper.Calderon, Cesar, and Luis Serven. 2003. “TheOutput Cost of Latin America’s InfrastructureGap.” In The Limits of Stabilization:Infrastructure, Public Deficits and Growthin Latin America, William Easterly and LuisServen (editors). Palo Alto, CA: StanfordUniversity Press, and The World Bank.———. 2004. “The Effects of InfrastructureDevelopment on Growth and Income Distribution.”Policy Research Working Paper No.3400. World Bank, Washington, DC.Call, Charles. 2000. “Sustainable Development inCentral America: The Challenges of Violence,Injustice and Insecurity.” Hamburg Institut fürIberoamerika-Kunde, CA 2020 Working PaperNo. 8.Campos, Edgardo, and Sanjay Pradhan. 1996.“Budgetary Institutions and ExpenditureOutcomes. Binding Governments to FiscalPerformance.” Policy Research Working PaperNo. 1646. World Bank, Washington, DC.Cardenas, Enrique, Jose. Ocampo, andRosemary Thorp. 2000. The Export Age:The Latin American Economies in the LateNineteenth and Early Twentieth Centuries.(Vol. 1, An Economic History of TwentiethCentury Latin America). New York: PalgravePress.Cashin, Paul Anthony, and Rupa Duttagupta.2008. “The Anatomy of Banking Crises.” IMFWorking Paper WP/08/93.Chand, Sheetal, and Karl Moene. 1999. “ControllingFiscal Corruption.” World Development27(7): 1129–40.Chetwynd, Eric, Frances Chetwynd, and BertramSpector. 2003. “Corruption and Poverty: AReview of Recent Literature.” United StatesAgency for International Development(USAID) Final Report.Chhibber, Ajay, and Gaurav Nayyar. 2007.“Pro-Poor Growth: Explaining the Cross-Country Variation in the Growth Elasticityof Poverty.” Brooks World Poverty Institute(BWPI) Working Paper No. 14, The Universityof Manchester.Chong, Alberto, and Cesar Calderon. 2000.“Causality and Feedback between InstitutionalMeasures and Economic Growth.” Economicsand Politics 12 (1): 69–81.Christiaensen, Luc, Lionel Demery, and StefanoPaternostro. 2002. “Growth, Distribution, andPoverty in Africa: Messages from the 1990s.”Policy Research Working Paper No. 2810.World Bank, Washington, DC.Clemens, Michael, Steven Radelet and RikhilBhavnani. 2004. “Counting Chickens whenThey Hatch: The Short Term Effect of Aidon Growth.” Center for Global DevelopmentWorking Paper No. 44.Cnossen, Sijbren. 2004. “VAT in South Africa:What Kind of Rate Structure?” InternationalVAT Monitor 15 (1): 19–24.Collier, Paul. 2007. The Bottom Billion: Why thePoorest Countries Are Failing and WhatCan Be Done About It. New-York and Oxford:Oxford University Press.Collier, Paul, and Jan Dehn. 2001. “Aid, Shocksand Growth.” Policy Research Working PaperNo. 2688. World Bank, Washington, DC.Collier, Paul, and David Dollar. 2002. “Aid Allocationand Poverty Reduction.” EuropeanEconomic Review 46 (8): 1475–1500.———. 2004. “Development Effectiveness.”Economic Journal 114 (496): F244–F271.Collier, Paul, and Anke Hoeffler. 2002. “Aid,Policy and Peace: Reducing the Risks of CivilConflict.” Journal of Defense Economics andPeace 13 (6): 435–50.———. 2004. “Aid, Policy and Growth inPost-Conflict Societies.” European EconomicReview 48 (5): 1125–45.Conley, Timothy G., and Christopher R. Udry. 2007.“Learning About a New Technology: Pineapplein Ghana.” Economic Growth Center DiscussionPaper No. 817, Yale University.1 2 3

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tCuddington, John T. 1993. “Modeling theMacroeconomic Effects of AIDS, with anApplication to Tanzania.” The World BankEconomic Review 7 (2): 173–189.Cuddington, John T., and John D. Hancock. 1994.“Assessing the Impact of AIDS on the GrowthPath of the Malawian Economy.” Journal ofDevelopment Economics 43 (2): 363–68.Dalgaard, Carl-Johan, and Henrik Hansen. 2001.“On Aid, Growth and Good Policies.” Journalof Development Studies 37 (6): 17–41.Dalgaard, Carl-Johan, Henrik Hansen, and FinnTarp. 2003. “On the Empirics of Foreign Aidand Growth.” The Economic Journal 114(496): F191–F216.Dehejia, Rajeev, and Roberta Gatti. 2002. “ChildLabor: The Role of Income Variability andAccess to Credit across Countries.” NBERWorking Paper No. 9018.De la Croix, David, and Clara Delavallade. 2006.“Growth, Public Investment and Corruptionwith Failing Institutions.” Center forOperations Research and Econometrics(CORE) Discussion Paper No. 2006/101.Demirgüç-Kunt, Asli, Edward J. Kane, and LucLaeven. 2006. “Deposit Insurance Designand Implementation: Policy Lessons fromResearch and Practice.” Policy ResearchWorking Paper No. 3969. World Bank,Washington, DC.De Nicoló, Gianni, Patrick Honohan, and AlainIze. 2005. “Dollarization of Bank Deposits:Causes and Consequences.” Journal ofBanking and Finance 29 (7): 1697–1727.Deolalikar, Anil, Alex Brillantes Jr., Raghav Gaiha,Ernesto Pernia, and Mary Racelis. 2002.“Poverty Reduction and the Role of Institutionsin Developing Asia.” Asian DevelopmentBank Economics and Research DepartmentWorking Paper No. 10.De Soto, Hernando. 2000. The Mystery ofCapital: Why Capitalism Triumphs in theWest and Fails Everywhere Else. New York:Random House.Devas, Nick, Simon Delay, and Michael Hubbard.2001. “Revenue Authorities: Are They theRight Vehicle for Improved Tax Administration?”Public Administration and Development21 (3): 211–22.Dillinger, William. 1992. Urban Property TaxReform. Guidelines and Recommendations.World Bank, Washington, DC.Di Tella, Rafael, and Ernesto Schargrodsky. 2003.“The Role of Wages and Auditing during aCrackdown on Corruption in the City ofBuenos Aires.” Journal of Law and Economics46 (1): 269–92.Dixit, Avinash. 2007. “Evaluating Recipes forDevelopment Success.” World Bank ResearchObserver 22 (2): 131–57.Djankov, Simeon, Rafael La Porta, FlorencioLopez-de Silanes, and Andrei Shleifer. 2009.“Disclosure by Politicians.” NBER WorkingPaper No. 14703.Djankov, Simeon, Caralee McLiesh and AndreiShleifer. 2007. “Private Credit in 129Countries.” Journal of Financial Economics84 (2): 299–329.Dollar, David, and Aart Kraay. 2002. “Growth isGood for the Poor.” Journal of EconomicGrowth 7 (3): 195–225.Dollar, David, and Victoria Levin. 2005. “Sowingand Reaping: Institutional Quality and ProjectOutcomes in Developing Countries.” PolicyResearch Working Paper No. 3524. WorldBank, Washington, DC.Doppelhofer, Gernot, Ronald Miller, andXavier Sala-i Martin. 2004. “Determinantsof Long-Term Growth: A Bayesian Averagingof Classical Estimates (BACE) Approach.”American Economic Review 94 (4): 813–35.Dreze, Jacques, and Amartya Sen. 1989. Hungerand Public Action. New York: Oxford UniversityPress.———. 1995. India: Economic Developmentand Social Opportunity. New Delhi: OxfordUniversity Press.———. 2002. India Development and Participation.New Delhi: Oxford University Press.Duncan-Waite, Imani, and Michael Woolcock.2008. “Arrested Development: The PoliticalOrigins and Socio-Economic Foundations ofCommon Violence in Jamaica.” Brooks WorldPoverty Institute (BWPI) Working Paper 46,University of Manchester, U.K.Due, John. 1988. Indirect Taxation in DevelopingCountries. Baltimore: Johns HopkinsUniversity Press.1 2 4

R E F E R E N C E SDuflo, Esther, Michael Kremer, and JonathanRobinson. 2007. “Why Don’t Farmers UseFertilizer? Experimental Evidence fromKenya.” Massachusetts Institute of Technologyand Harvard University, Working Paper.———. 2008. “How High Are Rates of Return toFertilizer? Evidence from Field Experiments inKenya.” American Economic Review 98 (2):482–88.Dyck, Alexander, David Moss, and Luigi Zingales.2008. “Media versus Special Interests.” NBERWorking Paper No. 14360.Easterly, William. 2001. The Elusive Quest forGrowth: Economists’ Adventures andMisadventures in the Tropics. Cambridge,Massachusetts: MIT Press.———. 2003. “Can Foreign Aid Buy Growth?”Journal of Economic Perspectives 17 (3):23–48.———. 2006. “Inequality Does Cause Underdevelopment:Insights from a New Instrument.”Mimeo.Easterly, William, and Stanley Fischer. 2001.“Inflation and the Poor.” Journal of Money,Credit and Banking 33 (2): 160–78.Easterly, William, and Ross Levine. 1997.“Africa’s Growth Tragedy: Policies and EthnicDivisions.” Quarterly Journal of Economics112 (4): 1203–50.———. 2003. “Tropics, Germs, and Crops: HowEndowments Influence Economic Development.”Journal of Monetary Economics 50(1): 3–39.Easterly, William, Ross Levine, and DavidRoodman. 2004. “Aid, Policies, and Growth:Comment.” American Economic Review 94(3): 774–80.Easterly, William, and Sergio Rebelo. 1993. “FiscalPolicy and Economic Growth: An EmpiricalInvestigation.” NBER Working Paper No. 4499.Easterly, William, Carlos Rodriguez, and KlausSchmidt-Hebbel (editors). 1995. Public SectorDeficits and Macroeconomic Performance.New York: Oxford University Press.Eastwood, Robert, and Michael Lipton. 2001.“Pro-Poor Growth and Pro-Growth PovertyReduction: Meaning, Evidence, and PolicyImplications.” Asian Development Review 18(2): 22–58. The Asian Development Bank.Edmonds, E.V. 2007. “Child Labor.” NBER WorkingPaper No. 12926.Eijffinger, Sylvester, and Petra Geraats. 2005.“How Transparent Are Central Banks?”European Journal of Political Economy 22(1): 1–21.Emerson, Patrick, and Andre Souza. 2003. “IsThere a Child Labor Trap? IntergenerationalPersistence of Child Labor in Brazil.” EconomicDevelopment and Cultural Change 51 (2):375–98.Epaulard, Anne. 2003. “Macroeconomic Performanceand Poverty Reduction.” IMF WorkingPaper WP/03/72.Esty, David C., and Michael E. Porter. 2005.“National Environmental Performance: AnEmpirical Analysis of Policy Results andDeterminants.” Environment and DevelopmentEconomics 10 (4): 391–434.Evans, Peter, and James Rauch. 1999. “Bureaucracyand Growth: A Cross-National Analysisof the Effects of ‘Weberian’ State Structureson Economic Growth.” American SociologicalReview 64 (5): 748–65.Ferraz, Claudio, and Federico Finan. 2008.“Exposing Corrupt Politicians: The Effects ofBrazil’s Publicly Released Audits on ElectoralOutcomes.” Quarterly Journal of Economics123 (2): 703–46.Fischer, Stanley. 1991. “Growth, Macroeconomicsand Development.” NBER Working PaperNo. 3702.———. 1993. “The Role of MacroeconomicFactors in Growth.” Journal of MonetaryEconomics 32 (3): 485–512.Field, Erica. 2002. “Entitled to Work: UrbanProperty Rights and Labor Supply in Peru.”Princeton, N.J.: Princeton University.Fields, Gary, Paul Cichello, Samuel Freije, MartaMenéndez, and David Newhouse. 2002.“Escaping Poverty: Household IncomeDynamics in Indonesia, South Africa, Spain,and Venezuela.” In Pathways Out of Poverty:Private Firms and Economic Mobility inDeveloping Countries. Gary S. Fields, andGuy Pfefferman (editors). Washington D.C.:International Finance Corporation.Fjeldstad, Odd-Helge, and Mick Moore. 2007.“Taxation and State-Building: Poor Countries1 2 5

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tin a Globalised World.” Chr. MichelsenInstitute Working Paper No. 2007:11.Fleisig, Heywood. 1995. “The Right to Borrow:Legal and Regulatory Barriers that Limit Accessto Credit by Small Farms and Businesses.”Public Policy for the Private Sector Note 44.The World Bank, Washington DC.Foster, James, and Miguel Székely. 2008. “IsEconomic Growth Good for the Poor?Tracking Low Incomes Using General Means.”International Economic Review 49 (4):1143–72.Foster, Mick, Adrian Fozzard, Felix Naschold,and Tim Conway. 2002. “How, When and WhyDoes Poverty Get Budget Priority? PovertyReduction Strategy and Public Expenditurein Five African Countries.” Overseas DevelopmentInstitute Working Paper No. 168.Fozzard, Adrian, and Mick Foster. 2001.“Changing Approaches to Public ExpenditureManagement in Low-Income Aid DependentCountries.” World Institute for DevelopmentEconomics Research (WIDER) DiscussionPaper No. 2001/17.Freund, Caroline, and Nadia Rocha. 2010. “WhatConstrains Africa’s Exports?” The World Bank,Washington, DC.Fullerton, Don. 2008. “Distributional Effects ofEnvironmental and Energy Policy: An Introduction.”NBER Working Paper No. 14241.Fullerton, Don, and Seung-Rae Kim. 2006.“Environmental Investment and Policywith Distortionary Taxes and EndogenousGrowth.” NBER Working Paper No. 12070.Gallup, John L., and Jeffrey D. Sachs, with AndrewMellinger. 1999. “Geography and EconomicDevelopment.” Center for InternationalDevelopment Working Paper No. 1, HarvardUniversity.Gemmell, Norman, and Oliver Morrissey. 2003.“Tax Structure and the Incidence on the Poorin Developing Countries.” Centre for Researchon Economic Development and InternationalTrade Research Paper No. 03/18.Gertler, Paul. 2004. “Do Conditional CashTransfers Improve Child Health? Evidencefrom PROGRESA’s Control RandomizedExperiment.” American Economic ReviewPapers and Proceedings 94 (2): 336–41.Ghura, Dhaneshwar, Carlos Leite, andCharalambos Tsangarides. 2002. “Is GrowthEnough? Macroeconomic Policy and PovertyReduction.” IMF Working Paper WP/02/118.Glewwe, Paul, Alaka Holla, and Michael Kremer.2008. “Teacher Incentives in the DevelopingWorld.” Mimeo.Glewwe, Paul, Michael Kremer, and Sylvie Moulin.2007. “Many Children Left Behind? Textbooksand Test Scores in Kenya.” NBER WorkingPaper No. 13300.Goldberg, Pinelopi, and Nina Pavcnik. 2004.“Trade, Inequality, and Poverty: What Do WeKnow? Evidence from Recent Trade LiberalizationEpisodes in Developing Countries.”NBER Working Paper No. 10593.———. 2005. “The Effects of the ColombianTrade Liberalization on Urban Poverty.” NBERWorking Paper No. 11081.Guillaumont, Patrick, and Lisa Chauvet. 2001.“Aid and Performance: A Reassessment.” TheJournal of Development Studies, Taylor andFrancis Journals 37 (6): 66–92.Gupta, Sanjeev, Benedict Clements, EmanueleBaldacci, and Carlos Mulas-Granados. 2005.“Fiscal Policy, Expenditure Composition, andGrowth in Low-Income Countries.” Journalof International Money and Finance 24 (3):441–63.Gupta, Sanjeev, Benedict Clements, AlexanderPivovarsky, and Erwin Tiongson. 2003.“Foreign Aid and Revenue Response: Doesthe Composition of Aid Matter?” IMF WorkingPaper WP/03/176.Gupta, Sanjeev, Hamid Davoodi, and Rosa AlonsoTerme. 1998. “Does Corruption Affect IncomeEquality and Poverty?” IMF Working PaperWP/98/76.———. 2002. “Does Corruption Affect IncomeDistribution?” Economic Governance 3 (1):23–45.Gupta, Sanjeev, Luis de Mello, and Raju Sharan.2000. “Corruption and Military Spending.”IMF Working Paper WP/00/23.Gyimah-Brempong, Kwabena, and SamariaMunoz de Camacho. 2006. “Corruption,Growth, and Income Distribution: AreThere Regional Differences?” Economics ofGovernance 7 (3): 245–69.1 2 6

R E F E R E N C E SHaacker, Markus. 2002. “The EconomicConsequences of HIV/AIDS in SouthernAfrica.” IMF Working Paper WP/02/38.———. 2004. “HIV/AIDS: The Impact on the SocialFabric and the Economy.” In The Macroeconomicsof HIV/AIDS, Markus Haacker, editor.Washington D.C.: The International MonetaryFund.Hall, Robert, and Charles Jones. 1999. “WhyDo Some Countries Produce So Much MoreOutput per Worker Than Others?” TheQuarterly Journal of Economics 114 (1):83–116.Hansen, Henrik, and Finn Tarp. 2000. “AidEffectiveness Disputed.” Journal of InternationalDevelopment 12 (3): 375–98.———. 2001. “Aid and Growth Regressions.”Journal of Development Economics 64 (2):547–70.Hanushek, Eric, and Dennis Kimko. 2000.“Schooling, Labor-Force Quality, and theGrowth of Nations.” American EconomicReview 90 (5): 1184–1208.Hanushek, Eric, and Ludger Woesmann. 2008.“The Role of Cognitive Skills in EconomicDevelopment.” Journal of Economic Literature46 (3): 607–68.Harrison, Ann. 2006. “Globalization and Poverty.”NBER Working Paper No. 12347.Hashemi, Syed M., Sidney Ruth Schuler, andAnn P. Riley. 1996. “Rural Credit Programsand Women’s Empowerment in Bangladesh.”World Development 24 (4): 635–53.Hausmann, Ricardo, and Michael Gavin. 1996.“Securing Stability and Growth in a Shock-Prone Region: The Policy Challenges for LatinAmerica.” In Securing Stability and Growthin Latin America, Ricardo Hausmann andHelmut Reisen (eds.). Paris: Organization forEconomic Cooperation and Development(OECD).Hay, Jonathan R., Andrei Shleifer, and RobertW. Vishny. 1996. “Privatization in TransitionEconomies, Toward a Theory of LegalReform.” European Economic Review 40(3–5): 559–67.Hazell, Peter, and Lawrence Haddad. 2001.“Agricultural Research and Poverty Reduction.”Food, Agriculture, and the EnvironmentDiscussion Paper 34. International Food PolicyResearch Institute (IFPRI), Washington, D.C.He, Fang, Leigh L. Linden, and Margaret MacLeod.2007. “Helping Teach What Teachers Don’tKnow: An Assessment of the Pratham EnglishLanguage Program.” Columbia University.Mimeo.Heady, Christopher. 2001. “Taxation Policy inLow-Income Countries.” WIDER DiscussionPaper No. 2001/81.Headey, Derek. 2005. “Foreign Aid and ForeignPolicy: How Donors Undermine the Effectivenessof Overseas Development Assistance.”Centre for Efficiency and Productivity AnalysisWorking Paper Series No. 05/2005, Australia,School of Economics, University ofQueensland.Heinemann, Alessandra, and Dorte Verner.2006. “Crime and Violence in Development.A Literature Review of Latin America and theCaribbean.” Policy Research Working Paper4041. The World Bank, Washington, DC.Hellman, Joel, Geraint Jones, and DanielKaufmann. 2000. “Seize the State, Seize theDay: State Capture, Corruption, and Influencein Transition.” Policy Research Working PaperNo. 2444. The World Bank, Washington, DC.Hettich, Frank. 1998. “Growth Effects of aRevenue-Neutral Environmental Tax Reform.”Journal of Economics 67 (3): 287–316.Hoff, Karla, and Joseph Stiglitz. 2001. “ModernEconomic Theory and Development.” InFrontiers of Development Economics: TheFuture in Perspective. New York: OxfordUniversity Press.Hoggarth, Glenn, Ricardo Reis, and VictoriaSaporta. 2002. “Costs of Banking SystemInstability: Some Empirical Evidence.” Journalof Banking and Finance 26 (5): 825–55.Holmes, K.R., B. T. Johnson, and M. Kirkpatrick.1997. The 1997 Index of Economic Freedom.New York, New York: The Heritage Foundation,Dow Jones and Company, Incorporated.Honohan, Patrick, editor. 2003. Taxation ofFinancial Intermediation: Theory andPractice for Emerging Economies. New York:Oxford University Press and Washington, D.C.:The World Bank.1 2 7

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tHonohan, Patrick. 2004a. Financial Sector Policyand the Poor. World Bank Working Paper No.43. The World Bank, Washington, DC.———. 2004b. “Financial Development, Growthand Poverty: How Close are the Links?” InFinancial Development and EconomicGrowth: Explaining the Links, CharlesGoodhart (ed.). London: Palgrave.———. 2008a. “Cross-Country Variation inHousehold Access to Financial Services.”Journal of Banking and Finance 32(11):2493–2500.———. 2008b. “Risk Management and the Costsof the Banking Crisis.” National InstituteEconomic Review 206 (1): 15–24.———. 2009. “Country Policy and InstitutionalAssessment (CPIA) Indicators and the Role ofFinance in the Development Process.” Draftbackground paper for The CPIA: An Evaluation,Independent Evaluation Group, TheWorld Bank.Honohan, Patrick, and Daniela Klingebiel. 2003.“The Fiscal Cost Implications of an AccommodatingApproach to Banking Crises.” Journalof Banking and Finance 27 (8):1539–60.Hossain, Mahabub. 1988. “Credit for Alleviation ofRural Poverty: The Grameen Bank in Bangladesh.”Research Report 65. International FoodPolicy Research Institute, Washington, DC.Huntington, Samuel. 1968. Political Order inChanging Societies. New Haven: Yale UniversityPress.Ilahi, Nadeem, Peter Orazem, and GuilhermeSedlacek. 2001. “The Implications of ChildLabor for Adult Wages, Income and Poverty:Retrospective Evidence from Brazil.” TheInternational Monetary Fund. Mimeo.IEG (Independent Evaluation Group). 2008.Environmental Sustainability: An Evaluationof World Bank Group Support. Washington,D.C.: The World Bank.Inter-American Development Bank. 1997“Studies of the Magnitude, Costs and PoliciesAgainst Violence in Six Places: Lima, Rio deJaneiro, San Salvador, Mexico, Venezuela andColombia.” Prepared for the Chief EconomicOfficer of the Inter-American DevelopmentBank. Inter-American Development Bank,Washington, DC.International Monetary Fund. 2000. “MozambiqueFiscal Transparency Study.” Aide Memoire.Irz, Xavier, Lin Lin, Colin Thirtle, and SteveWiggins. 2001. “Agricultural ProductivityGrowth and Poverty Alleviation.” DevelopmentPolicy Review 19 (4): 449–66.Isern, Jennifer, David Porteous, Raul Hernandez-Coss, and Chinyere Egwuagu. 2005. “AML/CFTRegulation: Implications for Financial ServiceProviders That Serve Low-Income People.”The Consultative Group to Assist the Poor(CGAP) Focus Note 29.Jamison, Dean, Jeffrey Sachs, and Jia Wang.2001. “The Effect of the AIDS Epidemic onEconomic Welfare in Sub-Saharan Africa.”World Health Organization Commission onMacroeconomics and Health, Working Group1, Working Paper No. 13.Jenkins, Glenn. 1994. “Modernization of Tax Administration:Revenue Boards and Privatizationas Instruments of Change.” Bulletin for InternationalFiscal Documentation 48 (2): 75–81.Johnson, Simon, John McMillan, and ChristopherWoodruff. 2002. “Property Rights andFinance.” The American Economic Review92 (5): 1335–56.Kalemli-Ozcan, Sebnem, Harl E. Ryder, and DavidN. Weil. 2000. “Mortality Decline, HumanCapital Investment, and Economic Growth.”Journal of Development Economics 62 (1):1–23.Kambou, Gerard, Shantayanan Devarajan, andMead Over. 1992. “The Economic Impact ofAIDS in an African Country: Simulations witha Computable General Equilibrium Model ofCameroon.” Journal of African Economies 1(1): 109–130.Kanbur, Ravi. 2004. “Growth, Inequality andPoverty: Some Hard Questions.” Mimeo.———. 2005. “Reforming the Formula: A ModestProposal for Introducing DevelopmentOutcomes in IDA Allocation Procedures.”Revised version published in Revue d’Economiedu Developpment.Kaufmann, Daniel. 2003. “Rethinking Governance:Empirical Lessons Challenge Orthodoxy.”Discussion draft. The World Bank.Kaufmann, Daniel, Aart Kraay, and Pablo Zoido-Lobaton. 1999. “Governance Matters.” Policy1 2 8

R E F E R E N C E SResearch Working Paper No. 2196. The WorldBank, Washington, DC.———. 2002. “Governance Matters II: UpdatedIndicators for 2000/01.” Policy ResearchWorking Paper No. 2772. The World Bank,Washington, DC.Kaufmann, Daniel, Aart Kraay, and MassimoMastruzzi. 2003. “Governance Matters III.”Policy Research Working Paper No. 3106. TheWorld Bank, Washington, DC.Kerekes, Carrie, and Claudia Williamson. 2008.“Unveiling de Soto’s Mystery: Property Rights,Capital Formation, and Development.”Journal of Institutional Economics 4 (3):299–325.Khandker, Shahidur. 1998. Fighting Poverty withMicrocredit: Experience in Bangladesh. NewYork: Oxford University Press.Khandker, Shahidur, 2005. “Microfinance andPoverty: Evidence Using Panel Data fromBangladesh.” The World Bank EconomicReview 19 (2): 263–86.Khandker, Shahidur, and Mark Pitt. 2003.“The Impact of Group-Based Credit onPoor Households: An Analysis of PanelData from Bangladesh.” The World Bank,Washington, DC.Kiser, Edgar, and Audrey Sacks. 2007. “ImprovingTax Administration in Contemporary AfricanStates: Lessons from History.” Preparedfor The Thunder of History: Taxation inComparative and Historical Perspective.Northwestern University, May 4–5.Klapper, Leora, Luc Laeven, and RaghuramRajan. 2006. “Entry Regulation as a Barrierto Entrepreneurship.” Journal of FinancialEconomics 82 (3): 591–629.Klasen, Stephan. 2002. “Low Schooling forGirls, Slower Growth for All?” World BankEconomic Review 16 (3): 345–73.Klasen, Stephan, and Francesca Lamanna.2003. “The Impact of Gender Inequality inEducation and Employment on EconomicGrowth: Updated Estimates.” Mimeo.Knack, Stephen, and Philip Keefer. 1995. “Institutionsand Economic Performance: CrossCountry Tests Using Alternative InstitutionalMeasures.” Economics and Politics 7 (3):207–27.Knowles, Stephan, Paula Lorgelly, and DorianOwen. 2002. “Are Educational Gender Gapsa Brake on Economic Development: SomeCross-Country Empirical Evidence.” OxfordEconomic Papers 54 (1): 118–49.Krebs, Tom. 2003. “Human Capital Risk andEconomic Growth.” Quarterly Journal ofEconomics 118 (2):709–44.Kremer, Michael, and Alaka Holla. 2008. “Pricingand Access: Lessons from RandomizedEvaluations in Education and Health.” Centerfor Global Development Working Paper No.158.Kremer, Michael, Jessica Leino, Edward Miguel,and Alix Peterson Zwane. 2008. “SpringCleaning: A Randomized Evaluation of SourceWater Quality Improvement.” Harvard University,Department of Economics. Mimeo.Krusell, Per, and Jose-Voctor Ríos-Rull. 1996.“Vested Interests in a Positive Theory ofStagnation and Growth.” The Review ofEconomic Studies 63 (2): 301–29.Kuznets, Simon. 1955. “Economic Growth andIncome Inequality.” American EconomicReview 45 (1): 1–28.Landes, David. 1983. Revolution in Time.Cambridge, MA: Harvard University Press.Laeven, Luc, and Fabian Valencia. 2008. “The Useof Blanket Guarantees in Banking Crises.”IMF Working Paper WP/08/250.La Porta, Rafael, Florencio Lopez-de-Silanes,and Andrei Shleifer. 2008. “The EconomicConsequences of Legal Origins.” Journal ofEconomic Literature 46 (2): 285–332.La Porta, Rafael, Florencio Lopez-de-Silanes,Andrei Shleifer, and Robert W. Vishny. 1998.“Law and Finance.” Journal of PoliticalEconomy 106 (6): 1113–55.———. 1999. “The Quality of Government.”Journal of Law, Economics and Organization15: 222–79.———. 2000. “Investor Protection and CorporateGovernance.” Journal of Financial Economics58: 3–27.Leff, Nathaniel. 1964. “Economic Developmentthrough Bureaucratic Corruption.” TheAmerican Behavioural Scientist 8 (3): 8–14.Levine, Ross. 1997. “Financial Developmentand Economic Growth: Views and Agenda.”1 2 9

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tJournal of Economic Literature 35 (2):688–726.———. 1998. “The Legal Environment, Banks,and Long-Run Economic Growth.” Journalof Money, Credit, and Banking 30 (3):596–613.———. 2003. “More on Finance and Growth:More Finance, More Growth?” The FederalReserve Bank of St. Louis.———. 2005. “Law, Endowments and PropertyRights.” Journal of Economic Perspectives 19(3):61–88.Levine, Ross, Norman Loayza, and Thorsten Beck.2000. “Financial Intermediation and Growth:Causality and Causes.” Journal of MonetaryEconomics 16 (46).Levine, Ross, and Sara Zervos. 1998. “StockMarkets, Bank and Economic Growth.”American Economic Review 88 (3): 537–58.Lewis, W. Arthur. 1954. “Economic Developmentwith Unlimited Supplies of Labour.” ManchesterSchool of Economic and Social Sciences22 (2): 139–91.Lien, Da Hsiang Donald. 1986. “A Note onCompetitive Bribery Game.” EconomicsLetters 22: 337–41.Lin, Justin Yifu. 1992. “Rural Reforms and AgriculturalGrowth in China.” American EconomicReview 82 (1): 34–51.Littlefield, Elizabeth, Jonathan Morduch, andSyed Hashemi. 2003. “Is Microfinance anEffective Strategy to Reach the MillenniumDevelopment Goals?” CGAP Focus Note 24.Loayza, Norman. 1996. “The Economics of theInformal Sector: A Simple Model and SomeEmpirical Evidence from Latin America.”Carnegie-Rochester Conference Series onPublic Policy 45 (1): 129–62.Londono, Juan Luis. 1996. “Violencia y CapitalSocial.” Paper presented at the Second AnnualConference of the World Bank on Developmentin Latin America and the Caribbean.Bogotá, Colombia.Lopez de Silanes Florencio, Simon Johnson,Rafael La Porta, and Andrei Shleifer. 2000.“Tunnelling.” NBER Working Paper No. 7523.Lui, Francis. 1985. “An Equilibrium Queuing Modelof Bribery.” Journal of Political Economics 93(4): 760–81.Luoto, Jill, Craig McIntosh, and Bruce Wydick.2007. “Credit Information Systems inLess-Developed Countries: A Test withMicrofinance in Guatemala.” EconomicDevelopment and Cultural Change 55 (2):313–34.Maine, Henry. 1980. Popular Government.London: John Murray.Martinez-Vazquez, Jorge, and Jameson Boex.2001. “Russia’s Transition to a New Federalism.”World Bank Institute, Washington, DC.Mauro, Paolo. 1995. “Corruption and Growth.”Quarterly Journal of Economics 110 (3):681–712.Mazingo, Christopher. 1999. “Effects of PropertyRights on Economic Activity: Lessons fromthe Stolypin Land Reform.” MassachusettsInstitute of Technology. Mimeo.McGillivray, Mark, Simon Feeny, Niels Hermes,and Robert Lensink. 2005. “It Works; ItDoesn’t; It Can, But That Depends? 50 Yearsof Controversy over the MacroeconomicImpact of Development Aid.” World Institutefor Development Economics Research PaperNo. 2005/54. United Nations University.McGillivray, Mark, and Oliver Morrissey.2001a. “Aid Illusion and Public Sector FiscalBehaviour.” The Journal of DevelopmentStudies 37 (6): 118–36.———. 2001b. “Fiscal Effects of Aid.” WorldInstitute for Development Economic ResearchDiscussion Paper No. 2001/61, United NationsUniversity.McLure, Charles. 1999 “Tax Holidays and InvestmentIncentives: A Comparative Analysis.”Bulletin for International Fiscal Documentation53(8–9) 326–39.Menezes-Filho, Naercio, and Ligia Vasconcellos.2004. “Operationalising Pro-Poor Growth: ACase Study on Brazil.” Prepared as part of theOperationalizing Pro-Poor Growth researchprogram.Mikesell, John. 2003. International Experienceswith Administration of Local Taxes: A Reviewof Practices and Issues. School of Public andEnvironmental Affairs, Indiana University.Miller, Margaret, editor. 2003. Credit ReportingSystems and the International Economy.Cambridge, Massachusetts: MIT Press.1 3 0

R E F E R E N C E SMocan, Naci. 2004. “What Determines Corruption?International Evidence from MicroData.” NBER Working Paper No. 10460.Mokyr, Joel. 1990. The Lever of Riches: TechnologicalCreativity and Economic Progress.New York: Oxford University Press.Montiel, Peter, and Luis Servén. 2006. “MacroeconomicStability in Developing Countries: HowMuch Is Enough? “World Bank ResearchObserver 21 (2): 151–78. Oxford UniversityPress.Mookherjee, Dilip. 1995. “Reforms in Income TaxEnforcement in Mexico.” IRIS-India WorkingPaper No. 6. University of Maryland.Mookherjee, Dileep, and Arindam Das-Gupta.1995. “Reforming Indian Income Tax Enforcement.”IRIS-India Working Paper No. 3,University of Maryland.Moore, Mick. 2004. “Taxation and the PoliticalAgenda, North and South.” Forum forDevelopment Studies 31 (1): 7–32.Moser, Caroline. 1996. “Urban Poverty andViolence: Consolidation or Erosion of SocialCapital?” Second Annual World Bank Conferenceon Development in Latin America,Bogota.Moser, Caroline, and Jeremy Holland. 1997.Urban Poverty and Violence in Jamaica.Washington, DC.: The World Bank.Muralidharan, Karthik, and Venkatesh Sundararaman.2007. “Teacher Incentives in DevelopingCountries: Experimental Evidence fromIndia.” The World Bank, Washington, DC.September. Mimeo.North, Douglas. 1990. Institutions, InstitutionalChange and Economic Performance. NewYork: Cambridge University Press.Ocampo, Jose. 2003. “Structural Dynamics andEconomic Growth in Developing Countries.”Mimeo.Olken, Benjamin. 2007. “Monitoring Corruption:Evidence from a Field Experiment in Indonesia.”Journal of Political Economy 115 (2):200–49.OED (Operations Evaluation Department). 2001.“Review of the Performance-Based AllocationSystem, IDA10–12.” The World Bank,Washington, DC, February 14.Otsuka, Keijiro. 2002, “Poverty Reduction Issues:Village Economy Perspective.” Asian DevelopmentReview 19 (1): 98–116.Over, Mead. 1992. The Macroeconomic Impact ofAIDS in Sub-Saharan Africa. Working Paper.Population and Human Resources Department.The World Bank, Washington, DC.Panday, Pranab Kumar. 2008. “RepresentationWithout Participation: Quotas for Women inBangladesh.” International Political ScienceReview 29 (4): 489–512.Parry, Ian, Hilary Sigman, Margaret Walls, andRoberton Williams. 2005. “The Incidence ofPollution Control Policies.” Rutgers University,Department of Economics, DepartmentalWorking Paper No. 2005–04.Perotti, Roberto. 1992. “Fiscal Policy, IncomeDistribution, and Growth.” Review ofEconomics and Statistics 60 (4): 755–76.———. 1993. “Political Equilibrium, IncomeDistribution, and Growth.” Review ofEconomic Studies 60 (4): 755–56.———. 1996. “Growth, Income Distribution,and Democracy: What the Data Say.” Journalof Economic Growth 1 (2): 149–87.Persson, T., and G. Tabellini. 1994. “Is InequalityHarmful for Growth?” The AmericanEconomic Review 84 (3): 600–21.Porto, Guido. 2005. “Informal Export Barriers andPoverty.” Journal of International Economics66 (2): 447–70.Posner, Richard. 1981. “The Economics ofPrivacy.” American Economic Review Papersand Proceedings 71 (2): 405–9.———. 1998. “Creating a Legal Framework forEconomic Development.” The World BankResearch Observer 13 (1): 1–11.Rajan, Raghuram, and Arvind Subramaniam.2008. “Aid and Growth: What Does the Cross-Country Evidence Really Show?” The Reviewof Economics and Statistics 90 (4): 643–65.Rajkumar, Andrew, and Vinaya Swaroop. 2002.“Public Spending and Outcomes: Does GovernanceMatter?” Policy Research Working PaperNo. 2840. The World Bank, Washington, DC.Ram, Rati. 2003. “Roles of Bilateral and MultilateralAid in Economic Growth of DevelopingCountries.” Kyklos 56 (1):95–110.Rauch, James, and Peter Evans. 2000. “BureaucraticStructure and Bureaucratic Performance1 3 1

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n tin Less Developed Countries.” Journal ofPublic Economics 75: 49–71.Ravallion, Martin. 2001. “Growth, Inequality andPoverty: Looking Beyond Averages.” WorldDevelopment 29 (11): 1803–15.———. 2004. “Looking Beyond Averages in theTrade and Poverty Debate.” Policy ResearchWorking Paper No. 3461. The World Bank,Washington, DC.Ravallion, Martin, and Gaurav Datt. 2002. “WhyHas Economic Growth Been More Pro-Poor inSome States of India than Others?” Journal ofDevelopment Economics 68: 381–400.Reddy, Sanjay, and Camelia Minoiu. 2006.“Development Aid and Economic Growth: APositive Long-Run Relation.” Working PaperNo. 29. United Nations, Department ofEconomics and Social Affairs.Reinikka, Ritva, and Jakob Svensson. 2004.“Local Capture: Evidence from a CentralGovernment Transfer Program in Uganda.”Quarterly Journal of Economics 119:679–705.Rigobon, Roberto, and Dani Rodrik. 2004. “Ruleof Law, Democracy, Openness, and Income:Estimating the Interrelationships.” NBERWorking Paper No. 10750.Rodrik, Dani. 1999. “Institutions for High-QualityGrowth: What They Are and How to AcquireThem.” Draft paper prepared for The InternationalMonetary Fund Conference on Second-Generation Reforms. Washington, D.C.,November 8–9.———. 2000. “Comments on “Trade, Growth,and Poverty” by David Dollar and Aart Kraay(editors). Mimeo.———. 2003. “Growth Strategies.” NBERWorking Paper No. 10050.Rodrik, Dani, Arvind Subramanian, and FrancescoTrebbi. 2002. “Institutions Rule: The Primacyof Institutions over Geography and Integrationin Economic Development.” NBERWorking Paper No. 9305.Roodman, David. 2007a. “The Anarchy ofNumbers: Aid, Development, and Cross-Country Empirics.” The World BankEconomic Review 21 (2): 255–77.———. 2007b. “Macro Aid EffectivenessResearch: A Guide for the Perplexed.” WorkingPaper No. 134. Center for Global Development,December.Rosegrant, Mark, and Peter Hazell. 2000.“Transforming the Rural Asian Economy:The Unfinished Revolution.” In Study ofRural Asia (Vol. 1), Asian DevelopmentBank, copublished by the Oxford UniversityPress.Rosenstein-Rodan, Paul. 1943. “Problems ofIndustrialisation of Eastern and South-Eastern Europe.” The Economic Journal 53(210/211): 202–11.Rostow, W. Walt. 1960. The Stages of EconomicGrowth: A Non-Communist Manifesto.Cambridge: Cambridge University Press.Sachs, Jeffrey. 2003. “Institutions Don’t Rule:Direct Effects of Geography on Per CapitaIncome.” NBER Working Paper No. 9490.———. 2005. The End of Poverty: EconomicPossibilities for Our Time. New York: PenguinUSA.———. 2008. Common Wealth: Economics fora Crowded Planet. New York: Penguin USA.Sachs, Jeffrey D., and Andrew Warner. 1995.“Economic Reform and the Process of GlobalIntegration.” Brookings Papers on EconomicActivity. (1): 1–118.Schneidman, Miriam. 1996. “Targeting At-Risk-Youth: Rationales, Approaches to ServiceDelivery, and Monitoring and EvaluationIssues.” Latin America and Caribbean Humanand Social Development Group Paper SeriesNo. 2. The World Bank, Washington, DC.Sen, Amartya. 1992. Inequality Re-examined.Cambridge: Harvard University Press.———. 1997. On Economic Inequality. Oxford:Clarendon Press.———. 1999. Development as Freedom. Oxford:Oxford University Press.Shavell, Steven. 2003. “Economic Analysis ofProperty Law.” NBER Working Paper No. 9695.Shleifer, Andrei, and Robert Vishny. 1993.“Corruption.” The Quarterly Journal ofEconomics 108 (3): 599–617.Skoufias, Emmanuel, Benjamin Davis, and Sergiode la Vega. 2001. “Targeting the Poor in Mexico:An Evaluation of the Selection of Householdsinto PROGRESA.” World Development 29(10): 1769–84.1 3 2

R E F E R E N C E SSteets, Julia. 2008. “Adaptation and Refinement ofthe World Bank’s Country Policy and InstitutionalAssessment.” Global Public PolicyInstitute on behalf of the German FederalMinistry for Economic Cooperation andDevelopment.Stern, Nicholas. 2001. A Strategy for Development.Washington, D.C.: The World Bank.Stevens, Michael. 1999. “Interview.” The WorldBank, Washington, DC, January 6.Summers, Lawrence. 2003. “Godkin Lectures.”Harvard University. Mimeo.Tanzi, Vito. 1998. “Corruption around the World:Causes, Scope, and Cures.” IMF WorkingPaper WP/98/63.Todaro, Michael. 1997. Economic Development.London: Longman.Topalova, Petia. 2004. “Factor Immobility andRegional Effects of Trade Liberalization:Evidence from India.” MIT. Mimeo.———. 2005. “Trade Liberalization, Poverty andInequality: Evidence from Indian Districts.”NBER Working Paper No. 11614.Treisman, Daniel. 2000. “The Causes of Corruption:A Cross-National Study.” Journal ofPublic Economics 76 (3): 399–457.Tressel, Thierry, and Enrica Detragiache. 2008.“Do Financial Sector Reforms Lead to FinancialDevelopment? Evidence from a New Dataset.”IMF Working Paper WP/08/265.Van Rijckeghem, Caroline, and Beatrice Weder.1997. “Corruption and the Rate of Temptation:Do Low Wages in the Civil Service CauseCorruption?” IMF Working Paper WP/97/73.———. 2001. “Bureaucratic Corruption andthe Rate of Temptation: Do Wages in theCivil Service Affect Corruption, and by HowMuch?” Journal of Development Economics65 (2): 307–31.Vandenbussche, Jerome, Philippe Aghion, andCostas Meghir. 2006. “Growth, Distanceto Frontier and Composition of HumanCapital.” Journal of Economic Growth 11(2): 97–127.Vermeersch, Christel, and Michael Kremer.2003. “School Meals, Educational Achievementand School Competition: Evidencefrom a Randomized Evaluation.” Universityof Oxford.Wei, Shang-Jin. 2000. “How Taxing Is Corruptionon International Investors?” Review ofEconomics and Statistics 82 (1): 1–11.Weil, David N. 2005. “Accounting for the Effect ofHealth on Economic Growth.” NBER WorkingPaper No. 11455.Welch, Karen Horn, Margaret McMillan, and DaniRodrik. 2004. “When Economic Reform GoesWrong: Cashews in Mozambique.” BrookingsTrade Forum 2003.Werlin, Herbert. 1979. “The Consequencesof Corruption: Ghanaian Experience.” InBureaucratic Corruption in Sub-SaharanAfrica, Monday U. Ekpo, editor,. Washington,D.C.: University Press of America.White, H., and E. Anderson. 2002. “Growth versusDistribution: Does the Pattern of GrowthMatter?” Development Policy Review 19 (3):267–89.Williamson, John. 1990. “What Washington Meansby Policy Reform.” In Latin American Adjustment:How Much Has Happened? WashingtonD.C.: Institute for International Economics.———. 2004. “The Washington Consensusas Policy Prescription for Development.” Alecture in the series “Practitioners of Development”delivered at The World Bank on January13, 2004.Winters, Alain, Neil McCulloch, and AndrewMcKay. 2004. “Trade Liberalization andPoverty: The Evidence So Far.” Journal ofEconomic Literature 42 (1): 72–115.Woolard, Ingrid, and Stephan Klasen. 2005.“Determinants of Income Mobility andHousehold Poverty Dynamics in South Africa.”The Journal of Development Studies 41 (5):865–97.World Bank. 1992. Development and Environment.New York: Oxford University Press.———. 1993. The East Asian Miracle: EconomicGrowth and Public Policy. New York andOxford: Oxford University Press for The WorldBank.———. 1994. World Development Report, 1994:Infrastructure for Development. Washington,D.C.: The World Bank.———. 1997. World Development Report, 1997:The State in a Changing World. Washington,D.C.: The World Bank.1 3 3

T h e W o r l d B a n k ’ s C o u n t r y P o l i c y a n d I n s t i t u t i o n a l A s s e s s m e n t———. 1998. Beyond the Washington Consensus:Institutions Matter. New York: OxfordUniversity Press.———. 1999. World Development Report1998/99: Knowledge for Development. NewYork: Oxford University Press.———. 2000a. Can Africa Claim the 21stCentury? New York: Oxford University Press.———. 2000b. World Development Report2000/2001: Attacking Poverty. New York:Oxford University Press.———. 2000c. Poverty Reduction and StrategyPaper Source Book.———. 2001a. Engendering Development. NewYork: Oxford University Press.———. 2001b. “Enhancing IDA’s Performance-Based Allocation System.” September————.2001c. Poverty Reduction StrategySourcebook: A Resource to Assist Countriesin Developing Poverty Reduction Strategies.Washington, D.C.: The World Bank.———. 2002a. Globalization, Growth andPoverty. Washington, D.C.: The World Bank.———. 2002b. “Linking IDA Support to CountryPerformance.” Third Annual Report on IDA’sCountry Assessment and Allocation Process.April 30.———. 2002c. World Bank Group Work inLow-Income Countries Under Stress: A TaskForce Report. Washington, D.C.: The WorldBank.———. 2003a. “Country Policy and InstitutionalAssessment 2003: Assessment Questionnaire.”Washington, D.C.: The World Bank.———. 2003b. “IDA’s Performance-Based AllocationSystem: Current and Emerging Issues.”October.———. 2004a. “Country Policy and InstitutionalAssessment: An External Panel Review.” TheWorld Bank, April 5.———. 2004b. “IDA’s Performance-Based AllocationSystem: IDA Rating Disclosure andFine-Tuning the Governance Factor.” September.———. 2004c. “IDA’s Performance-Based AllocationSystem: Update on Outstanding Issues.”February.————.2004d. “Disclosing IDA CountryPerformance Ratings.” Washington, D.C.: TheWorld Bank. Operations Policy and CountryServices.———. 2004e. “Reducing Poverty: SustainingGrowth.” Scaling up Poverty Reduction. AGlobal Learning Process and Conference inShanghai, The World Bank, May 25–27.———. 2004f. 2003 Annual Review of DevelopmentEffectiveness. The Effectiveness of BankSupport for Policy Reform.———. 2004g. World Development Report 2004:Making Services Work for Poor People. NewYork, NY: Oxford University Press.———. 2004h. Building State Capacity inAfrica: New Approaches, Emerging Lessons.Washington, D.C.: World Bank Institute, TheWorld Bank.———. 2005. Economic Growth in the 1990s:Learning from a Decade of Reform. TheInternational Bank for Reconstruction andDevelopment/The World Bank.———. 2006a. “Chairman’s Summary: IDA 14Mid-Term Review Meeting.” November.———. 2006b. World Development Report. NewYork: Oxford University Press.———. 2007. “Country Policy and InstitutionalAssessment 2007: Assessment Questionnaire.”The World Bank, Washington, DC.———. 2008a. Commission on Growth andDevelopment: “The Growth Report. Strategiesfor Sustained Growth and InclusiveDevelopment.” Discussion Paper. The WorldBank, Washington, DC.———. 2008b. “Country Policy and InstitutionalAssessment 2008: Assessment Questionnaire.”Operations Policy and Country Services.September 5. World Bank, Washington, DC———. 2008c. Demirguc-Kunt, Asli, ThorstenBeck, and Patrick Honohan, Finance for All?Policies and Pitfalls in Expanding Access.Washington, D.C.———. 2008d. Environmental Health and ChildSurvival: Epidemiology, Economics, Experiences.Washington, D.C.: The World Bank.———. 2008e. “IDA: The Platform for AchievingResults at the Country Level.” Additions toIDA Resources: the Fifteenth Replenishment.February.———. 2008f. Poverty and the Environment:Understanding Linkages at the Household1 3 4

R E F E R E N C E SLevel. Washington, D.C.: The World Bank.World Economic Forum. 2008. Global CompetitivenessReport 2008–2009.Young, Alwyn. 2004. “The Gift of the Dying: TheTragedy of AIDS and the Welfare of FutureAfrican Generations.” NBER Working PaperNo. 10991.Zwane, Alix Peterson, and Michael Kremer. 2007.“What Works in Fighting Diarrheal Diseasesin Developing Countries? A Critical Review.”NBER Working Paper No. 12987.1 3 5

IEG PUBLICATIONSAnnual Review of Development Effectiveness 2009: Achieving Sustainable DevelopmentAddressing the Challenges of Globalization: An Independent Evaluation of the World Bank’s Approach to Global ProgramsAssessing World Bank Support for Trade, 1987–2004: An IEG EvaluationBooks, Building, and Learning Outcomes: An Impact Evaluation of World Bank Support to Basic Education in GhanaBridging Troubled Waters: Assessing the World Bank Water Resources StrategyClimate Change and the World Bank Group—Phase I: An Evaluation of World Bank Win-Win energy Policy ReformsDebt Relief for the Poorest: An Evaluation Update of the HIPC InitiativeA Decade of Action in Transport: An Evaluation of World Bank Assistance to the Transport Sector, 1995–2005The Development Potential of Regional Programs: An Evaluation of World Bank Support of Multicountry OperationsDevelopment Results in Middle-Income Countries: An Evaluation of World Bank SupportDoing Business: An Independent Evaluation—Taking the Measure of the World Bank–IFC Doing Business IndicatorsEgypt: Positive Results from Knowledge Sharing and Modest Lending—An IEG Country Assistance Evaluation 1999 –2007Engaging with Fragile States: An IEG Review of World Bank Support to Low-Income Countries Under StressEnvironmental Sustainability: An Evaluation of World Bank Group SupportEvaluation of World Bank Assistance to Pacific Member Countries, 1992–2002Extractive Industries and Sustainable Development: An Evaluation of World Bank Group ExperienceFinancial Sector Assessment Program: IEG Review of the Joint World Bank and IMF InitiativeFrom Schooling Access to Learning Outcomes: An Unfinished Agenda—An Evaluation of World Bank Supportto Primary EducationHazards of Nature, Risks to Development: An IEG Evaluation of World Bank Assistance for Natural DisastersHow to Build M&E Systems to Support Better GovernmentIEG Review of World Bank Assistance for Financial Sector ReformAn Impact Evaluation of India’s Second and Third Andhra Pradesh Irrigation Projects: A Case of Poverty Reductionwith Low Economic ReturnsImproving Effectiveness and Outcomes for the Poor in Health, Nutrition, and PopulationImproving the Lives of the Poor through Investment in CitiesImproving Municipal Management for Cities to Succeed: An IEG Special StudyImproving the World Bank’s Development Assistance: What Does Evaluation Show:Maintaining Momentum to 2015: An Impact Evaluation of Interventions to Improve Maternal and Child Health andNutrition Outcomes in BangladeshNew Renewable Energy: A Review of the World Bank’s AssistancePakistan: An Evaluation of the World Bank’s AssistancePension Reform and the Development of Pension Systems: An Evaluation of World Bank AssistanceThe Poverty Reduction Strategy Initiative: An Independent Evaluation of the World Bank’s Support Through 2003The Poverty Reduction Strategy Initiative: Findings from 10 Country Case Studies of World Bank and IMF SupportPower for Development: A Review of the World Bank Group’s Experience with Private Participation in the ElectricitySectorPublic Sector Reform: What Works and Why? An IEG Evaluation of World Bank SupportSmall States: Making the Most of Development Assistance—A Synthesis of World Bank FindingsSocial Funds: Assessing EffectivenessSourcebook for Evaluating Global and Regional Partnership ProgramsUsing Knowledge to Improve Development Effectiveness: An Evaluation of World Bank Economic and Sector Work andTechnical Assistance, 2000–2006Using Training to Build Capacity for Development: An Evaluation of the World Bank’s Project-Based and WBI TrainingThe Welfare Impact of Rural Electrification: A Reassessment of the Costs and Benefits—An IEG Impact EvaluationWorld Bank Assistance to Agriculture in Sub-Saharan Africa: An IEG ReviewWorld Bank Assistance to the Financial Sector: A Synthesis of IEG EvaluationsWorld Bank Group Guarantee Instruments 1990–2007: An Independent EvaluationThe World Bank in Turkey: 1993–2004—An IEG Country Assistance EvaluationWorld Bank Engagement at the State Level: The Cases of Brazil, India, Nigeria, and RussiaAll IEG evaluations are available, in whole or in part, in languages other than English. For our multilingual section, please visithttp://www.worldbank.org/ieg.

ISBN 978-0-8213-8427-5