Evaluation of the Southern and Eastern Africa Regional Centre for ...

18and also any strategic planning documents that set out the programme vision, werereviewed and where possible, any theory of change identified.2) Survey- this largely quantitative method is the bedrock for the study because itinforms almost all aspects of the intervention, especially the impact questions. Keycharacteristics of the graduate cohort and key themes from findings capture thebeneficiaries’ views and also their contribution to broader processes. As proposed bythe ToR the survey provides background information on the candidates, the type oforganisation/institution they worked for before attending the women’s lawdiploma/master, information about the careers of the former students, their use ofthe theoretical knowledge in their current work, their consideration of the mostuseful aspects of their course in women’s law, the impact of the training as assessedby themselves, examples given of how they have used their knowledge, their ownoutputs (publications, papers, newspaper, radio, TV, etc), their level of contact withother former students, and their opinions about how the effect of the training bestcould be improved. In addition to these questions, we have asked about concreteachievements and contributions made to women’s law in particular and what kinds ofnetworks they are in.Some questions in the survey have gone to their current employers. Employers wereasked to assess the usefulness of the training in the work of former participants ofthe programme as well as the usefulness of the training on policy formulation. In thetwo countries where field work was conducted, we followed up on these issues ininterviews where it was deemed relevant and necessary in order to get more feedbackto assess the impact. Some of the questions in the survey were open-ended, such asgiving examples of how the candidates have used the knowledge gained and theiropinions about how training could be improved.The survey questions were formulated by Jones and Hellevik based on the questionsoutlined in the TOR and in the NIBR tender. Hellevik made the technical input intothe survey instrument used, namely, the Easyquest programme. The Easyquestprogramme is a web-based electronic survey instrument which is user friendly, cheapand makes the client the proprietor of the raw data. Tronstad and Aasen acted as testusers and gave feed back to the test questionnaire and the questions and their orderin the questionnaire was further refined before it was sent out. Hellevik organised thefinal layout and technical issues regarding the questionnaire and it was distributed to181 respondents on November 12, 2012. Two reminders were distributed torespondents before the survey was closed on December 13. The analysis of the datawas also done by use of Easyquest, supplemented by use of Excel.Previously the poor response rates in the MTR handicapped the validity ofquestionnaire sent out. The NIBR evaluation team were therefore aware that somedecrease in response would be anticipated, which in some cases may be due tomigration and HIV/AIDS. The major challenge the team anticipated was to get as upto date as possible contact information for respondents. We therefore started thework by sending out an early tracer e-mail to all former students for whom we had e-mail addresses, requesting people to reply “Yes” for having received the e-mail. Thistracer e-mail also allowed us to see which e-mail addresses were out of use. Based onthe results of this tracer e-mail, we followed up with the information manager of theCentre and the local consultant in Harare to find alternative e-mail addresses for