Strategy Survival Guide

More documents

Recommendations

Info

Strategy Survival Guide Version 2.1 Prime Minister’s Strategy Unit home | strategy development | strategy skills | site index Strategy Skills > Building an Evidence Base Collecting data - Surveys > in practice Surveys are a means of developing a broad, representative understanding of a situation, social attitudes or prevalent behaviour. It is helpful to first identify whether survey data is actually required or whether it is more appropriate to use data collected through other means such as focus groups, interviews with experts or practitioners or email discussion groups. If a survey data is considered necessary, a search should be conducted for previous surveys that have been undertaken that could provide raw data required. The ONS’s Guide to Official Statistics is a good starting point. If the data does already exist this would save considerable time and expense If a survey is to be conducted, it may be necessary to commission a market research company to undertake the work. This can be particularly helpful if a large amount of data needs to be collected in a short period of time. The company will also have experience of what makes a good survey, and can feed best practice into its design. However, it will be expensive and will also take some time to tender for the job, design the survey and train the market researchers to conduct it successfully. This timing should be incorporated into the project plan. Types of Survey Data Most surveys contain cross-sectional data. This provides a snapshot at a point in time. A typical crosssectional survey asks a random sample of the population the same questionnaire. As long as the sample is statistically representative, then it will give a clear guide to what answers the whole population would have given to the same questions. The larger the sample, the more confident you can be that the survey accurately represents the population's viewpoint. Alternatively a longitudinal survey may be appropriate. These trace the same individuals over time. They may range from short-term panel studies, such as when the same people are asked the same questions before and after a big event, to comprehensive studies that track individuals – and even whole families or households – over a life-time, enabling causal links to be more confidently established than when based on one-off surveys. Longitudinal data can therefore be used to analyse the impacts of policy over time (for instance over an individual's lifetime or between generations) and also permit the analysis of how policy interventions may affect the future. Things to consider when designing a survey Designing a survey is a complex task and should usually be done in collaboration with a government social researcher or specialist market research firm. Before starting to design a survey, there are a number of questions that need to be considered: • The purpose of the survey: a survey can either be descriptive or explanatory. A descriptive survey describes the distribution within a population of certain characteristics, attitudes or experience. An explanatory survey investigates the relationship between two or more variables. Explanatory surveys require that all variables that might be important are identified and measured during the data collection process. • A structured or an unstructured approach: structured approaches are useful for hypothesis testing. Unstructured approaches are more useful for acquiring population data in an area where little research has been done. • Quantitative or qualitative data: which type of data is more appropriate? Strategy Survival Guide – Strategy Skills Page 118
• The "population" and “sub-groups” to be studied: the sample to be surveyed needs to be carefully selected to ensure that the findings are similar to those found amongst your target population. There are three basic types of sampling: 1. Probability sampling. This includes random sampling, systematic sampling (similar to random sampling but some element of selection e.g. every 100th person in directory), and cluster sampling (e.g. pupils in a particular school). Consideration needs to be given to the 'sampling frame’ – such as the voting register, telephone book etc. If the sampling frame is biased, such as richer people being ex-directory or poorer people avoiding the voting register, then this problem will be reflected in the sample. 2. Non-probability sampling. This can be useful when there is insufficient information about the population (i.e. there is uncertainty about how many people or events make up the population) or the population is intrinsically difficult to survey e.g. the homeless. Non probability sampling techniques include purposive sampling (e.g. the sample is handpicked) or snowballing (those identified for inclusion in the sample nominate others). Caution must be taken in generalising from such samples. 3. Stratified sampling. This involves dividing the sampling frame into segments and 'oversampling’ sections of the population. For example, a survey might deliberately over-sample young people or ethnic minorities in order to ensure that there are sufficient in the sample to make reliable statistical comparisons. Such samples can be 're-weighted’ to give averages that are representative of the whole population. Stratified sampling is usually necessary for sub group analysis. • Optimum sample size: the sample size needs to be an adequate size, in order to generalise from the survey's findings. Provided that the sample size is representative of the target population, the larger the sample size, the more confident you can be that the results are an accurate reflection of the population as a whole. The key factor is the absolute size of the sample, rather than the proportion of the population that gets included in the sample. Adequate samples can be estimated from the expected variation in the major variables of interest, and will therefore depend on the specific question or hypothesis to be tested. As a general rule of thumb, adequate samples will generally involve more than 30 events or people. Most market research companies use samples of around 1000-2000. However, other factors to consider when deciding on the sample size include the likely response rate, the desired level of accuracy, sub-divisions in the data etc. For example, if the survey seeks to discover not only the general attitude towards an issue, but also that of married men under 40, single parents etc, then a larger sample will be needed. Advice from a statistician or social researcher will help to ensure that the chosen sample size will yield reliable and relevant data. • Data collection method: there are a variety of different methods for actually collecting survey data. Each has pros and cons: 1. self-completion postal questionnaires: this can be expensive and the typically low response rates, can result is a selection bias and hence doubt in the validity of the findings. 2. face to face interviews: market researchers may approach people in the street, or call at people’s homes. On other occasions contact will be made in advance by phone or letter. Response rate is usually higher than for postal surveys but face to face interviews tend to be more expensive. Decisions will need to be made about whether the interviews are to be structured, unstructured or partially structured. 3. telephone interviews: these are quicker and cheaper than face to face interviews, but have the highest non-response rate because people are less inhibited about saying no over the phone. Checklist for Designing a Survey or Questionnaire 1. Wording of the questions: • Style of question should be suited to target group e.g. children or professionals. The table below provides some alternative styles. • Respondents should only be required to answer about themselves, not others • Avoid the use of leading questions Strategy Survival Guide – Strategy Skills Page 119
Page 1:
Strategy Strategy Survival Survival
Page 4 and 5:
Strategy Survival Guide Version 2.1
Page 6 and 7:
Building an Evidence Base 113 Data
Page 8 and 9:
A Framework for Strategic Direction
Page 10 and 11:
These four components need to be co
Page 12 and 13:
Page 14 and 15:
Page 16 and 17:
Page 18 and 19:
Page 20 and 21:
Page 22 and 23:
Page 24 and 25:
Page 26 and 27:
Page 28 and 29:
Page 30 and 31:
Page 32 and 33:
Page 34 and 35:
Page 36 and 37:
Page 38 and 39:
Page 40 and 41:
Page 42 and 43:
Page 44 and 45:
Page 46 and 47:
Page 48 and 49:
Page 50 and 51:
Recruiting a team In Practice: SU G
Page 52 and 53:
• Commitment to a common purpose
Page 54 and 55:
Away-days A project away-day is ano
Page 56 and 57:
WHY ...are you telling me this? SO
Page 58 and 59:
Page 60 and 61:
Try to put into practice the princi
Page 62 and 63:
Encouraging creativity In Practice:
Page 64 and 65:
There are clear differences between
Page 66 and 67:
Page 68 and 69:
Page 70 and 71: Strategy Survival Guide Version 2.1
Page 80 and 81: Identifying key stakeholders and th
Page 82 and 83: Engaging with stakeholders Having i
Page 86 and 87: • What channels do we know don’
Page 90 and 91: Preparing the Story Once you have a
Page 94 and 95: development process include brainst
Page 100 and 101: Reinforcing Loops A + Balancing Loo
Page 102 and 103: Structuring the thinking - Systems
Page 104 and 105: Opportunities • Where are the goo
Page 108 and 109: Structuring the thinking - PESTLE I
Page 110 and 111: Revolution This is creativity at it
Page 112 and 113: For example, Sub-problem: My collea
Page 114 and 115: Structuring the thinking - Creativi
Page 118 and 119: in The Magenta Book) are rigorous m
Page 122 and 123: • Avoid asking the same question
Page 124 and 125: Expectations can be shaped by a num
Page 126 and 127: Collecting data - Surveys In Practi
Page 130 and 131: Collecting data - Interviews & focu
Page 132 and 133: Step One: Define the problem explic
Page 134 and 135: • Are the units correct? Step Sev
Page 136 and 137: Analysing data - Modelling In Pract
Page 140 and 141: The Five Forces Framework can be us
Page 142 and 143: References Structure, Conduct, Perf
Page 146 and 147: Symbols • What language and jargo
Page 150 and 151: • Put a contact name, telephone n
Page 152 and 153: Learning from others - Internationa
Page 154 and 155: 4. Functional Benchmarking or Gener
Page 158 and 159: Strengths • Quantitative forecast
Page 162 and 163: 3. Bring drivers together into a vi
Page 164 and 165: Scenario development In Practice 1:
Page 166 and 167: Scenarios used in the Strategy Unit
Page 168 and 169: groups at random. In other words, t
Page 170 and 171:
Page 172 and 173:
events. There are a number of diffe
Page 174 and 175:
Multi-criteria analysis In Practice
Page 176 and 177:
Page 178 and 179:
Cost-benefit & cost-effectiveness a
Page 180 and 181:
Page 182 and 183:
For further material, see the Ratio
Page 184 and 185:
Page 186 and 187:
Compelling vision for action - the
Page 188 and 189:
Page 190 and 191:
Page 192 and 193:
Designing an implementation plan In
Page 194 and 195:
Creating Public Value http://www.nu
Page 196 and 197:
National Statistics http://www.stat
Page 198 and 199:
Page 200 and 201:
Page 202 and 203:
Page 204 and 205:
Driver trees 97 Econometric modelli
Page 206 and 207:
management 17, 20, 27, 34, 45 outpu
Page 208:
Strategy Unit, Admiralty Arch, The
show all

Strategy Survival Guide

Create successful ePaper yourself

Delete template?

Save as template?